Abstract
Teleoperation allows a human to remotely operate a robot to perform complex and potentially dangerous tasks such as defusing a bomb, repairing a nuclear reactor, or maintaining the exterior of a space station. Existing teleoperation approaches generally rely on computer monitors to display sensor data and joysticks or keyboards to actuate the robot. These approaches use 2D interfaces to view and interact with a 3D world, which can make using them difficult for complex or delicate tasks. To address this problem, we introduce a virtual reality interface that allows users to remotely teleoperate a physical robot in real-time. Our interface allows users to control their point of view in the scene using virtual reality, increasing situational awareness (especially of object contact), and to directly move the robot’s end effector by moving a hand controller in 3D space, enabling fine-grained dexterous control. We evaluated our interface on a cup-stacking manipulation task with 18 participants, comparing the relative effectiveness of a keyboard and mouse interface, virtual reality camera control, and positional hand tracking. Our system reduces task completion time from 153 s (\(\pm 44\)) to 53 s (\(\pm 37\)), a reduction of 66%, while improving subjective assessments of system usability and workload. Additionally, we have shown the effectiveness of our system over long distances, successfully completing a cup stacking task from over 40 miles away. Our paper contributes a quantitative assessment of robot grasping teleoperation across desktop and virtual reality interfaces.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
Our code is at https://github.com/h2r/ros_reality.
- 2.
If a requested end effector pose is not possible to attain due to collision or not being in the robot’s work space, then the robot does not move.
- 3.
The WASD keys governed horizontal movement, Q and E moved the arm down and up, and R and F opened and closed the grippers. The shift key switched translational movement to rotational. This control scheme is based on control schemes used in space-flight simulator games like Kerbal Space Progam.
References
Interactive Manipulation IM Description. http://projects.csail.mit.edu/pr2/wiki/index.php?title=Interactive_Manipulation
IVRE - An Immersive Virtual Robotics Environment. https://cirl.lcsr.jhu.edu/research/human-machine-collaborative-systems/ivre/
Mind Meld. https://www.youtube.com/watch?v=kZlg0QvKkQQ
PR2 Surrogate. http://wiki.ros.org/pr2_surrogate
Bangor, A., Kortum, P.T., Miller, J.T.: An empirical evaluation of the system usability scale. Int. J. Hum.-Comput. Interact. 24(6), 574–594 (2008)
Brooke, J., et al.: Sus-a quick and dirty usability scale. Usability Evaluation in Industry, vol. 189(194), pp. 4–7 (1996)
Calli, B., Singh, A., Walsman, A., Srinivasa, S., Abbeel, P., Dollar, A.M.: The ycb object and model set: towards common benchmarks for manipulation research. In: 2015 International Conference on Advanced Robotics (ICAR), pp. 510–517. IEEE (2015)
Chen, Y.-P., Lee, S.-Y., Howard, A.M.: Effect of virtual reality on upper extremity function in children with cerebral palsy: a meta-analysis. Pediatr. Phys. Ther. 26(3), 289–300 (2014)
N. H. P. R. Group et al. Task load index (nasa-tlx) v1. 0 computerised version. NASA Ames Research Centre (1987)
Kam, H.R., Lee, S.-H., Park, T., Kim, C.-H.: Rviz: a toolkit for real domain data visualization. Telecommun. Syst. 60(2), 337–345 (2015)
Koenig, N., Howard, A.: Design and use paradigms for gazebo, an open-source multi-robot simulator. In: 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2004. (IROS 2004). Proceedings, vol. 3, pp. 2149–2154. IEEE (2004)
Lohse, K.R., Hilderman, C.G., Cheung, K.L., Tatla, S., Van der Loos, H.M.: Virtual reality therapy for adults post-stroke: a systematic review and meta-analysis exploring virtual environments and commercial games in therapy. PloS one 9(3), e93318 (2014)
Moroney, W.F., Biers, D.W., Eggemeier, F.T., Mitchell, J.A.: A comparison of two scoring procedures with the nasa task load index in a simulated flight task. In: Proceedings of the IEEE 1992 National Aerospace and Electronics Conference, 1992. NAECON 1992, pp. 734–740. IEEE (1992)
Pratt, G., Manzo, J.: The darpa robotics challenge [competitions]. IEEE Robot. Autom. Mag. 20(2), 10–12 (2013)
Quigley, M., Conley, K., Gerkey, B., Faust, J., Foote, T., Leibs, J., Wheeler, R., Ng, A.Y.: Ros: an open-source robot operating system. In: ICRA Workshop on Open Source Software, vol. 3, p. 5. Kobe (2009)
Serrano, D.: Introduction to ros–robot operating system
Zucker, M., Joo, S., Grey, M.X., Rasmussen, C., Huang, E., Stilman, M., Bobick, A.: A general-purpose system for teleoperation of the drc-hubo humanoid robot. J. Field Robot. 32(3), 336–351 (2015)
Acknowledgements
This material is based upon work supported by the Defense Advanced Research Projects Agency (DARPA) under Grants No. W911NF-15-1-0503, YFA: D15AP00104, YFA: GR5245014, and D15AP00102, as well as NASA under Grants No. GR5227035.
Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of DARPA.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Whitney, D., Rosen, E., Phillips, E., Konidaris, G., Tellex, S. (2020). Comparing Robot Grasping Teleoperation Across Desktop and Virtual Reality with ROS Reality. In: Amato, N., Hager, G., Thomas, S., Torres-Torriti, M. (eds) Robotics Research. Springer Proceedings in Advanced Robotics, vol 10. Springer, Cham. https://doi.org/10.1007/978-3-030-28619-4_28
Download citation
DOI: https://doi.org/10.1007/978-3-030-28619-4_28
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-28618-7
Online ISBN: 978-3-030-28619-4
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)