Shared-Control Teleoperation Paradigms on a Soft-Growing Robot Manipulator
Authors: 
Fabio Stroppa, Mario Selvaggio, Nathaniel Agharese, Ming Luo, Laura H. Blumenschein, Elliot W. Hawkes, Allison M. OkamuraAuthors Info & Claims
Abstract
Semi-autonomous telerobotic systems allow both humans and robots to exploit their strengths while enabling personalized execution of a remote task. For soft robots with kinematic structures dissimilar to those of human operators, it is unknown how the allocation of control between the human and the robot changes the performance. This work presents a set of interaction paradigms between a human and a remote soft-growing robot manipulator, with demonstrations in both real and simulated scenarios. The soft robot can grow and retract by eversion and inversion of its tubular body, a property we exploit in the interaction paradigms. We implemented and tested six different human-robot interaction paradigms, with full teleoperation at one extreme and gradually adding autonomy to various aspects of the task execution. All paradigms are demonstrated by two experts and two naive operators. Results show that humans and the soft robot manipulator can effectively split their control along different degrees of freedom while acting simultaneously to accomplish a task. In the simple pick-and-place task studied in this work, performance improves as the control is gradually given to the robot’s autonomy, especially when the robot can correct certain human errors. However, human engagement is maximized when the control over a task is at least partially shared. Finally, when the human operator is assisted by haptic guidance, which is computed based on soft robot tip position errors, we observed that the improvement in performance is dependent on the expertise of the human operator.
References
[1]
de Graaf, M.M., Allouch, S.B., van Dijk, J. (2016) Long-term acceptance of social robots in domestic environments: Insights from a user’s perspective. In: AAAI Spring Symposia
[2]
Scalise, R., Bisk, Y., Forbes, M., Yi, D., Choi, Y., Srinivasa, S. (2018) Balancing shared autonomy with human-robot communication. arXiv:1805.07719
[3]
Schilling, M., Kopp, S., Wachsmuth, S., Wrede, B., Ritter, H., Brox, T., Nebel, B., Burgard, W. (2016) Towards a multidimensional perspective on shared autonomy. In: AAAI fall symposium series, pp. 338–344
[4]
Coad MM, Thomasson RP, Blumenschein LH, Usevitch NS, Hawkes EW, and Okamura AM Retraction of soft growing robots without buckling IEEE Robotics & Automation Letters 2020 5 2 2115-2122
[5]
El-Hussieny, H., Mehmood, U., Mehdi, Z., Jeong, S.-G., Usman, M., Hawkes, E.W., Okamura, A.M., Ryu, J.-H. (2018) Development and evaluation of an intuitive flexible interface for teleoperating soft growing robots. In: IEEE/RSJ international conference on intelligent robots and systems, pp. 4995–5002
[6]
Mörtl A, Lawitzky M, Kucukyilmaz A, Sezgin M, Basdogan C, and Hirche S The role of roles: Physical cooperation between humans and robots The International Journal of Robotics Research 2012 31 13 1656-1674
[7]
Corrales J, Gomez GG, Torres F, and Perdereau V Cooperative tasks between humans and robots in industrial environments International Journal of Advanced Robotic Systems 2012 9 3 94
[8]
Pitzer, B., Styer, M., Bersch, C., DuHadway, C., Becker, J. (2011) Towards perceptual shared autonomy for robotic mobile manipulation. In: IEEE international conference on robotics and automation, pp. 6245–6251
[9]
Kortenkamp D, Keirn-Schreckenghost D, and Bonasso RP Adjustable control autonomy for manned space flight IEEE Aerospace Conference (AeroConf) 2000 7 629-640
[10]
Sellner, B., Simmons, R., Singh, S. (2005) User modelling for principled sliding autonomy in human-robot teams. In: Multi-Robot Systems. From Swarms to Intelligent Automata vol. 3, pp. 197–208. Springer
[11]
Dias, M.B., Kannan, B., Browning, B., Jones, E., Argall, B., Dias, M.F., Zinck, M., Veloso, M., Stentz, A. (2008) Sliding autonomy for peer-to-peer human-robot teams. In: International Conference on Intelligent Autonomous Systems, pp. 332–341
[12]
Rieffel J, Knox D, Smith S, and Trimmer B Growing and evolving soft robots Artificial life 2014 20 1 143-162
[13]
Coad MM, Blumenschein LH, Cutler S, Zepeda JAR, Naclerio ND, El-Hussieny H, Mehmood U, Ryu J-H, Hawkes EW, and Okamura AM Vine robots: Design, teleoperation, and deployment for navigation and exploration IEEE Robotics & Automation Magazine 2020 27 3 120-132
[14]
Hawkes EW, Blumenschein LH, Greer JD, and Okamura AM A soft robot that navigates its environment through growth Science Robotics 2017 2 8 3028
[15]
Sadeghi A, Mondini A, and Mazzolai B Toward self-growing soft robots inspired by plant roots and based on additive manufacturing technologies Soft Robotics 2017 4 3 211-223
[16]
Tsukagoshi H, Arai N, Kiryu I, and Kitagawa A Tip growing actuator with the hose-like structure aiming for inspection on narrow terrain International Journal of Automation Technology 2011 5 4 516-522
[17]
Dehghani H, Welch CR, Pourghodrat A, Nelson CA, Oleynikov D, Dasgupta P, and Terry BS Design and preliminary evaluation of a self-steering, pneumatically driven colonoscopy robot Journal of Medical Engineering & Technology 2017 41 3 223-236
[18]
Blumenschein LH, Gan LT, Fan JA, Okamura AM, and Hawkes EW A tip-extending soft robot enables reconfigurable and deployable antennas IEEE Robotics & Automation Letters 2018 3 2 949-956
[19]
Luong, J., Glick, P., Ong, A., deVries, M.S., Sandin, S., Hawkes, E.W., Tolley, M.T. (2019) Eversion and retraction of a soft robot towards the exploration of coral reefs. In: IEEE International Conference on Soft Robotics (RoboSoft), pp. 801–807
[20]
Naclerio, N.D., Hubicki, C.M., Aydin, Y.O., Goldman, D.I., Hawkes, E.W. (2018) Soft robotic burrowing device with tip-extension and granular fluidization. In: 2018 IEEE/RSJ international conference on intelligent robots and systems (IROS), pp. 5918–5923. IEEE
[21]
Stroppa, F., Luo, M., Yoshida, K., Coad, M.M., Blumenschein, L.H., Okamura, A.M. (2020) Human interface for teleoperated object manipulation with a soft growing robot. In: IEEE International Conference on Robotics and Automation, pp. 726–732
[22]
Goodrich MA and Schultz AC Human-robot interaction: a survey Foundations and Trends in Human-Computer Interaction 2008 1 3 203-275
[23]
Selvaggio M, Cognetti M, Nikolaidis S, Ivaldi S, and Siciliano B Autonomy in physical human-robot interaction: A brief survey IEEE Robotics and Automation Letters 2021 6 4 7989-7996
[24]
Yang G-Z, Cambias J, Cleary K, Daimler E, Drake J, Dupont PE, Hata N, Kazanzides P, Martel S, Patel RV, Santos VJ, and Taylor RH Medical robotics—regulatory, ethical, and legal considerations for increasing levels of autonomy Science Robotics 2017 2 4 8638
[25]
Kanda T and Ishiguro H Human-robot Interaction in Social Robotics 2017 CRC Press
[26]
Schilling, M., Burgard, W., Muelling, K., Wrede, B., Ritter, H.: Shared autonomy—learning of joint action and human-robot collaboration. Frontiers in Neurorobotics 13, 16 (2019)
[27]
Bruemmer, D.J., Dudenhoeffer, D.D., Marble, J.L.(2002) Dynamic-autonomy for urban search and rescue. In: AAAI mobile robot competition, pp. 33–37
[28]
Desai, M., Yanco, H.A. (2005) Blending human and robot inputs for sliding scale autonomy. In: IEEE International Workshop on Robot and Human Interactive Communication (ROMAN), pp. 537–542
[29]
Anderson, S., Peters, S., Iagnemma, K., Overholt, J. (2010) Semi-autonomous stability control and hazard avoidance for manned and unmanned ground vehicles. In: The 27th Army Science Conference, pp. 1–8
[30]
Selvaggio, M., Giordano, P.R., Ficuciello, F., Siciliano, B. (2019) Passive task-prioritized shared-control teleoperation with haptic guidance. In: IEEE International Conference on Robotics and Automation, pp. 430–436
[31]
Kofman J, Wu X, Luu TJ, and Verma S Teleoperation of a robot manipulator using a vision-based human-robot interface IEEE Transactions on Industrial Electronics 2005 52 5 1206-1219
[32]
Griffin, W.B., Provancher, W.R., Cutkosky, M.R. (2005) Feedback strategies for telemanipulation with shared control of object handling forces. Presence: Teleoperators & Virtual Environments 14(6), 720–731
[33]
Li, M., Okamura, A.M. (2003)Recognition of operator motions for real-time assistance using virtual fixtures. In: IEEE Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, pp. 125–131
[34]
Kragic D, Marayong P, Li M, Okamura AM, and Hager GD Human-machine collaborative systems for microsurgical applications The International Journal of Robotics Research 2005 24 9 731-741
[35]
Shen J, Ibanez-Guzman J, Ng TC, and Chew BS A collaborative-shared control system with safe obstacle avoidance capability IEEE Conference on Robotics, Automation and Mechatronics 2004 1 119-123
[36]
Dragan AD and Srinivasa SS A policy-blending formalism for shared control The International Journal of Robotics Research 2013 32 7 790-805
[37]
Javdani, S., Srinivasa, S.S., Bagnell, J.A. (2015) Shared autonomy via hindsight optimization. In: Robotics Science and Systems. NIH Public Access
[38]
Crandall, J.W., Goodrich, M.A. (2002) Characterizing efficiency of human robot interaction: A case study of shared-control teleoperation. In: IEEE/RSJ international conference on intelligent robots and systems, pp. 1290–1295
[39]
Aarno, D., Ekvall, S., Kragic, D. (2005) Adaptive virtual fixtures for machine-assisted teleoperation tasks. In: IEEE International Conference on Robotics and Automation, pp. 1139–1144
[40]
Selvaggio, M., Notomista, G., Chen, F., Gao, B., Trapani, F., Caldwell, D. (2016) Enhancing bilateral teleoperation using camera-based online virtual fixtures generation. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 1483–1488
[41]
Selvaggio M, Abi-Farraj F, Pacchierotti C, Robuffo Giordano P, and Siciliano B Haptic-based shared-control methods for a dual-arm system IEEE Robotics & Automation Letters 2018 3 4 4249-4256
[42]
Selvaggio, M., Ghalamzan E., A.M., Moccia, R., Ficuciello, F., Siciliano, B. (2019) Haptic-guided shared control for needle grasping optimization in minimally invasive robotic surgery. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 3617–3623
[43]
O’Malley, M.K., Gupta, A., Gen, M., Li, Y.(2006) Shared control in haptic systems for performance enhancement and training. Journal of Dynamic Systems, Measurement, and Control 128
[44]
Squeri, V., Basteris, A., Sanguineti, V. (2011) Adaptive regulation of assistance ‘as needed’ in robot-assisted motor skill learning and neuro-rehabilitation. In: IEEE International Conference on Rehabilitation Robotics, pp. 1–6
[45]
Guidali, M., Schlink, P., Duschau-Wicke, A., Riener, R. (2011) Online learning and adaptation of patient support during adl training. In: IEEE International Conference on Rehabilitation Robotics, pp. 1–6
[46]
Stroppa, F., Marcheschi, S., Mastronicola, N., Loconsole, C., Frisoli, A. (2017) Online adaptive assistance control in robot-based neurorehabilitation therapy. In: IEEE International Conference on Rehabilitation Robotics, pp. 628–633
[47]
Greer, J.D., Morimoto, T.K., Okamura, A.M., Hawkes, E.: A soft, steerable continuum robot that grows via tip extension. Soft Robotics 6(1), 95–108 (2019)
[48]
Coad, M.M. (2021) Design, modeling, and control of vine robots for exploration of unknown environments. PhD thesis, Stanford University
[49]
Gruebele, A.M., Zerbe, A.C., Coad, M.M., Okamura, A.M., Cutkosky, M.R. (2020) Distributed sensor networks deployed using soft growing robots. In: IEEE International Conference on Soft Robotics (RoboSoft), p
[50]
Rohmer, E., Singh, S.P., Freese, M. (2013) V-rep: A versatile and scalable robot simulation framework. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 1321–1326
[51]
Nuño E, Basañez L, and Ortega R Passivity-based control for bilateral teleoperation: A tutorial Automatica 2011 47 3 485-495
[52]
Chen, J.Y., Haas, E.C., Barnes, M.J. (2007) Human performance issues and user interface design for teleoperated robots. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) 37(6), 1231–1245
Recommendations
Interactive Teleoperation Interface for Semi-autonomous Control of Robot Arms
CRV '14: Proceedings of the 2014 Canadian Conference on Computer and Robot VisionWe propose and develop an interactive semi-autonomous control of robot arms. Our system controls two interactions: (1) A user can naturally control a robot arm by a direct linkage to the arm motion from the tracked human skeleton. (2) An autonomous ...
Trilateral teleoperation control of kinematically redundant robotic manipulators
Teleoperation control of kinematically redundant robots requires a strategy for resolving their redundancy. A trilateral two-master/one-slave control approach is proposed for delay-free applications in which the first master controls a primary task ...
A shared control method for obstacle avoidance with mobile robots and its interaction with communication delay
Teleoperation allows human operators to safely extend themselves to remote environments that are typically difficult or dangerous to access. The remote environments are often unstructured i.e. not having clear roads or paths to follow and only ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
© The Author(s), under exclusive licence to Springer Nature B.V. 2023. Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
Publisher
Kluwer Academic Publishers
United States
Publication History
Published: 22 September 2023
Accepted: 16 June 2023
Received: 04 March 2022
Author Tags
Qualifiers
- Research-article
Funding Sources
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 07 Mar 2025