Abstract
Robotic systems typically follow a rigid approach to task execution, in which they perform the necessary steps in a specific order, but fail when having to cope with issues that arise during execution. We propose an approach that handles such cases through dialogue and human-robot collaboration. The proposed approach contributes a hierarchical control architecture that 1) autonomously detects and is cognizant of task execution failures, 2) initiates a dialogue with a human helper to obtain assistance, and 3) enables collaborative human-robot task execution through extended dialogue in order to 4) ensure robust execution of hierarchical tasks with complex constraints, such as sequential, non-ordering, and multiple paths of execution. The architecture ensures that the constraints are adhered to throughout the entire task execution, including during failures. The recovery of the architecture from issues during execution is validated by a human-robot team on a building task.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Arkin, R.C.: An Behavior-based Robotics, 1st edn. MIT Press, Cambridge (1998)
Beetz, M., et al.: Robotic roommates making pancakes. In: 2011 11th IEEE-RAS International Conference on Humanoid Robots, pp. 529–536, October 2011
Blankenburg, J., et al.: A distributed control architecture for collaborative multi-robot task allocation. In: 2017 IEEE-RAS 17th International Conference on Humanoid Robotics (Humanoids), pp. 585–592, November 2017
Bradski, G., Kaehler, A.: Learning OpenCV: Computer vision with the OpenCV library. O’Reilly Media, Inc. (2008)
Breazeal, C., Hoffman, G., Lockerd, A.: Teaching and working with robots as a collaboration. In: Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems-Volume 3, pp. 1030–1037. IEEE Computer Society (2004)
Fong, T., Thorpe, C., Baur, C.: Multi-robot remote driving with collaborative control. IEEE Trans. Ind. Electron. 50(4), 699–704 (2003)
Fong, T., Thorpe, C., Baur, C.: Robot, asker of questions. Robot. Autonomous Syst. 42(3), 235–243 (2003). https://doi.org/10.1016/S0921-8890(02)00378-0, socially Interactive Robots
Fraser, L., Rekabdar, B., Nicolescu, M., Nicolescu, M., Feil-Seifer, D., Bebis, G.: A compact task representation for hierarchical robot control. In: 2016 IEEE-RAS 16th International Conference on Humanoid Robots (Humanoids), pp. 697–704, November 2016. https://doi.org/10.1109/HUMANOIDS.2016.7803350
Gassend, B.: sound\_play ros package. http://wiki.ros.org/sound_play
Hayes, B., Scassellati, B.: Autonomously constructing hierarchical task networks for planning and human-robot collaboration. In: 2016 IEEE International Conference on Robotics and Automation (ICRA), pp. 5469–5476. IEEE (2016)
Hayes, B., Shah, J.A.: Improving robot controller transparency through autonomous policy explanation. In: 2017 12th ACM/IEEE International Conference on Human-Robot Interaction (HRI), pp. 303–312. IEEE (2017)
Hernandez, C., et al.: Team delft’s robot winner of the amazon picking challenge 2016. In: Behnke, S., Sheh, R., Sarıel, S., Lee, D.D. (eds.) RoboCup 2016: Robot World Cup XX, pp. 613–624. Springer International Publishing, Cham (2017)
Huggins-Daines, D., Kumar, M., Chan, A., Black, A.W., Ravishankar, M., Rudnicky, A.I.: Pocketsphinx: a free, real-time continuous speech recognition system for hand-held devices. In: 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings, vol. 1, p. I. IEEE (2006)
Knepper, R.A., Tellex, S., Li, A., Roy, N., Rus, D.: Recovering from failure by asking for help. Auton. Robots 39(3), 347–362 (2015). https://doi.org/10.1007/s10514-015-9460-1
Mohseni-Kabir, A., Rich, C., Chernova, S., Sidner, C.L., Miller, D.: Interactive hierarchical task learning from a single demonstration. In: Proceedings of the Tenth Annual ACM/IEEE International Conference on Human-Robot Interaction, pp. 205–212. ACM (2015)
Quigley, M., et al.: ROS: an open-source Robot Operating System. In: ICRA Workshop on Open Source Software, vol. 3 (2009)
Rybski, P.E., Yoon, K., Stolarz, J., Veloso, M.M.: Interactive robot task training through dialog and demonstration. In: Proceedings of the ACM/IEEE International Conference on Human-Robot Interaction, pp. 49–56. ACM (2007)
Schillinger, P., Kohlbrecher, S., von Stryk, O.: Human-robot collaborative high-level control with application to rescue robotics. In: 2016 IEEE International Conference on Robotics and Automation (ICRA), pp. 2796–2802. IEEE (2016)
Sucan, I.A., Chitta, S.: Moveit! (2013). http://moveit.ros.org
Acknowledgment
This work was supported by the National Science Foundation (IIS-1757929) and by the Office of Naval Research (ONR) award #N00014-16-1-2312.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Blankenburg, J., Zagainova, M., Simmons, S.M., Talavera, G., Nicolescu, M., Feil-Seifer, D. (2020). Human-Robot Collaboration and Dialogue for Fault Recovery on Hierarchical Tasks. In: Wagner, A.R., et al. Social Robotics. ICSR 2020. Lecture Notes in Computer Science(), vol 12483. Springer, Cham. https://doi.org/10.1007/978-3-030-62056-1_13
Download citation
DOI: https://doi.org/10.1007/978-3-030-62056-1_13
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-62055-4
Online ISBN: 978-3-030-62056-1
eBook Packages: Computer ScienceComputer Science (R0)