Nothing Special   »   [go: up one dir, main page]

skip to main content
research-article
Open access

Probabilistic Human Intent Recognition for Shared Autonomy in Assistive Robotics

Published: 11 December 2019 Publication History

Abstract

Effective human-robot collaboration in shared autonomy requires reasoning about the intentions of the human partner. To provide meaningful assistance, the autonomy has to first correctly predict, or infer, the intended goal of the human collaborator. In this work, we present a mathematical formulation for intent inference during assistive teleoperation under shared autonomy. Our recursive Bayesian filtering approach models and fuses multiple non-verbal observations to probabilistically reason about the intended goal of the user without explicit communication. In addition to contextual observations, we model and incorporate the human agent’s behavior as goal-directed actions with adjustable rationality to inform intent recognition. Furthermore, we introduce a user-customized optimization of this adjustable rationality to achieve user personalization. We validate our approach with a human subjects study that evaluates intent inference performance under a variety of goal scenarios and tasks. Importantly, the studies are performed using multiple control interfaces that are typically available to users in the assistive domain, which differ in the continuity and dimensionality of the issued control signals. The implications of the control interface limitations on intent inference are analyzed. The study results show that our approach in many scenarios outperforms existing solutions for intent inference in assistive teleoperation and otherwise performs comparably. Our findings demonstrate the benefit of probabilistic modeling and the incorporation of human agent behavior as goal-directed actions where the adjustable rationality model is user customized. Results further show that the underlying intent inference approach directly affects shared autonomy performance, as do control interface limitations.

References

[1]
Jake K. Aggarwal and Michael S. Ryoo. 2011. Human activity analysis: A review. ACM Computing Surveys 43, 3 (2011), 16.
[2]
Brenna D. Argall. 2016. Modular and adaptive wheelchair automation. In Experimental Robotics. Springer Tracks in Advanced Robotics, Vol. 109. Springer, 835--848.
[3]
Reuben M. Aronson, Santini Thiago, Thomas C. Kubler, Enkelejda Kasneci, Siddhartha Srinivasa, and Henny Admoni. 2018. Eye-hand behavior in human-robot shared manipulation. In Proceedings of 13th ACM/IEEE International Conference on Human-Robot Interaction (HRI’18).
[4]
Chris L. Baker, Rebecca Saxe, and Joshua B. Tenenbaum. 2009. Action understanding as inverse planning. Cognition 113, 3 (2009), 329--349.
[5]
Cheng-Shiu Chung, Hongwu Wang, and Rory A. Cooper. 2013. Functional assessment and performance evaluation for assistive robotic manipulators: Literature review. Journal of Spinal Cord Medicine 36, 4 (2013), 273--289.
[6]
Arridhana Ciptadi, Tucker Hermans, and James M. Rehg. 2013. An in depth view of saliency. In Proceedings of the British Machine Vision Conference (BMVC’13).
[7]
Gergely Csibra and György Gergely. 2007. ‘Obsessed with goals’: Functions and mechanisms of teleological interpretation of actions in humans. Acta Psychologica 124, 1 (2007), 60--78.
[8]
Anca Dragan and Siddhartha Srinivasa. 2014. Integrating human observer inferences into robot motion planning. Autonomous Robots 37, 4 (2014), 351--368.
[9]
Anca D. Dragan and Siddhartha S. Srinivasa. 2013. A policy-blending formalism for shared control. International Journal of Robotics Research 32, 7 (2013), 790--805.
[10]
Chinemelu Ezeh, Pete Trautman, Louise Devigne, Valentin Bureau, Marie Babel, and Tom Carlson. 2017. Probabilistic vs linear blending approaches to shared control for wheelchair driving. In Proceedings of the IEEE International Conference on Rehabilitation Robotics (ICORR’17).
[11]
Alvin I. Goldman. 1992. In defense of the simulation theory. Mind 8 Language 7, 1--2 (1992), 104--119.
[12]
Michael A. Goodrich and Dan R. Olsen. 2003. Seven principles of efficient human robot interaction. In Proceedings of the IEEE International Conference on Systems, Man, and Cybernetics.
[13]
Deepak Gopinath, Siddarth Jain, and Brenna D. Argall. 2017. Human-in-the-loop optimization of shared autonomy in assistive robotics. IEEE Robotics and Automation Letters 2, 1 (2017), 247--254.
[14]
Chien-Ming Huang and Bilge Mutlu. 2016. Anticipatory robot control for efficient human-robot collaboration. In Proceedings of 11th ACM/IEEE International Conference on Human-Robot Interaction (HRI’16).
[15]
Andreas Hula, P. Read Montague, and Peter Dayan. 2015. Monte Carlo planning method estimates planning horizons during interactive social exchange. PLoS Computational Biology 11, 6 (2015), e1004254.
[16]
Siddarth Jain and Brenna Argall. 2014. Automated perception of safe docking locations with alignment information for assistive wheelchairs. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS’14).
[17]
Siddarth Jain and Brenna Argall. 2016. Grasp detection for assistive robotic manipulation. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA’16).
[18]
Siddarth Jain and Brenna Argall. 2018. Recursive Bayesian human intent recognition in shared-control robotics. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS’18).
[19]
Siddarth Jain, Ali Farshchiansadegh, Alexander Broad, Farnaz Abdollahi, Ferdinando Mussa-Ivaldi, and Brenna Argall. 2015. Assistive robotic manipulation through shared autonomy and a body-machine interface. In Proceedings of the IEEE International Conference on Rehabilitation Robotics (ICORR’15).
[20]
Shervin Javdani, Henny Admoni, Stefania Pellegrinelli, Siddhartha S. Srinivasa, and J. Andrew Bagnell. 2018. Shared autonomy via hindsight optimization for teleoperation and teaming. International Journal of Robotics Research 37, 7 (June 2018), 717--742.
[21]
Shervin Javdani, Siddhartha Srinivasa, and Andrew Bagnell. 2015. Shared autonomy via hindsight optimization. In Proceedings of Robotics: Science and Systems.
[22]
Oussama Khatib. 1986. Real-time obstacle avoidance for manipulators and mobile robots. In Autonomous Robot Vehicles. Springer, 396--404.
[23]
Dae-Jin Kim, Rebekah Hazlett-Knudsen, Heather Culver-Godfrey, Greta Rucks, Tara Cunningham, David Portee, John Bricout, Zhao Wang, and Aman Behal. 2012. How autonomy impacts performance and satisfaction: Results from a study with spinal cord injured subjects using an assistive robot. IEEE Transactions on Systems, Man, and Cybernetics—Part A: Systems and Humans 42, 1 (2012), 2--14.
[24]
Scott Kirkpatrick, C. Daniel Gelatt, and Mario P. Vecchi. 1983. Optimization by simulated annealing. Science 220, 4598 (1983), 671--680.
[25]
Danica Kragic, Panadda Marayong, Ming Li, Allison M. Okamura, and Gregory D. Hager. 2005. Human-machine collaborative systems for microsurgical applications. International Journal of Robotics Research 24, 9 (2005), 731--741.
[26]
Dylan P. Losey, Craig G. McDonald, Edoardo Battaglia, and Marcia K. O’Malley. 2018. A review of intent detection, arbitration, and communication aspects of shared control for physical human--robot interaction. Applied Mechanics Reviews 70, 1 (2018), 010804.
[27]
Mathew Monfort, Anqi Liu, and Brian D. Ziebart. 2015. Intent prediction and trajectory forecasting via predictive inverse linear-quadratic regulation. In Proceedings of the 29th AAAI Conference on Artificial Intelligence (AAAI’15).
[28]
Chris Moore and Valerie Corkum. 1994. Social understanding at the end of the first year of life. Developmental Review 14, 4 (1994), 349--372.
[29]
Katharina Muelling, Arun Venkatraman, Jean-Sebastien Valois, John Downey, Jeffrey Weiss, Shervin Javdani, Martial Hebert, Andrew Schwartz, Jennifer Collinger, and Andrew Bagnell. 2015. Autonomy infused teleoperation with application to BCI manipulation. In Proceedings of Robotics: Science and Systems.
[30]
Katharina Muelling, Arun Venkatraman, Jean-Sebastien Valois, John E. Downey, Jeffrey Weiss, Shervin Javdani, Martial Hebert, Andrew B. Schwartz, Jennifer L. Collinger, and J. Andrew Bagnell. 2017. Autonomy infused teleoperation with application to brain computer interface controlled manipulation. Autonomous Robots 41, 6 (2017), 1401--1422.
[31]
Claudia Pérez-D’Arpino and Julie A. Shah. 2015. Fast target prediction of human reaching motion for cooperative human-robot manipulation tasks using time series classification. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA’15).
[32]
Radu Bogdan Rusu, Wim Meeussen, Sachin Chitta, and Michael Beetz. 2009. Laser-based perception for door and handle identification. In Proceedings of the IEEE International Conference on Advanced Robotics.
[33]
Pete Trautman. 2015. Assistive planning in complex, dynamic environments: A probabilistic approach. In Proceedings of the IEEE International Conference on Systems, Man, and Cybernetics (SMC’15).
[34]
Katherine Tsui, Holly Yanco, David Kontak, and Linda Beliveau. 2008. Development and evaluation of a flexible interface for a wheelchair mounted robotic arm. In Proceedings of the 3rd ACM/IEEE International Conference on Human Robot Interaction (HRI’08).
[35]
Zhikun Wang, Katharina Mülling, Marc Peter Deisenroth, Heni Ben Amor, David Vogt, Bernhard Schölkopf, and Jan Peters. 2013. Probabilistic movement modeling for intention inference in human--robot interaction. International Journal of Robotics Research 32, 7 (2013), 841--858.
[36]
David Whitney, Miles Eldon, John Oberlin, and Stefanie Tellex. 2016. Interpreting multimodal referring expressions in real time. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA’16).
[37]
Erkang You and Kris Hauser. 2011. Assisted teleoperation strategies for aggressively controlling a robot arm with 2D input. In Proceedings of Robotics: Science and Systems.
[38]
Brian D. Ziebart, Andrew L. Maas, J. Andrew Bagnell, and Anind K. Dey. 2008. Maximum entropy inverse reinforcement learning. In Proceedings of the 23rd AAAI Conference on Artificial Intelligence (AAAI’08). 1433--1438.

Cited By

View all
  • (2024)Offline Risk-sensitive RL with Partial Observability to Enhance Performance in Human-Robot TeamingProceedings of the 23rd International Conference on Autonomous Agents and Multiagent Systems10.5555/3635637.3662852(58-67)Online publication date: 6-May-2024
  • (2024)Attention-Based Variational Autoencoder Models for Human–Human Interaction Recognition via GenerationSensors10.3390/s2412392224:12(3922)Online publication date: 17-Jun-2024
  • (2024)A survey of communicating robot learning during human-robot interactionThe International Journal of Robotics Research10.1177/02783649241281369Online publication date: 7-Oct-2024
  • Show More Cited By

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Transactions on Human-Robot Interaction
ACM Transactions on Human-Robot Interaction  Volume 9, Issue 1
March 2020
181 pages
EISSN:2573-9522
DOI:10.1145/3375676
Issue’s Table of Contents
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 December 2019
Accepted: 01 August 2019
Revised: 01 May 2019
Received: 01 May 2018
Published in THRI Volume 9, Issue 1

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Human intent recognition
  2. assistive robotics
  3. assistive teleoperation
  4. human-robot interaction
  5. intent inference
  6. probabilistic modeling
  7. shared autonomy

Qualifiers

  • Research-article
  • Research
  • Refereed

Funding Sources

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)868
  • Downloads (Last 6 weeks)89
Reflects downloads up to 21 Nov 2024

Other Metrics

Citations

Cited By

View all
  • (2024)Offline Risk-sensitive RL with Partial Observability to Enhance Performance in Human-Robot TeamingProceedings of the 23rd International Conference on Autonomous Agents and Multiagent Systems10.5555/3635637.3662852(58-67)Online publication date: 6-May-2024
  • (2024)Attention-Based Variational Autoencoder Models for Human–Human Interaction Recognition via GenerationSensors10.3390/s2412392224:12(3922)Online publication date: 17-Jun-2024
  • (2024)A survey of communicating robot learning during human-robot interactionThe International Journal of Robotics Research10.1177/02783649241281369Online publication date: 7-Oct-2024
  • (2024)Gaze-Based Intention Estimation: Principles, Methodologies, and Applications in HRIACM Transactions on Human-Robot Interaction10.1145/365637613:3(1-30)Online publication date: 26-Sep-2024
  • (2024)SARI: Shared Autonomy across Repeated InteractionACM Transactions on Human-Robot Interaction10.1145/365199413:2(1-36)Online publication date: 14-Jun-2024
  • (2024)Who’s in Charge Here? A Survey on Trustworthy AI in Variable Autonomy Robotic SystemsACM Computing Surveys10.1145/364509056:7(1-32)Online publication date: 9-Apr-2024
  • (2024)Bridging HRI Theory and Practice: Design Guidelines for Robot Communication in Dairy FarmingProceedings of the 2024 ACM/IEEE International Conference on Human-Robot Interaction10.1145/3610977.3634991(137-146)Online publication date: 11-Mar-2024
  • (2024)Independence in the Home: A Wearable Interface for a Person with Quadriplegia to Teleoperate a Mobile ManipulatorProceedings of the 2024 ACM/IEEE International Conference on Human-Robot Interaction10.1145/3610977.3634964(542-551)Online publication date: 11-Mar-2024
  • (2024)Intentional User Adaptation to Shared Control AssistanceProceedings of the 2024 ACM/IEEE International Conference on Human-Robot Interaction10.1145/3610977.3634953(4-12)Online publication date: 11-Mar-2024
  • (2024)A Laser-Guided Interaction Interface for Providing Effective Robot Assistance to People With Upper Limbs ImpairmentsIEEE Robotics and Automation Letters10.1109/LRA.2024.34307099:9(7653-7660)Online publication date: Sep-2024
  • Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Login options

Full Access

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media