extended-abstract

Public Access

Investigating Learning from Demonstration in Imperfect and Real World Scenarios

Authors:

Erin Hedlund-Botti,

Matthew C. GombolayAuthors Info & Claims

HRI '23: Companion of the 2023 ACM/IEEE International Conference on Human-Robot Interaction

Pages 769 - 771

https://doi.org/10.1145/3568294.3579980

Published: 13 March 2023 Publication History

Abstract

As the world's population is aging and there are growing shortages of caregivers, research into assistive robots is increasingly important. Due to differing needs and preferences, which may change over time, end-users will need to be able to communicate their preferences to a robot. Learning from Demonstration (LfD) is one method that enables non-expert users to program robots. While a powerful tool, prior research in LfD has made assumptions that break down in real-world scenarios. In this work, we investigate how to learn from suboptimal and heterogeneous demonstrators, how users react to failure with LfD, and the feasibility of LfD with a target population of older adults.

References

[1]

Saleema Amershi, Maya Cakmak, W. Bradley Knox, and Todd Kulesza. 2014. Power to the people: The role of humans in interactive machine learning. AI Magazine, Vol. 35, 4 (2014), 105--120. https://doi.org/10.1609/aimag.v35i4.2513

Digital Library

[2]

Brenna Argall, Sonia Chernova, Manuela M. Veloso, and Brett Browning. 2009. A survey of robot learning from demonstration. Robotics and Autonomous Systems, Vol. 57, 5 (May 2009), 469--483. https://doi.org/10.1016/j.robot.2008.10.024

Digital Library

[3]

Christoph Bartneck, Elizabeth Croft, and Dana Kulic. 2009. Measurement instruments for the anthropomorphism, animacy, likeability, perceived intelligence, and perceived safety of robots. International Journal of Social Robotics, Vol. 1, 1 (2009), 71--81. https://doi.org/10.1007/s12369-008-0001-3

[4]

Jenay M. Beer, Akanksha Prakash, Cory-Ann Smarr, Tracy L. Mitzner, Charles C. Kemp, and Wendy A. Rogers. 2012. "Commanding Your Robot" Older Adults' Preferences for Methods of Robot Control. Proceedings of the Human Factors and Ergonomics Society. .. Annual Meeting. Human Factors and Ergonomics Society. Annual meeting, Vol. 56, 1 (Sept. 2012), 1263--1267. https://doi.org/10.1177/1071181312561224

[5]

John Brooke. 1995. SUS: A quick and dirty usability scale. Usability Eval. Ind., Vol. 189 (11 1995).

[6]

Daniel Brown, Wonjoon Goo, Prabhat Nagarajan, and Scott Niekum. 2019. Extrapolating Beyond Suboptimal Demonstrations via Inverse Reinforcement Learning from Observations. In Proceedings of the 36th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 97), Kamalika Chaudhuri and Ruslan Salakhutdinov (Eds.). PMLR, 783--792.

[7]

Daniel S. Brown, Wonjoon Goo, and Scott Niekum. 2020. Better-than-Demonstrator Imitation Learning via Automatically-Ranked Demonstrations. In Proceedings of the Conference on Robot Learning (Proceedings of Machine Learning Research, Vol. 100), Leslie Pack Kaelbling, Danica Kragic, and Komei Sugiura (Eds.). PMLR, 330--359.

[8]

Letian Chen, Rohan Paleja, and Matthew Gombolay. 2021. Learning from Suboptimal Demonstration via Self-Supervised Reward Regression. In Proceedings of the 2020 Conference on Robot Learning (Proceedings of Machine Learning Research, Vol. 155), Jens Kober, Fabio Ramos, and Claire Tomlin (Eds.). PMLR, 1262--1277.

[9]

Sonia Chernova and Andrea L. Thomaz. 2014. Robot Learning from Human Teachers. Morgan & Claypool Publishers.

Digital Library

[10]

Sonia Chernova and Manuela M. Veloso. 2009. Interactive Policy Learning through Confidence-Based Autonomy. Journal of Artificial Intelligence Research, Vol. 34 (2009), 1--25.

Digital Library

[11]

Joseph DelPreto, Jeffrey I. Lipton, Lindsay Sanneman, Aidan J. Fay, Christopher Fourie, Changhyun Choi, and Daniela Rus. 2020. Helping Robots Learn: A Human-Robot Master-Apprentice Model Using Demonstrations via Virtual Reality Teleoperation. In 2020 IEEE International Conference on Robotics and Automation (ICRA). 10226--10233. https://doi.org/10.1109/ICRA40945.2020.9196754

[12]

Peter A. Hancock, Deborah R. Billings, Kristin E. Schaefer, Jessie Y.C. Chen, Ewart J. De Visser, and Raja Parasuraman. 2011. A Meta-Analysis of Factors Affecting Trust in Human-Robot Interaction. Human Factors, Vol. 53, 5 (2011), 517--527. https://doi.org/10.1177/0018720811417254 ISBN: 0018720811417.

[13]

Sandra G Hart and Lowell E Staveland. 1988. Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. Human mental workload, Vol. 1, 3 (1988), 139--183.

[14]

Erin Hedlund, Michael Johnson, and Matthew Gombolay. 2021. The Effects of a Robot's Performance on Human Teachers for Learning from Demonstration Tasks. In Proceedings of the 2021 ACM/IEEE International Conference on Human-Robot Interaction (HRI '21). Association for Computing Machinery, New York, NY, USA, 207--215. https://doi.org/10.1145/3434073.3444664 event-place: Boulder, CO, USA.

Digital Library

[15]

Guy Hoffman. 2019. Evaluating Fluency in Human-Robot Collaboration. IEEE Transactions on Human-Machine Systems, Vol. 49, 3 (2019), 209--218. https://doi.org/10.1109/THMS.2019.2904558

[16]

Jiun-Yin Jian, Ann Bisantz, and Colin Drury. 2000. Foundations for an Empirically Determined Scale of Trust in Automated Systems. International Journal of Cognitive Ergonomics, Vol. 4 (2000), 53--71. https://doi.org/10.1207/S15327566IJCE0401_04

[17]

Michael Laskey, Caleb Chuck, Jonathan Lee, Jeffrey Mahler, Sanjay Krishnan, Kevin Jamieson, Anca Dragan, and Ken Goldberg. 2017. Comparing human-centric and robot-centric sampling for robot deep learning from demonstrations. In 2017 IEEE International Conference on Robotics and Automation (ICRA). 358--365. https://doi.org/10.1109/ICRA.2017.7989046

Digital Library

[18]

Meredith Mealer, Ellen L. Burnham, Colleen J. Goode, Barbara Rothbaum, and Marc Moss. 2009. The prevalence and impact of post traumatic stress disorder and burnout syndrome in nurses. Depression and Anxiety, Vol. 26, 12 (2009), 1118--1126. https://doi.org/10.1002/da.20631

[19]

Neville Moray, Toshiyuki Inagaki, and Makoto Itoh. 2000. Adaptive Automation, trust, and self-confidence in fault management of time-critical tasks. Journal of Experimental Psychology: Applied, Vol. 6, 1 (2000), 44--58. https://doi.org/10.1037/0278--7393.6.1.44

[20]

Donald A. Norman and Stephen W. Draper. 1986. User Centered System Design; New Perspectives on Human-Computer Interaction. L. Erlbaum Associates Inc., USA.

Digital Library

[21]

Harish Ravichandar, Athanasios S. Polydoros, Sonia Chernova, and Aude Billard. 2020. Recent Advances in Robot Learning from Demonstration. Annual Review of Control, Robotics, and Autonomous Systems, Vol. 3, 1 (May 2020), 297--330. https://doi.org/10.1146/annurev-control-100819-063206 Publisher: Annual Reviews.

[22]

Maha Salem, Gabriella Lakatos, Farshid Amirabdollahian, and Kerstin Dautenhahn. 2015. Would You Trust a (Faulty) Robot?: Effects of Error, Task Type and Personality on Human-Robot Cooperation and Trust. ACM/IEEE International Conference on Human-Robot Interaction, Vol. 2015-March (2015), 141--148. https://doi.org/10.1145/2696454.2696497 ISBN: 9781450328821 Publisher: ACM.

Digital Library

[23]

Claude Sammut. 1992. Automatically Constructing Control Systems by Observing Human Behaviour. Second International Inductive Logic Programming Workshop (1992).

[24]

Joe Saunders, Dag Sverre Syrdal, Kheng Lee Koay, Nathan Burke, and Kerstin Dautenhahn. 2016. ?Teach Me--Show Me"-End-User Personalization of a Smart Home and Companion Robot. IEEE Transactions on Human-Machine Systems, Vol. 46, 1 (2016), 27--40. https://doi.org/10.1109/THMS.2015.2445105

[25]

Mariah L. Schrum, Erin Hedlund-Botti, and Matthew Gombolay. 2022a. Reciprocal MIND MELD: Improving Learning From Demonstration via Personalized, Reciprocal Teaching. In 6th Annual Conference on Robot Learning.

[26]

Mariah L. Schrum, Erin Hedlund-Botti, Nina Moorman, and Matthew C. Gombolay. 2022b. MIND MELD: Personalized Meta-Learning for Robot-Centric Imitation Learning. In Proceedings of the 2022 ACM/IEEE International Conference on Human-Robot Interaction (HRI '22). IEEE Press, 157--165.

[27]

Cory-Ann Smarr, Tracy L. Mitzner, Jenay M. Beer, Akanksha Prakash, Tiffany L. Chen, Charles C. Kemp, and Wendy A. Rogers. 2014. Domestic Robots for Older Adults: Attitudes, Preferences, and Potential. International journal of social robotics, Vol. 6, 2 (April 2014), 229--247. https://doi.org/10.1007/s12369-013-0220-0

[28]

Jonathan Spencer, Sanjiban Choudhury, Matthew Barnes, Matthew Schmittle, Mung Chiang, Peter Ramadge, and Siddhartha Srinivasa. 2020. Learning from Interventions: Human-robot interaction as both explicit and implicit feedback. In Robotics: Science and Systems, Marc Toussaint, Antonio Bicchi, and Tucker Hermans (Eds.). MIT Press Journals, United States. https://doi.org/10.15607/RSS.2020.XVI.055

[29]

Adriana Tapus, Maja J. Mataric, and Brian Scassellati. 2007. Socially assistive robotics [Grand Challenges of Robotics]. IEEE Robotics & Automation Magazine, Vol. 14, 1 (March 2007), 35--42. https://doi.org/10.1109/MRA.2007.339605 Conference Name: IEEE Robotics & Automation Magazine.

[30]

Andrea L. Thomaz and Cynthia Breazeal. 2008. Teachable Robots: Understanding Human Teaching Behavior to Build More Effective Robot Learners. Artif. Intell., Vol. 172, 6--7 (April 2008), 716--737. https://doi.org/10.1016/j.artint.2007.09.009 io

Digital Library

Cited By

Gong XFeng DXu KZhai YYao CWang WDing BWang HSalakhutdinov RKolter ZHeller KWeller AOliver NScarlett JBerkenkamp F(2024)Iterative regularized policy optimization with imperfect demonstrationsProceedings of the 41st International Conference on Machine Learning10.5555/3692070.3694360(55547-55568)Online publication date: 21-Jul-2024
https://dl.acm.org/doi/10.5555/3692070.3694360
Kumar RHuang YYao NGrollman DBroadbent EJu WSoh HWilliams T(2024)Unveiling the Dynamics of Human Decision-Making: From Strategies to False Beliefs in Collaborative Human-Robot Co-Learning TasksCompanion of the 2024 ACM/IEEE International Conference on Human-Robot Interaction10.1145/3610978.3640743(632-636)Online publication date: 11-Mar-2024
https://dl.acm.org/doi/10.1145/3610978.3640743

Index Terms

Investigating Learning from Demonstration in Imperfect and Real World Scenarios
1. Human-centered computing
  1. Accessibility
    1. Accessibility design and evaluation methods
  2. Human computer interaction (HCI)
    1. Empirical studies in HCI
    2. HCI design and evaluation methods
      1. User studies

Recommendations

Impacts of Robot Learning on User Attitude and Behavior
HRI '23: Proceedings of the 2023 ACM/IEEE International Conference on Human-Robot Interaction

With an aging population and a growing shortage of caregivers, the need for in-home robots is increasing. However, it is intractable for robots to have all functionalities pre-programmed prior to deployment. Instead, it is more realistic for robots to ...
The Effects of a Robot's Performance on Human Teachers for Learning from Demonstration Tasks
HRI '21: Proceedings of the 2021 ACM/IEEE International Conference on Human-Robot Interaction

Learning from Demonstration (LfD) algorithms seek to enable end-users to teach robots new skills through human demonstration of a task. Previous studies have analyzed how robot failure affects human trust, but not in the context of the human teaching ...
What Makes a Good Demonstration for Robot Learning Generalization?
HRI '21 Companion: Companion of the 2021 ACM/IEEE International Conference on Human-Robot Interaction

Robot learning from demonstration (LfD) is a common approach that allows robots to perform tasks after observing teacher's demonstrations. Thus, users without a robotics background could use LfD to teach robots. However, such users may provide low-...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

HRI '23: Companion of the 2023 ACM/IEEE International Conference on Human-Robot Interaction

March 2023

612 pages

ISBN:9781450399708

DOI:10.1145/3568294

General Chairs:
Ginevra Castellano
Uppsala University, Sweden
,
Laurel Riek
University of California San Diego, USA
,
Program Chairs:
Maya Cakmak
University of Washington, USA
,
Iolanda Leite
KTH Royal Institute of Technology, Sweden

Copyright © 2023 Owner/Author.

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 March 2023

Check for updates

Author Tags

Qualifiers

Extended-abstract

Funding Sources

National Science Foundation
Konica Minolta
NASA

Conference

HRI '23

Sponsor:

HRI '23: ACM/IEEE International Conference on Human-Robot Interaction

March 13 - 16, 2023

Stockholm, Sweden

Acceptance Rates

Overall Acceptance Rate 268 of 1,124 submissions, 24%

Upcoming Conference

HRI '25

Sponsor:
sigai
sigai

ACM/IEEE International Conference on Human-Robot Interaction

March 4 - 6, 2025

Melbourne , VIC , Australia

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
224
Total Downloads

Downloads (Last 12 months)122
Downloads (Last 6 weeks)23

Reflects downloads up to 02 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Gong XFeng DXu KZhai YYao CWang WDing BWang HSalakhutdinov RKolter ZHeller KWeller AOliver NScarlett JBerkenkamp F(2024)Iterative regularized policy optimization with imperfect demonstrationsProceedings of the 41st International Conference on Machine Learning10.5555/3692070.3694360(55547-55568)Online publication date: 21-Jul-2024
https://dl.acm.org/doi/10.5555/3692070.3694360
Kumar RHuang YYao NGrollman DBroadbent EJu WSoh HWilliams T(2024)Unveiling the Dynamics of Human Decision-Making: From Strategies to False Beliefs in Collaborative Human-Robot Co-Learning TasksCompanion of the 2024 ACM/IEEE International Conference on Human-Robot Interaction10.1145/3610978.3640743(632-636)Online publication date: 11-Mar-2024
https://dl.acm.org/doi/10.1145/3610978.3640743

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Figures

Tables

Media

View Table of Conten