research-article

Open access

Curiosity Did Not Kill the Robot: A Curiosity-based Learning System for a Shopkeeper Robot

Authors:

Malcolm Doering,

Takayuki Kanda,

Hiroshi IshiguroAuthors Info & Claims

ACM Transactions on Human-Robot Interaction (THRI), Volume 8, Issue 3

Article No.: 15, Pages 1 - 24

https://doi.org/10.1145/3326462

Published: 23 July 2019 Publication History

All formats PDF

Abstract

Learning from human interaction data is a promising approach for developing robot interaction logic, but behaviors learned only from offline data simply represent the most frequent interaction patterns in the training data, without any adaptation for individual differences. We developed a robot that incorporates both data-driven and interactive learning. Our robot first learns high-level dialog and spatial behavior patterns from offline examples of human--human interaction. Then, during live interactions, it chooses among appropriate actions according to its curiosity about the customer's expected behavior, continually updating its predictive model to learn and adapt to each individual. In a user study, we found that participants thought the curious robot was significantly more humanlike with respect to repetitiveness and diversity of behavior, more interesting, and better overall in comparison to a non-curious robot.

References

[1]

H. Admoni and B. Scassellati. 2014. Data-driven model of nonverbal behavior for socially assistive human-robot interactions. In Proceedings of the ACM International Conference on Multimodal Interaction (ICMI’14). ACM, New York, NY, 196--199.

Digital Library

[2]

C. Breazeal, N. Depalma, J. Orkin, S. Chernova, and M. Jung. 2013. Crowdsourcing human-robot interaction: New methods and system evaluation in a public environment. J. Hum.-Robot Interact. 2, 1 (2013), 82--111.

Digital Library

[3]

A. Breuing and I. Wachsmuth. 2012. Let's talk topically with artificial agents! Providing agents with humanlike topic awareness in everyday dialog situations. In Proceedings of the International Conference on Agents and Artificial Intelligence (ICAART’12).

[4]

D. Brscic, T. Kanda, T. Ikeda, and T. Miyashita. 2013. Person tracking in large public spaces using 3-D range sensors. IEEE Trans. Hum.-Mach. Syst. 43, 6 (2013), 522--534.

[5]

M. Cakmak and A. L. Thomaz. 2012. Designing robot learners that ask good questions. In Proceedings of the 7th Annual ACM/IEEE International Conference on Human--Robot Interaction. ACM, New York, NY, 17--24.

Digital Library

[6]

M. T. Chan, R. Gorbet, P. Beesley, and D. Kulič. 2015. Curiosity-based learning algorithm for distributed interactive sculptural systems. In Proceedings of the 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS’15). IEEE, Los Alamitos, CA, 3435--3441.

[7]

C.-W. Chang, J.-H. Lee, P.-Y. Chao, C.-Y. Wang, and G.-D. Chen. 2010. Exploring the possibility of using humanoid robots as instructional tools for teaching a second language in primary school. Educ. Technol. Soc. 13, 2 (2010), 13--24.

[8]

S. Chernova, N. Depalma, E. Morant, and C. Breazeal. 2011. Crowdsourcing human-robot interaction: Application from virtual to physical worlds. In Proceedings of the IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN’11). IEEE, 21--26.

[9]

D. A. Cohn, Z. Ghahramani, and M. I. Jordan. 1996. Active learning with statistical models. J. Artif. Intell. Res. 4 (1996), 129--145.

[10]

M. E. Foster, S. Keizer, Z. Wang, and O. Lemon. 2012. Machine learning of social states and skills for multi-party human-robot interaction. In Proceedings of the Workshop on Machine Learning for Interactive Systems (MLIS’12). 9.

[11]

D. Fox, W. Burgard, and S. Thrun. 1997. The dynamic window approach to collision avoidance. IEEE Robot. Autom. Mag. 4, 1 (1997), 23--33.

[12]

G. Gordon, C. Breazeal, and S. Engel. 2015. Can children catch curiosity from a social robot? In Proceedings of the 10th Annual ACM/IEEE International Conference on Human-Robot Interaction. ACM, New York, NY, 91--98.

Digital Library

[13]

E. T. Hall. 1966. The Hidden Dimension. Doubleday, Garden City, NY.

[14]

T. Hester and P. Stone. 2017. Intrinsically motivated model learning for developing curious robots. Artif. Intell. 247 (2017), 170--186.

Digital Library

[15]

S. Ioffe and C. Szegedy. 2015. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In Proceedings of the International Conference on Machine Learning, 448--456.

Digital Library

[16]

K. Jokinen, H. Tanaka, and A. Yokoo. 1998. Context management with topics for spoken dialogue systems. In Proceedings of the International Conference on Computational Linguistics (COLING’98). ACL, New York, NY, 631--637.

Digital Library

[17]

T. Kanda, T. Hirano, D. Eaton, and H. Ishiguro. 2004. Interactive robots as social partners and peer tutors for children: A field trial. Hum.-Comput. Interact. 19, 1 (2004), 61--84.

Digital Library

[18]

F. Kaplan and P.-Y. Oudeyer. 2011. From Hardware and Software to Kernels and Envelopes: A Concept Shift for Robotics, Developmental Psychology, and Brain Sciences. Cambridge University Press, Cambridge.

[19]

H. Kawai, T. Toda, J. Ni, M. Tsuzaki, and K. Tokuda. 2004. XIMERA: A new TTS from ATR based on corpus-based technologies. In Proceedings of the 5th ISCA Workshop on Speech Synthesis.

[20]

T. Kitade, S. Satake, T. Kanda, and M. Imai. 2013. Understanding suitable locations for waiting. In Proceedings of the 8th ACM/IEEE International Conference on Human--Robot Interaction IEEE Press, Los Alamitos, CA, 57--64.

Digital Library

[21]

H. Kozima, M. P. Michalowski, and C. Nakagawa. 2009. Keepon. Int. J. Soc. Robot. 1, 1 (2009), 3--18.

[22]

T. K. Landauer, P. W. Foltz, and D. Laham. 1998. An introduction to latent semantic analysis. Disc. Process. 25, 2--3 (1998), 259--284.

[23]

R. Langevin. 1971. Is curiosity a unitary construct? Can. J. Psychol. 25, 4 (1971), 360.

[24]

E. Law, V. Cai, Q. F. Liu, S. Sasy, J. Goh, A. Blidaru, and D. Kulic. 2017. A wizard-of-oz study of curiosity in human-robot interaction. In Proceedings of the IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN’17).

[25]

I. Leite, A. Pereira, A. Funkhouser, B. Li, and J. F. Lehman. 2016. Semi-situated learning of verbal and nonverbal content for repeated human-robot interaction. In Proceedings of the ACM International Conference on Multimodal Interaction (ICMI’14). ACM, New York, NY, 13--20.

Digital Library

[26]

P. Liu, D. F. Glas, T. Kanda, and H. Ishiguro. 2016. Data-driven HRI: Learning social behaviors by example from human--human interaction. IEEE Trans. Robotics 32 (2016), 988--1008.

[27]

P. Liu, D. F. Glas, T. Kanda, and H. Ishiguro. 2017a. Learning proactive behavior for interactive social robots. Auton. Robots 42, 5 (2017), 1067--1085.

Digital Library

[28]

P. Liu, D. F. Glas, T. Kanda, and H. Ishiguro. 2017b. Two demonstrators are better than one-a social robot that learns to imitate people with different interaction styles. IEEE Trans. Cogn. Dev. Syst. Early access.

[29]

K. Madani, C. Sabourin, and D. M. Ramík. 2016. Artificial curiosity emerging human-like behavior: Toward fully autonomous cognitive robots. In Computational Intelligence Springer, Berlin, 501--516.

[30]

M. P. Michalowski, S. Sabanovic, and H. Kozima. 2007. A dancing robot for rhythmic social interaction. In Proceedings of the 2007 2nd ACM/IEEE International Conference on Human-Robot Interaction (HRI’07). IEEE, Los Alamitos, CA, 89--96.

Digital Library

[31]

S. Müller, S. Sprenger, and H.-M. Gross. 2014. Online adaptation of dialog strategies based on probabilistic planning. In Proceedings of the 23rd IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN’14). IEEE, Los Alamitos, CA, 692--697.

[32]

Y. Nagai, C. Muhl, and K. J. Rohlfing. 2008. Toward designing a robot that learns actions from parental demonstrations. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA’08). IEEE, Los Alamitos, CA, 3545--3550.

[33]

P.-Y. Oudeyer, F. Kaplan, and V. V. Hafner. 2007. Intrinsic motivation systems for autonomous mental development. IEEE Trans. Evol. Comput. 11 (2007), 265--286.

Digital Library

[34]

R. P. Petrick and M. E. Foster. 2012. What would you like to drink? Recognising and planning with social states in a robot bartender domain. In Proceedings of the International Workshop on Cognitive Robotics (CogRob’12).

[35]

A. H. Qureshi, Y. Nakamura, Y. Yoshikawa, and H. Ishiguro. 2018. Intrinsically motivated reinforcement learning for human--robot interaction in the real-world. Neur. Netw. 107 (2018), 23--33.

[36]

R. Rojas. 1996. The backpropagation algorithm. In Neural Networks. Springer, Berlin, 149--182.

[37]

C. A. Rothkopf and D. H. Ballard. 2010. Credit assignment in multiple goal embodied visuomotor behavior. Front. Psychol. 1, 173 (2010).

[38]

J. M. Santos, T. Krajník, and T. Duckett. 2017. Spatio-temporal exploration strategies for long-term autonomy of mobile robots. Robot. Auton. Syst. 88 (2017), 116--126.

Digital Library

[39]

J. Schmidhuber. 2013. Maximizing fun by creating data with easily reducible subjective complexity. In Intrinsically Motivated Learning in Natural and Artificial Systems. Springer, 95--128.

[40]

B. Settles. 2012. Active learning. Synth. Lect. Artif. Intell. Mach. Learn. 6, 1 (2012), 1--114.

[41]

K. Severinson-Eklundh, A. Green, and H. Hüttenrauch. 2003. Social and collaborative aspects of interaction with a service robot. Robot. Auton. Syst. 42, 3 (2003), 223--234.

[42]

C. Shi, T. Kanda, M. Shimada, F. Yamaoka, H. Ishiguro, and N. Hagita. 2010. Easy development of communicative behaviors in social robots. In Proceedings of the 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS’10). 5302--5309.

[43]

T. Sinha, Z. Bai, and J. Cassell. 2017. Curious minds wonder alike: Studying multimodal behavioral dynamics to design social scaffolding of curiosity. Arxiv Preprint Arxiv:1705.00204.

[44]

J. Thomason, A. Padmakumar, J. Sinapov, J. Hart, P. Stone, and R. J. Mooney. 2017. Opportunistic active learning for grounding natural language descriptions. In Proceedings of the Conference on Robot Learning. 67--76.

[45]

A. L. Thomaz and C. Breazeal. 2006. Reinforcement learning with human teachers: Evidence of feedback and guidance with implications for learning performance. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI’06). 1000--1005.

Digital Library

[46]

A. L. Thomaz and C. Breazeal. 2008. Teachable robots: Understanding human teaching behavior to build more effective robot learners. Artif. Intell. 172, 6--7 (2008), 716--737.

Digital Library

[47]

R. Toris, D. Kent, and S. Chernova. 2014. The robot management system: A framework for conducting human-robot interaction studies through crowdsourcing. J. Hum.-Robot Interact. 3, 2 (2014), 25--49.

Digital Library

[48]

R. Triebel, K. Arras, R. Alami, L. Beyer, S. Breuers, R. Chatila, M. Chetouani, D. Cremers, V. Evers, and M. Fiore. 2016. Spencer: A socially aware service robot for passenger guidance and help in busy airports. In Field and Service Robotics. Springer, Berlin, 607--622.

[49]

Q. A. Wang. 2008. Probability distribution and entropy as a measure of uncertainty. J. Phys. A: Math. Theor. 41, 6 (2008), 065004.

[50]

J. D. Williams, A. Raux, D. Ramachandran, and A. W. Black. 2013. The dialog state tracking challenge. In Proceedings of the Special Interest Group on Discourse and Dialogue Conference (SIGdial’13). 404--413.

[51]

J. D. Williams and S. Young. 2007. Partially observable markov decision processes for spoken dialog systems. Comput. Speech Lang. 21, 2 (2007), 393--422.

Digital Library

[52]

F. Yamaoka, T. Kanda, H. Ishiguro, and N. Hagita. 2008. How close?: Model of proximity control for information-presenting robots. In Proceedings of the 3rd ACM/IEEE International Conference on Human--Robot Interaction. ACM, New York, NY, 137--144.

Digital Library

[53]

J. E. Young, E. Sharlin, and T. Igarashi. 2013. Teaching robots style: Designing and evaluating style-by-demonstration for interactive robotic locomotion. Hum.--Comput. Interact. 28, 5 (2013), 379--416.

Digital Library

[54]

T. Zhang, R. Ramakrishnan, and M. Livny. 1996. BIRCH: An efficient data clustering method for very large databases. In ACM Sigmod Record, Vol. 25. ACM, New York, NY, 103--114.

Digital Library

Cited By

Chen RMinato TSakai KKanda T(2024)Meet the Motivational Robot That Predicts Your Future FeelingsACM Transactions on Human-Robot Interaction10.1145/370746614:2(1-32)Online publication date: 6-Dec-2024
https://dl.acm.org/doi/10.1145/3707466
Pan XDoering MKanda TGrollman DBroadbent EJu WSoh HWilliams T(2024)What Is Your Other Hand Doing, Robot? A Model of Behavior for Shopkeeper Robot's Idle HandProceedings of the 2024 ACM/IEEE International Conference on Human-Robot Interaction10.1145/3610977.3634986(552-560)Online publication date: 11-Mar-2024
https://dl.acm.org/doi/10.1145/3610977.3634986
Ravishankar JDoering MKanda TGrollman DBroadbent EJu WSoh HWilliams T(2024)Zero-Shot Learning to Enable Error Awareness in Data-Driven HRIProceedings of the 2024 ACM/IEEE International Conference on Human-Robot Interaction10.1145/3610977.3634940(592-601)Online publication date: 11-Mar-2024
https://dl.acm.org/doi/10.1145/3610977.3634940
Show More Cited By

Index Terms

Curiosity Did Not Kill the Robot: A Curiosity-based Learning System for a Shopkeeper Robot

Recommendations

Robot Curiosity in Human-Robot Interaction (RCHRI)
HRI '22: Proceedings of the 2022 ACM/IEEE International Conference on Human-Robot Interaction

One of the fundamental modes of learning in children is through curiosity. Children (and adults) interact with new people, learn about novel objects, activities and other stimuli through curiosity and other intrinsic motivations. Creating autonomous ...
Can Children Catch Curiosity from a Social Robot?
HRI '15: Proceedings of the Tenth Annual ACM/IEEE International Conference on Human-Robot Interaction

Curiosity is key to learning, yet school children show wide variability in their eagerness to acquire information. Recent research suggests that other people have a strong influence on children's exploratory behavior. Would a curious robot elicit ...
To kill a mockingbird robot
HRI '07: Proceedings of the ACM/IEEE international conference on Human-robot interaction

Robots are being introduced in our society but their social status is still unclear. A critical issue is if the robot's exhibition of intelligent life-like behavior leads to the users' perception of animacy. The ultimate test for the life-likeness of a ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Transactions on Human-Robot Interaction

ACM Transactions on Human-Robot Interaction Volume 8, Issue 3

September 2019

128 pages

EISSN:2573-9522

DOI:10.1145/3349339

Editors:
Odest Chadwicke Jenkins
University of Michigan, USA
,
Selma Sabanovic
Indiana University, USA

Issue’s Table of Contents

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 July 2019

Accepted: 01 April 2019

Revised: 01 January 2019

Received: 01 June 2018

Published in THRI Volume 8, Issue 3

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

JST, ERATO

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

14
Total Citations
View Citations
1,526
Total Downloads

Downloads (Last 12 months)173
Downloads (Last 6 weeks)16

Reflects downloads up to 13 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Chen RMinato TSakai KKanda T(2024)Meet the Motivational Robot That Predicts Your Future FeelingsACM Transactions on Human-Robot Interaction10.1145/370746614:2(1-32)Online publication date: 6-Dec-2024
https://dl.acm.org/doi/10.1145/3707466
Pan XDoering MKanda TGrollman DBroadbent EJu WSoh HWilliams T(2024)What Is Your Other Hand Doing, Robot? A Model of Behavior for Shopkeeper Robot's Idle HandProceedings of the 2024 ACM/IEEE International Conference on Human-Robot Interaction10.1145/3610977.3634986(552-560)Online publication date: 11-Mar-2024
https://dl.acm.org/doi/10.1145/3610977.3634986
Ravishankar JDoering MKanda TGrollman DBroadbent EJu WSoh HWilliams T(2024)Zero-Shot Learning to Enable Error Awareness in Data-Driven HRIProceedings of the 2024 ACM/IEEE International Conference on Human-Robot Interaction10.1145/3610977.3634940(592-601)Online publication date: 11-Mar-2024
https://dl.acm.org/doi/10.1145/3610977.3634940
Niu XIto ANose T(2024)Scheduled Curiosity-Deep Dyna-Q: Efficient Exploration for Dialog Policy LearningIEEE Access10.1109/ACCESS.2024.3376418(1-1)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3376418
Shin MJang MCho MRyu JCastellano GRiek LCakmak MLeite I(2023)Uncertainty-Resolving Questions for Social RobotsCompanion of the 2023 ACM/IEEE International Conference on Human-Robot Interaction10.1145/3568294.3580077(226-230)Online publication date: 13-Mar-2023
https://dl.acm.org/doi/10.1145/3568294.3580077
Hedayati HSeo SKanda TRea DAndrist SNakano YIshiguro HCastellano GRiek LCakmak MLeite I(2023)Symbiotic Society with Avatars (SSA)Companion of the 2023 ACM/IEEE International Conference on Human-Robot Interaction10.1145/3568294.3579964(953-955)Online publication date: 13-Mar-2023
https://dl.acm.org/doi/10.1145/3568294.3579964
Doering MBrščić DKanda T(2021)Data-Driven Imitation Learning for a Shopkeeper Robot with Periodically Changing Product InformationACM Transactions on Human-Robot Interaction10.1145/345188310:4(1-20)Online publication date: 14-Jul-2021
https://dl.acm.org/doi/10.1145/3451883
Bennett C(2021)Evoking an Intentional Stance during Human-Agent Social Interaction: Appearances Can Be Deceiving2021 30th IEEE International Conference on Robot & Human Interactive Communication (RO-MAN)10.1109/RO-MAN50785.2021.9515420(362-368)Online publication date: 8-Aug-2021
https://dl.acm.org/doi/10.1109/RO-MAN50785.2021.9515420
Iqbal TRiek L(2021)Temporal Anticipation and Adaptation Methods for Fluent Human-Robot Teaming2021 IEEE International Conference on Robotics and Automation (ICRA)10.1109/ICRA48506.2021.9561763(3736-3743)Online publication date: 30-May-2021
https://dl.acm.org/doi/10.1109/ICRA48506.2021.9561763
Morimoto DEven JKanda TBelpaeme TYoung JGunes HRiek L(2020)Can a Robot Handle Customers with Unreasonable Complaints?Proceedings of the 2020 ACM/IEEE International Conference on Human-Robot Interaction10.1145/3319502.3374830(579-587)Online publication date: 9-Mar-2020
https://dl.acm.org/doi/10.1145/3319502.3374830
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

Figures

Tables

Media

View Issue’s Table of Contents