The Impact of Robot's Body Language on Customer Experience: An Analysis in a Cafe Setting
Authors: 
Nguyen Tan Viet Tuyen, Mateusz Adamski, Yuchen Wang, Shintaro Okazaki, Oya CeliktutanAuthors Info & Claims
Pages 248 - 252
Abstract
Nonverbal communication plays a crucial role in human-robot interaction (HRI) and have been widely used for robots in service environments. While few studies have addressed the understanding customer's acceptance of robots under many different interaction conditions, the impact of robots' nonverbal interaction modalities (i.e., a combination of body language, voice, and touch) on customers' experience has not been investigated truly. To this end, in this paper, we introduce an HRI framework that aims to assist customers in their food and beverage choices in a real-world cafe setting. With this framework, the contribution of this paper are two folds. We introduce a time-synchronised multisensory HRI dataset comprising the interactions between a social robot and customers in a real-world environment. We conduct a user study to evaluate the configuration of multimodal HRI framework, particularly nonverbal gestures, and its contribution to customers' interaction experience in this specific marketing setting.
Supplementary Material
In this presentation video, we introduce a multimodal HRI framework designed for the Pepper humanoid robot to assist customers in their drink and food choices in a real-world cafe setting. Based on a subjective study conducted, we discuss the impact of the robot's nonverbal gestures on customer interaction experience in this specific marketing setting.
- Download
- 48.33 MB
References
[1]
Justine Cassell, Hannes Högni Vilhjálmsson, and Timothy Bickmore. 2004. Beat: the behavior expression animation toolkit. In Life-Like Characters. Springer, 163--185.
[2]
Laurens De Gauquier, Malaika Brengman, Kim Willems, Hoang-Long Cao, and Bram Vanderborght. 2021. In or out? A field observational study on the placement of entertaining robots in retailing. International Journal of Retail & Distribution Management, Vol. 49, 7 (2021), 846--874.
[3]
Roland Graef, Mathias Klier, Andreas Obermeier, and Jan Felix Zolitschka. 2022. What to buy, pepper?--Bridging the physical and the digital world with recommendations from humanoid robots. Journal of Decision Systems (2022), 1--27.
[4]
Dai Hasegawa, Naoshi Kaneko, Shinichi Shirakawa, Hiroshi Sakuta, and Kazuhiko Sumi. 2018. Evaluation of speech-to-gesture generation using bi-directional LSTM network. In Proceedings of the 18th International Conference on Intelligent Virtual Agents. 79--86.
[5]
Koji Kamei, Kazuhiko Shinozawa, Tetsushi Ikeda, Akira Utsumi, Takahiro Miyashita, and Norihiro Hagita. 2010. Recommendation from robots in a real-world retail shop. In International conference on multimodal interfaces and the workshop on machine learning for multimodal interaction. 1--8.
[6]
Taras Kucherenko, Dai Hasegawa, Gustav Eje Henter, Naoshi Kaneko, and Hedvig Kjellström. 2019. Analyzing input and output representations for speech-driven gesture generation. In Proceedings of the 19th ACM International Conference on Intelligent Virtual Agents. 97--104.
[7]
Taras Kucherenko, Patrik Jonell, Sanne van Waveren, Gustav Eje Henter, Simon Alexandersson, Iolanda Leite, and Hedvig Kjellström. 2020. Gesticulator: A framework for semantically-aware speech-driven gesture generation. In Proceedings of the 2020 International Conference on Multimodal Interaction. 242--250.
[8]
Stacy Marsella, Yuyu Xu, Margaux Lhommet, Andrew Feng, Stefan Scherer, and Ari Shapiro. 2013. Virtual character performance from speech. In Proceedings of the 12th ACM SIGGRAPH/Eurographics symposium on computer animation. 25--35.
[9]
Takahiro Matsumoto, Satoru Satake, Takayuki Kanda, Michita Imai, and Norihiro Hagita. 2012. Do you remember that shop? computational model of spatial memory for shopping companion robots. In Proceedings of the seventh annual ACM/IEEE international conference on Human-Robot Interaction. 447--454.
[10]
Nestlé. 2014. Nestlé to use humanoid robot to sell Nescafé in Japan. https://www.nestle.com/media/news/nestle-humanoid-robot-nescafe-japan Retrieved Oct 1, 2022
[11]
Tan Viet Tuyen Nguyen and Oya Celiktutan. 2022. Agree or Disagree? Generating Body Gestures from Affective Contextual Cues during Dyadic Interactions. In International Symposium on Robot and Human Interactive Communication (RO-MAN). IEEE.
[12]
Marketta Niemelä, Päivi Heikkilä, and Hanna Lammi. 2017. A social service robot in a shopping mall: expectations of the management, retailers and consumers. In Proceedings of the Companion of the 2017 ACM/IEEE International Conference on Human-Robot Interaction. 227--228.
[13]
Shane Saunderson and Goldie Nejat. 2019. How robots influence humans: A survey of nonverbal communication in social human--robot interaction. International Journal of Social Robotics, Vol. 11, 4 (2019), 575--608.
[14]
Sichao Song, Baba Jun, Junya Nakanishi, Yuichiro Yoshikawa, and Hiroshi Ishiguro. 2022. Service Robots in a Bakery Shop: A Field Study. arXiv preprint arXiv:2208.09260 (2022).
[15]
Nguyen Tan Viet Tuyen, Armagan Elibol, and Nak Young Chong. 2020. Conditional generative adversarial network for generating communicative robot gestures. In 2020 29th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN). IEEE, 201--207.
[16]
Jun Xu, Tao Mei, Ting Yao, and Yong Rui. 2016. Msr-vtt: A large video description dataset for bridging video and language. In Proceedings of the IEEE conference on computer vision and pattern recognition. 5288--5296.
[17]
Youngwoo Yoon, Woo-Ri Ko, Minsu Jang, Jaeyeon Lee, Jaehong Kim, and Geehyuk Lee. 2019. Robots learn social skills: End-to-end learning of co-speech gesture generation for humanoid robots. In 2019 International Conference on Robotics and Automation (ICRA). IEEE, 4303--4309.
Index Terms
- The Impact of Robot's Body Language on Customer Experience: An Analysis in a Cafe Setting
Recommendations
A Multimodal Dataset for Robot Learning to Imitate Social Human-Human Interaction
HRI '23: Companion of the 2023 ACM/IEEE International Conference on Human-Robot InteractionHumans tend to use various nonverbal signals to communicate their messages to their interaction partners. Previous studies utilised this channel as an essential clue to develop automatic approaches for understanding, modelling and synthesizing individual ...
Lexical Entrainment in Multi-party Human–Robot Interaction
Social RoboticsAbstractThis paper reports lexical entrainment in a multi-party human–robot interaction, wherein one robot and two humans serve as participants. Humans tend to use the same terms as their interlocutors while making conversation. This phenomenon is called ...
Human-robot collaborative tutoring using multiparty multimodal spoken dialogue
HRI '14: Proceedings of the 2014 ACM/IEEE international conference on Human-robot interactionIn this paper, we describe a project that explores a novel experimental setup towards building a spoken, multi-modally rich, and human-like multiparty tutoring robot. A human-robot interaction setup is designed, and a human-human dialogue corpus is ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
March 2023
612 pages
ISBN:9781450399708
DOI:10.1145/3568294
- General Chairs:
- Ginevra Castellano,
- Laurel Riek,
- Program Chairs:
- Maya Cakmak,
- Iolanda Leite
Copyright © 2023 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 13 March 2023
Check for updates
Author Tags
Qualifiers
- Short-paper
Funding Sources
- The Royal Society
- EPSRC
Conference
HRI '23: ACM/IEEE International Conference on Human-Robot Interaction
March 13 - 16, 2023
Stockholm, Sweden
Acceptance Rates
Overall Acceptance Rate 268 of 1,124 submissions, 24%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 135Total Downloads
- Downloads (Last 12 months)63
- Downloads (Last 6 weeks)3
Reflects downloads up to 19 Nov 2024
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in