We introduce a new conversational Human-Robot-Interaction (HRI) dataset with a real-behaving robot inducing interactive behavior with and between humans. Our scenario involves a humanoid robot NAO¹ explaining paintings in a room and then quizzing the participants, who are naive users. As perceiving nonverbal cues, apart from the spoken words, plays a major role in social interactions and socially-interactive robots, we have extensively annotated the dataset. It has been recorded and annotated to benchmark many relevant perceptual tasks, towards enabling a robot to converse with multiple humans, such as speaker localization and speech segmentation; tracking, pose estimation, nodding, visual focus of attention estimation in visual domain; and an audio-visual task such as addressee detection. NAO system states are also available. As compared to recordings done with a static camera, this corpus involves the head-movement of a humanoid robot (due to gaze change, nodding), posing challenges to visual processing. Also, the significant background noise present in a real HRI setting makes auditory tasks challenging.

References

[1]

X. Alameda-Pineda et al. The RAVEL data set. In ICMI 2011 Workshop on Multimodal Corpora, Alicante, Spain, Nov 2011.

Google Scholar

[2]

E. Arnaud et al. The CAVA corpus: synchronised stereoscopic and binaural datasets with head movements. In Proc. ICMI. ACM, 2008.

Digital Library

Google Scholar

[3]

T. Fong et al. A survey of socially interactive robots. Robotics and autonomous systems, 42(3):143--166, 2003.

Google Scholar

[4]

D. Jayagopi et al. The vernissage corpus: A multimodal human-robot-interaction dataset. In Idiap research report (Idiap-RR-33--2012), 2012.

Google Scholar

[5]

M. Lohse et al. Systemic interaction analysis (SInA) in HRI. In Proc. Human-Robot Interaction (HRI), San Diego, CA, USA, 2009.

Digital Library

Google Scholar

[6]

Y. Mohammad et al. The h3r explanation corpus human-human and base human-robot interaction dataset. In Proc. ISSNIP. IEEE, 2008.

Google Scholar

[7]

K. Pitsch et al. Attitude of german museum visitors towards an interactive art guide robot. In Proc. HRI. ACM, 2011.

Digital Library

Google Scholar

[8]

J. Wienke et al. A framework for the acquisition of multimodal HRI data sets with a whole-system perspective. In LREC, 2012.

Google Scholar

[9]

Z. Zivkovic et al. From sensors to human spatial concepts. Robotics and Autonomous Systems, 55(5):357--358, 2007.

Digital Library

Google Scholar

Cited By

View all

Soneda YMatsuda YArakawa YYasumoto KHarle RFarrahi KLane N(2019)M3B corpusAdjunct Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable Computers10.1145/3341162.3345588(825-834)Online publication date: 9-Sep-2019
https://dl.acm.org/doi/10.1145/3341162.3345588
Le Minh TShimizu NMiyazaki TShinoda K(2018)Deep learning based multi-modal addressee recognition in visual scenes with utterancesProceedings of the 27th International Joint Conference on Artificial Intelligence10.5555/3304415.3304635(1546-1553)Online publication date: 13-Jul-2018
https://dl.acm.org/doi/10.5555/3304415.3304635
Sheikhi SKhalidov VKlotz DWrede BOdobez JEpps JChen FOviatt SMase KSears AJokinen KSchuller B(2013)Leveraging the robot dialog state for visual focus of attention recognitionProceedings of the 15th ACM on International conference on multimodal interaction10.1145/2522848.2522881(107-110)Online publication date: 9-Dec-2013
https://dl.acm.org/doi/10.1145/2522848.2522881

Index Terms

The vernissage corpus: a conversational human-robot-interaction dataset
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
2. Hardware
  1. Communication hardware, interfaces and storage
    1. Signal processing systems

Recommendations

The unobtrusive group interaction (UGI) corpus
MMSys '19: Proceedings of the 10th ACM Multimedia Systems Conference

Studying group dynamics requires fine-grained spatial and temporal understanding of human behavior. Social psychologists studying human interaction patterns in face-to-face group meetings often find themselves struggling with huge volumes of data that ...
A Multimodal Dataset for Robot Learning to Imitate Social Human-Human Interaction
HRI '23: Companion of the 2023 ACM/IEEE International Conference on Human-Robot Interaction

Humans tend to use various nonverbal signals to communicate their messages to their interaction partners. Previous studies utilised this channel as an essential clue to develop automatic approaches for understanding, modelling and synthesizing individual ...
MCIC: Multimodal Conversational Intent Classification for E-commerce Customer Service
Natural Language Processing and Chinese Computing
Abstract
Conversational intent classification (CIC) plays a significant role in dialogue understanding, and most previous works only focus on the text modality. Nevertheless, in real conversations of E-commerce customer service, users often send images (...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

HRI '13: Proceedings of the 8th ACM/IEEE international conference on Human-robot interaction

March 2013

452 pages

ISBN:9781467330558

General Chairs:
Hideaki Kuzuoka
University of Tsukuba, Japan
,
Vanessa Evers
University of Twente, The Netherlands
,
Program Chairs:
Michita Imai
Keio University, Japan
,
Jodi Forlizzi
Carnegie Mellon University, USA

In-Cooperation

AAAI: American Association for Artificial Intelligence
Human Factors & Ergonomics Soc: Human Factors & Ergonomics Soc

Publisher

IEEE Press

Publication History

Published: 03 March 2013

Check for updates

Author Tags

Qualifiers

Abstract

Conference

HRI'13

Sponsor:

HRI'13: ACM/IEEE International Conference on Human-Robot Interaction

March 3 - 6, 2013

Tokyo, Japan

Acceptance Rates

Overall Acceptance Rate 268 of 1,124 submissions, 24%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
135
Total Downloads

Downloads (Last 12 months)2
Downloads (Last 6 weeks)0

Reflects downloads up to 29 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Soneda YMatsuda YArakawa YYasumoto KHarle RFarrahi KLane N(2019)M3B corpusAdjunct Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable Computers10.1145/3341162.3345588(825-834)Online publication date: 9-Sep-2019
https://dl.acm.org/doi/10.1145/3341162.3345588
Le Minh TShimizu NMiyazaki TShinoda K(2018)Deep learning based multi-modal addressee recognition in visual scenes with utterancesProceedings of the 27th International Joint Conference on Artificial Intelligence10.5555/3304415.3304635(1546-1553)Online publication date: 13-Jul-2018
https://dl.acm.org/doi/10.5555/3304415.3304635
Sheikhi SKhalidov VKlotz DWrede BOdobez JEpps JChen FOviatt SMase KSears AJokinen KSchuller B(2013)Leveraging the robot dialog state for visual focus of attention recognitionProceedings of the 15th ACM on International conference on multimodal interaction10.1145/2522848.2522881(107-110)Online publication date: 9-Dec-2013
https://dl.acm.org/doi/10.1145/2522848.2522881

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

The unobtrusive group interaction (UGI) corpus

A Multimodal Dataset for Robot Learning to Imitate Social Human-Human Interaction

MCIC: Multimodal Conversational Intent Classification for E-commerce Customer Service