research-article

Evaluating multimodal human-robot interaction: a case study of an early humanoid prototype

Authors:

Gudberg K. Jonsson,

Kristinn R. ThorissonAuthors Info & Claims

MB '10: Proceedings of the 7th International Conference on Methods and Techniques in Behavioral Research

Article No.: 9, Pages 1 - 4

https://doi.org/10.1145/1931344.1931353

Published: 24 August 2010 Publication History

Get Access

Abstract

Multimodal natural behavior of humans presents a complex yet highly coordinated set of interacting processes. Providing robots with such interactive skills is a challenging and worthy goal and numerous such efforts are currently underway; evaluating the progress in this direction, however, continues to be a challenge. General methods for measuring the performance of artificially intelligent systems would be of great benefit to the research community. In this paper we describe an approach to evaluating human-robot multimodal natural behavior. The approach is based on a detailed scoring and spatio-temporal analysis of the structure and patterning of live behavior, at multiple temporal scales, down to the decisecond level. The approach is tested in a case study involving an early virtual robot prototype, Gandalf, which is capable of real-time verbal and non-verbal interaction with people. Our analysis includes a comparison to a comparable human-human dyadic interaction scenario. Our main objective is to develop a methodology for comparing the quality and effectiveness of human-robot interaction between a wide variety of such systems. Early results indicate that our approach holds significant promise as a future methodology for evaluating complex systems that have a natural counterpart.

References

[1]

Bromberg, M. & Landréé, A. Analyse de la Structure Interactionnelle et des Stratégies discursive dans untalk-show. Psychologie Francaise, 38, 2 (1993), 99--109.

Google Scholar

[2]

Hadzibeganovic, T., Van den Noort, M. W. M. L., Cannas, S. A., Bosch, M. P. C., Jonsson, G. K., & Magnusson, M. S. The missing neurocognitive and artificial general intelligence bases of Robo Cup research: What still needs to be done before 2050? Proceedings of the 3rd Austrian Robo Cup Workshop 2008. Villach, Austria.

Google Scholar

[3]

Johnson, W. L., Vilhjalmsson, H. and Marsella, M. Serious Games for Language Learning: How Much Game, How Much AI? 12th International Conference on Artificial Intelligence in Education, 2005).

Digital Library

Google Scholar

[4]

Jonsdottir, G. R. and Thórisson, K. R. Teaching Computers to Conduct Spoken Interviews: Breaking the Realtime Barrier With Learning. IVA '09, 446--459; LNAI 5773.

Digital Library

Google Scholar

[5]

Jonsson, G. K. Personality and Self-Esteem in Social Interaction. In From Communication to Presence: Cognition, Emotions and Culture towards the Ultimate Communicative Experience. Edited by Giuseppe Riva et al. IOS Press, 2006. ISBN 1-58603-662-9.

Google Scholar

[6]

Kerepesi, A., Kubinyi, E., Jonsson, G. K., Magnusson, M. S., Miklósi, Á. Behavioural comparison of human-animal (dog) and human-robot (AIBO) interactions. Behavioral Processes, 73(2006), 92--99.

Crossref

Google Scholar

[7]

Magnusson, M. S. "Hidden Real-Time Patterns in Intra- and Inter-Individual Behavior: Description and Detection." European Journal of Psychological Assessment, 12, 2 (1996). 112--123.

Crossref

Google Scholar

[8]

Magnusson, M. S. Discovering hidden time Patterns in Behavior: T-patterns and their detection. Behavior Research Methods, Instruments & Computers, 32 (2000), 93--110.

Google Scholar

[9]

Magnusson, M. S. Structure and Communication in Interaction. In G. Riva, M. T. Anguera, B. K. Wiederhold, F. Mantovani (eds.) 2006. From Communication to Presence: Cognition, Emotions and Culture Towards the Ultimate Communicative Experience. Útgefandi: IOS Press October 1, 2006.

Google Scholar

[10]

McGrew, W. C. An Ethological Study of Children's Behaviour. New York: Lawrence Erlbaum Associates, 1972.

Google Scholar

[11]

Ng-Thow-Hing, V., K. R. Thórisson, R. K. Sarvadevabhatla, J. Wormer, T. List. The Cognitive Map architecture for facilitating human-robot interaction in humanoid robots. IEEE Robotics & Automation, 16, 1 (2009), 55--66.

Google Scholar

[12]

Thorisson, K. R. Communicative Humanoids: A Computational Model of Psycho-Social Dialogue Skills. Unpublished Ph.D. Thesis, Media Laboratory, Massachusetts Institute of Technology, 1996.

Digital Library

Google Scholar

[13]

Thorisson, K. R. A Mind Model for Multimodal Communicative Creatures and Humanoids. International Journal of Applied Artificial Intelligence, 13, 4--5 (1999), 449--486.

Crossref

Google Scholar

Cited By

View all

Sáiz-Manzanares MAlonso-Martínez LMarticorena-Sánchez R(2022)A Systematic Review of the Use of T-Pattern and T-String Analysis (TPA) With Theme: An Analysis Using Mixed Methods and Data Mining TechniquesFrontiers in Psychology10.3389/fpsyg.2022.94390713Online publication date: 22-Jul-2022
https://doi.org/10.3389/fpsyg.2022.943907
Thórisson KGislason OJonsdottir GThorisson H(2010)A multiparty multimodal architecture for realtime turntakingProceedings of the 10th international conference on Intelligent virtual agents10.5555/1889075.1889117(350-356)Online publication date: 20-Sep-2010
https://dl.acm.org/doi/10.5555/1889075.1889117
Thórisson KGislason OJonsdottir GThorisson H(2010)A Multiparty Multimodal Architecture for Realtime TurntakingIntelligent Virtual Agents10.1007/978-3-642-15892-6_37(350-356)Online publication date: 2010
https://doi.org/10.1007/978-3-642-15892-6_37

Index Terms

Evaluating multimodal human-robot interaction: a case study of an early humanoid prototype

Recommendations

Human-robot collaborative tutoring using multiparty multimodal spoken dialogue
HRI '14: Proceedings of the 2014 ACM/IEEE international conference on Human-robot interaction

In this paper, we describe a project that explores a novel experimental setup towards building a spoken, multi-modally rich, and human-like multiparty tutoring robot. A human-robot interaction setup is designed, and a human-human dialogue corpus is ...
Comparison of Human-Human and Human-Robot Turn-Taking Behaviour in Multiparty Situated Interaction
UM3I '14: Proceedings of the 2014 workshop on Understanding and Modeling Multiparty, Multimodal Interactions

In this paper, we present an experiment where two human subjects are given a team-building task to solve together with a robot. The setting requires that the speakers' attention is partly directed towards objects on the table between them, as well as to ...
Enabling Multimodal Human–Robot Interaction for the Karlsruhe Humanoid Robot

In this paper, we present our work in building technologies for natural multimodal human-robot interaction. We present our systems for spontaneous speech recognition, multimodal dialogue processing, and visual perception of a user, which includes ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

MB '10: Proceedings of the 7th International Conference on Methods and Techniques in Behavioral Research

August 2010

183 pages

ISBN:9781605589268

DOI:10.1145/1931344

Editors:
Emilia Barakova
User Experiences Department, Philips Research Europe, The Netherlands
,
Boris de Ruyter
Faculty of Industrial Design, Eindhoven University of Technology, The Netherlands
,
Andrew Spink
Noldus Information Technology, The Netherlands

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 24 August 2010

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Seventh Framework Programme

Conference

Measuring Behavior

Sponsor:

Measuring Behavior: Selected papers from the Proceedings of the 7th International Conference on Methods and Techniques in Behavioral research

August 24 - 27, 2010

Eindhoven, The Netherlands

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
136
Total Downloads

Downloads (Last 12 months)8
Downloads (Last 6 weeks)0

Reflects downloads up to 21 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Sáiz-Manzanares MAlonso-Martínez LMarticorena-Sánchez R(2022)A Systematic Review of the Use of T-Pattern and T-String Analysis (TPA) With Theme: An Analysis Using Mixed Methods and Data Mining TechniquesFrontiers in Psychology10.3389/fpsyg.2022.94390713Online publication date: 22-Jul-2022
https://doi.org/10.3389/fpsyg.2022.943907
Thórisson KGislason OJonsdottir GThorisson H(2010)A multiparty multimodal architecture for realtime turntakingProceedings of the 10th international conference on Intelligent virtual agents10.5555/1889075.1889117(350-356)Online publication date: 20-Sep-2010
https://dl.acm.org/doi/10.5555/1889075.1889117
Thórisson KGislason OJonsdottir GThorisson H(2010)A Multiparty Multimodal Architecture for Realtime TurntakingIntelligent Virtual Agents10.1007/978-3-642-15892-6_37(350-356)Online publication date: 2010
https://doi.org/10.1007/978-3-642-15892-6_37

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Human-robot collaborative tutoring using multiparty multimodal spoken dialogue

Comparison of Human-Human and Human-Robot Turn-Taking Behaviour in Multiparty Situated Interaction

Enabling Multimodal Human–Robot Interaction for the Karlsruhe Humanoid Robot