Article

The role of spoken feedback in experiencing multimodal interfaces as human-like

Authors:

Pernilla Qvarfordt,

Nils DahlbäckAuthors Info & Claims

ICMI '03: Proceedings of the 5th international conference on Multimodal interfaces

Pages 250 - 257

https://doi.org/10.1145/958432.958478

Published: 05 November 2003 Publication History

Abstract

If user interfaces should be made human-like vs. tool-like has been debated in the HCI field, and this debate affects the development of multimodal interfaces. However, little empirical study has been done to support either view so far. Even if there is evidence that humans interpret media as other humans, this does not mean that humans experience the interfaces as human-like. We studied how people experience a multimodal timetable system with varying degree of human-like spoken feedback in a Wizard-of-Oz study. The results showed that users' views and preferences lean significantly towards anthropomorphism after actually experiencing the multimodal timetable system. The more human-like the spoken feedback is the more participants preferred the system to be human-like. The results also showed that the users experience matched their preferences. This shows that in order to appreciate a human-like interface, the users have to experience it.

References

[1]

Brennan, S. E., and Ohaeri, J. O. Effects of message style on users' attribution toward agents. In Proceedings of the Conference on Human Factors in Computing Systems: CHI'94 (1994), ACM Press, pp. 281--282.

Digital Library

[2]

Burgoon, J. K., Bonito, J. A., Bengtsson, B., Cederberg, C., Lundberg, M., and Allspach, L. Interactivity in human-computer interaction: A study of credibility, understanding, and influence. Computers in Human Behaviour 16 (2000), 553--574.

[3]

Cassell, J. Emboded conversational agents, representation and intelligence in user interfaces. AI Magazine Winter (2001), 67--83.

Digital Library

[4]

Cassell, J., Bickmore, T., Billinghurst, M., Campbell, L., Chang, K., Vilhjalmsson, H., and Yan, H. Embodiment in conversational interfaces: Rea. In Proceeding of the Conference on Human Factors in Computing Systems: CHI'99 (Pittsburgh, 1999), pp. 520--527.

Digital Library

[5]

Clark, H. C., and Schaeffer, E. F. Contributing to discourse. Cognitive Science 13 (1989), 259--294.

[6]

Cohen, P. R. The role of natural language in a multimodal interface. In Proceedings of the Symposium on User Interface Software and Technology: UIST'92 (1992), ACM, pp. 143--149.

Digital Library

[7]

Dahlbäck, N., Jönsson, A., and Ahrenberg, L. Wizard of oz studies - why and how. Knowledge-Based Systems 6, 4 (December 1993), 258--266.

[8]

Dehn, D. M., and van Mulken, S. The impact of animated interface agents: a review of empirical research. International Journal of Human-Computer Studies 52 (2000), 1--22.

Digital Library

[9]

Frohlich, D. M. The design space of interface. In Multimedia: Systems, interaction and Application, L. Kjelldahl, Ed. Springer-Verlag, Berlin, 1991, pp. 53--69.

[10]

Heckman, C. E., and Wobbrock, J. O. Put your best face forward: Anthropomorphic agents, e-commerce consumers, and the law. In Proceedings of the International Conference on Autonomous Agents (2000), pp. 435--442.

Digital Library

[11]

Horvitz, E. Principles for mixed-initiative user interfaces. In Proceedings of the Conference on Human Factors in Computing Systems: CHI'99 (Pittsburgh, 1999), pp. 159--166.

Digital Library

[12]

Isbister, K., and Nass, C. Consistency of personality in interactive characters: Verbal cues, non-verbal cues, and user characteristics. International Journal of Human-Computer Studies 53 (2000), 251--267.

Digital Library

[13]

Jönsson, A., and Dahlbäck, N. Talking to a computer is not like talking to your best friend. In Proceedings of the First Scandinavian Conference on Artificial Intelligence (Tromso, Norway, March 9-11, 1988).

[14]

Laurel, B. Interface as mimesis. In User Centered Systems Design: New Perspectives on Human Computer Interface Design, D. A. Norman and S. W. Draper, Eds. Lawrence Erlbaum Associates, Hilldale, NJ, 1986, pp. 87--124.

[15]

Lester, J. C., Converse, S. A., Kahler, S. E., Barlow, S. T., Stone, B. A., and Bhogal, R. S. The persona effect: Affective impact of animated pedagogical agents. In Proceedings of the Conference on Human Factors in Computing Systems, CHI '97, (1997), 359--366.

Digital Library

[16]

Maybury, M. T. Intelligent user interfaces for all. In User Interfaces for All, Concepts, Methods, and Tools, C. Stephanidis, Ed. Lawrence Erlbaum Associates, Publishers, Mahwah, NJ, 2001, pp. 65--80.

[17]

Oviatt, S. Multimodal interfaces for dynamic interactive maps. In Proceedings of the Conference on Human-Factors in Computing Systems: CHI'96 (1996), New York, ACM Press, pp. 95--102.

Digital Library

[18]

Oviatt, S., Cohen, P., Wu, L., Vergo, J., Duncan, L., Suhm, B., Bers, J., Holzman, T., Winograd, T., Landay, J., Larson, J., and Ferro, D. Designing the user interface for multimodal speech and pen-based gesture application: State-of-the-art systems and future research direction. Human-Computer Interaction 15 (2000), 263--322.

Digital Library

[19]

Reeves, B., and Nass, C. The Media Equation - How People Treat Computers, Television, and New Media Like Real People and Places. CSLI Publications and Cambridge University Press, Cambridge, New York, 1996.

Digital Library

[20]

Rickenberg, R., and Reeves, B. The effect of animated characters on anxiety, task performance, and evaluation of user interfaces. In Proceedings of the Conference on Human Factors in Computing Systems (CHI'2000) (2000), pp. 49--56.

Digital Library

[21]

Schaumburg, H. Computers as tools or as social actors? - The users' perspective on anthropomorphic agent. International Journal of Cooperative Information Systems 10, 1, 2 (2001), 217--234.

[22]

Shechtman, N., and Horowitz, L. M. Media inequality in conversation: How people behave differently when interacting with computers and people. In Proceedings of the Conference on Human Factors in Computing Systems, (CHI'03) (2003), pp. 281--288.

Digital Library

[23]

Suhm, B., Myers, B., and Waibel, A. Multimodal error correction for speech user interfaces. ACM Transactions on Computer-Human Interaction 8, 1 (March 2001), 60--98.

Digital Library

[24]

Takeuchi, A., and Nagao, K. Communicative facial displays as new conversational modality. In Proceedings of the Conference on Human Factors in Computing Systems (INTERCHI'93) (1993), pp. 187--193.

Digital Library

[25]

Walker, J. H., Sproull, L., and Subramani, R. Using a human in an interface. In Proceeding of the Conference on Human Factors in Computing Systems: CHI'94 (1994), pp. 85--91.

Digital Library

[26]

Weizenbaum, J. ELIZA-a computer program for the study of natural language communication between man and machine. Communication of the ACM 9, 1 (1966), 36--45.

Digital Library

Cited By

Viswanathan SGuillot FChang MGrasso ARenders J(2022)Addressing Hiccups in Conversations with Recommender SystemsProceedings of the 2022 ACM Designing Interactive Systems Conference10.1145/3532106.3533491(1243-1259)Online publication date: 13-Jun-2022
https://dl.acm.org/doi/10.1145/3532106.3533491
Reicherts LRogers YCapra LWood EDuong TSebire N(2022)It's Good to Talk: A Comparison of Using Voice Versus Screen-Based Interactions for Agent-Assisted TasksACM Transactions on Computer-Human Interaction10.1145/348422129:3(1-41)Online publication date: 14-Jan-2022
https://dl.acm.org/doi/10.1145/3484221
Doyle PClark LCowan BKitamura YQuigley AIsbister KIgarashi TBjørn PDrucker S(2021)What Do We See in Them? Identifying Dimensions of Partner Models for Speech Interfaces Using a Psycholexical ApproachProceedings of the 2021 CHI Conference on Human Factors in Computing Systems10.1145/3411764.3445206(1-14)Online publication date: 6-May-2021
https://dl.acm.org/doi/10.1145/3411764.3445206
Show More Cited By

Index Terms

The role of spoken feedback in experiencing multimodal interfaces as human-like
1. Hardware
  1. Communication hardware, interfaces and storage
    1. Sound-based input / output
2. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

A Wizard of Oz study for an AR multimodal interface
ICMI '08: Proceedings of the 10th international conference on Multimodal interfaces

In this paper we describe a Wizard of Oz (WOz) user study of an Augmented Reality (AR) interface that uses multimodal input (MMI) with natural hand interaction and speech commands. Our goal is to use a WOz study to help guide the creation of a ...
From vocal to multimodal dialogue management
ICMI '06: Proceedings of the 8th international conference on Multimodal interfaces

Multimodal, speech-enabled systems pose different research problems when compared to unimodal, voice-only dialogue systems. One of the important issues is the question of how a multimodal interface should look like in order to make the multimodal ...
Towards human-like spoken dialogue systems

This paper presents an overview of methods that can be used to collect and analyse data on user responses to spoken dialogue system components intended to increase human-likeness, and to evaluate how well the components succeed in reaching that goal. ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ICMI '03: Proceedings of the 5th international conference on Multimodal interfaces

November 2003

318 pages

ISBN:1581136218

DOI:10.1145/958432

Conference Chair:
Sharon Oviatt
Oregon Health & Science University
,
Program Chairs:
Trevor Darrell
Massachusetts Institute of Technology
,
Mark Maybury
MITRE
,
Wolfgang Wahlster
DFKI, Germany

Copyright © 2003 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 November 2003

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

ICMI-PUI03

Sponsor:

ICMI-PUI03: International Conference on Multimodal User Interfaces

November 5 - 7, 2003

British Columbia, Vancouver, Canada

Acceptance Rates

ICMI '03 Paper Acceptance Rate 45 of 130 submissions, 35%;

Overall Acceptance Rate 453 of 1,080 submissions, 42%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
774
Total Downloads

Downloads (Last 12 months)7
Downloads (Last 6 weeks)1

Reflects downloads up to 12 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Viswanathan SGuillot FChang MGrasso ARenders J(2022)Addressing Hiccups in Conversations with Recommender SystemsProceedings of the 2022 ACM Designing Interactive Systems Conference10.1145/3532106.3533491(1243-1259)Online publication date: 13-Jun-2022
Reicherts LRogers YCapra LWood EDuong TSebire N(2022)It's Good to Talk: A Comparison of Using Voice Versus Screen-Based Interactions for Agent-Assisted TasksACM Transactions on Computer-Human Interaction10.1145/348422129:3(1-41)Online publication date: 14-Jan-2022
Doyle PClark LCowan BKitamura YQuigley AIsbister KIgarashi TBjørn PDrucker S(2021)What Do We See in Them? Identifying Dimensions of Partner Models for Speech Interfaces Using a Psycholexical ApproachProceedings of the 2021 CHI Conference on Human Factors in Computing Systems10.1145/3411764.3445206(1-14)Online publication date: 6-May-2021
Li CYeh SChang TTsai MChen KChang YBernhaupt RMueller FVerweij DAndres JMcGrenere JCockburn AAvellino IGoguey ABjørn PZhao SSamson BKocielnik R(2020)A Conversation Analysis of Non-Progress and Coping Strategies with a Banking Task-Oriented ChatbotProceedings of the 2020 CHI Conference on Human Factors in Computing Systems10.1145/3313831.3376209(1-12)Online publication date: 21-Apr-2020
Myers CFurqan ANebolsky JCaro KZhu JMandryk RHancock MPerry MCox A(2018)Patterns for How Users Overcome Obstacles in Voice User InterfacesProceedings of the 2018 CHI Conference on Human Factors in Computing Systems10.1145/3173574.3173580(1-7)Online publication date: 21-Apr-2018
Kipp AKummert FKuzuoka HOno TImai MYoung J(2014)Dynamic dialog system for human robot collaborationProceedings of the second international conference on Human-agent interaction10.1145/2658861.2658928(225-228)Online publication date: 29-Oct-2014
Yee NBailenson JRickertsen KRosson MGilmore D(2007)A meta-analysis of the impact of the inclusion and realism of human-like faces on user experiences in interfacesProceedings of the SIGCHI Conference on Human Factors in Computing Systems10.1145/1240624.1240626(1-10)Online publication date: 29-Apr-2007
(2006)Human-centered visualization environmentsundefinedOnline publication date: 5-Mar-2006

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents