research-article

What Makes a Good Conversation?: Challenges in Designing Truly Conversational Agents

Authors:

Diego Garaialde,

Justin Edwards,

Brendan Spillane,

Emer Gilmartin,

Christine Murad,

Cosmin Munteanu,

Benjamin R. CowanAuthors Info & Claims

CHI '19: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems

Paper No.: 475, Pages 1 - 12

https://doi.org/10.1145/3290605.3300705

Published: 02 May 2019 Publication History

Abstract

Conversational agents promise conversational interaction but fail to deliver. Efforts often emulate functional rules from human speech, without considering key characteristics that conversation must encapsulate. Given its potential in supporting long-term human-agent relationships, it is paramount that HCI focuses efforts on delivering this promise. We aim to understand what people value in conversation and how this should manifest in agents. Findings from a series of semi-structured interviews show people make a clear dichotomy between social and functional roles of conversation, emphasising the long-term dynamics of bond and trust along with the importance of context and relationship stage in the types of conversations they have. People fundamentally questioned the need for bond and common ground in agent communication, shifting to more utilitarian definitions of conversational qualities. Drawing on these findings we discuss key challenges for conversational agent design, most notably the need to redefine the design parameters for conversational agent interaction.

Supplementary Material

MP4 File (paper475.mp4)

Download
335.77 MB

References

[1]

Ali Abdolrahmani, Ravi Kuber, and Stacy M Branham. 2018. Siri Talks at You: An Empirical Investigation of Voice-Activated Personal Assistant (VAPA) Usage by Individuals Who Are Blind. In Proceedings of the 20th International ACM SIGACCESS Conference on Computers and Accessibility. ACM, 249--258.

Digital Library

[2]

Frank Bentley, Chris Luvogt, Max Silverman, Rushani Wirasinghe, Brooke White, and Danielle Lottrjdge. 2018. Understanding the LongTerm Use of Smart Speaker Assistants. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 2, 3 (2018), 91.

Digital Library

[3]

Timothy Bickmore and Justine Cassell. 2001. Relational agents: a model and implementation of building user trust. In Proceedings of the SIGCHI conference on Human factors in computing systems. ACM, 396--403.

Digital Library

[4]

Timothy Bickmore and Justine Cassell. 2005. Social dialngue with embodied conversational agents. In Advances in natural multimodal dialogue systems. Springer, 23--54.

[5]

Timothy W Bickmore, Lisa Caruso, and Kerri Clough-Gorr. 2005. Acceptance and usability of a relational agent interface by urban older adults. In CHI'05 extended abstracts on Human factors in computing systems. ACM, 1212--1215.

Digital Library

[6]

Timothy W Bickmore, Suzanne E Mitchell, Brian W Jack, Michael K Paasche-Orlow, Laura M Pfeifer, and Julie O'Donnell. 2010. Response to a relational agent by hospital patients with depressive symptoms. Interacting with computers 22, 4 (2010), 289--298.

Digital Library

[7]

Timothy W Bickmore and Rosalind W Picard. 2004. Towards caring machines. In CHI'04 extended abstracts on Human factors in computing systems. ACM, 1489--1492.

Digital Library

[8]

Holly P Branigan, Martin J Pickering, Jamie Pearson, Janet F McLean, and Ash Brown. 2011. The role of beliefs in lexical alignment: Evidence from dialogs with humans and computers. Cognition 121, 1 (2011), 41--57.

[9]

Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology. Qualitative research in psychology 3, 2 (2006), 77--101.

[10]

Cynthia Breazeal. 2003. Emotion and sociable humanoid robots. International Journal of Human-Computer Studies 59, 1 (July 2003), 119--155.

Digital Library

[11]

Gillian Brown and George Yule. 1983. Discourse Analysis. Cambridge University Press. Google-Books-ID: 9gUrbzov9x0C.

[12]

Penelope Brown and Stephen C Levinson. 1987. Politeness: Some universals in language usage. Vol. 4. Cambridge university press.

[13]

A. Bruce, I. Nourbakhsh, and R. Simmons. 2002. The role of expressiveness and attention in human-robot interaction. In Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292), Vol. 4. 4138--4142 vol.4.

[14]

BusinessWire. 2018. Global Intelligent Virtual Assistant Market 20182023:. https://www.businesswire.com/news/home/20180723005506/ en/Global-Intelligent-Virtual-Assistant-Market-2018--2023-Market

[15]

C. Cheepen. 1988. The predictability of informal conversation. Pinter. https://books.google.ie/books?id=ai23AAAAIAAJ

[16]

H.H. Clark. 1996. Using Language. Cambridge University Press. https: //books.google.ie/books?id=DiWBGOP-YnoC

[17]

Herbert H Clark. 1992. Arenas of language use. University of Chicago Press.

[18]

Herbert H Clark, Susan E Brennan, et al. 1991. Grounding in communication. Perspectives on socially shared cognition 13, 1991 (1991), 127--149.

[19]

Leigh Clark. 2018. Social Boundaries of Appropriate Speech in HCI: A Politeness Perspective. In Proceedings of the 32nd British Human Computer Interaction Conference. British Computer Society.

Digital Library

[20]

Benjamin R Cowan, Nadia Pantidi, David Coyle, Kellie Morrissey, Peter Clarke, Sara Al-Shehri, David Earley, and Natasha Bandeira. 2017. What Can I Help You With?: Infrequent Users' Experiences of Intelligent Personal Assistants. In Proceedings of the 19th International Conference on Human-Computer Interaction with Mobile Devices and Services. ACM, 43.

Digital Library

[21]

L. Devillers, S. Rosset, G. Dubuisson Duplessis, L. Bechade, Y. Yemez, B. B. Turker, M. Sezgin, E. Erzin, K. El Haddad, S. Dupont, P. Deleglise, Y. Esteve, C. Lailler, E. Gilmartin, and N. Campbell. 2018. Multifaceted Engagement in Social Interaction with a Machine: The JOKER Project. In 2018 13th IEEE International Conference on Automatic Face Gesture Recognition (FG 2018). 697--701.

[22]

Hugh Dubberly and Paul Pangaro. 2009. What is conversation, and how can we design for it? interactions 16, 4 (2009), 22--28.

Digital Library

[23]

Robin Dunbar. 1998. Grooming, gossip, and the evolution of language. Harvard University Press.

[24]

Aarthi Easwara Moorthy and Kim-Phuong L Vu. 2015. Privacy concerns for use of voice activated personal assistant in the public space. International Journal of Human-Computer Interaction 31, 4 (2015), 307-- 335.

[25]

Suzanne Eggins and Diana Slade. 2005. Analysing Casual Conversation. Equinox Publishing Ltd. Google-Books-ID: cTEkm6WzfNgC.

[26]

Hao Fang, Hao Cheng, Elizabeth Clark, Ariel Holtzman, Maarten Sap, Mari Ostendorf, Yejin Choi, and Noah A Smith. 2017. Sounding Board -- University of Washington's Alexa Prize Submission. (2017), 12.

[27]

Emer Gilmartin, Marine Collery, Ketong Su, Yuyun Huang, Christy Elias, Benjamin R. Cowan, and Nick Campbell. 2017. Social Talk: Making Conversation with People and Machine. In Proceedings of the 1st ACM SIGCHI International Workshop on Investigating Social Interactions with Artifcial Agents (ISIAA 2017). ACM, New York, NY, USA, 31--32.

Digital Library

[28]

Emer Gilmartin, Benjamin R. Cowan, Carl Vogel, and Nick Campbell. 2018. Explorations in multiparty casual social talk and its relevance for social human machine dialogue. Journal on Multimodal User Interfaces (Sept. 2018).

[29]

Rachel Gockley, Allison Bruce, Jodi Forlizzi, Marek Michalowski, Anne Mundell, Stephanie Rosenthal, Brennan Sellner, Reid Simmons, Kevin Snipes, Alan C Schultz, and others. 2005. Designing robots for longterm social interaction. In Intelligent Robots and Systems, 2005.(IROS 2005). 2005 IEEE/RSJ International Conference on. IEEE, 1338--1343.

[30]

Erving Gofman. 1967. Interaction Ritual : Essays in Face-to-Face Behavior. Routledge.

[31]

Adam Jaworski. 2014. Silence and small talk. In Small talk. Routledge, 130--152.

[32]

Mardi Kidwell. 2000. Common ground in cross-cultural communication: Sequential and institutional contexts in front desk service encounters. Issues in Applied Linguistics 11, 1 (2000).

[33]

Yoshinori Kuno, Kazuhisa Sadazuka, Michie Kawashima, Keiichi Yamazaki, Akiko Yamazaki, and Hideaki Kuzuoka. 2007. Museum Guide Robot Based on Sociological Interaction Analysis. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '07). ACM, New York, NY, USA, 1191--1194.

Digital Library

[34]

Rosemarijn Looije, Mark A. Neerincx, and Fokie Cnossen. 2010. Persuasive robotic assistant for health self-management of older adults: Design and evaluation of social behaviors. International Journal of Human-Computer Studies 68, 6 (June 2010), 386--397.

Digital Library

[35]

Ewa Luger and Abigail Sellen. 2016. Like having a really bad PA: the gulf between user expectation and experience of conversational agents. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. ACM, 5286--5297.

Digital Library

[36]

Bronislaw Malinowski. 1994. The problem of meaning in primitive languages. Language and literacy in social practice: A reader (1994), 1--10.

[37]

Michael McTear, Zoraida Callejas, and David Griol. 2016. The conversational interface: Talking to smart devices. Springer.

Digital Library

[38]

Rishabh Mehrotra, A Hassan Awadallah, AE Kholy, and Imed Zitouni. 2017. Hey Cortana! Exploring the use cases of a Desktop based Digital Assistant. In SIGIR 1st International Workshop on Conversational Approaches to Information Retrieval (CAIR'17), Vol. 4.

[39]

Roger K. Moore. 2017. Is Spoken Language All-or-Nothing? Implications for Future Speech-Based Human-Machine Interaction. In Dialogues with Social Robots: Enablements, Analyses, and Evaluation, Kristiina Jokinen and Graham Wilcock (Eds.). Springer Singapore, Singapore, 281--291.

[40]

Cosmin Munteanu, Pourang Irani, Sharon Oviatt, Matthew Aylett, Gerald Penn, Shimei Pan, Nikhil Sharma, Frank Rudzicz, Randy Gomez, Ben Cowan, et al. 2017. Designing Speech, Acoustic and Multimodal Interactions. In Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems. ACM, 601--608.

Digital Library

[41]

Ioannis Papaioannou, Amanda Cercas Curry, Jose L Part, Igor Shalyminov, Xinnuo Xu, Yanchao Yu, Ondrej Duek, Verena Rieser, and Oliver Lemon. 2017. Alana: Social dialogue using an ensemble model and a ranker trained on user feedback. Alexa Prize Proceedings (2017).

[42]

Martin Porcheron, Joel E Fischer, Stuart Reeves, and Sarah Sharples. 2018. Voice Interfaces in Everyday Life. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, 640.

Digital Library

[43]

Amanda Purington, Jessie G Taft, Shruti Sannon, Natalya N Bazarova, and Samuel Hardman Taylor. 2017. Alexa is my new BFF: social roles, user satisfaction, and personifcation of the amazon echo. In Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems. ACM, 2853--2859.

Digital Library

[44]

Stuart Reeves. 2012. Envisioning Ubiquitous Computing. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '12). ACM, New York, NY, USA, 1573--1582.

Digital Library

[45]

Stuart Reeves. 2017. Some conversational challenges of talking with machines. In In: Talking with Conversational Agents in Collaborative Action Workshop.

[46]

Tom A. Rodden, Joel E. Fischer, Nadia Pantidi, Khaled Bachour, and Stuart Moran. 2013. At home with agents: exploring attitudes towards future smart energy infrastructures. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems - CHI '13. ACM Press, Paris, France, 1173.

Digital Library

[47]

A. M. Sabelli, T. Kanda, and N. Hagita. 2011. A conversational robot in an elderly care center: An ethnographic study. In 2011 6th ACM/IEEE International Conference on Human-Robot Interaction (HRI). 37--44.

Digital Library

[48]

Harvey Sacks. 1987. On the preferences for agreement and contiguity in sequences in conversation. Talk and social organisation (1987), 54--69.

[49]

Harvey Sacks, Emanuel A Scheglof, and Gail Jeferson. 1978. A simplest systematics for the organization of turn taking for conversation. In Studies in the organization of conversational interaction. Elsevier, 7--55.

[50]

Martin Saerbeck, Tom Schut, Christoph Bartneck, and Maddy D Janse. 2010. Expressive robots in education: varying the degree of social supportive behavior of a robotic tutor. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 1613--1622.

Digital Library

[51]

Klaus P. Schneider. 1988. Small talk: Analyzing phatic discourse. Vol. 1. Hitzeroth.

[52]

Brendan Spillane, Emer Gilmartin, Christian Saam, Ketong Su, Benjamin R Cowan, Séamus Lawless, and Vincent Wade. 2017. Introducing ADELE: a personalized intelligent companion. In Proceedings of the 1st ACM SIGCHI International Workshop on Investigating Social Interactions with Artifcial Agents. ACM, 43--44.

Digital Library

[53]

Adriana Tapus, Maja J Mataric, and Brian Scassellati. 2007. Socially assistive robotics {grand challenges of robotics}. IEEE Robotics & Automation Magazine 14, 1 (2007), 35--42.

[54]

Janice Y. Tsai, Tawfq Ammaria, Abraham Wallin, and Jofsh Kaye. 2018. Alexa, play some music: Categorization of Alexa Commands. In Workshop on Voice-based Conversational UX Studies and Design (CHI'18).

[55]

Laura Pfeifer Vardoulakis, Lazlo Ring, Barbara Barry, Candace L Sidner, and Timothy Bickmore. 2012. Designing relational agents as long term social companions for older adults. In International Conference on Intelligent Virtual Agents. Springer, 289--302.

Digital Library

[56]

Svetlana Yarosh, Stryker Thompson, Kathleen Watson, Alice Chase, Ashwin Senthilkumar, Ye Yuan, and AJ Brush. 2018. Children asking questions: speech interface reformulations and personifcation preferences. In Proceedings of the 17th ACM Conference on Interaction Design and Children. ACM, 300--312.

Digital Library

[57]

Avner Ziv. 2010. The Social Function of Humor in Interpersonal Relationships. Society 47, 1 (Jan. 2010), 11--18.

Cited By

Karimova GKim Y(2024)The Typology of ChatbotsDesign and Development of Emerging Chatbot Technology10.4018/979-8-3693-1830-0.ch003(51-61)Online publication date: 15-Mar-2024
https://doi.org/10.4018/979-8-3693-1830-0.ch003
B. Sable AKapse D(2024)A Framework Design for Centralised Monitoring of Patient Disease Diagnosis for Better ImprovementInternational Journal of Engineering and Advanced Technology10.35940/ijeat.D4438.1304042413:4(47-52)Online publication date: 30-Apr-2024
https://doi.org/10.35940/ijeat.D4438.13040424
Jeon J(2024)The Effect of AI Agent Gender on Trust and GroundingJournal of Theoretical and Applied Electronic Commerce Research10.3390/jtaer1901003719:1(692-704)Online publication date: 21-Mar-2024
https://doi.org/10.3390/jtaer19010037
Show More Cited By

Index Terms

What Makes a Good Conversation?: Challenges in Designing Truly Conversational Agents
1. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

Addressing Hiccups in Conversations with Recommender Systems
DIS '22: Proceedings of the 2022 ACM Designing Interactive Systems Conference

Conversational Agents (CAs) employing voice as their main interaction mode produce natural language utterances with the aim of mimicking human conversations. To unveil hiccups in conversations with recommender systems, we observed users interacting with ...
Multiagent system for joke generation: Humor and emotions combined in human-agent conversation

In this paper we present an innovative work on a multiagent joking conversational system. In our research so far we have shown that implementing humor into a chatterbot can visibly improve its performance. The results presented in this paper are the ...
A Semi-automatic Wizard of Oz Technique for Let'sFly Spoken Dialogue System
TSD '08: Proceedings of the 11th international conference on Text, Speech and Dialogue

The paper presents Let'sFly spoken dialogue system intended for natural human-computer interaction via telephone lines in travel planning domain. The system uses ASR, keyword spotting and TTS methods for continuous Russian speech and a dialogue manager ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '19: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems

May 2019

9077 pages

ISBN:9781450359702

DOI:10.1145/3290605

General Chairs:
Stephen Brewster
University of Glasgow, Scotland, UK
,
Geraldine Fitzpatrick
TU Wien, Austria
,
Program Chairs:
Anna Cox
University College London, UK
,
Vassilis Kostakos
University of Melbourne, Australia

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 02 May 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Badges

Honorable Mention

Author Tags

Qualifiers

Research-article

Funding Sources

Irish Research Council

Conference

CHI '19

Sponsor:

SIGCHI

CHI '19: CHI Conference on Human Factors in Computing Systems

May 4 - 9, 2019

Glasgow, Scotland Uk

Acceptance Rates

CHI '19 Paper Acceptance Rate 703 of 2,958 submissions, 24%;

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI '25

Sponsor:
sigchi

CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

276
Total Citations
View Citations
6,694
Total Downloads

Downloads (Last 12 months)1,280
Downloads (Last 6 weeks)173

Reflects downloads up to 20 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Karimova GKim Y(2024)The Typology of ChatbotsDesign and Development of Emerging Chatbot Technology10.4018/979-8-3693-1830-0.ch003(51-61)Online publication date: 15-Mar-2024
https://doi.org/10.4018/979-8-3693-1830-0.ch003
B. Sable AKapse D(2024)A Framework Design for Centralised Monitoring of Patient Disease Diagnosis for Better ImprovementInternational Journal of Engineering and Advanced Technology10.35940/ijeat.D4438.1304042413:4(47-52)Online publication date: 30-Apr-2024
https://doi.org/10.35940/ijeat.D4438.13040424
Jeon J(2024)The Effect of AI Agent Gender on Trust and GroundingJournal of Theoretical and Applied Electronic Commerce Research10.3390/jtaer1901003719:1(692-704)Online publication date: 21-Mar-2024
https://doi.org/10.3390/jtaer19010037
Doi TNishikawa Y(2024)Usability Analysis of Smart Speakers from a Learnability Perspective for Novel UsersApplied System Innovation10.3390/asi70300367:3(36)Online publication date: 25-Apr-2024
https://doi.org/10.3390/asi7030036
Zhou JPorat Tvan Zalk N(2024)Humans Mindlessly Treat AI Virtual Agents as Social Beings, but This Tendency Diminishes Among the Young: Evidence From a Cyberball ExperimentHuman Behavior and Emerging Technologies10.1155/2024/88649092024:1Online publication date: 27-Sep-2024
https://doi.org/10.1155/2024/8864909
Seaborn KSawa YWatanabe M(2024)Coimagining the Future of Voice Assistants with Cultural SensitivityHuman Behavior and Emerging Technologies10.1155/2024/32387372024(1-21)Online publication date: 25-Mar-2024
https://doi.org/10.1155/2024/3238737
Joshi RBengler K(2024)Crafting Human-AI Interaction: A Rhetorical Approach to Adaptive Interaction in Conversational AgentsProceedings of the 12th International Conference on Human-Agent Interaction10.1145/3687272.3688297(314-322)Online publication date: 24-Nov-2024
https://dl.acm.org/doi/10.1145/3687272.3688297
Kumar HMusabirov IReza MShi JWang XWilliams JKuzminykh ALiut M(2024)Guiding Students in Using LLMs in Supported Learning Environments: Effects on Interaction Dynamics, Learner Performance, Confidence, and TrustProceedings of the ACM on Human-Computer Interaction10.1145/36870388:CSCW2(1-30)Online publication date: 8-Nov-2024
https://dl.acm.org/doi/10.1145/3687038
Leong JTang JCutrell EJunuzovic SBaribault GInkpen K(2024)Dittos: Personalized, Embodied Agents That Participate in Meetings When You Are UnavailableProceedings of the ACM on Human-Computer Interaction10.1145/36870338:CSCW2(1-28)Online publication date: 8-Nov-2024
https://dl.acm.org/doi/10.1145/3687033
Do HBrachman MDugan CJohnson JLauer JRai PPan Q(2024)Grounding with Structure: Exploring Design Variations of Grounded Human-AI Collaboration in a Natural Language InterfaceProceedings of the ACM on Human-Computer Interaction10.1145/36869028:CSCW2(1-27)Online publication date: 8-Nov-2024
https://dl.acm.org/doi/10.1145/3686902
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents