research-article

(Non-)Interacting with conversational agents: perceptions and motivations of using chatbots and voice assistants

Authors:

Thomas LudwigAuthors Info & Claims

MuC '20: Proceedings of Mensch und Computer 2020

Pages 321 - 331

https://doi.org/10.1145/3404983.3405513

Published: 06 September 2020 Publication History

Abstract

Conversational agents (CAs) such as Siri, Alexa, and Google Assistant are increasingly penetrating everyday life. From a Human-Computer Interaction (HCI) perspective, designing CAs that appropriately support the way they are used within daily life is still challenging. While initial design guidelines for human-AI interaction exist, we still know little about how users actually perceive CAs within their daily lives and what aspects motivate their usage of such tools. Within our research, we therefore conducted an interview study with 29 participants to uncover daily positive and negative experiences with CAs. By revealing how users currently perceive CAs, we identify quality criteria that could inform their future design. By evaluating these criteria with respect to existing research discourses about user experience (UX) guidelines for CAs, we contribute to the field by extending these guidelines from an end-user's perspective.

References

[1]

Saleema Amershi, Dan Weld, Mihaela Vorvoreanu, Adam Fourney, Besmira Nushi, Penny Collisson, Jina Suh, Shamsi Iqbal, Paul N Bennett, Kori Inkpen, Jaime Teevan, Ruth Kikin-Gil, and Eric Horvitz. 2019. Guidelines for Human-AI Interaction. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems - CHI '19, 1--13.

Digital Library

[2]

Prashanti Angara, Miguel Jiménez, Kirti Agarwal, Harshit Jain, Roshni Jain, Ulrike Stege, Sudhakar Ganti, Hausi A Müller, and Joanna W Ng. 2017. Foodie Fooderson a Conversational Agent for the Smart Kitchen. In CASCON '17 Proceedings of the 27th Annual International Conference on Computer Science and Software Engineering, 247--253. Retrieved from http://dl.acm.org/citation.cfm?id=3172795.3172825

Digital Library

[3]

Tiago Araújo and Beatriz Casais. 2020. Customer Acceptance of Shopping-Assistant Chatbots. In Smart Innovation, Systems and Technologies, Rocha Álvaro, José Luís Reis, Marc K. Peter and Zorica Bogdanović (eds.). Springer Nature, Singapore, 278--287.

[4]

Zahra Ashktorab, Mohit Jain, Q. Vera Liao, and Justin D. Weisz. 2019. Resilient Chatbots: Repair Strategy Preferences for Conversational Breakdowns. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems - CHI '19, ACM Press, New York, New York, USA, 1--12.

Digital Library

[5]

Baidu. Baidu Announces Fourth Quarter and Fiscal Year 2019 Results. Retrieved March 20, 2020 from http://ir.baidu.com/news-releases/news-release-details/baidu-announces-fourth-quarter-and-fiscal-year-2019-results

[6]

Monique Faye Baier and Michael Burmester. 2019. Not Just About the User. In Proceedings of Mensch und Computer 2019 on - MuC'19, ACM Press, New York, New York, USA, 349--359.

Digital Library

[7]

Frank Bentley, Chris Luvogt, Max Silverman, Rushani Wirasinghe, Brooke White, and Danielle Lottridge. 2018. Understanding the Long-Term Use of Smart Speaker Assistants. Proc. ACM Interactive, Mobile, Wearable Ubiquitous Technol. 2, 3 (September 2018), 1--24.

Digital Library

[8]

Petter Bae Brandtzaeg and Asbjørn Følstad. 2017. Why People Use Chatbots. In Internet Science, Ioannis Kompatsiaris, Jonathan Cave, Anna Satsiou, Georg Carle, Antonella Passani, Efstratios Kontopoulos, Sotiris Diplaris and Donald McMillan (eds.). Springer, Cham, 377--392.

Digital Library

[9]

Justine Cassell and Kristinn R. Thorisson. 1999. The power of a nod and a glance: Envelope vs. emotional feedback in animated conversational agents. Appl. Artif. Intell. 13, 4--5 (May 1999), 519--538.

[10]

Ana Paula Chaves and Marco Aurelio Gerosa. 2018. Single or multiple conversational agents? An interactional coherence comparison. In Conference on Human Factors in Computing Systems - Proceedings, 1--13.

Digital Library

[11]

Yu-ting Cheng, Guo-Ling Tsai, Heng-an Lin, Yi-Ching Huang, Rung Huei Liang, and Lin-lin Chen. 2019. Call Me by My Name: Exploring Roles of Sci-fi Voice Agents. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems - CHI '19, 1--6.

Digital Library

[12]

Leigh Clark, Nadia Pantidi, Orla Cooney, Philip Doyle, Diego Garaialde, Justin Edwards, Brendan Spillane, Emer Gilmartin, Christine Murad, Cosmin Munteanu, Vincent Wade, and Benjamin R. Cowan. 2019. What Makes a Good Conversation? Challenges in Designing Truly Conversational Agents. 1--12.

Digital Library

[13]

Cleverbot. Cleverbot.com - a clever bot - speak to an AI with some Actual Intelligence? Retrieved April 2, 2020 from https://www.cleverbot.com/

[14]

Kathleen S. Crittenden and Richard J Hill. 1971. Coding Reliability and Validity of Interview Data. Am. Sociol. Rev. 36, 6 (1971), 1073--1080.

[15]

Robert Dale. 2016. The return of the chatbots. Nat. Lang. Eng. 22, 5 (2016), 811--817.

[16]

Florian Daniel, Maristella Matera, Vittorio Zaccaria, and Alessandro Dell'Orto. 2018. Toward truly personal chatbots. In Proceedings of the 1st International Workshop on Software Engineering for Cognitive Services - SE4COG '18, ACM Press, New York, New York, USA, 31--36.

Digital Library

[17]

Stephan Diederich, Alfred Benedikt Brendel, and Lutz M. Kolbe. 2019. Towards Designing an Anthropomorphic Conversational Agent for Assisting Job Interview Preparation. In Proceedings of the Workshop on Designing User Assistance in Interactive Intelligent Systems, 0--3.

[18]

Stephan Diederich, Alfred Benedikt Brendel, and Lutz M. Kolbe. 2019. On Conversational Agents in Information Systems Research: Analyzing the Past to Guide Future Work. In Proceedings of Internationale Tagung Wirtschaftsinformatik, Siegen, Germany.

[19]

Ahmed Fadhil and Adolfo Villafiorita. 2017. An Adaptive Learning with Gamification & Conversational UIs: The Rise of CiboPoliBot. In Adjunct Publication of the 25th Conference on User Modeling, Adaptation and Personalization - UMAP '17, ACM Press, New York, New York, USA, 408--412.

Digital Library

[20]

Ethan Fast, Binbin Chen, Julia Mendelsohn, Jonathan Bassen, and Michael S. Bernstein. 2018. Iris: A Conversational Agent for Complex Tasks Ethan. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems - CHI '18, ACM Press, New York, New York, USA, 1--12.

Digital Library

[21]

Jasper Feine, Ulrich Gnewuch, Stefan Morana, and Alexander Maedche. 2019. A Taxonomy of Social Cues for Conversational Agents. Int. J. Hum. Comput. Stud. 132, July (2019), 138--161.

Digital Library

[22]

Jennifer Fereday and Eimear Muir-Cochrane. 2006. Demonstrating Rigor Using Thematic Analysis: A Hybrid Approach of Inductive and Deductive Coding and Theme Development. Int. J. Qual. Methods 5, 1 (2006), 80--92.

[23]

Asbjørn Følstad, Cecilie Bertinussen Nordheim, and Cato Alexander Bjørkli. 2018. What Makes Users Trust a Chatbot for Customer Service? An Exploratory Interview Study. In Internet Science, Svetlana S. Bodrunova (ed.). Springer International Publishing, Cham, 194--208.

[24]

Asbjørn Følstad and Marita Skjuve. 2019. Chatbots for Customer Service: User Experience and Motivation. In Proceedings of the 1st International Conference on Conversational User Interfaces - CUI '19, ACM Press, New York, New York, USA, 1--9.

Digital Library

[25]

M. D Gall, J. P. Gall, and W. R. Borg. 2003. Educational Research: An Introduction. A & B Publications, Boston, MA.

[26]

U Gnewuch, S Morana, M Adam, and A Mädche. 2017. Towards Designing Cooperative and Social Conversational Agents for Customer Service. In Proceedings of the International Conference on Information Systems.

[27]

Margot J. van der Goot and Tyler Pilgrim. 2020. Exploring Age Differences in Motivations for and Acceptance of Chatbot Communication in a Customer Service Context. In Chatbot Research and Design. CONVERSATIONS 2019. Lecture Notes in Computer Science, Følstad A. et al. (ed.). Springer, Cham, 173--186.

Digital Library

[28]

Sviatlana Höhn. 2019. Artificial Companion for Second Language Conversation. Springer International Publishing, Cham.

[29]

Mohit Jain, Pratyush Kumar, Ramachandra Kota, and Shwetak N. Patel. 2018. Evaluating and informing the design of chatbots. DIS 2018 - Proc. 2018 Des. Interact. Syst. Conf. (2018), 895--906.

Digital Library

[30]

Karolina Kuligowska. 2015. Department of Information Systems and Economic Analysis, Faculty of Economic Sciences, University of Warsaw, Warsaw, Poland. Prof. Cent. Bus. Res. 2, 2 (2015), 1--16.

[31]

Minha Lee, Sander Ackermans, Nena van As, Hanwen Chang, Enzo Lucas, and Wijnand IJsselsteijn. 2019. Caring for Vincent:: A Chatbot for Self-compassion. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems - CHI '19, ACM Press, New York, New York, USA, 1--13.

Digital Library

[32]

Amanda Lenhart, Rich Ling, Scott Campbell, and Kristen Purcell. 2010. Teens and Mobile Phones. Pew Internet & American Life Project. Retrieved from https://files.eric.ed.gov/fulltext/ED525059.pdf

[33]

Q. Vera Liao, Muhammed Mas Ud Hussain, Praveen Chandar, Matthew Davis, Yasaman Khazaen, Marco Patricio Crasso, Dakuo Wang, Michael Muller, N. Sadat Shami, and Werner Geyer. 2018. All work and no play? Conversations with a question-and-answer chatbot in the wild. Conf. Hum. Factors Comput. Syst. - Proc. 2018-April, (2018), 1--13.

Digital Library

[34]

R López-Cózar, Jan Silovsky, and David Griol. 2010. F2 - New Technique for Recognition of User Emotional States in Spoken Dialogue Systems. In Proceedings of SIGDIAL 2010: the 11th Annual Meeting of the Special Interest Group on Discourse and Dialogue, 281--288. Retrieved from http://dl.acm.org/citation.cfm?id=1944559

[35]

Ewa Luger and Abigail Sellen. 2016. "Like Having a Really Bad PA": The Gulf between User Expectation and Experience of Conversational Agents. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, ACM, New York, NY, USA, 5286--5297.

Digital Library

[36]

Michael McTear, Zoraida Callejas, and David Griol. 2016. The Conversational Interface. Springer International Publishing, Cham.

[37]

Andreea Muresan and Henning Pohl. 2019. Chats with Bots. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems, ACM, New York, NY, USA, 1--6.

Digital Library

[38]

Pandorabots. Mitsuku. Retrieved April 2, 2020 from https://www.pandorabots.com/mitsuku/

[39]

Diana Perez-Marin and Ismael Pascual-Nieto. 2011. Conversational Agents and Natural Language Interaction: Techniques and Effective Practices (1st ed.). IGI Global, Hershey.

[40]

Lara S. G. Piccolo, Martino Mensio, and Harith Alani. 2018. Chasing the Chatbots: Directions for Interaction and Design Research. Conversations 2018: 2nd International Workshop on Chatbot Research.

[41]

Nicole M. Radziwill and Morgan C. Benton. 2017. Evaluating Quality of Chatbots and Intelligent Conversational Agents. (2017). Retrieved from http://arxiv.org/abs/1704.04579

[42]

Sherry Ruan, Liwei Jiang, Justin Xu, Bryce Joe-Kun Tham, Zheng-neng Qiu, Yeshuang Zhu, Elizabeth L Murnane, Emma Brunskill, and James A Landay. 2019. QuizBot. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems - CHI '19, ACM Press, New York, New York, USA, 1--13.

Digital Library

[43]

Juliana Schroeder and Matthew Schroeder. 2018. Trusting in Machines: How Mode of Interaction Affects Willingness to Share Personal Information with Machines. In Proceedings of the 51st Hawaii International Conference on System Sciences.

[44]

Bayan Shawar and Eric Atwell. 2007. Chatbots: Are they Really Useful? LDV Forum 22, (2007), 29--49.

[45]

Timo Strohmann, Simon Fischer, Dominik Siemon, Florian Brachten, Christoph Lattemann, Susanne Robra-Bissantz, and Stefan Stieglitz. 2018. Virtual Moderation Assistance: Creating Design Guidelines for Virtual Assistants Supporting Creative Workshops. Proc. Pacific Asia Conf. Inf. Syst. (2018), 3580--3594.

[46]

Stergios Tegos, Stavros Demetriadis, and Thrasyvoulos Tsiatsos. 2012. Using a conversational agent for promoting collaborative language learning. Proc. 2012 4th Int. Conf. Intell. Netw. Collab. Syst. INCoS 2012 (2012), 162--165.

Digital Library

[47]

Zoya Teirstein. Duolingo's new crew of AI tutors will help you learn languages - The Verge. Retrieved March 26, 2020 from https://www.theverge.com/2016/10/6/13188326/duolingo-language-tutor-chatbor-ai-announced

[48]

Gabriele Trovato, Josué G. Ramos, Helio Azevedo, Artemis Moroni, Silvia Magossi, Hiroyuki Ishii, Reid Simmons, and Atsuo Takanishi. 2015. "Olá, my name is Ana": A study on Brazilians interacting with a receptionist robot. Proc. 17th Int. Conf. Adv. Robot. ICAR 2015 (2015), 66--71.

[49]

Francisco A.M. Valério, Tatiane G. Guimarães, Raquel O. Prates, and Heloisa Candello. 2017. Here's what i can do: Chatbots' strategies to convey their features to users. ACM Int. Conf. Proceeding Ser. September (2017).

Digital Library

[50]

Joseph Weizenbaum. 1983. ELIZA --- a computer program for the study of natural language communication between man and machine. Commun. ACM 26, 1 (January 1983), 23--28.

Digital Library

[51]

Ying Xu and Mark Warschauer. 2019. Young Children's Reading and Learning with Conversational Agents. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems, ACM, New York, NY, USA, 1--8.

Digital Library

[52]

Xi Yang, Marco Aurisicchio, and Weston Baxter. 2019. Understanding Affective Experiences with Conversational Agents. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems - CHI '19, ACM Press, New York, New York, USA, 1--12.

Digital Library

[53]

Zhou Yu, Leah Nicolich-Henkin, Alan W Black, and Alexander Rudnicky. 2016. A Wizard-of-Oz Study on A Non-Task-Oriented Dialog Systems That Reacts to User Engagement. In Proceedings of the 17th Annual Meeting of the Special Interest Group on Discourse and Dialogue, Association for Computational Linguistics, Stroudsburg, PA, USA, 55--63.

[54]

Li Zhou, Jianfeng Gao, Di Li, and Heung-Yeung Shum. 2020. The Design and Implementation of XiaoIce, an Empathetic Social Chatbot. Comput. Linguist. 46, 1 (March 2020), 53--93.

Digital Library

Cited By

Seaborn KUrakami JPennefather PMiyake N(2024)Qualitative Approaches to Voice UXACM Computing Surveys10.1145/365866656:12(1-34)Online publication date: 20-Apr-2024
https://dl.acm.org/doi/10.1145/3658666
Ha JJeon HHan DSeo JOh C(2024)CloChat: Understanding How People Customize, Interact, and Experience Personas in Large Language ModelsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642472(1-24)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642472
Anchitaalagammai JKavitha SMurali SGururaj KKiruthik J(2024)Predictive Health Assistant: AI-Driven Disease Projection Tool2024 11th International Conference on Reliability, Infocom Technologies and Optimization (Trends and Future Directions) (ICRITO)10.1109/ICRITO61523.2024.10522266(1-6)Online publication date: 14-Mar-2024
https://doi.org/10.1109/ICRITO61523.2024.10522266
Show More Cited By

Index Terms

(Non-)Interacting with conversational agents: perceptions and motivations of using chatbots and voice assistants
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Empirical studies in HCI

Recommendations

Evaluating and Informing the Design of Chatbots
DIS '18: Proceedings of the 2018 Designing Interactive Systems Conference

Text messaging-based conversational agents (CAs), popularly called chatbots, received significant attention in the last two years. However, chatbots are still in their nascent stage: They have a low penetration rate as 84% of the Internet users have not ...
Elderly Users' Interaction with Conversational Agent
HAI '19: Proceedings of the 7th International Conference on Human-Agent Interaction

Exploring how users behave when they interact with a conversational agent has been a popular research topic recently. However, very few studies have focused on the unique features of old people's interaction with an agent. In this paper, we report the ...
Exploring Effects of Conversational Fillers on User Perception of Conversational Agents
CHI EA '19: Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems

Through technological advancements in various areas of our lives, Conversational Agents progressed in their human-likeness. In the field of HCI, however, the use of conversational fillers (e.g., "um," "uh," etc.) by Conversational Agents have not been ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

MuC '20: Proceedings of Mensch und Computer 2020

September 2020

523 pages

ISBN:9781450375405

DOI:10.1145/3404983

General Chairs:
Bernhard Preim
Universität Magdeburg
,
Andreas Nürnberger
Universität Magdeburg
,
Christian Hansen
Universität Magdeburg

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 06 September 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

EFRE

Conference

MuC'20

MuC'20: Mensch und Computer 2020

September 6 - 9, 2020

Magdeburg, Germany

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

20
Total Citations
View Citations
1,151
Total Downloads

Downloads (Last 12 months)212
Downloads (Last 6 weeks)18

Reflects downloads up to 09 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Seaborn KUrakami JPennefather PMiyake N(2024)Qualitative Approaches to Voice UXACM Computing Surveys10.1145/365866656:12(1-34)Online publication date: 20-Apr-2024
https://dl.acm.org/doi/10.1145/3658666
Ha JJeon HHan DSeo JOh C(2024)CloChat: Understanding How People Customize, Interact, and Experience Personas in Large Language ModelsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642472(1-24)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642472
Anchitaalagammai JKavitha SMurali SGururaj KKiruthik J(2024)Predictive Health Assistant: AI-Driven Disease Projection Tool2024 11th International Conference on Reliability, Infocom Technologies and Optimization (Trends and Future Directions) (ICRITO)10.1109/ICRITO61523.2024.10522266(1-6)Online publication date: 14-Mar-2024
https://doi.org/10.1109/ICRITO61523.2024.10522266
Shi YDeng B(2024)Finding the sweet spot: Exploring the optimal communication delay for AI feedback toolsInformation Processing & Management10.1016/j.ipm.2023.10357261:2(103572)Online publication date: Mar-2024
https://doi.org/10.1016/j.ipm.2023.103572
Brock LDe Bortoli LBellei EDe Marchi A(2024)Conversational agents and momentary user experience: an assessment using an electroencephalography deviceUniversal Access in the Information Society10.1007/s10209-024-01144-9Online publication date: 19-Aug-2024
https://doi.org/10.1007/s10209-024-01144-9
Weber PMahmood FAhmadi Mvon Jan VLudwig TWieching R(2023)Fridolin: participatory design and evaluation of a nutrition chatbot for older adultsi-com10.1515/icom-2022-004222:1(33-51)Online publication date: 14-Mar-2023
https://doi.org/10.1515/icom-2022-0042
Seymour WZhan XCoté MSuch J(2023)A Systematic Review of Ethical Concerns with Voice AssistantsProceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society10.1145/3600211.3604679(131-145)Online publication date: 8-Aug-2023
https://dl.acm.org/doi/10.1145/3600211.3604679
Choi YLee MLee S(2023)Toward a Multilingual Conversational Agent: Challenges and Expectations of Code-mixing Multilingual UsersProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581445(1-17)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581445
Zhang RLi KHao YWang YLai ZGuimbretière FZhang C(2023)EchoSpeech: Continuous Silent Speech Recognition on Minimally-obtrusive Eyewear Powered by Acoustic SensingProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3580801(1-18)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3580801
Wald RPiotrowski JAraujo Tvan Oosten J(2023)Virtual assistants in the family home. Understanding parents’ motivations to use virtual assistants with their Child(dren)Computers in Human Behavior10.1016/j.chb.2022.107526139:COnline publication date: 1-Feb-2023
https://dl.acm.org/doi/10.1016/j.chb.2022.107526
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents