research-article

Open access

Effects of Persuasive Dialogues: Testing Bot Identities and Inquiry Strategies

Authors:

Zhou YuAuthors Info & Claims

CHI '20: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems

Pages 1 - 13

https://doi.org/10.1145/3313831.3376843

Published: 23 April 2020 Publication History

All formats PDF

Abstract

Intelligent conversational agents, or chatbots, can take on various identities and are increasingly engaging in more human-centered conversations with persuasive goals. However, little is known about how identities and inquiry strategies influence the conversation's effectiveness. We conducted an online study involving 790 participants to be persuaded by a chatbot for charity donation. We designed a two by four factorial experiment (two chatbot identities and four inquiry strategies) where participants were randomly assigned to different conditions. Findings showed that the perceived identity of the chatbot had significant effects on the persuasion outcome (i.e., donation) and interpersonal perceptions (i.e., competence, confidence, warmth, and sincerity). Further, we identified interaction effects among perceived identities and inquiry strategies. We discuss the findings for theoretical and practical implications for developing ethical and effective persuasive chatbots. Our published data, codes, and analyses serve as the first step towards building competent ethical persuasive chatbots.

Supplementary Material

ZIP File (paper714aux.zip)

post-survey.docx: the post-survey material in the study: supplementary.pdf: the supplementary material containing the experimental conditions, the chat interface, examples sentences for the persuasive appeals and the interaction effect illustrations

Download
1.54 MB

MP4 File (a714-shi-presentation.mp4)

Download
58.29 MB

References

[1]

Elisabeth André, Thomas Rist, Susanne Van Mulken, Martin Klesen, and Stefan Baldes. 2000. The automated design of believable dialogues for animated presentation teams. Embodied conversational agents (2000), 220--255.

[2]

Frank R Bentley, Nediyana Daskalova, and Brooke White. 2017. Comparing the reliability of Amazon Mechanical Turk and Survey Monkey to traditional market research surveys. In Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems. ACM, 1092--1099.

Digital Library

[3]

Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. 2017. Enriching Word Vectors with Subword Information. Transactions of the Association for Computational Linguistics 5 (2017), 135--146.

[4]

Michael Buhrmester, Tracy Kwang, and Samuel D Gosling. 2011. Amazon's Mechanical Turk: A new source of inexpensive, yet high-quality, data? Perspectives on psychological science 6, 1 (2011), 3--5.

[5]

J. Cieciuch and E. Davidov. 2012. A comparison of the invariance properties of the PVQ-40 and the PVQ-21 to measure human values across German and Polish Samples. Survey Research Methods 6, 1 (2012), 37--48.

[6]

Kevin Corti and Alex Gillespie. 2015. Revisiting Milgramâ's cyranoid method: experimenting with hybrid human agents. The Journal of social psychology 155, 1 (2015), 30--56.

[7]

Antonella De Angeli, Graham I Johnson, and Lynne Coventry. 2001. The unfriendly user: exploring social reactions to chatterbots. In Proceedings of The International Conference on Affective Human Factors Design, London. 467--474.

[8]

Chad Edwards, Autumn Edwards, Patric R Spence, and Ashleigh K Shelton. 2014. Is that a bot running the social media feed? Testing the differences in perceptions of communication quality for a human agent and a bot agent on Twitter. Computers in Human Behavior 33 (2014), 372--376.

Digital Library

[9]

Brian J Fogg. 2002. Persuasive technology: using computers to change what we think and do. Ubiquity 2002, December (2002), 5.

Digital Library

[10]

Eun Go and S Shyam Sundar. 2019. Humanizing chatbots: The effects of visual, identity and conversational cues on humanness perceptions. Computers in Human Behavior 97 (2019), 304--316.

Digital Library

[11]

Gaston Godin, Paschal Sheeran, Mark Conner, and Marc Germain. 2008. Asking questions changes behavior: mere measurement effects on frequency of blood donation. Health Psychology 27, 2 (2008), 179.

[12]

Lewis R. Goldberg. 1992. The development of markers for the Big-Five factor structure. Psychological Assessment 4, 1 (1992), 26--42.

[13]

California Governor. 2018. California new Autobot Law, Cal. Bus. & Prof. Code 17940, et seq. (SB 1001). (2018). https://leginfo.legislature.ca.gov/faces/billTextClient.xhtml?bill_id=201720180SB1001.

[14]

J. Graham, B. A. Nosek, J. Haidt, R. Iyer, S. Koleva, and P. H. Ditto. 2011. Mapping the moral domain. Journal of Personality and Social Psychology 101, 2 (2011), 366--385.

[15]

Kurt Gray and Daniel M Wegner. 2012. Feeling robots and human zombies: Mind perception and the uncanny valley. Cognition 125, 1 (2012), 125--130.

[16]

Shih S. I. Hamilton, K. and S. Mohammed. 2016. The development and validation of the rational and intuitive decision styles scale. Journal of personality assessment 98, 5 (2016), 523--535.

[17]

He He, Derek Chen, Anusha Balakrishnan, and Percy Liang. 2018. Decoupling strategy and generation in negotiation dialogues. arXiv preprint arXiv:1808.09637 (2018).

[18]

Sally Hibbert, Andrew Smith, Andrea Davies, and Fiona Ireland. 2007. Guilt appeals: Persuasion knowledge and charitable giving. Psychology & Marketing 24, 8 (2007), 723--742.

[19]

Christopher Hidey, Elena Musi, Alyssa Hwang, Smaranda Muresan, and Kathy McKeown. 2017. Analyzing the semantic types of claims and premises in an online persuasive forum. In Proceedings of the 4th Workshop on Argument Mining. 11--21.

[20]

Christopher Thomas Hidey and Kathleen McKeown. 2018. Persuasive influence detection: The role of argument sequencing. In Thirty-Second AAAI Conference on Artificial Intelligence.

[21]

Jacob B Hirsh, Sonia K Kang, and Galen V Bodenhausen. 2012. Personalized persuasion: Tailoring persuasive appeals to recipients-personality traits. Psychological science 23, 6 (2012), 578--581.

[22]

Annabell Ho, Jeff Hancock, and Adam S Miner. 2018. Psychological, relational, and emotional effects of self-disclosure after conversations with a chatbot. Journal of Communication 68, 4 (2018), 712--733.

[23]

Clayton J Hutto and Eric Gilbert. 2014. Vader: A parsimonious rule-based model for sentiment analysis of social media text. In Eighth international AAAI conference on weblogs and social media.

[24]

Siwei Lai, Liheng Xu, Kang Liu, and Jun Zhao. 2015. Recurrent Convolutional Neural Networks for Text Classification. In Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence (AAAI'15). AAAI Press, 2267--2273. http://dl.acm.org/citation.cfm?id=2886521.2886636

Digital Library

[25]

Jong-Eun Roselyn Lee and Clifford I Nass. 2010. Trust in computers: The computers-are-social-actors (CASA) paradigm and trustworthiness perception in human-computer communication. In Trust and technology in a ubiquitous modern environment: Theoretical and methodological perspectives. IGI Global, 1--15.

[26]

Min Kyung Lee, Ji Tae Kim, and Leah Lizarondo. 2017. A human-centered approach to algorithmic services: Considerations for fair and motivating smart community service management that allocates donations to non-profit organizations. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. ACM, 3365--3376.

Digital Library

[27]

Winter Mason and Siddharth Suri. 2012. Conducting behavioral research on Amazon's Mechanical Turk. Behavior research methods 44, 1 (2012), 1--23.

[28]

Alexander Miller, Will Feng, Dhruv Batra, Antoine Bordes, Adam Fisch, Jiasen Lu, Devi Parikh, and Jason Weston. 2017. ParlAI: A Dialog Research Software Platform. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing: System Demonstrations. 79--84.

[29]

S Mori and CP Leblond. 1970. Electron microscopic identification of three classes of oligodendrocytes and a preliminary study of their proliferative activity in the corpus callosum of young rats. Journal of Comparative Neurology 139, 1 (1970), 1--29.

[30]

Clifford Nass and Youngme Moon. 2000. Machines and mindlessness: Social responses to computers. Journal of social issues 56, 1 (2000), 81--103.

[31]

Harri Oinas-Kukkonen and Marja Harjumaa. 2008. Towards deeper understanding of persuasion in software and information systems. In First international conference on advances in computer-human interaction. IEEE, 200--205.

Digital Library

[32]

Harri Oinas-Kukkonen and Marja Harjumaa. 2009. Persuasive systems design: Key issues, process model, and system features. Communications of the Association for Information Systems 24, 1 (2009), 28.

[33]

Rita Orji, Lennart E Nacke, and Chrysanne Di Marco. 2017. Towards personality-driven persuasive health games and gamified systems. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. ACM, 1015--1027.

Digital Library

[34]

Rita Orji, Gustavo F Tondello, and Lennart E Nacke. 2018. Personalizing persuasive strategies in gameful systems to gamification user types. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, 435.

Digital Library

[35]

Rita Orji, Julita Vassileva, and Regan L Mandryk. 2014. Modeling the efficacy of persuasive strategies for different gamer types in serious games for health. User Modeling and User-Adapted Interaction 24, 5 (2014), 453--498.

Digital Library

[36]

James W Pennebaker, Martha E Francis, and Roger J Booth. 2001. Linguistic inquiry and word count: LIWC 2001. Mahway: Lawrence Erlbaum Associates 71, 2001 (2001), 2001.

[37]

Aleksandra Przegalinska, Leon Ciechanowski, Mikolaj Magnuski, and Peter Gloor. 2018. Muse headband: Measuring tool or a collaborative gadget? In Collaborative Innovation Networks. Springer, 93--101.

[38]

Alec Radford, Rafal Jozefowicz, and Ilya Sutskever. 2017. Learning to generate reviews and discovering sentiment. arXiv preprint arXiv:1704.01444 (2017).

[39]

Carol A Scott. 1977. Modifying socially-conscious behavior: The foot-in-the-door technique. Journal of Consumer Research 4, 3 (1977), 156--164.

[40]

Paul W Sherman. 1980. The meaning of nepotism. The American Naturalist 116, 4 (1980), 604--606.

[41]

Weiyan Shi, Kun Qian, Xuewei Wang, and Zhou Yu. 2019. How to Build User Simulators to Train RL-based Dialog Systems. arXiv preprint arXiv:1909.01388 (2019).

[42]

Weiyan Shi and Zhou Yu. 2018. Sentiment adaptive end-to-end dialog systems. arXiv preprint arXiv:1804.10731 (2018).

[43]

David E Sprott, Eric R Spangenberg, Lauren G Block, Gavan J Fitzsimons, Vicki G Morwitz, and Patti Williams. 2006. The question--behavior effect: What we know and where we go from here. Social Influence 1, 2 (2006), 128--137.

[44]

Chenhao Tan, Vlad Niculae, Cristian Danescu-Niculescu-Mizil, and Lillian Lee. 2016. Winning arguments: Interaction dynamics and persuasion strategies in good-faith online discussions. In Proceedings of the 25th international conference on world wide web. International World Wide Web Conferences Steering Committee, 613--624.

Digital Library

[45]

Yla R Tausczik and James W Pennebaker. 2010. The psychological meaning of words: LIWC and computerized text analysis methods. Journal of language and social psychology 29, 1 (2010), 24--54.

[46]

Xuewei Wang, Weiyan Shi, Richard Kim, Yoojung Oh, Sijia Yang, Jingwen Zhang, and Zhou Yu. 2019. Persuasion for Good: Towards a Personalized Persuasive Dialogue System for Social Good. arXiv preprint arXiv:1906.06725 (2019).

[47]

Adam Waytz, John Cacioppo, and Nicholas Epley. 2010. Who sees human? The stability and importance of individual differences in anthropomorphism. Perspectives on Psychological Science 5, 3 (2010), 219--232.

[48]

Patti Williams, Lauren G Block, and Gavan J Fitzsimons. 2006. Simply asking questions about health behaviors increases both healthy and unhealthy behaviors. Social Influence 1, 2 (2006), 117--127.

[49]

Diyi Yang, Jiaao Chen, Zichao Yang, Dan Jurafsky, and Eduard Hovy. 2019. Let's Make Your Request More Persuasive: Modeling Persuasive Strategies via Semi-Supervised Neural Nets on Crowdfunding Platforms. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). 3620--3630.

[50]

Zhou Yu, Xinrui He, Alan W Black, and Alexander I Rudnicky. 2016. User Engagement Study with Virtual Agents Under Different Cultural Contexts. In International Conference on Intelligent Virtual Agents. Springer, 364--368.

[51]

Tangming Yuan, David Moore, and Alec Grierson. 2008. A human-computer dialogue system for educational debate: A computational dialectics approach. International Journal of Artificial Intelligence in Education 18, 1 (2008), 3--26.

Digital Library

Cited By

Izadi SForouzanfar M(2024)Error Correction and Adaptation in Conversational AI: A Review of Techniques and Applications in ChatbotsAI10.3390/ai50200415:2(803-841)Online publication date: 4-Jun-2024
https://doi.org/10.3390/ai5020041
Schubhan MAltmeyer MRogers KDegraen DLessel PKrüger A(2024)Auditory, Visual, or Both? Comparing Visual and Auditory Representations of Game Elements in a Gamified Image-Tagging TaskProceedings of the ACM on Human-Computer Interaction10.1145/36770598:CHI PLAY(1-28)Online publication date: 15-Oct-2024
https://dl.acm.org/doi/10.1145/3677059
Wei JKim SJung HKim Y(2024)Leveraging Large Language Models to Power Chatbots for Collecting User Self-Reported DataProceedings of the ACM on Human-Computer Interaction10.1145/36373648:CSCW1(1-35)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3637364
Show More Cited By

Index Terms

Effects of Persuasive Dialogues: Testing Bot Identities and Inquiry Strategies
1. Computing methodologies
  1. Artificial intelligence
    1. Natural language processing
      1. Discourse, dialogue and pragmatics
2. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

Distilling dialogues: a method using natural dialogue corpora for dialogue systems development
ANLC '00: Proceedings of the sixth conference on Applied natural language processing

We report on a method for utilising corpora collected in natural settings. It is based on distilling (re-writing) natural dialogues to elicit the type of dialogue that would occur if one the dialogue participants was a computer instead of a human. The ...
(Semi-)Automatic Analysis of Dialogues
ICAART 2014: Proceedings of the 6th International Conference on Agents and Artificial Intelligence - Volume 1

We study human-human dialogues and human-computer dialogues with the aim to determine which dialogue acts and communicative strategies do the participants of interaction use, and which structural parts does a dialogue include. In order to simplify the ...
Acoustic-Prosodic Entrainment and Rapport in Collaborative Learning Dialogues
MLA '14: Proceedings of the 2014 ACM workshop on Multimodal Learning Analytics Workshop and Grand Challenge

In spoken dialogue analysis, the speech signal is a rich source of information. We explore in this paper how low level features of the speech signal, such as pitch, loudness, and speaking rate, can inform a model of student interaction in collaborative ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '20: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems

April 2020

10688 pages

ISBN:9781450367080

DOI:10.1145/3313831

General Chairs:
Regina Bernhaupt
Eindhoven University of Technology, Netherlands
,
Florian 'Floyd' Mueller
Monash University, Australia
,
David Verweij
Newcastle University, UK
,
Josh Andres
RMIT, Australia
,
Program Chairs:
Joanna McGrenere
University of British Columbia, Canada
,
Andy Cockburn
University of Canterbury, New Zealand
,
Ignacio Avellino
University of Maryland Baltimore County, USA
,
Alix Goguey
Grenoble Alpes University, France
,
Pernille Bjørn
University of Copenhagen, Denmark
,
Shengdong (Shen) Zhao
National University of Singapore, Singapore
,
Briane Paul Samson
Future University Hakodate, Japan & De La Salle University, Philippines
,
Rafal Kocielnik
University of Washington, USA

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 April 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CHI '20

Sponsor:

SIGCHI

CHI '20: CHI Conference on Human Factors in Computing Systems

April 25 - 30, 2020

HI, Honolulu, USA

Acceptance Rates

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI '25

Sponsor:
sigchi

CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

40
Total Citations
View Citations
3,020
Total Downloads

Downloads (Last 12 months)975
Downloads (Last 6 weeks)167

Reflects downloads up to 12 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Izadi SForouzanfar M(2024)Error Correction and Adaptation in Conversational AI: A Review of Techniques and Applications in ChatbotsAI10.3390/ai50200415:2(803-841)Online publication date: 4-Jun-2024
https://doi.org/10.3390/ai5020041
Schubhan MAltmeyer MRogers KDegraen DLessel PKrüger A(2024)Auditory, Visual, or Both? Comparing Visual and Auditory Representations of Game Elements in a Gamified Image-Tagging TaskProceedings of the ACM on Human-Computer Interaction10.1145/36770598:CHI PLAY(1-28)Online publication date: 15-Oct-2024
https://dl.acm.org/doi/10.1145/3677059
Wei JKim SJung HKim Y(2024)Leveraging Large Language Models to Power Chatbots for Collecting User Self-Reported DataProceedings of the ACM on Human-Computer Interaction10.1145/36373648:CSCW1(1-35)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3637364
Mishra KFirdaus MEkbal A(2024) Please Donate to Save a Life : Inducing Politeness to Handle Resistance in Persuasive Dialogue Agents IEEE/ACM Transactions on Audio, Speech, and Language Processing10.1109/TASLP.2024.335703232(2202-2212)Online publication date: 2024
https://doi.org/10.1109/TASLP.2024.3357032
Yu RFeng JWang KYang LFeng J(2024)Who are you talking to? How chatbot identity disclosure affects service satisfactionJournal of Travel & Tourism Marketing10.1080/10548408.2024.236975541:8(1090-1106)Online publication date: 15-Jul-2024
https://doi.org/10.1080/10548408.2024.2369755
Hauptmann CKrenzer AVölkel JPuppe F(2024)Argumentation effect of a chatbot for ethical discussions about autonomous AI scenariosKnowledge and Information Systems10.1007/s10115-024-02074-x66:6(3607-3637)Online publication date: 13-Mar-2024
https://doi.org/10.1007/s10115-024-02074-x
Zeng DLegaspi RSun YDong XIkeda KSpirtes PZhang K(2024)Counterfactual Reasoning Using Predicted Latent Personality Dimensions for Optimizing Persuasion OutcomePersuasive Technology10.1007/978-3-031-58226-4_22(287-300)Online publication date: 10-Apr-2024
https://doi.org/10.1007/978-3-031-58226-4_22
Huseynov F(2023)Chatbots in Digital MarketingContemporary Approaches of Digital Marketing and the Role of Machine Intelligence10.4018/978-1-6684-7735-9.ch003(46-72)Online publication date: 30-Jun-2023
https://doi.org/10.4018/978-1-6684-7735-9.ch003
Ferreira MBarbosa B(2023)A Review on Chatbot Personality and Its Expected Effects on UsersTrends, Applications, and Challenges of Chatbot Technology10.4018/978-1-6684-6234-8.ch010(222-243)Online publication date: 24-Feb-2023
https://doi.org/10.4018/978-1-6684-6234-8.ch010
Chatterjee CChugunova MGhosh MSinghal AWang L(2023)Human mediation leads to higher compliance in digital mental health: field evidence from IndiaFrontiers in Behavioral Economics10.3389/frbhe.2023.12324622Online publication date: 28-Sep-2023
https://doi.org/10.3389/frbhe.2023.1232462
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents