research-article

Chatbots Language Design: The Influence of Language Variation on User Experience with Tourist Assistant Chatbots

Authors:

Ana Paula Chaves,

Marco Aurelio GerosaAuthors Info & Claims

ACM Transactions on Computer-Human Interaction, Volume 29, Issue 2

Article No.: 13, Pages 1 - 38

https://doi.org/10.1145/3487193

Published: 16 January 2022 Publication History

Abstract

Chatbots are often designed to mimic social roles attributed to humans. However, little is known about the impact of using language that fails to conform to the associated social role. Our research draws on sociolinguistic to investigate how a chatbot’s language choices can adhere to the expected social role the agent performs within a context. We seek to understand whether chatbots design should account for linguistic register. This research analyzes how register differences play a role in shaping the user’s perception of the human-chatbot interaction. We produced parallel corpora of conversations in the tourism domain with similar content and varying register characteristics and evaluated users’ preferences of chatbot’s linguistic choices in terms of appropriateness, credibility, and user experience. Our results show that register characteristics are strong predictors of user’s preferences, which points to the needs of designing chatbots with register-appropriate language to improve acceptance and users’ perceptions of chatbot interactions.

Supplementary Material

chaves (chaves.zip)

Supplemental movie, appendix, image and software files for, Chatbots Language Design: The Influence of Language Variation on User Experience with Tourist Assistant Chatbots

Download
1.30 MB

References

[1]

B. Abu Shawar and E. Atwell. 2004. Evaluation of chatbot information system. In Proceedings of the 8th Maghrebian Conference on Software Engineering and Artificial Intelligence. Centre de Publication Universitaire, Tunis, 12.

[2]

Papathanassis Alexis. 2017. R-Tourism: Introducing the potential impact of robotics and service automation in tourism. Ovidius University Annals, Series Economic Sciences 17, 1 (2017), 211–216.

[3]

Laurence Anthony. 2005. AntConc: Design and development of a freeware corpus analysis toolkit for the technical writing classroom. In Proceedings of the International Professional Communication Conference, 2005.IEEE, 729–737.

[4]

Markus Appel, David Izydorczyk, Silvana Weber, Martina Mara, and Tanja Lischetzke. 2020. The uncanny of mind in a machine: Humanoid robots as tools, agents, and experiencers. Computers in Human Behavior 102 (2020), 274–286.

Digital Library

[5]

Theo Araujo. 2018. Living up to the chatbot hype: The influence of anthropomorphic design cues and communicative agency framing on conversational agent and company perceptions. Computers in Human Behavior 85 (2018), 183–189.

[6]

Shlomo Argamon. 2019. Register in computational language research. Register Studies 1, 1 (2019), 100–135.

[7]

Shlomo Argamon, Moshe Koppel, and Galit Avneri. 1998. Routing documents according to style. In Proceedings of the 1st International Workshop on Innovative Information Systems. Citeseer, 85–92.

[8]

Mikhail Mikhaĭlovich Bakhtin. 2010. Speech Genres and Other Late Essays. University of Texas Press, Austin, TX.

[9]

Divyaa Balaji. 2019. Assessing User Satisfaction with Information Chatbots: A Preliminary Investigation. Master’s thesis. University of Twente. Retrieved from https://essay.utwente.nl/79785/1/Balaji_MA_BMS.pdf.

[10]

Douglas Bates, Martin Mächler, Ben Bolker, and Steve Walker. 2015. Fitting linear mixed-effects models using lme4. Journal of Statistical Software 67, 1 (2015), 1–48. DOI:https://doi.org/10.18637/jss.v067.i01

[11]

Douglas Biber. 1988. Variation across Speech and Writing. Cambridge University Press, Cambridge.

[12]

Douglas Biber. 1995. Dimensions of Register Variation: A Cross-Linguistic Comparison. Cambridge University Press, New York, NY.

[13]

Douglas Biber. 2012. Register as a predictor of linguistic variation. Corpus Linguistics and Linguistic Theory 8, 1 (2012), 9–37.

[14]

Douglas Biber. 2017. MAT–Multidimensional Analysis Tagger. Retrieved 27 August, 2021 from https://goo.gl/u7h9gb.

[15]

Douglas Biber. 2019. Text-linguistic approaches to register variation. Register Studies 1, 1 (2019), 42–75.

[16]

Douglas Biber and Susan Conrad. 2019. Register, Genre, and Style (2nd ed.). Cambridge University Press, New York, NY.

[17]

Douglas Biber, Susan Conrad, and Viviana Cortes. 2004. If you look at...: Lexical bundles in university teaching and textbooks. Applied Linguistics 25, 3 (2004), 371–405.

[18]

Douglas Biber and Jesse Egbert. 2016. Using multi-dimensional analysis to study register variation on the searchable web. Corpus Linguistics Research 2, 21 (2016), 1–23.

[19]

D. Biber and J. Egbert. 2018. Register Variation Online. Cambridge University Academic Press, Cambridge.

[20]

Douglas Biber and Bethany Gray. 2010. Challenging stereotypes about academic writing: Complexity, elaboration, explicitness. Journal of English for Academic Purposes 9, 1 (2010), 2–20.

[21]

Douglas Biber, Bethany Gray, and Kornwipa Poonpon. 2011. Should we use characteristics of conversation to measure grammatical complexity in L2 writing development? Tesol Quarterly 45, 1 (2011), 5–35.

[22]

Douglas Biber, Stig Johansson, Geoffrey Leech, Susan Conrad, Edward Finegan, and Randolph Quirk. 1999. Longman Grammar of Spoken and Written English. Vol. 2, Pearson Longman, London.

[23]

Nina Böcker. 2019. Usability of Information-Retrieval Chatbots and the Effects of Avatars on Trust. B.S. thesis. University of Twente.

[24]

Susan Bosher and Melissa Bowles. 2008. The effects of linguistic modification on ESL students’ comprehension of nursing course test items-a collaborative process is used to modify multiple-choice questions for comprehensibility without damaging the integrity of the item. Nursing Education Perspectives 29, 4 (2008), 174.

[25]

Petter Bae Brandtzaeg and Asbjørn Følstad. 2018. Chatbots: Changing user needs and motivations. Interactions 25, 5 (2018), 38–43.

Digital Library

[26]

Dimitrios Buhalis and Soo Hyun Jun. 2011. E-tourism. In Contemporary Tourism Reviews. Chris Cooper (Ed.), Goodfellow Publishers Limited, Woodeaton, Oxford, 1–38.

[27]

Jhonny Cerezo, Juraj Kubelka, Romain Robbes, and Alexandre Bergel. 2019. Building an expert recommender chatbot. In Proceedings of the 1st International Workshop on Bots in Software Engineering. IEEE, New York, NY, 59–63.

Digital Library

[28]

Ana Paula Chaves. 2020. GitHub Repository. Retrieved 27 August, 2021 from https://github.com/chavesana/chatbots-register.

[29]

Ana Paula Chaves, Eck Doerry, Jesse Egbert, and Marco Gerosa. 2019. It’s how you say it: Identifying appropriate register for chatbot language design. In Proceedings of the 7th International Conference on Human-Agent Interaction. ACM, New York, NY, 1–8. DOI:https://doi.org/10.1145/3349537.3351901

Digital Library

[30]

Ana Paula Chaves, Jesse Egbert, and Marco Aurelio Gerosa. 2019. Chatting like a robot: The relationship between linguistic choices and users’ experiences. In Proceedings of the ACM CHI 2019 Workshop on Conversational Agents: Acting on the Wave of Research and Development. Retrieved from https://convagents.org/.

[31]

Ana Paula Chaves and Marco Aurelio Gerosa. 2018. Single or multiple conversational agents? An interactional coherence comparison. In Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, NY, 191:1–191:13.

Digital Library

[32]

Ana Paula Chaves and Marco Aurelio Gerosa. 2020. How should my chatbot interact? A survey on social characteristics in human–chatbot interaction design. International Journal of Human–Computer Interaction 0, 0 (2020), 1–30. DOI:https://doi.org/10.1080/10447318.2020.1841438

[33]

Tianqi Chen and Carlos Guestrin. 2016. XGBoost: A scalable tree boosting system. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining . ACM, New York, NY, 785–794. DOI:https://doi.org/10.1145/2939672.2939785

Digital Library

[34]

Zhifa Chen, Yichen Lu, Mika P. Nieminen, and Andrés Lucero. 2020. Creating a chatbot for and with migrants: Chatbot personality drives co-design activities. In Proceedings of the 2020 ACM Designing Interactive Systems Conference. ACM, New York, NY, 219–230.

Digital Library

[35]

R. H. B. Christensen. 2019. ordinal—Regression models for ordinal data. R package version 2019.12-10. Retrieved 27 August, 2021 from https://CRAN.R-project.org/package=ordinal.

[36]

Leon Ciechanowski, Aleksandra Przegalinska, Mikolaj Magnuski, and Peter Gloor. 2018. In the shades of the uncanny valley: An experimental study of human–chatbot interaction. Future Generation Computer Systems 92 (2018), 539–548.

[37]

Susan Conrad and Douglas Biber. 2014. Multi-Dimensional Studies of Register Variation in English. Routledge, New York, NY.

[38]

Cynthia L. Corritore, Robert P. Marble, Susan Wiedenbeck, Beverly Kracher, and Ashwin Chandran. 2005. Measuring online trust of websites: Credibility, perceived ease of use, and risk. In Proceedings of the Americas Conference on Information Systems. Association for Information Systems, Atlanta, GA, 370.

[39]

Benjamin R. Cowan, Philip Doyle, Justin Edwards, Diego Garaialde, Ali Hayes-Brady, Holly P. Branigan, João Cabral, and Leigh Clark. 2019. What’s in an accent? The impact of accented synthetic speech on lexical choice in human-machine dialogue. In Proceedings of the 1st International Conference on Conversational User Interfaces. ACM, New York, NY, 1–8.

Digital Library

[40]

Hyo Jin Do, Seon Hye Yang, Boo-Gyoung Choi, Wayne T. Fu, and Brian P. Bailey. 2021. Do you have time for a quick chat? Designing a conversational interface for sexual harassment prevention training. In Proceedings of the 26th International Conference on Intelligent User Interfaces. ACM, New York, NY, 542–552.

Digital Library

[41]

AnHai Doan, Alon Halevy, and Zachary Ives. 2012. String matching. In Principles of Data Integration. AnHai Doan, Alon Halevy, and Zachary Ives (Eds.), Morgan Kaufmann, Boston, Chapter 4, 95–119. DOI:https://doi.org/10.1016/B978-0-12-416044-6.00004-1

Digital Library

[42]

Daniëlle Duijst. 2017. Can We Improve the User Experience of Chatbots with Personalisation. Master’s thesis. University of Amsterdam.

[43]

Willem Duijvelshoff. 2017. Use-cases and ethics of chatbots on Plek: A social intranet for organizations. In Proceedings of the Workshop on Chatbots and Artificial Intelligence.

[44]

Gregory Dyke, Iris Howley, David Adamson, Rohit Kumar, and Carolyn Penstein Rosé. 2013. Towards academically productive talk supported by conversational agents. In Intelligent Tutoring Systems. Stefano A. Cerri, William J. Clancey, Giorgos Papadourakis, and Kitty Panourgia (Eds.), Springer, Berlin, Heidelberg, 531–540.

Digital Library

[45]

Justin Edwards and Elaheh Sanoubari. 2019. A need for trust in conversational interface research. In Proceedings of the 1st International Conference on Conversational User Interfaces. ACM, New York, NY, 1–3.

Digital Library

[46]

Ela Elsholz, Jon Chamberlain, and Udo Kruschwitz. 2019. Exploring language style in chatbots to increase perceived product value and user engagement. In Proceedings of the 2019 Conference on Human Information Interaction and Retrieval. ACM, New York, NY, 301–305.

Digital Library

[47]

Facebook. 2018. Tech for Tourism-Rresearch Page. Retrieved 27 August, 2021 from https://www.facebook.com/VisitFlagstaff/.

[48]

Jasper Feine, Ulrich Gnewuch, Stefan Morana, and Alexander Maedche. 2019. A taxonomy of social cues for conversational agents. International Journal of Human-Computer Studies 132 (2019), 138–161.

Digital Library

[49]

Kraig Finstad. 2010. The usability metric for user experience. Interacting with Computers 22, 5 (2010), 323–327.

Digital Library

[50]

Kathleen Kara Fitzpatrick, Alison Darcy, and Molly Vierhile. 2017. Delivering cognitive behavior therapy to young adults with symptoms of depression and anxiety using a fully automated conversational agent (Woebot): A randomized controlled trial. JMIR Mental Health 4, 2 (2017), e7785.

[51]

Flagstaff. 2019. The City of Flagstaff–Arizona. Retrieved 27 August, 2021 from http://www.flagstaff.az.gov/2/Community-Profile.

[52]

B. J. Fogg. 2003. Computers as persuasive social actors. In Persuasive Technology. B. J. Fogg (Ed.), Morgan Kaufmann, San Francisco, Chapter 5, 89–120.

[53]

Asbjørn Følstad and Petter Bae Brandtzæg. 2017. Chatbots and the new world of HCI. Interactions 24, 4 (2017), 38–42.

Digital Library

[54]

Asbjørn Følstad and Marita Skjuve. 2019. Chatbots for customer service: User experience and motivation. In Proceedings of the 1st International Conference on Conversational User Interfaces. ACM, New York, NY, 1–9.

Digital Library

[55]

Jodi Forlizzi, John Zimmerman, Vince Mancuso, and Sonya Kwak. 2007. How interface agents affect interaction between humans and computers. In Proceedings of the 2007 Conference on Designing Pleasurable Products and Interfaces. ACM, New York, NY, 209–221.

Digital Library

[56]

Jerome Friedman, Trevor Hastie, and Rob Tibshirani. 2010. Regularization paths for generalized linear models via coordinate descent. Journal of Statistical Software 33, 1 (2010), 1–22. DOI:https://doi.org/10.18637/jss.v033.i01

[57]

Boris Galitsky, Dmitry Ilvovsky, and Elizaveta Goncharova. 2019. On a chatbot providing virtual dialogues. In Proceedings of the International Conference on Recent Advances in Natural Language Processing. INCOMA Ltd., 382–387.

[58]

Damianos Gavalas and Michael Kenteris. 2011. A web-based pervasive recommendation system for mobile tourist guides. Personal and Ubiquitous Computing 15, 7 (2011), 759–770.

Digital Library

[59]

Ulrich Gnewuch, Stefan Morana, and Alexander Maedche. 2017. Towards designing cooperative and social conversational agents for customer service. In Proceedings of the International Conference on Information Systems 2017. Association for Information Systems, 13.

[60]

Eun Go and S. Shyam Sundar. 2019. Humanizing Chatbots: The effects of visual, identity and conversational cues on humanness perceptions. Computers in Human Behavior 97 (Aug. 2019), 304–316.

Digital Library

[61]

Google Developers. 2021. Conversation Design by Google. Retrieved March 3, 2021 from https://developers.google.com/assistant/conversation-design/what-is-conversation-design.

[62]

Grand View Research. 2017. Chatbot market size to reach $1.25 Billion by 2025. CAGR: 24.3%. Retrieved 27 August, 2021 from https://www.grandviewresearch.com/press-release/global-chatbot-market.

[63]

Jonathan Grudin and Richard Jacques. 2019. Chatbots, humbots, and the quest for artificial general intelligence. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. ACM, New York, NY, 1–11.

Digital Library

[64]

Xiaodong Gu, Kyunghyun Cho, Jung-Woo Ha, and Sunghun Kim. 2018. Dialogwae: Multimodal response generation with conditional wasserstein auto-encoder. In International Conference on Learning Representations. OpenReview. https://openreview.net/forum?id=BkgBvsC9FQ.

[65]

Michael A. K. Halliday and Ruqaiya Hasan. 1992. Cohesion in English. Penguin, London.

[66]

Yugo Hayashi. 2016. The effect of “mood”: Group-based collaborative problem solving by taking different perspectives. In Proceedings of the 38th Annual Conference of the Cognitive Science Society. The Cognitive Science Society, 818–823.

[67]

E. S. Heyselaar and T. Bosse. 2019. Using theory of mind to assess users’ sense of agency in social chatbots. In Proceedings of the Conversations 2019: 3rd International Workshop on Chatbot Research. Springer, Cham, 1–13.

[68]

Jennifer Hill, W. Randolph Ford, and Ingrid G. Farreras. 2015. Real conversations with artificial intelligence: A comparison between human–human online conversations and human–chatbot conversations. Computers in Human Behavior 49, C (2015), 245–250.

Digital Library

[69]

Rens Hoegen, Deepali Aneja, Daniel McDuff, and Mary Czerwinski. 2019. An end-to-end conversational style matching agent. In Proceedings of the 19th ACM International Conference on Intelligent Virtual Agents . ACM, New York, NY, 111–118. DOI:https://doi.org/10.1145/3308532.3329473

Digital Library

[70]

Torsten Hothorn, Peter Buehlmann, Sandrine Dudoit, Annette Molinaro, and Mark Van Der Laan. 2006. Survival ensembles. Biostatistics 7, 3 (2006), 355–373.

[71]

Susan Hunston. 2010. Corpus Approaches to Evaluation: Phraseology and Evaluative Language. Routledge, New York, NY.

[72]

Shinhee Hwang, Beomjun Kim, and Keeheon Lee. 2019. A data-driven design framework for customer service chatbot. In Proceedings of the International Conference on Human–Computer Interaction. Springer, Cham, 222–236.

[73]

Ken Hyland. 2012. Bundles in academic discourse. Annual Review of Applied Linguistics 32, 1 (2012), 150–169.

[74]

ISO 9241-11. 2018. Ergonomics of Human-System Interaction: Part 11: Usability: Definitions and Concepts. International Organization for Standardization, ISO.

[75]

S. Ivanov and C. Webster. 2017. Adoption of robots, artificial intelligence and service automation by travel, tourism and hospitality companies–a cost-benefit analysis. In Proceedings of the International Scientific Conference “Contemporary Tourism–Traditions and Innovations. Sofia University, 19–21.

[76]

Muayyad Jabri, Allyson D. Adrian, and David Boje. 2008. Reconsidering the role of conversations in change communication: A contribution based on Bakhtin. Journal of Organizational Change Management 21, 6 (2008), 667–685.

[77]

Mohit Jain, Pratyush Kumar, Ramachandra Kota, and Shwetak N. Patel. 2018. Evaluating and informing the design of chatbots. In Proceedings of the 2018 Designing Interactive Systems Conference. ACM, New York, NY, 895–906.

Digital Library

[78]

Ana Jakic, Maximilian Oskar Wagner, and Anton Meyer. 2017. The impact of language style accommodation during social media interactions on brand trust. Journal of Service Management 28, 3 (2017), 418–441.

[79]

Marie-Claire Jenkins, Richard Churchill, Stephen Cox, and Dan Smith. 2007. Analysis of user interaction with service oriented chatbot systems. In Human–Computer Interaction. HCI Intelligent Multimodal Interaction Environments. Julie A. Jacko (Ed.), Springer, Berlin, 76–83.

Digital Library

[80]

Ridong Jiang and Rafael E. Banchs. 2017. Towards improving the performance of chat oriented dialogue system. In Proceedings of the 2017 International Conference on Asian Language Processing. IEEE, New York, NY, 23–26.

[81]

Martin Joos. 1967. The Five Clocks. Vol. 58, Harcourt, Brace & World, New York, NY.

[82]

George Kamberelis. 1995. Genre as institutionally informed social practice. Journal of Contemporary Legal Issues 6 (1995), 115.

[83]

Merel Keijsers and Christoph Bartneck. 2018. Mindless Robots get bullied. In Proceedings of the 2018 ACM/IEEE International Conference on Human-Robot Interaction. ACM, New York, NY, 205–214.

Digital Library

[84]

Adam Kilgarriff. 2005. Language is never, ever, ever, random. Corpus Linguistics and Linguistic Theory 1, 2 (2005), 263–276.

[85]

Jurek Kirakowski, Anthony Yiu, and P. O’Donnell. 2009. Establishing the hallmarks of a convincing chatbot-human dialogue. In Human-Computer Interaction, Inaki Maurtua. InTech, London.

[86]

Julia Kiseleva, Kyle Williams, Ahmed Hassan Awadallah, Aidan C. Crook, Imed Zitouni, and Tasos Anastasakos. 2016. Predicting user satisfaction with intelligent assistants. In Proceedings of the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, New York, NY, 45–54.

Digital Library

[87]

Takanori Komatsu, Rie Kurosawa, and Seiji Yamada. 2012. How does the difference between users’ expectations and perceptions about a robotic agent affect their behavior? International Journal of Social Robotics 4, 2 (2012), 109–116.

[88]

Sherrie Y. X. Komiak and Izak Benbasat. 2006. The effects of personalization and familiarity on trust and adoption of recommendation agents. MIS Quarterly 30, 4 (2006), 941–960.

Digital Library

[89]

Robert M. Krauss and Chi-Yue Chiu. 1998. Language and social behavior. In The Handbook of Social Psychology. D. T. Gilbert, S. T. Fiske, and G. Lindzey (Eds.), McGraw-Hill, New York, NY, 41–88.

[90]

William Labov, Sharon Ash, and Charles Boberg. 2005. The Atlas of North American English: Phonetics, Phonology and Sound Change. Walter de Gruyter, Boston, MA.

[91]

Tania C. Lang. 2000. The effect of the Internet on travel consumer purchasing behaviour and implications for travel agencies. Journal of Vacation Marketing 6, 4 (2000), 368–385.

[92]

Mirosława Lasek and Szymon Jessa. 2013. Chatbots for customer service on Hotels’ websites. Information Systems in Management 2, 2 (2013), 146–158.

[93]

Minha Lee, Gale Lucas, Johnathan Mell, Emmanuel Johnson, and Jonathan Gratch. 2019. What’s on your virtual mind?: Mind perception in human-agent negotiations. In Proceedings of the 19th ACM International Conference on Intelligent Virtual Agents . ACM, New York, NY, 38–45. DOI:https://doi.org/10.1145/3308532.3329465

Digital Library

[94]

Min Kyung Lee, Sara Kiesler, and Jodi Forlizzi. 2010. Receptionist or information kiosk: How do people talk with a robot? In Proceedings of the 2010 ACM Conference on Computer Supported Cooperative Work. ACM, New York, NY, 31–40.

Digital Library

[95]

SeoYoung Lee and Junho Choi. 2017. Enhancing user experience with conversational agent for movie recommendation: Effects of self-disclosure and reciprocity. International Journal of Human-Computer Studies 103, C (2017), 95–105.

Digital Library

[96]

Geoffrey N. Leech and Mick Short. 2007. Style in Fiction: A Linguistic Introduction to English Fictional Prose. Pearson Education, London.

[97]

Yanran Li, Hui Su, Xiaoyu Shen, Wenjie Li, Ziqiang Cao, and Shuzi Niu. 2017. DailyDialog: A manually labelled multi-turn dialogue dataset. In Proceedings of the International Joint Conference on Natural Language Processing. Asian Federation of Natural Language Processing, 986–995.

[98]

Vera Q. Liao, Matthew Davis, Werner Geyer, Michael Muller, and N. Sadat Shami. 2016. What can you do?: Studying social-agent orientation and agent proactive interactions with an agent for employees. In Proceedings of the 2016 ACM Conference on Designing Interactive Systems . ACM, New York, NY, 264–275.

Digital Library

[99]

Grace I. Lin and Marilyn A. Walker. 2017. Stylistic variation in television dialogue for natural language generation. In Proceedings of the Workshop on Stylistic Variation. Association for Computational Linguistics, 85–93.

[100]

Greg Linden, Steve Hanks, and Neal Lesh. 1997. Interactive assessment of user preference models: The automated travel assistant. In User Modeling. A. Jameson, C. Paris, and C. Tasso (Eds.), Springer, Vienna, 67–78.

[101]

Michael H. Long and Steven Ross. 1993. Modifications That Preserve Language and Content.Technical Report. ERIC.

[102]

Jaclyn Loo. 2017. The future of travel: New consumer behavior and the technology giving it flight. Google/Phocuswright Travel Study 2017. Retrieved on 27 August 2021 from https://www.thinkwithgoogle.com/consumer-insights/consumer-trends/new-consumer-travel-assistance/?utm_source=HospitalityTrends.

[103]

Ewa Luger and Abigail Sellen. 2016. Like having a really bad PA: The gulf between user expectation and experience of conversational agents. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. ACM, New York, NY, 5286–5297.

Digital Library

[104]

Rhonda W. Mack, Julia E. Blose, and Bing Pan. 2008. Believe it or not: Credibility of blogs in tourism. Journal of Vacation Marketing 14, 2 (2008), 133–144.

[105]

François Mairesse and Marilyn A. Walker. 2009. Can Conversational Agents Express Big Five Personality Traits through Language?:Evaluating a Psychologically-Informed Language Generator. Cambridge & Sheffield, Cambridge University Engineering Department & Department of Computer Science, University of Sheffield.

[106]

Bronislaw Malinowski. 1923. The Problem of Meaning in Primitive Languages. Harcourt, Brace & World, Inc, New York, Chapter Supplement to C.K., 296–336.

[107]

Irina Maslowski, Delphine Lagarde, and Chloé Clavel. 2017. In-the-wild chatbot corpus: From opinion analysis to interaction problem detection. In Proceedings of the International Conference on Natural Language, Signal and Speech Processing. International Science and General Applications, 115–120.

[108]

Dominic W. Massaro, Michael M. Cohen, Sharon Daniel, and Ronald A. Cole. 1999. Developing and evaluating conversational agents. In Human Performance and Ergonomics (2nd ed.). P. A. Hancock (Ed.), Academic Press, Cambridge, Chapter 7, 173–194.

[109]

Niamh McNamara and Jurek Kirakowski. 2006. Functionality, usability, and user experience. Interactions 13, 6 (2006), 26–28.

Digital Library

[110]

Joao Luis Zeni Montenegro, Cristiano André da Costa, and Rodrigo da Rosa Righi. 2019. Survey of conversational agents in health. Expert Systems with Applications 129 (2019), 56–67.

Digital Library

[111]

Junko Mori. 1992. About What and What about: The Distinction between Topic and Issue in Conversations. Ph.D. Dissertation. University of Delaware.

[112]

Thomas William Morris. 2002. Conversational agents for game-like virtual environments. In Artificial Intelligence and Interactive Entertainment, Ken Forbus and Magy Seif El-Nasr. AAAI Press, Palo Alto, CA, 82–86.

[113]

Kellie Morrissey and Jurek Kirakowski. 2013. ‘Realness’ in Chatbots: Establishing quantifiable criteria. In Proceedings of the International Conference on Human-Computer Interaction. Springer, Berlin, Heidelberg, 87–96.

Digital Library

[114]

Clifford Nass, Jonathan Steuer, and Ellen R. Tauber. 1994. Computers are social actors. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, NY, 72–78.

Digital Library

[115]

Mario Neururer, Stephan Schlögl, Luisa Brinkschulte, and Aleksander Groth. 2018. Perceptions on authenticity in chat bots. Multimodal Technologies and Interaction 2, 3 (2018), 60.

[116]

Craig Nevill and Timothy Bell. 1992. Compression of parallel texts. Information Processing & Management 28, 6 (1992), 781–793.

Digital Library

[117]

Kate G. Niederhoffer and James W. Pennebaker. 2002. Linguistic style matching in social interaction. Journal of Language and Social Psychology 21, 4 (2002), 337–360.

[118]

Stefan Palan and Christian Schitter. 2018. Prolific. ac—A subject pool for online experiments. Journal of Behavioral and Experimental Finance 17, C (2018), 22–27.

[119]

Brian Paltridge. 1994. Genre analysis and the identification of textual boundaries. Applied Linguistics 15, 3 (1994), 288–299.

[120]

Aneta Pawłowska. 2016. Tourists and social media: Already inseparable marriage or still a long-distance relationship? Analysis of focus group study results conducted among tourists using social media. World Scientific News 57 (2016), 106–115.

[121]

Dijana Peras. 2018. Chatbot evaluation metrics: Review paper. In Proceedings of the 36th International Scientific Conference on Economic and Social Development. Varazdin Development and Entrepreneurship Agency, Varazdin, 89.

[122]

Claudio S. Pinhanez. 2020. HCI research challenges for the next generation of conversational systems. In Proceedings of the 2nd Conference on Conversational User Interfaces. ACM, New York, NY, 1–4.

Digital Library

[123]

Filip Radlinski and Nick Craswell. 2017. A theoretical framework for conversational search. In Proceedings of the 2017 Conference on Conference Human Information Interaction and Retrieval.

Digital Library

[124]

Nicole M. Radziwill and Morgan C. Benton. 2017. Evaluating quality of chatbots and intelligent conversational agents. Software Quality Professional 19, 3 (2017), 25–36.

[125]

Thomas B. W. Reid. 1956. Linguistics, structuralism and philology. Archivum Linguisticum 8, 1 (1956), 28–37.

[126]

Ehud Reiter and Robert Dale. 2000. Building Natural Language Generation Systems. Cambridge University Press, New York, NY.

[127]

Valeria Resendez. 2020. A Very Formal Agent: How Culture, Mode of Dressing and Linguistic Style Influence the Perceptions Toward an Embodied Conversational Agent?Master’s thesis. University of Twente.

[128]

Francesco Ricci, Lior Rokach, and Bracha Shapira. 2011. Introduction to recommender systems handbook. In Recommender Systems Handbook. F. Ricci, L. Rokach, B. Shapira, and P. Kantor (Eds.), Springer, Boston, MA, 1–35.

[129]

David A. Robb, José Lopes, Stefano Padilla, Atanas Laskov, Francisco J. Chiyah Garcia, Xingkun Liu, Jonatan Scharff Willners, Nicolas Valeyrie, Katrin Lohan, David Lane, Pedro Patron, Yvan Petillot, Mike J. Chantler, and Helen Hastie. 2019. Exploring interaction with remote autonomous systems using conversational agents. In Proceedings of the 2019 on Designing Interactive Systems Conference. ACM, New York, NY, 1543–1556.

Digital Library

[130]

E. Sato. 2007. A guide to linguistic modification: Strategies for increasing English language learner access to academic content. LEP Partnership. Retrieved on 27 August 2021 from https://ncela.ed.gov/files/uploads/11/abedi_sato.pdf.

[131]

Xiaoyu Shen, Hui Su, Shuzi Niu, and Vera Demberg. 2018. Improving variational encoder-decoders in dialogue generation. In 32nd AAAI Conference on Artificial Intelligence. AAAI Press, Palo Alto, California, 5456–5463.

Digital Library

[132]

Heung-yeung Shum, Xiao-dong He, and Di Li. 2018. From Eliza to XiaoIce: Challenges and opportunities with social chatbots. Frontiers of Information Technology & Electronic Engineering 19, 1 (2018), 10–26.

[133]

Svetlana Stoyanchev, Alex Liu, and Julia Hirschberg. 2014. Towards natural clarification questions in dialogue systems. In Proceedings of the AISB Symposium on Questions, Discourse and Dialogue. Vol. 20, Society for the Study of Artificial Intelligence and Simulation of Behaviour, Brighton, 8.

[134]

Carolin Strobl, Anne-Laure Boulesteix, Thomas Kneib, Thomas Augustin, and Achim Zeileis. 2008. Conditional variable importance for random forests. BMC Bioinformatics 9, 307 (2008), 11. Retrieved from http://www.biomedcentral.com/1471-2105/9/307.

[135]

Carolin Strobl, Anne-Laure Boulesteix, Achim Zeileis, and Torsten Hothorn. 2007. Bias in random forest variable importance measures: Illustrations, sources and a solution. BMC Bioinformatics 8, 25 (2007), 21. Retrieved from http://www.biomedcentral.com/1471-2105/8/25.

[136]

Ekaterina Svikhnushina, Alexandru Placinta, and Pearl Pu. 2021. User expectations of conversational chatbots based on online reviews. In Proceedings of the Designing Interactive Systems Conference 2021. ACM, New York, NY, 1481–1491.

Digital Library

[137]

Ekaterina Svikhnushina and Pearl Pu. 2021. Key qualities of conversational chatbots–the PEACE model. In Proceedings of the 26th International Conference on Intelligent User Interfaces. ACM, New York, NY, 520–530.

Digital Library

[138]

Jill Sweeney and Joffre Swait. 2008. The effects of brand credibility on customer loyalty. Journal of Retailing and Consumer Services 15, 3 (2008), 179–193.

[139]

Bakhtiyar Hussain Syed. 2020. Adapting Language Models for Style Transfer. Master’s thesis. International Institute of Information Technology Hyderabad.

[140]

Benedikt Szmrecsanyi. 2011. Corpus-based dialectometry: A methodological sketch. Corpora 6, 1 (2011), 45–76.

[141]

Ella Tallyn, Hector Fried, Rory Gianni, Amy Isard, and Chris Speed. 2018. The ethnobot: Gathering ethnographies in the age of IoT. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, New York, NY, 604.

Digital Library

[142]

Gunay Tariverdiyeva. 2019. Chatbots’ Perceived Usability in Information Retrieval Tasks: An Exploratory Analysis.Master’s thesis. University of Twente.

[143]

Stergios Tegos, Stavros Demetriadis, and Thrasyvoulos Tsiatsos. 2016. An Investigation of conversational agent interventions supporting historical reasoning in primary education. In Proceedings of the International Conference on Intelligent Tutoring Systems. Alessandro Micarelli, John Stamper, and Kitty Panourgia (Eds.), Springer International Publishing, Cham, 260–266.

Digital Library

[144]

Indrani Medhi Thies, Nandita Menon, Sneha Magapu, Manisha Subramony, and Jacki O’neill. 2017. How do you want your chatbot? An exploratory Wizard-of-Oz study with young, urban Indians. In Proceedings of the Human–Computer Interaction–INTERACT . Bernhaupt R., Dalvi G., Joshi A., K. Balkrishan D., O’Neill J., and Winckler M. (Eds.). Lecture Notes in Computer Science, Vol. 10513. Springer, Cham, 441–459.

[145]

Think with Google. 2016. How Micro-Moments are Reshaping the Travel Customer Journey. Retrieved 27 August, 2021 from https://www.thinkwithgoogle.com/marketing-resources/micro-moments/micro-moments-travel-customer-journey/.

[146]

Paul Thomas, Mary Czerwinski, Daniel McDuff, Nick Craswell, and Gloria Mark. 2018. Style and alignment in information-seeking conversation. In Proceedings of the 2018 Conference on Human Information Interaction&Retrieval. ACM, New York, NY, 42–51.

Digital Library

[147]

Rebecca Ruiz Thomas Combrink, Melinda Bradford. 2018. 2017-2018 Flagstaff Visitor Survey. Technical Report. Alliance Bank Eonomic Policy Institute, The W. A. Franke College of Business, Northern Arizona University. Prepared for the Flagstaff Convention and Visitors Bureau, Arizona Office of Tourism.

[148]

Alexey Tikhonov and Ivan P. Yamshchikov. 2018. What is wrong with style transfer for texts? CoRR

[149]

Peter Trudgill. 2000. Sociolinguistics: An Introduction to Language and Society. Penguin Books Limited, London.

[150]

Thomas Tullis and William Albert. 2013. Measuring the User Experience, Second Edition: Collecting, Analyzing, and Presenting Usability Metrics (2nd ed.). Morgan Kaufmann Publishers Inc., San Francisco, CA.

Digital Library

[151]

Iis Tussyadiah. 2020. A review of research into automation in tourism: Launching the Annals of Tourism Research Curated Collection on Artificial Intelligence and Robotics in Tourism. Annals of Tourism Research 81, C (2020), 102883.

[152]

UNWTO. 2017. UNWTO Tourism Highlights, 2017 Edition. World Tourism Organization, Madrid.

[153]

M. S. Walgama and B. Hettige. 2017. Chatbots: The next generation in computer interfacing–A review. In Proceedings of the KDU International Research Conference. General Sir John Kotelawala Defence University, 7.

[154]

Peter Wallis and Bruce Edmonds. 2019. How language works & what machines can do about it. In Proceedings of the 1st International Conference on Conversational User Interfaces. ACM, New York, NY, 1–3.

Digital Library

[155]

Dan Wang, Zheng Xiang, and Daniel R. Fesenmaier. 2016. Smartphone use in everyday life and travel. Journal of Travel Research 55, 1 (2016), 52–63.

[156]

Leo Wanner, Elisabeth André, Josep Blat, Stamatia Dasiopoulou, Mireia Farrús, Thiago Fraga, Eleni Kamateri, Florian Lingenfelser, Gerard Llorach, Oriol Martínez, and G. Meditskos. 2017. Design of a knowledge-based agent as a social companion. Procedia Computer Science 121, C (2017), 920–926.

Digital Library

[157]

Justin D. Weisz, Mohit Jain, Narendra Nath Joshi, James Johnson, and Ingrid Lange. 2019. BigBlueBot: Teaching strategies for successful human-agent interactions. In Proceedings of the 24th International Conference on Intelligent User Interfaces. ACM, New York, NY, 448–459.

Digital Library

[158]

Wikibooks. 2020. Algorithm Implementation/Strings/Levenshtein distance—Wikibooks, The Free Textbook Project. Retrieved May 8, 2020 from https://en.wikibooks.org/w/index.php?title=Algorithm_Implementation/Strings/Levenshtein_distance&oldid=3678144.

[159]

Katerina Zdravkova. 2000. Conceptual framework for an intelligent chatterbot. In Proceedings of the 22nd International Conference on Information Technology Interfaces. IEEE, New York, NY, 189–194.

[160]

Tiancheng Zhao, Kyusong Lee, and Maxine Eskenazi. 2018. Unsupervised discrete sentence representation learning for interpretable neural dialog generation. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, Vol. 1: Long Papers. Association for Computational Linguistics, 1098–1107. DOI:https://doi.org/10.18653/v1/P18-1101

[161]

Pengfei Zhu, Zhuosheng Zhang, Jiangtong Li, Yafang Huang, and Hai Zhao. 2018. Lingke: A fine-grained multi-turn chatbot for customer service. In Proceedings of the 27th International Conference on Computational Linguistics: System Demonstrations. Association for Computational Linguistics, 108–112.

[162]

Darius Zumstein and Sophie Hundertmark. 2017. Chatbots-An interactive technology for personalized communication, transactions and services. IADIS International Journal on WWW/Internet 15, 1 (2017), 96–109.

Cited By

Joshi ADabre RKanojia DLi ZZhan HHaffari GDippold D(2025)Natural Language Processing for Dialects of a Language: A SurveyACM Computing Surveys10.1145/371206057:6(1-37)Online publication date: 10-Feb-2025
https://dl.acm.org/doi/10.1145/3712060
Li ZWu CLi JYuan Q(2025)Chatbot research in the fields of business and information systems: a systematic review and bibliometric analysisAslib Journal of Information Management10.1108/AJIM-05-2024-0427Online publication date: 17-Feb-2025
https://doi.org/10.1108/AJIM-05-2024-0427
Nayak ANair A(2025)Language translation effects in Chatbots: Evidence from a randomized field experiment on a mobile commerce platformJournal of Business Research10.1016/j.jbusres.2024.115158190(115158)Online publication date: Mar-2025
https://doi.org/10.1016/j.jbusres.2024.115158
Show More Cited By

Index Terms

Chatbots Language Design: The Influence of Language Variation on User Experience with Tourist Assistant Chatbots
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Empirical studies in HCI
    2. Interaction paradigms
      1. Natural language interfaces

Recommendations

User Engagement with Chatbots: A Discursive Psychology Approach
CUI '20: Proceedings of the 2nd Conference on Conversational User Interfaces

Conversational agents have transcended into multiple industries with increased ability for user engagement in intelligent conversation. Conversations with chatbots are different from interpersonal communication in terms of turn-taking, intentions, and ...
Leveraging Large Language Models to Power Chatbots for Collecting User Self-Reported Data
CSCW

Large language models (LLMs) provide a new way to build chatbots by accepting natural language prompts. Yet, it is unclear how to design prompts to power chatbots to carry on naturalistic conversations while pursuing a given goal such as collecting self-...
Exploring how politeness impacts the user experience of chatbots for mental health support
Abstract
Politeness is important in human–human interaction when asking people to engage in sensitive conversations. If politeness manifests similarly in human–chatbot interaction, it may play an important role in the design of sensitive chatbot ...
Highlights
- Politeness can both positively and negatively impact the chatbot user experience.
- The Personal politeness chatbot was experienced as caring and encouraging.
- The Passive politeness chatbot was experienced as too apologetic and ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Transactions on Computer-Human Interaction

ACM Transactions on Computer-Human Interaction Volume 29, Issue 2

April 2022

347 pages

ISSN:1073-0516

EISSN:1557-7325

DOI:10.1145/3505202

Editor:
Kristina Höök
KTH Royal Institute of Technology, Sweden

Issue’s Table of Contents

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 16 January 2022

Accepted: 01 September 2021

Revised: 01 July 2021

Received: 01 April 2021

Published in TOCHI Volume 29, Issue 2

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Refereed

Funding Sources

National Science Foundation

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

31
Total Citations
View Citations
3,134
Total Downloads

Downloads (Last 12 months)1,009
Downloads (Last 6 weeks)129

Reflects downloads up to 13 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Joshi ADabre RKanojia DLi ZZhan HHaffari GDippold D(2025)Natural Language Processing for Dialects of a Language: A SurveyACM Computing Surveys10.1145/371206057:6(1-37)Online publication date: 10-Feb-2025
https://dl.acm.org/doi/10.1145/3712060
Li ZWu CLi JYuan Q(2025)Chatbot research in the fields of business and information systems: a systematic review and bibliometric analysisAslib Journal of Information Management10.1108/AJIM-05-2024-0427Online publication date: 17-Feb-2025
https://doi.org/10.1108/AJIM-05-2024-0427
Nayak ANair A(2025)Language translation effects in Chatbots: Evidence from a randomized field experiment on a mobile commerce platformJournal of Business Research10.1016/j.jbusres.2024.115158190(115158)Online publication date: Mar-2025
https://doi.org/10.1016/j.jbusres.2024.115158
Fietta VRizzi SDe Luca CGios LPavesi MGabrielli SMonaro MForti S(2024)A Chatbot-Based Version of the World Health Organization–Validated Self-Help Plus Intervention for Stress Management: Co-Design and Usability TestingJMIR Human Factors10.2196/6461411(e64614)Online publication date: 18-Oct-2024
https://doi.org/10.2196/64614
Huang SLin YHe ZHuang CHuang T(2024)How Does Conversation Length Impact User’s Satisfaction? A Case Study of Length-Controlled Conversations with LLM-Powered ChatbotsExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3650823(1-13)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3650823
Calambur VJun DSchiaffino MZhang ZHuh-Yoo J(2024)A case for "little English" in Nurse Notes from the Telehealth Intervention Program for Seniors: Implications for Future Design and ResearchProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3641961(1-16)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3641961
Hankel SLiebrecht CKamoen N(2024)‘Hi Chatbot, let’s Talk about Politics!’ Examining the Impact of Verbal Anthropomorphism in Conversational Agent Voting Advice Applications (CAVAAs) on Higher and Lower Politically Sophisticated UsersInteracting with Computers10.1093/iwc/iwae031Online publication date: 24-Jul-2024
https://doi.org/10.1093/iwc/iwae031
Dippold D(2024)Making the Case for Audience Design in Conversational AI: Users’ Pragmatic Strategies and Rapport Expectations in Interaction with a Task-Oriented ChatbotApplied Linguistics10.1093/applin/amae033Online publication date: 8-May-2024
https://doi.org/10.1093/applin/amae033
Cingillioglu I(2024)What impacts matriculation decisions? Identifying students’ university choice factors on a global scale with Artificial IntelligenceStudies in Higher Education10.1080/03075079.2024.231987049:12(2637-2655)Online publication date: 16-Feb-2024
https://doi.org/10.1080/03075079.2024.2319870
Wang LXiao JLuo ZGuo YXu X(2024)The Impact of Default Options on Tourist Intention Post Tourism Chatbot Failure: The Role of Service Recovery and EmoticonTourism Management Perspectives10.1016/j.tmp.2024.10129953(101299)Online publication date: Sep-2024
https://doi.org/10.1016/j.tmp.2024.101299
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Full Text

View this article in Full Text.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View full text|Download PDF

View Issue’s Table of Contents