Let’s Chat Internal: User Acceptance of an In-Company Service Desk Chatbot
Pages 557 - 562
Abstract
Due to remote working models, which were strengthened by the COVID-19 pandemic, it became highly relevant to digitalize on-site service desk consultations. Virtual enterprise assistants which simplify the interaction with in-company services provide similar advantages to their popular peers in customer service: high availability and customer satisfaction, and low response time and costs. To find out how their usage affects user acceptance we developed an early high-fidelity prototype for an in-company UX service desk chatbot and evaluated it in an online survey (N=53) and user study (N=14) with employees from the industry using the Technology Acceptance Model. The results show that the prototype’s acceptance does not differ from the classical service and prior experiences with textual chatbots in a private usage context but was rated higher than a private sample bot.
References
[1]
BMW Group and MessengerPeople. 2021. FollowNow. https://follownow.bmwgroup.com/whatsapp/. Accessed: 08.06.2021.
[2]
Petter Bae Brandtzaeg and Asbjørn Følstad. 2018. Chatbots: Changing User Needs and Motivations. Interactions 25, 5 (Aug. 2018), 38–43. https://doi.org/10.1145/3236669
[3]
Leon Ciechanowski, Aleksandra Przegalinska, Mikołaj Magnuski, and Peter Gloor. 2018. In the Shades of the Uncanny Valley: An Experimental Study of Human-Chatbot Interaction. Future Generation Computer Systems 92 (02 2018), 539–548. https://doi.org/10.1016/j.future.2018.01.055
[4]
Jacob Cohen. 1998. Statistical Power Analysis for the Behavioral Sciences. Lawrence Erlbaum Associates, Mahwah, New Jersey, USA.
[5]
Lee J. Cronbach. 1951. Coefficient alpha and the internal structure of tests. Psychometrika 16, 3 (1951), 297–334. https://doi.org/10.1007/BF02310555
[6]
Robert Dale. 2016. The return of the chatbots. Natural Language Engineering 22, 5 (2016), 811–817. https://doi.org/10.1017/S1351324916000243
[7]
Fred D. Davis. 1985. A Technology Acceptance Model for Empirically Testing New End-user Information Systems: Theory and Results. Massachusetts Institute of Technology, Sloan School of Management, Cambridge, Massachusetts, United States.
[8]
Fred D. Davis. 1989. Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology. MIS Q. 13(1989), 319–340.
[9]
Dario Fiore, Matthias Baldauf, and Christian Thiel. 2019. ”Forgot Your Password Again?”: Acceptance and User Experience of a Chatbot for in-Company IT Support. In Proceedings of the 18th International Conference on Mobile and Ubiquitous Multimedia (Pisa, Italy) (MUM ’19). Association for Computing Machinery, New York, NY, USA, Article 37, 11 pages. https://doi.org/10.1145/3365610.3365617
[10]
Asbjørn Følstad and Petter Bae Brandtzaeg. 2020. Users’ experiences with chatbots: findings from a questionnaire study. Quality and User Experience 5, 1 (2020), 3. https://doi.org/10.1007/s41233-020-00033-2
[11]
Duen-Huang Huang and Hao-En Chueh. 2020. An Analysis of Use Intention of Pet Disease Consultation Chatbot. Association for Computing Machinery, New York, NY, USA, 1–5. https://doi-org.thi.idm.oclc.org/10.1145/3421682.3421693
[12]
Kamran Iqbal, Jonathan Fingold, WashingtonKayaker, zxyanliu, and Emily Olshefski. 2020. Introduction to Bot Framework Composer. https://docs.microsoft.com/de-de/composer/introduction. Accessed: 10.01.2021.
[13]
Mohit Jain, Pratyush Kumar, Ramachandra Kota, and Shwetak N. Patel. 2018. Evaluating and Informing the Design of Chatbots. In Proceedings of the 2018 Designing Interactive Systems Conference (Hong Kong, China) (DIS ’18). Association for Computing Machinery, New York, NY, USA, 895–906. https://doi.org/10.1145/3196709.3196735
[14]
Lorenz Cuno Klopfenstein, Saverio Delpriori, Silvia Malatini, and Alessandro Bogliolo. 2017. The Rise of Bots: A Survey of Conversational Interfaces, Patterns, and Paradigms. In Proceedings of the 2017 Conference on Designing Interactive Systems (Edinburgh, United Kingdom) (DIS ’17). Association for Computing Machinery, New York, NY, USA, 555–565. https://doi.org/10.1145/3064663.3064672
[15]
Jeya A. Kumar and Paula A. Silva. 2020. Work-in-Progress: A Preliminary Study on Students’ Acceptance of Chatbots for Studio-Based Learning. In 2020 IEEE Global Engineering Education Conference (EDUCON). IEEE, New York City, New York, USA, 1627–1631. https://doi.org/10.1109/EDUCON45650.2020.9125183
[16]
Robert Kusber. 2017. Chatbots – Conversational UX Platforms. Springer Fachmedien Wiesbaden, Wiesbaden, 231–244. https://doi.org/10.1007/978-3-658-15648-0_11
[17]
Q. Vera Liao, Muhammed Mas-ud Hussain, Praveen Chandar, Matthew Davis, Yasaman Khazaeni, Marco Patricio Crasso, Dakuo Wang, Michael Muller, N. Sadat Shami, and Werner Geyer. 2018. All Work and No Play?Association for Computing Machinery, New York, NY, USA, 1–13. https://doi-org.thi.idm.oclc.org/10.1145/3173574.3173577
[18]
Rensis Likert. 1932. A Technique for the Measurement of Attitudes. Number Nr. 136-165 in A Technique for the Measurement of Attitudes. Archives of Psychology, New York, USA. https://books.google.de/books?id=9rotAAAAYAAJ
[19]
Ewa Luger and Abigail Sellen. 2016. ”Like Having a Really Bad PA”: The Gulf between User Expectation and Experience of Conversational Agents. Association for Computing Machinery, New York, NY, USA, 5286–5297. https://doi.org/10.1145/2858036.2858288
[20]
Henry B. Mann and Donald R. Whitney. 1947. On a Test of Whether one of Two Random Variables is Stochastically Larger than the Other. Ann. Math. Statist. 18, 1 (03 1947), 50–60. https://doi.org/10.1214/aoms/1177730491
[21]
Raphael Meyer von Wolff, Sebastian Hobert, and Matthias Schumann. 2019. How May I Help You? - State of the Art and Open Research Questions for Chatbots at the Digital Workplace. In Proceedings of the 52nd Hawaii International Conference on System Sciences. Hawaii International Conference on System Sciences, Hawaii.
[22]
Microsoft Corporation. 2021. Azure Bot Service. https://azure.microsoft.com/de-de/services/bot-services/. Accessed: 10.01.2021.
[23]
Jakob Nielsen. 2020. 10 Usability Heuristics for User Interface Design. https://www.nngroup.com/articles/ten-usability-heuristics/ Accessed: 05-02-2021.
[24]
Donald A. Norman and Stephen W. Draper. 1986. User Centered System Design; New Perspectives on Human-Computer Interaction. L. Erlbaum Associates Inc., USA.
[25]
Tim Rietz, Ivo Benke, and Alexander Maedche. 2019. The Impact of Anthropomorphic and Functional Chatbot Design Features in Enterprise Collaboration Systems on User Acceptance. In 14th International Conference on Wirtschaftsinformatik (WI2019). Universität Siegen, Siegen, Germany, 1642–1656.
[26]
Neil J. Salkind, Kristin Rasmussen, Gale (Firm), and Sage Publications. 2007. Encyclopedia of Measurement and Statistics. Number Bd. 1 in A Sage reference publication. SAGE Publications, Thousand Oaks, California, USA. https://books.google.de/books?id=HQMPAQAAMAAJ
[27]
David Schatsky and Peter Gratzke. 2016. The conversational enterprise - how chatbots can answer questions for both customers and employees.https://www2.deloitte.com/content/dam/insights/us/articles/3453_Signals_chatbots/DUP_signals-for-strategists_chatbots.pdf.
[28]
Martin Schrepp. 2018. Data Analysis Tool (UEQ-S). https://www.ueq-online.org/Material/Short_UEQ_Data_Analysis_Tool.xlsx. Accessed: 24.01.2021.
[29]
Martin Schrepp, Andreas Hinderks, and Jörg Thomaschewski. 2017. Design and Evaluation of a Short Version of the User Experience Questionnaire (UEQ-S). International Journal of Interactive Multimedia and Artificial Intelligence 4 (01 2017), 103. https://doi.org/10.9781/ijimai.2017.09.001
[30]
Abhishek Singh, Karthik Ramasubramanian, and Shrey Shivam. 2019. Building an Enterprise Chatbot: Work with Protected Enterprise Data Using Open Source Frameworks. Apress, New York City, New York, USA. https://doi.org/10.1007/978-1-4842-5034-1
[31]
Tuva Lunde Smestad and Frode Volden. 2019. Chatbot Personalities Matters. In Internet Science, Svetlana S Bodrunova, Olessia Koltsova, Asbjørn Følstad, Harry Halpin, Polina Kolozaridi, Leonid Yuldashev, Anna Smoliarova, and Heiko Niedermayer (Eds.). Springer International Publishing, Cham, 170–181.
[32]
Emanuel Stoeckli, Falk Uebernickel, and Walter Brenner. 2018. Exploring Affordances of Slack Integrations and Their Actualization Within Enterprises - Towards an Understanding of How Chatbots Create Value. In Proceedings of the 51st Hawaii International Conference on System Sciences. Hawaii International Conference on System Sciences, Hawaii. https://doi.org/10.24251/HICSS.2018.255
[33]
Telekom Deutschland GmbH. 2020. FRAG MAGENTA – Service rund um die Uhr. https://www.telekom.de/hilfe/hilfe-bei-stoerungen/frag-magenta. Accessed: 05.01.2021.
[34]
Margot J. van der Goot and Tyler Pilgrim. 2020. Exploring Age Differences in Motivations for and Acceptance of Chatbot Communication in a Customer Service Context. In Chatbot Research and Design, Asbjørn Følstad, Theo Araujo, Symeon Papadopoulos, Effie Lai-Chong Law, Ole-Christoffer Granmo, Ewa Luger, and Petter Bae Brandtzaeg (Eds.). Springer International Publishing, Cham, 173–186.
[35]
Viswanath Venkatesh and Hillol Bala. 2008. Technology Acceptance Model 3 and a Research Agenda on Interventions. https://doi.org/10.1111/j.1540-5915.2008.00192.x
[36]
Viswanath Venkatesh and Fred Davis. 2000. A Theoretical Extension of the Technology Acceptance Model: Four Longitudinal Field Studies. Management Science 46 (02 2000), 186–204. https://doi.org/10.1287/mnsc.46.2.186.11926
[37]
Hugh Watson. 2017. Preparing for the cognitive generation of decision support. MIS Quarterly Executive 16 (01 2017), 153–169.
[38]
Philipp Wintersberger, Anna-Katharina Frison, Andreas Riener, and Tamara von Sawitzky. 2019. Fostering User Acceptance and Trust in Fully Automated Vehicles: Evaluating the Potential of Augmented Reality. Presence: Teleoper. Virtual Environ. 27, 1 (March 2019), 46–62. https://doi.org/10.1162/pres_a_00320
[39]
Woebot Health. 2021. Woebot Health. https://woebothealth.com. Accessed: 08.06.2021.
Recommendations
Chatbot Acceptance: A Latent Profile Analysis on Individuals' Trust in Conversational Agents
SIGMIS-CPR '19: Proceedings of the 2019 on Computers and People Research ConferenceAccording to industry reports, the lack of trust in non-human interaction prevents widespread Chatbot acceptance. Since the willingness and the ability to trust varies between individuals, this study examines to what extent the trust in Chatbots varies ...
User Acceptance of Computer Technology: A Comparison of Two Theoretical Models
Computer systems cannot improve organizational performance if they aren't used. Unfortunately, resistance to end-user systems by managers and professionals is a widespread problem. To better predict, explain, and increase user acceptance, we need to ...
Understanding the User Experience of Customer Service Chatbots: What Can We Learn from Customer Satisfaction Surveys?
Chatbot Research and DesignAbstractUnderstanding and improving user experience is key to strengthening uptake and realizing the potential of chatbots for customer service. In this paper, we investigate customer satisfaction surveys as a source of insight into such user experience. ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
September 2021
613 pages
ISBN:9781450386456
DOI:10.1145/3473856
Copyright © 2021 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 13 September 2021
Check for updates
Author Tags
Qualifiers
- Short-paper
- Research
- Refereed limited
Conference
MuC '21
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 387Total Downloads
- Downloads (Last 12 months)65
- Downloads (Last 6 weeks)10
Reflects downloads up to 10 Nov 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format