research-article

Comparing users' perception of different chatbot interaction paradigms: a case study

Authors:

Francisco A. M. Valério,

Tatiane G. Guimarães,

Raquel O. Prates,

Heloisa CandelloAuthors Info & Claims

IHC '20: Proceedings of the 19th Brazilian Symposium on Human Factors in Computing Systems

Article No.: 11, Pages 1 - 10

https://doi.org/10.1145/3424953.3426501

Published: 23 December 2020 Publication History

Get Access

Abstract

In this work, we used the results of applying the Semiotic Inspection Method (SIM) to popular chatbots[12, 13], which yielded eleven strategies for conveying features to users, as well as six sign classes used for designing their interaction. We conducted user studies comparing two prototype chatbots with same features and different approaches to interaction: one using Natural Language Processing, and other using the sign classes and strategies to guide the user interaction. After that, we interviewed the participants, asking about their preferred aspects of each chatbot and their opinions regarding some of these aspects, and, later, analyzed the results. These point to the effectiveness of the strategies and sign classes. Then, we discuss users' perceptions of different ways of interacting with chatbots and their communicative strategies.

References

[1]

Daniel G. Bobrow. 1964. Natural Language Input for a Computer Problem Solving System. Technical Report. Massachusetts Institute of Technology, Cambridge, MA, USA.

Crossref

Google Scholar

[2]

Heloisa Candello and Claudio Pinhanez. 2016. Designing Conversational Interfaces. In Simpósio Brasileiro sobre Fatores Humanos em Sistemas Computacionais - IHC 2016 (15th ed.). Vol. C - Livro dos Tutoriais. Sociedade Brasileira de Computação - SBC, Porto Alegre, RS, Brazil.

Google Scholar

[3]

Heloisa Candello, Claudio Pinhanez, and Flavio Figueiredo. 2017. Typefaces and the Perception of Humanness in Natural Language Chatbots. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). ACM, New York, NY, USA, 3476--3487.

Digital Library

Google Scholar

[4]

Clarisse S. de Souza. 2005. The Semiotic Engineering of Human-computer Interaction. MIT Press.

Google Scholar

[5]

Thiago Carvalho D'Ávila. 2018. Kino: an approach for rule-based chatbot development, monitoring and evaluation. Master's thesis. Universidade Federal de Minas Gerais, Belo Horizonte, Brazil.

Google Scholar

[6]

Mohit Jain, Pratyush Kumar, Ramachandra Kota, and Shwetak N. Patel. 2018. Evaluating and Informing the Design of Chatbots. In Proceedings of the 2018 Designing Interactive Systems Conference (DIS '18). ACM, New York, NY, USA.

Digital Library

Google Scholar

[7]

Jiepu Jiang, Ahmed H. Awadallah, Rosie Jones, Umut Ozertem, Imed Zitouni, Ranjitha G. Kulkarni, and Omar Z. Khan. 2015. Automatic Online Evaluation of Intelligent Assistants. In Proceedings of the 24th International Conference on World Wide Web (WWW '15). ACM, New York, NY, USA, 506--516.

Digital Library

Google Scholar

[8]

Michael McTear, Zoraida Callejas, and David Griol. 2016. Evaluating the Conversational Interface. In The Conversational Interface. Springer International Publishing, 379--402.

Digital Library

Google Scholar

[9]

Michael F. McTear. 2017. The Rise of the Conversational Interface: A New Kid on the Block?. In Future and Emerging Trends in Language Technology. Machine Learning and Big Data (Lecture Notes in Computer Science), José F Quesada, Francisco-Jesús Martín Mateos, and Teresa López Soto (Eds.). Springer International Publishing, 38--49.

Google Scholar

[10]

Raquel Prates, Clarisse de Souza, and Simone Barbosa. 2000. A method for evaluating the communicability of user interfaces. Interactions 7 (01 2000), 31--38.

Digital Library

Google Scholar

[11]

Francisco A. M. Valério. 2020. Chatbots and Communication: A Qualitative Analysis of Communicative Aspects of Conversational Interfaces. Master's thesis. Universidade Federal de Minas Gerais, Belo Horizonte, Brazil.

Google Scholar

[12]

Francisco A. M. Valério, Tatiane G. Guimarães, Raquel O. Prates, and Heloisa Candello. 2017. Here's What I Can Do: Chatbots' Strategies to Convey Their Features to Users. In Proceedings of the 16th Brazilian Symposium on Human Factors in Computer Systems (IHC '17). ACM, New York, NY, USA.

Digital Library

Google Scholar

[13]

Francisco A. M. Valério, Tatiane G. Guimarães, Raquel O. Prates, and Heloisa Candello. 2018. Chatbots Explain Themselves: Designers' Strategies for Conveying Chatbot Features to Users. SBC Journal on Interactive Systems 9, 3 (Dec. 2018), 61--79. https://seer.ufrgs.br/jis/article/view/80272

Google Scholar

[14]

Joseph Weizenbaum. 1966. ELIZA --- A Computer Program for the Study of Natural Language Communication Between Man and Machine. Commun. ACM 9, 1 (Jan. 1966), 36--45.

Digital Library

Google Scholar

Cited By

View all

Klein SPapies DUtz S(2024)How Interaction Mechanism and Error Responses Influence Users’ Responses to Customer Service ChatbotsInternational Journal of Human–Computer Interaction10.1080/10447318.2024.2351707(1-19)Online publication date: 20-May-2024
https://doi.org/10.1080/10447318.2024.2351707
Yang MGuo Y(2024)Research on User Experience Design of Artificial Intelligence (AI) Medical Consultation SystemHuman-Computer Interaction10.1007/978-3-031-60428-7_17(236-252)Online publication date: 1-Jun-2024
https://doi.org/10.1007/978-3-031-60428-7_17
Baskara F(2023)Chatbots and Flipped Learning: Enhancing Student Engagement and Learning Outcomes through Personalised Support and CollaborationIJORER : International Journal of Recent Educational Research10.46245/ijorer.v4i2.3314:2(223-238)Online publication date: 31-Mar-2023
https://doi.org/10.46245/ijorer.v4i2.331
Show More Cited By

Index Terms

Comparing users' perception of different chatbot interaction paradigms: a case study
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. HCI design and evaluation methods
      1. User studies
    2. Interaction paradigms
      1. Natural language interfaces

Recommendations

Here's What I Can Do: Chatbots' Strategies to Convey Their Features to Users
IHC '17: Proceedings of the XVI Brazilian Symposium on Human Factors in Computing Systems

Chatbots have been around since the 1960's, but recently they have risen in popularity especially due to new compatibility with social networks and messenger applications. Chatbots are different from traditional user interfaces, for they unveil ...
Comparing How a Chatbot References User Utterances from Previous Chatting Sessions: An Investigation of Users' Privacy Concerns and Perceptions
HAI '23: Proceedings of the 11th International Conference on Human-Agent Interaction

Chatbots are capable of remembering and referencing previous conversations, but does this enhance user engagement or infringe on privacy? To explore this trade-off, we investigated the format of how a chatbot references previous conversations with a ...
Is chatting with a sophisticated chatbot as good as chatting online or FTF with a stranger?
Abstract
Emotionally-responsive chatbots are marketed as agents with which one can form emotional connections. They can also become weak ties in the outer layers of one's acquaintance network and available for social support. In this experiment,...
Highlights
- Adults had positive reactions after talking to the emotionally responsive Replika chatbot.

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

IHC '20: Proceedings of the 19th Brazilian Symposium on Human Factors in Computing Systems

October 2020

519 pages

ISBN:9781450381727

DOI:10.1145/3424953

General Chairs:
Caroline Queiroz Santos
UFVJM
,
Maria Lúcia Bento Villela
UFVJM
,
Program Chairs:
Isabela Gasparini
Santa Catarina State University & Federal University of Rio Grande do Sul, Brazil & Telecom & Management SudParis, France
,
Tayana Uchôa Conte
Federal University of Amazonas & Federal University of Rio de Janeiro, Brazil

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

In-Cooperation

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 December 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

IHC '20

IHC '20: XIX Brazilian Symposium on Human Factors in Computing Systems

October 26 - 30, 2020

Diamantina, Brazil

Acceptance Rates

IHC '20 Paper Acceptance Rate 60 of 155 submissions, 39%;

Overall Acceptance Rate 331 of 973 submissions, 34%

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
365
Total Downloads

Downloads (Last 12 months)119
Downloads (Last 6 weeks)13

Reflects downloads up to 18 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Klein SPapies DUtz S(2024)How Interaction Mechanism and Error Responses Influence Users’ Responses to Customer Service ChatbotsInternational Journal of Human–Computer Interaction10.1080/10447318.2024.2351707(1-19)Online publication date: 20-May-2024
https://doi.org/10.1080/10447318.2024.2351707
Yang MGuo Y(2024)Research on User Experience Design of Artificial Intelligence (AI) Medical Consultation SystemHuman-Computer Interaction10.1007/978-3-031-60428-7_17(236-252)Online publication date: 1-Jun-2024
https://doi.org/10.1007/978-3-031-60428-7_17
Baskara F(2023)Chatbots and Flipped Learning: Enhancing Student Engagement and Learning Outcomes through Personalised Support and CollaborationIJORER : International Journal of Recent Educational Research10.46245/ijorer.v4i2.3314:2(223-238)Online publication date: 31-Mar-2023
https://doi.org/10.46245/ijorer.v4i2.331
Musi EO’Halloran KCarmi EYates S(2023)Developing Misinformation Immunity in a Post-Truth World: Human Computer Interaction for Data LiteracyTruth Claims Across Media10.1007/978-3-031-42064-1_11(245-271)Online publication date: 21-Dec-2023
https://doi.org/10.1007/978-3-031-42064-1_11
Packowski SAlfonso CSuthar PKarakowsky NOnuţ IZulkernine F(2021)Building game-playing chatbots using IBM watson assistantProceedings of the 31st Annual International Conference on Computer Science and Software Engineering10.5555/3507788.3507840(282-283)Online publication date: 22-Nov-2021
https://dl.acm.org/doi/10.5555/3507788.3507840

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Here's What I Can Do: Chatbots' Strategies to Convey Their Features to Users

Comparing How a Chatbot References User Utterances from Previous Chatting Sessions: An Investigation of Users' Privacy Concerns and Perceptions

Is chatting with a sophisticated chatbot as good as chatting online or FTF with a stranger?