research-article

It Takes a Village: Integrating an Adaptive Chatbot into an Online Gaming Community

Authors:

Joseph Seering,

Jessica HammerAuthors Info & Claims

CHI '20: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems

Pages 1 - 13

https://doi.org/10.1145/3313831.3376708

Published: 23 April 2020 Publication History

Abstract

While the majority of research in chatbot design has focused on creating chatbots that engage with users one-on-one, less work has focused on the design of conversational agents for online communities. In this paper we present results from a three week test of a social chatbot in an established online community. During this study, the chatbot "grew up" from "birth" through its teenage years, engaging with community members and "learning" vocabulary from their conversations. We discuss the design of this chatbot, how users' interactions with it evolved over the course of the study, and how it impacted the community as a whole. We discuss how we addressed challenges in developing a chatbot whose vocabulary could be shaped by users, and conclude with implications for the role of machine learning in social interactions in online communities and potential future directions for design of community-based chatbots.

References

[1]

Natã M Barbosa and Monchu Chen. 2019. Rehumanized Crowdsourcing: A Labeling Framework Addressing Bias and Ethics in Machine Learning. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. ACM, NY, NY, USA, 543.

Digital Library

[2]

Heloisa Candello, Claudio Pinhanez, Mauro Carlos Pichiliani, Melina Alberio Guerra, and Maira Gatti de Bayser. 2018. Having an Animated Coffee with a Group of Chatbots from the 19th Century. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems (CHI EA '18). ACM, NY, NY, USA, Article D206, 4 pages.

Digital Library

[3]

Hyojin Chin and Mun Yong Yi. 2019. Should an Agent Be Ignoring It?: A Study of Verbal Abuse Types and Conversational Agents' Response Styles. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems. ACM, NY, NY, USA, LBW2422.

Digital Library

[4]

Leigh Clark, Nadia Pantidi, Orla Cooney, Philip Doyle, Diego Garaialde, Justin Edwards, Brendan Spillane, Emer Gilmartin, Christine Murad, Cosmin Munteanu, Vincent Wade, and Benjamin R. Cowan. 2019. What Makes a Good Conversation?: Challenges in Designing Truly Conversational Agents. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19). ACM, NY, NY, USA, Article 475, 12 pages.

Digital Library

[5]

Kerstin Dautenhahn. 2007. Socially intelligent robots: Dimensions of human--robot interaction. Philosophical Transactions of the Royal Society of London B: Biological Sciences 362 (2007), 679--704.

[6]

Antonella De Angeli and Sheryl Brahnam. 2008. I hate you! Disinhibition with virtual partners. Interacting with computers 20, 3 (2008), 302--310.

[7]

Stefania Druga, Randi Williams, Cynthia Breazeal, and Mitchel Resnick. 2017. "Hey Google is It OK if I Eat You?": Initial Explorations in Child-Agent Interaction. In Proceedings of the 2017 Conference on Interaction Design and Children (IDC '17). ACM, NY, NY, USA, 595--600.

Digital Library

[8]

Samantha Finkelstein, Evelyn Yarzebinski, Callie Vaughn, Amy Ogan, and Justine Cassell. 2013. The effects of culturally congruent educational technologies on student achievement. In International Conference on Artificial Intelligence in Education. Springer, New York, NY, USA, 493--502.

[9]

Seth Glickman, Nathan McKenzie, Joseph Seering, Rachel Moeller, and Jessica Hammer. 2018. Design Challenges for Livestreamed Audience Participation Games. In Proceedings of the 2018 Annual Symposium on Computer-Human Interaction in Play (CHI PLAY '18). ACM, NY, NY, USA, 187--199.

Digital Library

[10]

Bettina Graf, Maike Krüger, Felix Müller, Alexander Ruhland, and Andrea Zech. 2015. Nombot: Simplify Food Tracking. In Proceedings of the 14th International Conference on Mobile and Ubiquitous Multimedia (MUM '15). ACM, NY, NY, USA, 360--363.

Digital Library

[11]

J Guo, S Lu, H Cai, W Zhang, Y Yu, and J Wang. 2018. Long text generation via adversarial training with leaked information. In Proceedings of the 32nd AAAI Conference on Artificial Intelligence. AAAI, Palo Alto, CA, USA, 5141--5148.

[12]

William A. Hamilton, Oliver Garretson, and Andruid Kerne. 2014. Streaming on Twitch: Fostering Participatory Communities of Play Within Live Mixed Media. In Proceedings of the 32Nd Annual ACM Conference on Human Factors in Computing Systems (CHI '14). ACM, NY, NY, USA, 1315--1324.

Digital Library

[13]

Daniel C. Howe. 2009. RiTa: Creativity Support for Computational Literature. In Proceedings of the Seventh ACM Conference on Creativity and Cognition (C&C '09). Association for Computing Machinery, New York, NY, USA, 205--210.

Digital Library

[14]

Mark R Johnson and Jamie Woodcock. 2019. 'It's like the gold rush': the lives and careers of professional video game streamers on Twitch.tv. Information, Communication & Society 22, 3 (2019), 336--351.

[15]

Charles Kiene, Jialun Aaron Jiang, and Benjamin Mako Hill. 2019. Technological Frames and User Innovation: Exploring Technological Change in Community Moderation Teams. Proc. ACM Hum.-Comput. Interact. 3, CSCW, Article 44 (Nov. 2019), 23 pages.

Digital Library

[16]

Sara Kiesler, Robert Kraut, Paul Resnick, and Aniket Kittur. 2012. Regulating behavior in online communities. In Building Successful Online Communities: Evidence-Based Social Design, Robert Kraut and Paul Resnick (Eds.). MIT Press, Cambridge, MA, USA, Chapter 4, 125--177.

[17]

Soomin Kim, Joonhwan Lee, and Gahgene Gweon. 2019. Comparing Data from Chatbot and Web Surveys: Effects of Platform and Conversational Style on Survey Response Quality. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19). ACM, NY, NY, USA, Article 86, 12 pages.

Digital Library

[18]

S. Kopp, L. Gesellensetter, and Wachsmuth I. Krämer, N.C. 2005. A Conversational Agent as Museum Guide -- Design and Evaluation of a Real-World Application. In Intelligent Virtual Agents. IVA 2005. Lecture Notes in Computer Science, vol 3661, T. Panayiotopoulos, J. Gratch, R. Aylett, D. Ballin, and T. Olivier, P. Rist (Eds.). Springer, Berlin, Germany.

[19]

Minha Lee, Sander Ackermans, Nena van As, Hanwen Chang, Enzo Lucas, and Wijnand IJsselsteijn. 2019. Caring for Vincent: A Chatbot for Self-Compassion. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19). ACM, NY, NY, USA, Article 702, 13 pages.

Digital Library

[20]

Pascal Lessel, Alexander Vielhauer, and Antonio Krüger. 2017. Expanding Video Game Live-Streams with Enhanced Communication Channels: A Case Study. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). ACM, NY, NY, USA, 1571--1576.

Digital Library

[21]

Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. 2016. A Diversity-Promoting Objective Function for Neural Conversation Models. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics, San Diego, California, 110--119.

[22]

Michal Luria, Samantha Reig, Xiang Zhi Tan, Aaron Steinfeld, Jodi Forlizzi, and John Zimmerman. 2019. Re-Embodiment and Co-Embodiment: Exploration of social presence for robots and conversational agents. In Proceedings of the 2019 on Designing Interactive Systems Conference. ACM, NY, NY, USA, 633--644.

Digital Library

[23]

Michael L Mauldin. 1994. Chatterbots, tinymuds, and the turing test: Entering the loebner prize competition. In AAAI-94 Proceedings, Vol. 94. AAAI, Palo Alto, CA, USA, 16--21.

[24]

Safiya Umoja Noble. 2018. Algorithms of Oppression. NYU Press, New York, NY, USA.

[25]

Rui Pan, Lyn Bartram, and Carman Neustaedter. 2016. TwitchViz: A Visualization Tool for Twitch Chatrooms. In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA '16). ACM, NY, NY, USA, 1959--1965.

Digital Library

[26]

Maria Roussou, Sara Perry, Akrivi Katifori, Stavros Vassos, Angeliki Tzouganatou, and Sierra McKinney. 2019. Transformation Through Provocation?. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19). ACM, NY, NY, USA, Article 627, 13 pages.

Digital Library

[27]

Saiph Savage, Andres Monroy-Hernandez, and Tobias Höllerer. 2016. Botivist: Calling Volunteers to Action Using Online Bots. In Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing (CSCW '16). ACM, NY, NY, USA, 813--822.

Digital Library

[28]

Ari Schlesinger, Kenton P. O'Hara, and Alex S. Taylor. 2018. Let's Talk About Race: Identity, Chatbots, and AI. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). ACM, NY, NY, USA, Article 315, 14 pages.

Digital Library

[29]

Alex Sciuto, Arnita Saini, Jodi Forlizzi, and Jason I Hong. 2018. Hey Alexa, What's Up?: A mixed-methods studies of in-home conversational agent usage. In Proceedings of the 2018 Designing Interactive Systems Conference. ACM, NY, NY, USA, 857--868.

Digital Library

[30]

Joseph Seering, Juan Pablo Flores, Saiph Savage, and Jessica Hammer. 2018. The Social Roles of Bots: Evaluating Impact of Bots on Discussions in Online Communities. Proc. ACM Hum.-Comput. Interact. 2, CSCW, Article 157 (Nov. 2018), 29 pages.

Digital Library

[31]

Joseph Seering, Michal Luria, Geoff Kaufman, and Jessica Hammer. 2019. Beyond Dyadic Interactions: Considering Chatbots As Community Members. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19). ACM, NY, NY, USA, Article 450, 13 pages.

Digital Library

[32]

Joseph Seering, Saiph Savage, Michael Eagle, Joshua Churchin, Rachel Moeller, Jeffrey P. Bigham, and Jessica Hammer. 2017. Audience Participation Games: Blurring the Line Between Player and Spectator. In Proceedings of the 2017 Conference on Designing Interactive Systems (DIS '17). ACM, NY, NY, USA, 429--440.

Digital Library

[33]

Joseph Seering, Tony Wang, Jina Yoon, and Geoff Kaufman. 2019. Moderator engagement and community development in the age of algorithms. New Media & Society 21, 7 (2019), 1417--1443.

[34]

Carolyn Snyder. 2003. Paper prototyping: The fast and easy way to design and refine user interfaces. Morgan Kaufmann, San Francisco, CA, USA.

[35]

Fumihide Tanaka and Shizuko Matsuzoe. 2012. Children Teach a Care-receiving Robot to Promote Their Learning: Field Experiments in a Classroom for Vocabulary Learning. J. Hum.-Robot Interact. 1, 1 (July 2012), 78--95.

Digital Library

[36]

T.L. Taylor. 2018. Watch Me Play: Twitch and the Rise of Game Live Streaming. Princeton University Press, Princeton, NJ, USA.

[37]

Na?a Terzimehif, Mohamed Khamis, Florian Bemmann, and Heinrich Hussmann. 2018. Lunchocracy: Improving Eating Dynamics in the Workplace Using a Bot-Based Anonymous Voting System. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems (CHI EA '18). ACM, NY, NY, USA, Article LBW022, 6 pages.

Digital Library

[38]

Richard Wallace. 2003. The elements of AIML style. Alice AI Foundation.

[39]

Richard S Wallace. 2009. The anatomy of ALICE. In Parsing the Turing Test. Springer, New York, NY, USA, 181--210.

[40]

Joseph Weizenbaum. 1966. ELIZA -- a Computer Program for the Study of Natural Language Communication Between Man and Machine. Commun. ACM 9, 1 (Jan. 1966), 36--45.

Digital Library

[41]

John Zimmerman, Jodi Forlizzi, and Shelley Evenson. 2007. Research Through Design As a Method for Interaction Design Research in HCI. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '07). ACM, NY, NY, USA, 493--502.

Digital Library

Cited By

Silva CRamos Fde Moraes RSantos E(2024)ChatGPT: Challenges and Benefits in Software Programming for Higher EducationSustainability10.3390/su1603124516:3(1245)Online publication date: 1-Feb-2024
https://doi.org/10.3390/su16031245
Piorkowski DOstrand RBrimijoin KHe JAlbert EHoude S(2024)Towards Interactive Guidance for Writing Training Utterances for Conversational AgentsProceedings of the 6th ACM Conference on Conversational User Interfaces10.1145/3640794.3665553(1-15)Online publication date: 8-Jul-2024
https://dl.acm.org/doi/10.1145/3640794.3665553
Gao ZThebault-Spieker J(2024)Investigating Influential Users' Responses to Permanent Suspension on Social MediaProceedings of the ACM on Human-Computer Interaction10.1145/36373568:CSCW1(1-41)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3637356
Show More Cited By

Index Terms

It Takes a Village: Integrating an Adaptive Chatbot into an Online Gaming Community
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. HCI design and evaluation methods
      1. User studies

Recommendations

Dialoging Resonance in Human-Chatbot Conversation: How Users Perceive and Reciprocate Recommendation Chatbot's Self-Disclosure Strategy
CSCW

Using chatbots to make recommendations is increasingly popular. The design of recommendation chatbots has mainly been taking an information-centric approach by focusing on the recommended content per se. Limited attention is on how social connection and ...
Chatbot with Touch and Graphics: An Interaction of Users for Emotional Expression and Turn-taking
CUI '20: Proceedings of the 2nd Conference on Conversational User Interfaces

Use of chatbots for emotional exchange is recently increasing in various domains. However, as existing chatbots have been considered in terms of natural language processing techniques for interaction with text-based chatting, chatbot interaction with ...
Chillbot: Content Moderation in the Backchannel
CSCW

Moderating online spaces effectively is not a matter of simply taking down content: moderators also provide private feedback and defuse situations before they cross the line into harm. However, moderators have little tool support for these activities, ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '20: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems

April 2020

10688 pages

ISBN:9781450367080

DOI:10.1145/3313831

General Chairs:
Regina Bernhaupt
Eindhoven University of Technology, Netherlands
,
Florian 'Floyd' Mueller
Monash University, Australia
,
David Verweij
Newcastle University, UK
,
Josh Andres
RMIT, Australia
,
Program Chairs:
Joanna McGrenere
University of British Columbia, Canada
,
Andy Cockburn
University of Canterbury, New Zealand
,
Ignacio Avellino
University of Maryland Baltimore County, USA
,
Alix Goguey
Grenoble Alpes University, France
,
Pernille Bjørn
University of Copenhagen, Denmark
,
Shengdong (Shen) Zhao
National University of Singapore, Singapore
,
Briane Paul Samson
Future University Hakodate, Japan & De La Salle University, Philippines
,
Rafal Kocielnik
University of Washington, USA

Copyright © 2020 Owner/Author.

This work is licensed under a Creative Commons Attribution International 4.0 License.

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 April 2020

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CHI '20

Sponsor:

SIGCHI

CHI '20: CHI Conference on Human Factors in Computing Systems

April 25 - 30, 2020

HI, Honolulu, USA

Acceptance Rates

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI '25

Sponsor:
sigchi

CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

34
Total Citations
View Citations
1,610
Total Downloads

Downloads (Last 12 months)275
Downloads (Last 6 weeks)33

Reflects downloads up to 13 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Silva CRamos Fde Moraes RSantos E(2024)ChatGPT: Challenges and Benefits in Software Programming for Higher EducationSustainability10.3390/su1603124516:3(1245)Online publication date: 1-Feb-2024
https://doi.org/10.3390/su16031245
Piorkowski DOstrand RBrimijoin KHe JAlbert EHoude S(2024)Towards Interactive Guidance for Writing Training Utterances for Conversational AgentsProceedings of the 6th ACM Conference on Conversational User Interfaces10.1145/3640794.3665553(1-15)Online publication date: 8-Jul-2024
https://dl.acm.org/doi/10.1145/3640794.3665553
Gao ZThebault-Spieker J(2024)Investigating Influential Users' Responses to Permanent Suspension on Social MediaProceedings of the ACM on Human-Computer Interaction10.1145/36373568:CSCW1(1-41)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3637356
Kaur JSharma PKumar VDuggal MDiamond-Smith NEl Ayadi AVosburg KSingh P(2024)Exploring the Role of Chatbots in Tackling COVID-19 Vaccine Hesitancy among Pregnant and Breastfeeding Women in Rural Northern IndiaProceedings of the ACM on Human-Computer Interaction10.1145/36373328:CSCW1(1-29)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3637332
Prinster GSmith CTan CKeegan B(2024)Community Archetypes: An Empirical Framework for Guiding Research Methodologies to Reflect User Experiences of Sense of Virtual Community on RedditProceedings of the ACM on Human-Computer Interaction10.1145/36373108:CSCW1(1-33)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3637310
Jin YChen LCai WZhao X(2024)CRS-Que: A User-centric Evaluation Framework for Conversational Recommender SystemsACM Transactions on Recommender Systems10.1145/36315342:1(1-34)Online publication date: 7-Mar-2024
https://dl.acm.org/doi/10.1145/3631534
Chang FWang GGu Z(2024)Navigating Challenges and Opportunities in Community-based Interventions for Promoting Active Ageing: A Systematic Review of the Roles of TechnologyInnovation in Aging10.1093/geroni/igae077Online publication date: 6-Sep-2024
https://doi.org/10.1093/geroni/igae077
Zhu CDastani MWang S(2024)A survey of multi-agent deep reinforcement learning with communicationAutonomous Agents and Multi-Agent Systems10.1007/s10458-023-09633-638:1Online publication date: 6-Jan-2024
https://doi.org/10.1007/s10458-023-09633-6
Dwivedi RYaqoob MKhan APriyadarshi PAhmad SAttri SMutharaju R(2023)In Search of a “Social-AQI”South Asia Multidisciplinary Academic Journal10.4000/samaj.8768Online publication date: 7-Jul-2023
https://doi.org/10.4000/samaj.8768
Mills RMangone ELesh NMohan DBaraitser P(2023)Chatbots to Improve Sexual and Reproductive Health: Realist SynthesisJournal of Medical Internet Research10.2196/4676125(e46761)Online publication date: 9-Aug-2023
https://doi.org/10.2196/46761
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents