research-article

"Hey Alexa, What's Up?": A Mixed-Methods Studies of In-Home Conversational Agent Usage

Authors:

Jason I. HongAuthors Info & Claims

DIS '18: Proceedings of the 2018 Designing Interactive Systems Conference

Pages 857 - 868

https://doi.org/10.1145/3196709.3196772

Published: 08 June 2018 Publication History

Abstract

In-home, place-based, conversational agents have exploded in popularity over the past three years. In particular, Amazon's conversational agent, Alexa, now dominates the market and is in millions of homes. This paper presents two complementary studies investigating the experience of households living with a conversational agent over an extended period of time. First, we gathered the history logs of 75 Alexa participants and quantitatively analyzed over 278,000 commands. Second, we performed seven in-home, contextual interviews of Alexa owners focusing on how their household interacts with Alexa. Our findings give the first glimpse of how households integrate Alexa into their lives. We found interesting behaviors around purchasing and acclimating to Alexa, in the number and physical placement of devices, and in daily use patterns. Participants also uniformly described interactions between children and Alexa. We conclude with suggestions for future improvement for intelligent conversational agents.

References

[1]

Sameera A. Abdul-Kader and John Woods. Survey on Chatbot Design Techniques in Speech Conversation Systems. International Journal of Advanced Computer Science and Applications 6, 7.

[2]

Gregory D. Abowd and Elizabeth D. Mynatt. 2000. Charting past, present, and future research in ubiquitous computing. ACM Transactions on Computer-Human Interaction 7, 1: 29--58.

Digital Library

[3]

Gregory Abowd, Keith Edwards, and Beki Grinter. 2003. Smart homes or homes that smart? ACM SIGCHI Bulletin - a supplement to interactions 2003: 13--13.

Digital Library

[4]

Bayan Abu Shawar and Eric Atwell. 2007. Chatbots: are they really useful? LDV Forum. Retrieved September 17, 2017 from https://www.researchgate.net/profile/Eric_Atwell/pub li cation/220046725_Chatbots_Are_they_Really_Useful / links/00b7d518ab7329add2000000/Chatbots-Arethey- Really-Useful.pdf

[5]

Corey Badcock. 2015. First Alexa Third-Party Skills Now Available for Amazon EchoNo Title. Alexa Blog. Retrieved from https://developer.amazon.com/blogs/post/TxC2VHK F EIZ9SG/First-Alexa-Third-Party-Skills-NowAvailable-for-Amazon-Echo

[6]

Izak Bensabat and Weiquan Wang. "Trust in and adoption of online recommendation agents." Journal of the association for information systems 6, no. 3 (2005):

[7]

Justine Cassell, Timothy Bickmore, Mark Billinghurst, Lee Campbell, Kenny Chang, Hannes Vilhjálmsson, and Hao Yan. "Embodiment in conversational interfaces: Rea." In Proceedings of the SIGCHI conference on Human Factors in Computing Systems, pp. 520--527. ACM, 1999.

Digital Library

[8]

Justine Cassell, Tom Stocky, Tim Bickmore, et al. 2002. MACK: Media lab autonomous conversational kiosk. Proceedings of Imagina '02.

[9]

Marshini Chetty, Ja-Young Sung, and Rebecca E. Grinter. 2007. How Smart Homes Learn: The Evolution of the Networked Home and Household. 127--144.

[10]

Andy Crabtree and Tom Rodden. Domestic Routines and Design for the Home. Computer Supported Cooperative Work (CSCW) 13, 2: 191-- 220.

Digital Library

[11]

Andy Crabtree and Peter Tolmie. 2016. A Day in the Life of Things in the Home. Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing - CSCW '16, 1736--1748.

Digital Library

[12]

Scott Davidoff, Min Kyung Lee, Charles Yiu, John Zimmerman, and Anind K. Dey. 2006. Principles of Smart Home Control. In Proceedings of the 8th international conference on Ubiquitous Computing. Springer-Verlag, 19--34.

Digital Library

[13]

Stefania Druga, Randi Williams, Cynthia Breazeal, and Mitchel Resnick. 2017. "Hey Google is it OK if I eat you?": Initial Explorations in Child-Agent Interaction. Proceedings of the 2017 Conference on Interaction Design and Children - IDC '17, ACM Press, 595--600.

Digital Library

[14]

Homer Dudley. 1939. Remaking Speech. The Journal of the Acoustical Society of America 11, 2: 169--177.

[15]

W. Keith Edwards and Rebecca E. Grinter. 2001. At Home with Ubiquitous Computing: Seven Challenges. Proceedings of the 3rd international conference on Ubiquitous Computing, Springer, 372. Retrieved September 16, 2017 from http://dl.acm.org/citation.cfm?id=741327&CFID=98 50 43092&CFTOKEN=74478828

Digital Library

[16]

Darrell Etherington. 2014. Amazon Echo Is A $199 Connected Speaker Packing An Always-On SiriStyle Assistant | TechCrunch. Techcrunch. Retrieved September 15, 2017 from https://techcrunch.com/2014/11/06/amazon-echo/

[17]

Adjamir M. Galvao, Flavia A. Barros, Andre M. M. Neves, and Geber L. Ramalho. 2004. PersonaAIML: An Architecture Developing Chatterbots with Personality. Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems - Volume 3: 1266-- 1267.

Digital Library

[18]

Juan Pablo Garcia Vazquez, Marcela D. Rodriguez, and Angel G. Andrade. 2010. Design dimensions of ambient information systems to assist elderly with their activities of daily living. Proceedings of the 12th ACM international conference adjunct papers on Ubiquitous computing - Ubicomp '10, ACM Press, 461.

Digital Library

[19]

William Gaver, Alex Wilkie, Andy Boucher, et al. 2008. Threshold devices. Proceeding of the twenty- sixth annual CHI conference on Human factors in computing systems - CHI '08, ACM Press, 1429.

Digital Library

[20]

Lars Gesellensetter, Nicole C. Krämer, and Ipke Wachsmuth. 2005. A conversational agent as museum guide - Design and evaluation of a realworld application. THE 5TH INTERNATIONAL WORKING CONFERENCE ON INTELLIGENT VIRTUAL AGENTS (IVA '05), Springer, 329--343. Retrieved March 5, 2017 from http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1 .1.93.7488

Digital Library

[21]

Evangelos Karapanos, John Zimmerman, Jodi Forlizzi, and Jean-Bernard Martens. 2009. User experience over time: an initial framework. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems: 729--738.

Digital Library

[22]

Stefan Kopp, Lars Gesellensetter, Nicole C. Krämer, and Ipke Wachsmuth. "A conversational agent as museum guide--design and evaluation of a realworld application." In International Workshop on Intelligent Virtual Agents, pp. 329--343. Springer, Berlin, Heidelberg, 2005.

Digital Library

[23]

Gustavo López, Luis Quesada, and Luis A. Guerrero. 2018. Alexa vs. Siri vs. Cortana vs. Google Assistant: A Comparison of Speech-Based Natural User Interfaces. . Springer, Cham, 241-- 250. Farhad Manjoo. 2015. Amazon Echo, a Personal Assistant in Need of Some Schooling. New York Times, B8. Retrieved from https://www.nytimes.com/2015/06/25/technology/per s onaltech/amazon-echo-aka-alexa-is-a-personal-aidein- need-of-schooling.html

[24]

Sven Meyer and Andry Rakontonirainy. 2003. A survey of research on context-aware homes. Proceedings of the Australasian information security workshop conference on ACSW frontiers 2003 - Volume 21, Australian Computer Society, 90. Retrieved September 13, 2017 from http://dl.acm.org/citation.cfm?id=828005

Digital Library

[25]

Kris Nagel, Cory D. Kidd, Thomas O'Connell, Anind Dey, and Gregory D. Abowd. 2001. The Family Intercom: Developing a Context-Aware Audio Communication System. . Springer, Berlin, Heidelberg, 176--183.

[26]

William Odom, John Zimmerman, Jodi Forlizzi, Hajin Choi, Stephanie Meier, and Angela Park. 2012. Investigating the presence, form and behavior of virtual possessions in the context of a teen bedroom. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems: 327--336.

Digital Library

[27]

Gerard Oleksik, David Frohlich, Lorna M. Brown, and Abigail Sellen. 2008. Sonic interventions. Proceeding of the twenty-sixth annual CHI conference on Human factors in computing systems CHI '08, ACM Press, 1419.

[28]

Charles L. Ortiz. 2014. The Road to Natural Conversational Speech Interfaces. IEEE Internet Computing 18, 2: 74--78.

[29]

Emanuel Parzen. On Estimation of a Probability Density Function and Mode. The Annals of Mathematical Statistics 33, 1065-- 1076.

[30]

Sarah Perez. 2017. Amazon to control 70 percent of the voice-controlled speaker market this year. Techcrunch.

[31]

Sarah Perez. 2017. 42 percent of smart speaker owners have bought a second device (or more) | TechCrunch. Techcrunch. Retrieved September 15, 2017 from https://techcrunch.com/2017/06/23/42percent-of- amazon-echo-owners-have-bought-asecond-device-or- more/

[32]

David Powers, Richard Leibbrandt, Martin Luerssen, Trent Lewis, and Mike Lawson. 2008. PETA. Proceedings of the 1st ACM international conference on PErvasive Technologies Related to Assistive Environments - PETRA '08, ACM Press, 1.

[33]

Amanda Purington, Jessie G. Taft, Shruti Sannon, Natalya N. Bazarova, and Samuel Hardman Taylor. 2017. "Alexa is my new BFF": Social Roles, User Satisfaction, and Personification of the Amazon Echo. Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems - CHI EA '17, ACM Press, 2853--2859.

Digital Library

[34]

Jason Del Rey. 2016. You're not alone: 51 percent of Amazon Echo owners have it in the kitchen. Recode.

[35]

Douglas. Schuler and Aki. Namioka. 1993. Participatory design?: principles and practices. L. Erlbaum Associates. Retrieved September 18, 2017 from http://dl.acm.org/citation.cfm?id=563076

Digital Library

[36]

Yankelovich and Nicole. 1996. How do users know what to say? interactions 3, 6: 32--43.

Digital Library

Cited By

Giudici MPadalino LPaolino GParatici IPascu AGarzotto F(2024)Designing Home Automation Routines Using an LLM-Based ChatbotDesigns10.3390/designs80300438:3(43)Online publication date: 13-May-2024
https://doi.org/10.3390/designs8030043
Seaborn KSawa YWatanabe M(2024)Coimagining the Future of Voice Assistants with Cultural SensitivityHuman Behavior and Emerging Technologies10.1155/2024/32387372024(1-21)Online publication date: 25-Mar-2024
https://doi.org/10.1155/2024/3238737
Shafei HTan C(2024)Enhancing Alexa Skill Testing Through Improved Utterance DiscoveryACM Transactions on Internet Technology10.1145/369820024:4(1-25)Online publication date: 18-Nov-2024
https://dl.acm.org/doi/10.1145/3698200
Show More Cited By

Index Terms

"Hey Alexa, What's Up?": A Mixed-Methods Studies of In-Home Conversational Agent Usage
1. Human-centered computing

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

DIS '18: Proceedings of the 2018 Designing Interactive Systems Conference

June 2018

1418 pages

ISBN:9781450351980

DOI:10.1145/3196709

General Chairs:
Ilpo Koskinen
University of Twente
,
Youn-kyung Lim
KAIST
,
Program Chairs:
Teresa Cerratto-Pargman
Stockholm University
,
Kenny Chow
The Hong Kong Polytechnic University
,
William Odom
Simon Fraser University

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 June 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

DIS '18

Sponsor:

SIGCHI

DIS '18: Designing Interactive Systems Conference 2018

June 9 - 13, 2018

Hong Kong, China

Acceptance Rates

DIS '18 Paper Acceptance Rate 107 of 487 submissions, 22%;

Overall Acceptance Rate 1,158 of 4,684 submissions, 25%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

273
Total Citations
View Citations
4,754
Total Downloads

Downloads (Last 12 months)261
Downloads (Last 6 weeks)46

Reflects downloads up to 18 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Giudici MPadalino LPaolino GParatici IPascu AGarzotto F(2024)Designing Home Automation Routines Using an LLM-Based ChatbotDesigns10.3390/designs80300438:3(43)Online publication date: 13-May-2024
https://doi.org/10.3390/designs8030043
Seaborn KSawa YWatanabe M(2024)Coimagining the Future of Voice Assistants with Cultural SensitivityHuman Behavior and Emerging Technologies10.1155/2024/32387372024(1-21)Online publication date: 25-Mar-2024
https://doi.org/10.1155/2024/3238737
Shafei HTan C(2024)Enhancing Alexa Skill Testing Through Improved Utterance DiscoveryACM Transactions on Internet Technology10.1145/369820024:4(1-25)Online publication date: 18-Nov-2024
https://dl.acm.org/doi/10.1145/3698200
Shalawadi SEchtler FRaptis D(2024)Dr. Convenience Love or: How I Learned to Stop Worrying and Love my Voice Assistant✱Proceedings of the 13th Nordic Conference on Human-Computer Interaction10.1145/3679318.3685364(1-14)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3679318.3685364
Chidziwisano GCummings APotnis DJalakasi M(2024)Understanding Urban Women's Interaction with Domestic Technologies in Malawi: Advancing Feminist Theories in HCIACM Journal on Computing and Sustainable Societies10.1145/36502042:2(1-25)Online publication date: 5-Mar-2024
https://dl.acm.org/doi/10.1145/3650204
Wu LLafreniere BGrossman TWhite TSantosa S(2024)Body Language for VUIs: Exploring Gestures to Enhance Interactions with Voice User InterfacesProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3660691(133-150)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3643834.3660691
Oh JIm HLee S(2024)Toward a Third-Kind Voice for Conversational Agents in an Era of Blurring Boundaries Between Machine and Human SoundsProceedings of the 6th ACM Conference on Conversational User Interfaces10.1145/3640794.3665880(1-7)Online publication date: 8-Jul-2024
https://dl.acm.org/doi/10.1145/3640794.3665880
Desai SDubiel MLeiva L(2024)Examining Humanness as a Metaphor to Design Voice User InterfacesProceedings of the 6th ACM Conference on Conversational User Interfaces10.1145/3640794.3665535(1-15)Online publication date: 8-Jul-2024
https://dl.acm.org/doi/10.1145/3640794.3665535
Kalirai MWilliams AKuzminykh A(2024)Toward Faceted Skill Recommendation in Intelligent Personal AssistantsProceedings of the 29th International Conference on Intelligent User Interfaces10.1145/3640543.3645201(640-649)Online publication date: 18-Mar-2024
https://dl.acm.org/doi/10.1145/3640543.3645201
Sun KLi JZou YRadesky JBrooks CSchaub F(2024)Unfulfilled Promises of Child Safety and Privacy: Portrayals and Use of Children in Smart Home MarketingProceedings of the ACM on Human-Computer Interaction10.1145/36374228:CSCW1(1-29)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3637422
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents