Nothing Special   »   [go: up one dir, main page]

skip to main content
10.1145/3441000.3441068acmotherconferencesArticle/Chapter ViewAbstractPublication PagesozchiConference Proceedingsconference-collections
research-article

Intelligibility Issues Faced by Smart Speaker Enthusiasts in Understanding What Their Devices Do and Why

Published: 15 February 2021 Publication History

Abstract

Studies of smart speakers highlight issues people face with understanding why unexpected behaviour occurs and with recovering from mistakes due to uninformative responses. Yet, our understanding of such intelligibility issues in smart speakers — difficulties in understanding the device's behaviour — remains limited, in particular, for long-term and frequent smart speaker users who may encounter more complex situations than first-time users. We conducted an online survey and interviews with smart speaker enthusiasts to investigate how they form an understanding of the device's behaviour and what strategies they use to recover from breakdowns. We identified seven different breakdown recovery strategies and found that enthusiasts particularly struggled with breakdowns in their IoT infrastructure. Informed by our results, we propose three research directions: infrastructural breakdowns as learning opportunities for understanding the smart speaker's behaviour; leveraging aspects of non-verbal communication as opportunities for design; and considering passive users’ intelligibility and control needs.

Supplementary Material

p314-avdic-supplement1 (p314-avdic-supplement1.pdf)
Online survey
p314-avdic-supplement2 (p314-avdic-supplement2.pdf)
Interview questions

References

[1]
Abdul, A., Vermeulen, J., Wang, D., Lim, B.Y. and Kankanhalli, M. 2018. Trends and Trajectories for Explainable, Accountable and Intelligible Systems: An HCI Research Agenda. Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (New York, NY, USA, 2018), 582:1–582:18.
[2]
Amershi, S., Weld, D., Vorvoreanu, M., Fourney, A., Nushi, B., Collisson, P., Suh, J., Iqbal, S., Bennett, P.N., Inkpen, K., Teevan, J., Kikin-Gil, R. and Horvitz, E. 2019. Guidelines for Human-AI Interaction. Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (2019), 19.
[3]
Anderson-Bashan, L., Megidish, B., Erel, H., Wald, I., Hoffman, G., Zuckerman, O. and Grishko, A. 2018. The Greeting Machine: An Abstract Robotic Object for Opening Encounters. 2018 27th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN) (Aug. 2018), 595–602.
[4]
Bellotti, V., Back, M., Edwards, W.K., Grinter, R.E., Henderson, A. and Lopes, C. 2002. Making Sense of Sensing Systems: Five Questions for Designers and Researchers. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (New York, NY, USA, 2002), 415–422.
[5]
Bellotti, V. and Edwards, K. 2001. Intelligibility and Accountability: Human Considerations in Context-Aware Systems. Human–Computer Interaction. 16, 2–4 (Dec. 2001), 193–212.
[6]
Beneteau, E., Richards, O.K., Zhang, M., Kientz, J.A., Yip, J. and Hiniker, A. 2019. Communication Breakdowns Between Families and Alexa. Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (New York, NY, USA, 2019), 243:1–243:13.
[7]
Bentley, F., Luvogt, C., Silverman, M., Wirasinghe, R., White, B. and Lottrjdge, D. 2018. Understanding the Long-Term Use of Smart Speaker Assistants. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies. 2, 3 (Sep. 2018), 1–24.
[8]
Blandford, A., Furniss, D. and Makri, S. 2016. Qualitative HCI Research: Going Behind the Scenes. Morgan & Claypool.
[9]
Branham, M., S. and Roy Rishin Mukkath, A. 2019. Reading Between the Guidelines: How Commercial Voice Assistant Guidelines Hinder Accessibility for Blind Users. ASSETS ’19 (Oct. 2019).
[10]
Brinkmann, S. and Kvale, S. 2014. Interviews - Learning the Craft of Qualitative Research Interviewing. Sage Publications Inc.
[11]
Bunt, A., Lount, M. and Lauzon, C. 2012. Are explanations always important?: a study of deployed, low-cost intelligent interactive systems. Proceedings of the 2012 ACM international conference on Intelligent User Interfaces (Feb. 2012), 169–178.
[12]
Cha, N., Kim, A., Park, C.Y., Kang, S., Park, M., Lee, J.-G., Lee, S. and Lee, U. 2020. ``Hello There! Is Now a Good Time to Talk?’’: Opportune Moments for Proactive Interactions with Smart Speakers. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies. 4, 3 (Sep. 2020), 28.
[13]
Chalmers, M. 2003. Seamful Design and Ubicomp Infrastructure. Proceedings of Ubicomp 2003 workshop at the crossroads: The interaction of HCI and systems issues in Ubicomp. (2003).
[14]
Chilana, P.K., Ko, A.J. and Wobbrock, J.O. 2012. LemonAid: selection-based crowdsourced contextual help for web applications. Proceedings of the 2012 ACM annual conference on Human Factors in Computing Systems - CHI ’12 (Austin, Texas, USA, 2012), 1549.
[15]
Cho, J. 2018. Mental Models and Home Virtual Assistants (HVAs). Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems (New York, NY, USA, 2018), SRC05:1–SRC05:6.
[16]
Cho, M., Lee, S. and Lee, K.-P. 2019. Once a Kind Friend is Now a Thing: Understanding How Conversational Agents at Home Are Forgotten. Proceedings of the 2019 on Designing Interactive Systems Conference (New York, NY, USA, 2019), 1557–1569.
[17]
Chuang, Y., Chen, L. and Liu, Y. 2018. Design vocabulary for human-IoT systems communication. CHI 2018 - Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems: Engage with CHI (Apr. 2018).
[18]
Corbin, J. and Strauss, A. 2015. Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory. SAGE Publications, Inc.
[19]
Cowan, B.R., Pantidi, N., Coyle, D., Morrissey, K., Clarke, P., Al-Shehri, S., Earley, D. and Bandeira, N. 2017. “What Can I Help You with?”: Infrequent Users’ Experiences of Intelligent Personal Assistants. Proceedings of the 19th International Conference on Human-Computer Interaction with Mobile Devices and Services (New York, NY, USA, 2017), 43:1–43:12.
[20]
Dey, A.K. 2001. Understanding and Using Context. Personal and Ubiquitous Computing. 5, 1 (Jan. 2001), 4–7.
[21]
Dey, A.K. and Newberger, A. 2009. Support for context-aware intelligibility and control. (Apr. 2009), 859–868.
[22]
Druga, S., Williams, R., Breazeal, C. and Resnick, M. 2017. “Hey Google is It OK if I Eat You?”: Initial Explorations in Child-Agent Interaction. Proceedings of the 2017 Conference on Interaction Design and Children (New York, NY, USA, 2017), 595–600.
[23]
Edwards, W.K., Newman, M.W. and Poole, E.S. 2010. The infrastructure problem in HCI. Proceedings of the 28th international conference on Human factors in computing systems - CHI ’10 (Atlanta, Georgia, USA, 2010), 423.
[24]
Fraser, C.A., Dontcheva, M., Winnemöller, H., Ehrlich, S. and Klemmer, S. 2016. DiscoverySpace: Suggesting Actions in Complex Software. Proceedings of the 2016 ACM Conference on Designing Interactive Systems - DIS ’16 (Brisbane, QLD, Australia, 2016), 1221–1232.
[25]
Greenberg, S., Marquardt, N., Ballendat, T., Diaz-Marino, R. and Wang, M. 2011. Proxemic interactions: the new ubicomp? interactions. 18, 1 (Jan. 2011), 42.
[26]
Gregor, P., Newell, A.F. and Zajicek, M. 2002. Designing for Dynamic Diversity: Interfaces for Older People. Proceedings of the Fifth International ACM Conference on Assistive Technologies (New York, NY, USA, 2002), 151–156.
[27]
Hornecker, E. and Buur, J. 2006. Getting a Grip on Tangible Interaction: A Framework on Physical Space and Social Interaction. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (New York, NY, USA, 2006), 437–446.
[28]
Jiang, J., Jeng, W. and He, D. 2013. How Do Users Respond to Voice Input Errors?: Lexical and Phonetic Query Reformulation in Voice Search. Proceedings of the 36th International ACM SIGIR Conference on Research and Development in Information Retrieval (New York, NY, USA, 2013), 143–152.
[29]
Ju, W. and Takayama, L. 2009. Approachability: How People Interpret Automatic Door Movement as Gesture. International Journal of Design. Vol. 3(2), Design and Emotion (Aug. 2009), 15.
[30]
Klein, L. 2015. Design for Voice Interfaces. O'Reilly Media, Inc.
[31]
Kulesza, T., Burnett, M., Wong, W.-K. and Stumpf, S. 2015. Principles of Explanatory Debugging to Personalize Interactive Machine Learning. Proceedings of the 20th International Conference on Intelligent User Interfaces - IUI ’15 (Atlanta, Georgia, USA, 2015), 126–137.
[32]
Lau, J., Zimmerman, B. and Schaub, F. 2018. Alexa, Are You Listening?: Privacy Perceptions, Concerns and Privacy-seeking Behaviors with Smart Speakers. Proc. ACM Hum.-Comput. Interact.2, CSCW (Nov. 2018), 102:1–102:31.
[33]
Lim, B.Y. and Dey, A.K. 2009. Assessing demand for intelligibility in context-aware applications. Proceedings of the 11th international conference on Ubiquitous computing - Ubicomp ’09 (Orlando, Florida, USA, 2009), 195.
[34]
Lim, B.Y. and Dey, A.K. 2011. Design of an Intelligible Mobile Context-aware Application. Proceedings of the 13th International Conference on Human Computer Interaction with Mobile Devices and Services (New York, NY, USA, 2011), 157–166.
[35]
Lim, B.Y. and Dey, A.K. 2011. Investigating Intelligibility for Uncertain Context-aware Applications. Proceedings of the 13th International Conference on Ubiquitous Computing (New York, NY, USA, 2011), 415–424.
[36]
Lim, B.Y. and Dey, A.K. 2010. Toolkit to support intelligibility in context-aware applications. Proceedings of the 12th ACM international conference on Ubiquitous computing - Ubicomp ’10 (Copenhagen, Denmark, 2010), 13.
[37]
Lim, B.Y., Dey, A.K. and Avrahami, D. 2009. Why and Why Not Explanations Improve the Intelligibility of Context-aware Intelligent Systems. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (New York, NY, USA, 2009), 2119–2128.
[38]
Lopatovska, I., Rink, K., Knight, I., Raines, K., Cosenza, K., Williams, H., Sorsche, P., Hirsch, D., Li, Q. and Martinez, A. 2018. Talk to me: Exploring user interactions with the Amazon Alexa. Journal of Librarianship and Information Science. (Mar. 2018), 0961000618759414.
[39]
Luger, E. and Sellen, A. 2016. “Like Having a Really Bad PA”: The Gulf Between User Expectation and Experience of Conversational Agents. Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (New York, NY, USA, 2016), 5286–5297.
[40]
Matejka, J., Li, W., Grossman, T. and Fitzmaurice, G. 2009. CommunityCommands: command recommendations for software applications. Proceedings of the 22nd annual ACM symposium on User interface software and technology - UIST ’09 (Victoria, BC, Canada, 2009), 193.
[41]
McMillan, D., Brown, B., Kawaguchi, I., Jaber, R., Solsona Belenguer, J. and Kuzuoka, H. 2019. Designing with Gaze: Tama – a Gaze Activated Smart-Speaker. Proceedings of the ACM on Human-Computer Interaction. 3, CSCW (Nov. 2019), 1–26.
[42]
Mennicken, S., Brillman, R., Thom, J. and Cramer, H. 2018. Challenges and Methods in Design of Domain-Specific Voice Assistants. AAAI Spring Symposium Series. (2018), 5.
[43]
Mennicken, S. and Huang, E.M. 2012. Hacking the Natural Habitat: An In-the-Wild Study of Smart Homes, Their Development, and the People Who Live in Them. Pervasive Computing. J. Kay, P. Lukowicz, H. Tokuda, P. Olivier, and A. Krüger, eds. Springer Berlin Heidelberg. 143–160.
[44]
Mennicken, S., Vermeulen, J. and Huang, E.M. 2014. From today's augmented houses to tomorrow's smart homes: new directions for home automation research. Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing - UbiComp ’14 Adjunct (Seattle, Washington, 2014), 105–115.
[45]
Murad, C., Munteanu, C., Cowan, B.R. and Clark, L. 2019. Revolution or Evolution? Speech Interaction and HCI Design Guidelines. IEEE Pervasive Computing. 18, 2 (Apr. 2019), 33–45.
[46]
Myers, B.A., Weitzman, D.A., Ko, A.J. and Chau, D.H. 2006. Answering why and why not questions in user interfaces. Proceedings of the SIGCHI conference on Human Factors in computing systems - CHI ’06 (Montréal, Québec, Canada, 2006), 397.
[47]
Myers, C., Furqan, A., Nebolsky, J., Caro, K. and Zhu, J. 2018. Patterns for How Users Overcome Obstacles in Voice User Interfaces. Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems - CHI ’18 (Montreal QC, Canada, 2018), 1–7.
[48]
Myers, C.M., Grethlein, D., Furqan, A., Ontañón, S. and Zhu, J. 2019. Modeling Behavior Patterns with an Unfamiliar Voice User Interface. Proceedings of the 27th ACM Conference on User Modeling, Adaptation and Personalization - UMAP ’19 (Larnaca, Cyprus, 2019), 196–200.
[49]
Nest Learning Thermostat: 2020. https://store.google.com/us/product/nest_learning_thermostat_3rd_gen?hl=en-US. Accessed: 2020-08-14.
[50]
Nielsen, J. 1994.10 Heuristics for User Interface Design: Article by Jakob Nielsen.
[51]
Niemantsverdriet, K., Broekhuijsen, M., van Essen, H. and Eggen, B. 2016. Designing for Multi-User Interaction in the Home Environment: Implementing Social Translucence. Proceedings of the 2016 ACM Conference on Designing Interactive Systems (New York, NY, USA, 2016), 1303–1314.
[52]
Norval, C. and Singh, J. 2019. Explaining automated environments: interrogating scripts, logs, and provenance using voice-assistants. Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable Computers - UbiComp/ISWC ’19 (London, United Kingdom, 2019), 332–335.
[53]
Nowacka, D., Wolf, K., Costanza, E. and Kirk, D. 2018. Working with an Autonomous Interface: Exploring the Output Space of an Interactive Desktop Lamp. Proceedings of the Twelfth International Conference on Tangible, Embedded, and Embodied Interaction - TEI ’18 (Stockholm, Sweden, 2018), 1–10.
[54]
van Oosterhout, A., Bruns Alonso, M. and Jumisko-Pyykkö, S. 2018. Ripple Thermostat: Affecting the Emotional Experience through Interactive Force Feedback and Shape Change. Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems - CHI ’18 (Montreal QC, Canada, 2018), 1–12.
[55]
Pelikan, H.R.M. and Broth, M. 2016. Why That Nao?: How Humans Adapt to a Conventional Humanoid Robot in Taking Turns-at-Talk. Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (New York, NY, USA, 2016), 4921–4932.
[56]
Porcheron, M., Fischer, J.E., Reeves, S. and Sharples, S. 2018. Voice Interfaces in Everyday Life. Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (New York, NY, USA, 2018), 640:1–640:12.
[57]
Pradhan, A., Mehta, K. and Findlater, L. 2018. “Accessibility Came by Accident”: Use of Voice-Controlled Intelligent Personal Assistants by People with Disabilities. Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (New York, NY, USA, 2018), 459:1–459:13.
[58]
Reeves, S. 2019. Conversation considered harmful? (Aug. 2019), 10.
[59]
Reeves, S., Porcheron, M. and Fischer, J. 2018. “This is Not What We Wanted”: Designing for Conversation with Voice Interfaces. Interactions. 26, 1 (Dec. 2018), 46–51.
[60]
Sciuto, A., Saini, A., Forlizzi, J. and Hong, J.I. 2018. “Hey Alexa, What's Up?”: A Mixed-Methods Studies of In-Home Conversational Agent Usage. Proceedings of the 2018 Designing Interactive Systems Conference (New York, NY, USA, 2018), 857–868.
[61]
Shneiderman, B., Plaisant, C., Cohen, M., Jacobs, S., Elmqvist, N. and Diakopoulos, N. 2016. Designing the User Interface: Strategies for Effective Human-Computer Interaction. Pearson.
[62]
Søndergaard, M.L.J. and Hansen, L.K. 2018. Intimate Futures: Staying with the Trouble of Digital Personal Assistants through Design Fiction. Proceedings of the 2018 on Designing Interactive Systems Conference 2018 - DIS ’18 (Hong Kong, China, 2018), 869–880.
[63]
Sung, J.-Y., Guo, L., Grinter, R.E. and Christensen, H.I. 2007. “My Roomba Is Rambo”: Intimate Home Appliances. UbiComp 2007: Ubiquitous Computing (2007), 145–162.
[64]
The European Parliament and the Council of the European Union 2020. Regulations.
[65]
Ur, B., McManus, E., Pak Yong Ho, M. and Littman, M.L. 2014. Practical trigger-action programming in the smart home. Proceedings of the 32nd annual ACM conference on Human factors in computing systems - CHI ’14 (Toronto, Ontario, Canada, 2014), 803–812.
[66]
U.S. Smart Speaker Ownership Rises 40% in 2018 to 66.4 Million and Amazon Echo Maintains Market Share Lead Says New Report from Voicebot: 2019. https://voicebot.ai/2019/03/07/u-s-smart-speaker-ownership-rises-40-in-2018-to-66-4-million-and-amazon-echo-maintains-market-share-lead-says-new-report-from-voicebot/. Accessed: 2019-05-20.
[67]
Vermeulen, J., Slenders, J., Luyten, K. and Coninx, K. 2009. I Bet You Look Good on the Wall: Making the Invisible Computer Visible. Ambient Intelligence (2009), 196–205.
[68]
Vermeulen, J., Vanderhulst, G., Luyten, K. and Coninx, K. 2010. PervasiveCrystal: Asking and Answering Why and Why Not Questions about Pervasive Computing Applications. 2010 Sixth International Conference on Intelligent Environments (Jul. 2010), 271–276.
[69]
Weiser, M. and Brown, J.S. 1997. The Coming Age of Calm Technology. Beyond Calculation: The Next Fifty Years of Computing. P.J. Denning and R.M. Metcalfe, eds. Springer New York. 75–85.
[70]
West, M., Kraut, R. and Ei Chew, H. 2019. I'd blush if I could: closing gender divides in digital skills through education. UNESCO.
[71]
Winograd, T., Flores, F. and Flores, F.F. 1986. Understanding Computers and Cognition: A New Foundation for Design. Intellect Books.
[72]
Wu, Y., Rough, D., Bleakley, A., Edwards, J., Cooney, O., Doyle, P.R., Clark, L. and Cowan, B.R. 2020. See What I'm Saying? Comparing Intelligent Personal Assistant Use for Native and Non-Native Language Speakers. 22nd International Conference on Human-Computer Interaction with Mobile Devices and Services (Oldenburg Germany, Oct. 2020), 1–9.
[73]
Yang, R. and Newman, M.W. 2013. Learning from a Learning Thermostat: Lessons for Intelligent Systems for the Home. Proceedings of the 2013 ACM International Joint Conference on Pervasive and Ubiquitous Computing (New York, NY, USA, 2013), 93–102.
[74]
Yankelovich, N. 1996. How do users know what to say? interactions. 3, 6 (Dec. 1996), 32–43.
[75]
You Can Now Use Google Home as an Intercom: 2017. https://uk.pcmag.com/news-analysis/91988/you-can-now-use-google-home-as-an-intercom. Accessed: 2019-07-11.

Cited By

View all
  • (2023)Metaphors in Voice User Interfaces: A Slippery FishACM Transactions on Computer-Human Interaction10.1145/360932630:6(1-37)Online publication date: 25-Sep-2023
  • (2021)Machine Body Language: Expressing a Smart Speaker’s Activity with Intelligible Physical MotionProceedings of the 2021 ACM Designing Interactive Systems Conference10.1145/3461778.3462031(1403-1418)Online publication date: 28-Jun-2021
  • (2021)Two Cases for TracesProceedings of the ACM on Human-Computer Interaction10.1145/34492895:CSCW1(1-28)Online publication date: 22-Apr-2021

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences
OzCHI '20: Proceedings of the 32nd Australian Conference on Human-Computer Interaction
December 2020
764 pages
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 February 2021

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Breakdowns
  2. Intelligent Personal Assistants
  3. Intelligibility
  4. Shared Usage
  5. Smart Speakers
  6. Voice User Interfaces

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Funding Sources

Conference

OzCHI '20

Acceptance Rates

Overall Acceptance Rate 362 of 729 submissions, 50%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)36
  • Downloads (Last 6 weeks)2
Reflects downloads up to 16 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2023)Metaphors in Voice User Interfaces: A Slippery FishACM Transactions on Computer-Human Interaction10.1145/360932630:6(1-37)Online publication date: 25-Sep-2023
  • (2021)Machine Body Language: Expressing a Smart Speaker’s Activity with Intelligible Physical MotionProceedings of the 2021 ACM Designing Interactive Systems Conference10.1145/3461778.3462031(1403-1418)Online publication date: 28-Jun-2021
  • (2021)Two Cases for TracesProceedings of the ACM on Human-Computer Interaction10.1145/34492895:CSCW1(1-28)Online publication date: 22-Apr-2021

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media