research-article

The Impact of User Characteristics and Preferences on Performance with an Unfamiliar Voice User Interface

Authors:

Chelsea M. Myers,

Anushay Furqan,

Jichen ZhuAuthors Info & Claims

CHI '19: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems

Paper No.: 47, Pages 1 - 9

https://doi.org/10.1145/3290605.3300277

Published: 02 May 2019 Publication History

Abstract

Voice User Interfaces (VUIs) are increasing in popularity. However, their invisible nature with no or limited visuals makes it difficult for users to interact with unfamiliar VUIs. We analyze the impact of user characteristics and preferences on how users interact with a VUI-based calendar, DiscoverCal. While recent VUI studies analyze user behavior through self-reported data, we extend this research by analyzing both VUI usage data and self-reported data to observe correlations between both data types. Results from our user study (n=50) led to four key findings: 1) programming experience did not have a wide-spread impact on performance metrics while 2) assimilation bias did, 3) participants with more technical confidence exhibited a trial-and-error approach, and 4) desiring more guidance from our VUI correlated with performance metrics that indicate cautious users.

References

[1]

John Brooke. 1996. SUS-A quick and dirty usability scale. Usability evaluation in industry 189, 194 (1996), 4--7.

[2]

Janghee Cho. 2018. Mental Models and Home Virtual Assistants (HVAs). Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems - CHI '18 (2018), 1--6. The Impact of User Characteristics and Preferences on Performance with an Unfamiliar Voice User Interface CHI 2019, May 4--9, 2019, Glasgow, Scotland Uk

Digital Library

[3]

J. Chu-Carroll. 2000. MIMIC: An adaptive mixed initiative spoken dialogue system for information queries. Proceedings of the sixth conference on Applied natural language processing 6 (2000), 97--104.

Digital Library

[4]

Eric Corbett and Astrid Weber. 2016. What Can I Say?: Addressing User Experience Challenges of a Mobile Voice User Interface for Accessibility. Proceedings of the 18th International Conference on HumanComputer Interaction with Mobile Devices and Services (2016), 72--82.

Digital Library

[5]

Benjamin R Cowan, Nadia Pantidi, David Coyle, Kellie Morrissey, Peter Clarke, Sara Al-Shehri, David Earley, and Natasha Bandeira. 2017. "What can i help you with?": Infrequent Users' Experiences of Intelligent Personal Assistants. Proceedings of the 19th International Conference on Human-Computer Interaction with Mobile Devices and Services - MobileHCI '17 (2017), 1--12.

Digital Library

[6]

Janet Feigenspan, Christian Kästner, Jörg Liebig, Sven Apel, and Stefan Hanenberg. 2012. Measuring programming experience. In Program Comprehension (ICPC), 2012 IEEE 20th International Conference on. IEEE, 73--82.

[7]

Anushay Furqan, Chelsea Myers, and Jichen Zhu. 2017. Learnability through Adaptive Discovery Tools in Voice User Interfaces. Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems - CHI EA '17 (2017), 1617--1623.

Digital Library

[8]

Debjyoti Ghosh, Pin Sym Foong, Shan Zhang, and Shengdong Zhao. 2018. Assessing the Utility of the System Usability Scale for Evaluating Voice-based User Interfaces. Proceedings of the Sixth International Symposium of Chinese CHI on - ChineseCHI '18 (2018), 11--15.

Digital Library

[9]

Ido Guy. 2016. Searching by Talking: Analysis of Voice Queries on Mobile Web Search. Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval. ACM (2016), 35--44.

Digital Library

[10]

Jiepu Jiang, Wei Jeng, and Daqing He. 2013. How Do Users Respond to Voice Input Errors?: Lexical and Phonetic Query Reformulation in Voice Search. Proceedings of the 36th International ACM SIGIR Conference on Research and Development in Information Retrieval (2013), 143--152.

Digital Library

[11]

Jie Kang, Kyle Condiff, Shuo Chang, Joseph A. Konstan, Loren Terveen, and F. Maxwell Harper. 2017. Understanding How People Use Natural Language to Ask for Recommendations. Proceedings of the 11th ACM conference on Recommender systems (2017), 229--237.

Digital Library

[12]

Clare M. Karat, Christine Halverson, Daniel Horn, and John Karat. 1999. Patterns of entry and correction in large vocabulary continuous speech recognition systems. Proceedings of the SIGCHI conference on Human factors in computing systems: the CHI is the limit (1999), 568--575.

Digital Library

[13]

Laurent Karsenty and Valerie Botherel. 2005. Transparency strategies to help users handle system errors. Speech Communication 45, 3 SPEC. ISS. (2005), 305--324.

[14]

Diane J Litman and Shimei Pan. 1999. Empirically evaluating an adaptable spoken dialogue system. In UM99 User Modeling. Springer, 55--64.

Digital Library

[15]

Diane J Litman and Shimei Pan. 2000. Predicting and adapting to poor speech recognition in a spoken dialogue system. Proceedings of the National Conference on Artificial Intelligence (2000), 722--728.

Digital Library

[16]

Diane J Litman and Shimei Pan. 2002. Designing and evaluating an adaptive spoken dialogue system. User Modeling and User-Adapted Interaction 12, 2--3 (2002), 111--137.

Digital Library

[17]

Ewa Luger and Abigail Sellen. 2016. "Like Having a Really Bad PA": The Gulf between User Expectation and Experience of Conversational Agents. Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems - CHI '16 (2016), 5286--5297.

Digital Library

[18]

Christine Murad, Cosmin Munteanu, Leigh Clark, and Benjamin R Cowan. 2018. Design guidelines for hands-free speech interaction. In Proceedings of the 20th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct. ACM, 269--276.

Digital Library

[19]

Chelsea Myers, Anushay Furqan, Jessica Nebolsky, Karina Caro, and Jichen Zhu. 2018. Patterns for How Users Overcome Obstacles in Voice User Interfaces. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, 6.

Digital Library

[20]

Amanda Purington, Jessie G. Taft, Shruti Sannon, Natalya N. Bazarova, and Samuel Hardman Taylor. 2017. "Alexa is my new BFF". Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems - CHI EA '17 (2017), 2853--2859.

Digital Library

[21]

Aung Pyae and Tapani N. Joelsson. 2018. Investigating the usability and user experiences of voice user interface. Proceedings of the 20th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct - MobileHCI '18 (2018), 127--131.

Digital Library

[22]

Nicole Yankelovich. 1996. How do users know what to say? Interactions 3, 6 (1996), 32--43.

Digital Library

[23]

Yu Zhong, T V Raman, Casey Burkhardt, Fadi Biadsy, and Jeffrey P Bigham. 2014. JustSpeak: Enabling Universal Voice Control on Android. Proceedings of the 11th Web for All Conference on - W4A '14 (2014), 1--4.

Digital Library

[24]

Randall Ziman and Greg Walsh. 2018. Factors Affecting Seniors' Perceptions of Voice-enabled User Interfaces. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems (CHI EA '18). ACM, New York, NY, USA, Article LBW591, 6 pages.

Digital Library

Cited By

Deshmukh AChalmeta R(2024)User Experience and Usability of Voice User Interfaces: A Systematic Literature ReviewInformation10.3390/info1509057915:9(579)Online publication date: 19-Sep-2024
https://doi.org/10.3390/info15090579
Doi TNishikawa Y(2024)Usability Analysis of Smart Speakers from a Learnability Perspective for Novel UsersApplied System Innovation10.3390/asi70300367:3(36)Online publication date: 25-Apr-2024
https://doi.org/10.3390/asi7030036
Kalirai MWilliams AKuzminykh A(2024)Toward Faceted Skill Recommendation in Intelligent Personal AssistantsProceedings of the 29th International Conference on Intelligent User Interfaces10.1145/3640543.3645201(640-649)Online publication date: 18-Mar-2024
https://dl.acm.org/doi/10.1145/3640543.3645201
Show More Cited By

Index Terms

The Impact of User Characteristics and Preferences on Performance with an Unfamiliar Voice User Interface
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Empirical studies in HCI
    2. Interaction paradigms
      1. Natural language interfaces

Recommendations

Patterns for How Users Overcome Obstacles in Voice User Interfaces
CHI '18: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems

Voice User Interfaces (VUIs) are growing in popularity. However, even the most current VUIs regularly cause frustration for their users. Very few studies exist on what people do to overcome VUI problems they encounter, or how VUIs can be designed to aid ...
Investigating the usability and user experiences of voice user interface: a case of Google home smart speaker
MobileHCI '18: Proceedings of the 20th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct

Recently, commercial Voice User Interfaces (VUIs) have been introduced to the market (e.g. Amazon Echo and Google Home). Although they have drawn much attention from users, little is known about their usability, user experiences, and usefulness. In this ...
Towards a human-computer interaction model for voice user interfaces
CLIHC '19: Proceedings of the IX Latin American Conference on Human Computer Interaction

The user interaction with computer systems has evolved over the years, from Command Line Interfaces (CLIs), Graphics User Interfaces (GUIs), Natural User Interfaces (NUIs) and actually Voice User Interfaces (VUIs). The use of VUIs is increasingly common,...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '19: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems

May 2019

9077 pages

ISBN:9781450359702

DOI:10.1145/3290605

General Chairs:
Stephen Brewster
University of Glasgow, Scotland, UK
,
Geraldine Fitzpatrick
TU Wien, Austria
,
Program Chairs:
Anna Cox
University College London, UK
,
Vassilis Kostakos
University of Melbourne, Australia

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 02 May 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CHI '19

Sponsor:

SIGCHI

CHI '19: CHI Conference on Human Factors in Computing Systems

May 4 - 9, 2019

Glasgow, Scotland Uk

Acceptance Rates

CHI '19 Paper Acceptance Rate 703 of 2,958 submissions, 24%;

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

38
Total Citations
View Citations
1,433
Total Downloads

Downloads (Last 12 months)147
Downloads (Last 6 weeks)5

Reflects downloads up to 30 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Deshmukh AChalmeta R(2024)User Experience and Usability of Voice User Interfaces: A Systematic Literature ReviewInformation10.3390/info1509057915:9(579)Online publication date: 19-Sep-2024
https://doi.org/10.3390/info15090579
Doi TNishikawa Y(2024)Usability Analysis of Smart Speakers from a Learnability Perspective for Novel UsersApplied System Innovation10.3390/asi70300367:3(36)Online publication date: 25-Apr-2024
https://doi.org/10.3390/asi7030036
Kalirai MWilliams AKuzminykh A(2024)Toward Faceted Skill Recommendation in Intelligent Personal AssistantsProceedings of the 29th International Conference on Intelligent User Interfaces10.1145/3640543.3645201(640-649)Online publication date: 18-Mar-2024
https://dl.acm.org/doi/10.1145/3640543.3645201
Wen SPing SWang JLiang HXu XYan Y(2024)AdaptiveVoice: Cognitively Adaptive Voice Interface for Driving AssistanceProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642876(1-18)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642876
Ehsan UPassi SLiao QChan LLee IMuller MRiedl M(2024)The Who in XAI: How AI Background Shapes Perceptions of AI ExplanationsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642474(1-32)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642474
MOTTA IQUARESMA M(2023)“A realidade nem sempre é frustrante”: Alinhando os modelos mentais de usuários com as reais capacidades de Assistentes de Voz (AVs)Blucher Design Proceedings10.5151/ped2022-1943537(2897-2911)Online publication date: Feb-2023
https://doi.org/10.5151/ped2022-1943537
Chen QGong YLu Y(2023)User Experience of Digital Voice Assistant: Conceptualization and MeasurementACM Transactions on Computer-Human Interaction10.1145/362278231:1(1-35)Online publication date: 29-Nov-2023
https://dl.acm.org/doi/10.1145/3622782
Jensen JAshbrook D(2023)Exploring Audio Icons for Content-Based Navigation in Voice User InterfacesProceedings of the 5th International Conference on Conversational User Interfaces10.1145/3571884.3604302(1-9)Online publication date: 19-Jul-2023
https://dl.acm.org/doi/10.1145/3571884.3604302
Murad CMunteanu CCowan BClark LPorcheron MFischer JCandello HLangevin R(2023)Is CUI Design Ready Yet? A Workshop on Community Practices and Gaps in CUI Design & Resource DevelopmentProceedings of the 5th International Conference on Conversational User Interfaces10.1145/3571884.3597440(1-5)Online publication date: 19-Jul-2023
https://dl.acm.org/doi/10.1145/3571884.3597440
Murad CCandello HMunteanu C(2023)What’s The Talk on VUI Guidelines? A Meta-Analysis of Guidelines for Voice User Interface DesignProceedings of the 5th International Conference on Conversational User Interfaces10.1145/3571884.3597129(1-16)Online publication date: 19-Jul-2023
https://dl.acm.org/doi/10.1145/3571884.3597129
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents