Nothing Special   »   [go: up one dir, main page]

skip to main content
10.1145/3640543.3645202acmconferencesArticle/Chapter ViewAbstractPublication PagesiuiConference Proceedingsconference-collections
research-article
Open access

Impact of Voice Fidelity on Decision Making: A Potential Dark Pattern?

Published: 05 April 2024 Publication History

Abstract

Manipulative design in user interfaces (conceptualized as dark patterns) has emerged as a significant impediment to the ethical design of technology and a threat to user agency and freedom of choice. While previous research focused on exploring these patterns in the context of graphical user interfaces, the impact of speech has largely been overlooked. We conducted a listening test (N = 50) to elicit participants’ preferences regarding different synthetic voices that varied in terms of synthesis method (concatenative vs. neural) and prosodic qualities (speech pace and pitch variance), and then evaluated their impact in an online decision-making study (N = 101). Our results indicate a significant effect of voice qualities on the participant’s choices, independently from the content of the available options. Our results also indicate that the voice’s perceived engagement, ease of understanding, and domain fit directly translate to its impact on participants’ behavior in decision-making tasks.

References

[1]
Tawfiq Ammari, Jofish Kaye, Janice Y Tsai, and Frank Bentley. 2019. Music, search, and IoT: How people (really) use voice assistants. ACM Transactions on Computer-Human Interaction (TOCHI) 26, 3 (2019), 1–28.
[2]
Matthew P Aylett, Per Ola Kristensson, Steve Whittaker, and Yolanda Vazquez-Alvarez. 2014. None of a CHInd: relationship counselling for HCI and speech technology. In CHI’14 Extended Abstracts on Human Factors in Computing Systems. 749–760.
[3]
Matthew P Aylett and Yolanda Vazquez-Alvarez. 2020. Voice puppetry: Speech synthesis adventures in human centred ai. In Proceedings of the 25th International Conference on Intelligent User Interfaces Companion. 108–109.
[4]
Alan Baddeley. 1992. Working memory. Science 255, 5044 (1992), 556–559.
[5]
Pascal Belin, Patricia EG Bestelmeyer, Marianne Latinus, and Rebecca Watson. 2011. Understanding voice perception. British Journal of Psychology 102, 4 (2011), 711–725.
[6]
Pascal Belin, Bibi Boehme, and Phil McAleer. 2017. The sound of trustworthiness: Acoustic-based modulation of perceived voice personality. PloS one 12, 10 (2017), e0185651.
[7]
Shlomo Benartzi, John Beshears, Katherine L Milkman, Cass R Sunstein, Richard H Thaler, Maya Shankar, Will Tucker-Ray, William J Congdon, and Steven Galing. 2017. Should governments invest more in nudging?Psychological science 28, 8 (2017), 1041–1055.
[8]
James Betker. 2023. Better speech synthesis through scaling. arXiv preprint arXiv:2305.07243 (2023).
[9]
European Data Protection Board. 2022. Guidelines 3/2022 on Dark patterns in social media platform interfaces: How to recognise and avoid them.https://edpb.europa.eu/system/files/2022-03/edpb_03-2022_guidelines_on_dark_patterns_in_social_media_platform_interfaces_en.pdf Last accessed 9 January 2023.
[10]
Kerstin Bongard-Blanchy, Arianna Rossi, Salvador Rivas, Sophie Doublet, Vincent Koenig, and Gabriele Lenzini. 2021. ” I am Definitely Manipulated, Even When I am Aware of it. It’s Ridiculous!”-Dark Patterns from the End-User Perspective. In Designing Interactive Systems Conference 2021. 763–776.
[11]
Christoph Bösch, Benjamin Erb, Frank Kargl, Henning Kopp, and Stefan Pfattheicher. 2016. Tales from the Dark Side: Privacy Dark Strategies and Privacy Dark Patterns.Proc. Priv. Enhancing Technol. 2016, 4 (2016), 237–254.
[12]
Julia Cambre, Jessica Colnago, Jim Maddock, Janice Tsai, and Jofish Kaye. 2020. Choice of voices: A large-scale evaluation of text-to-speech voice quality for long-form content. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–13.
[13]
Akash Chaudhary, Jaivrat Saroha, Kyzyl Monteiro, Angus G Forbes, and Aman Parnami. 2022. “Are You Still Watching?”: Exploring Unintended User Behaviors and Dark Patterns on Video Streaming Platforms. In Designing Interactive Systems Conference. 776–791.
[14]
Dasom Choi, Daehyun Kwak, Minji Cho, and Sangsu Lee. 2020. " Nobody speaks that fast!" An empirical study of speech rate in conversational agents for people with vision impairments. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–13.
[15]
Council of European Union. 2022. Regulation (EU) 2022/2065 of the European Parliament and of the Council of 19 October 2022 on a Single Market For Digital Services and amending Directive 2000/31/EC (Digital Services Act). https://eur-lex.europa.eu/eli/reg/2022/2065/oj.
[16]
Hengchen Dai, Silvia Saccardo, Maria A Han, Lily Roh, Naveen Raja, Sitaram Vangala, Hardikkumar Modi, Shital Pandya, Michael Sloyan, and Daniel M Croymans. 2021. Behavioural nudges increase COVID-19 vaccinations. Nature 597, 7876 (2021), 404–409.
[17]
Kahneman Daniel. 2017. Thinking, fast and slow.
[18]
Valdemar Danry, Pat Pataranutaporn, Yaoli Mao, and Pattie Maes. 2023. Don’t Just Tell Me, Ask Me: AI Systems that Intelligently Frame Explanations as Questions Improve Human Logical Discernment Accuracy over Causal AI explanations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13.
[19]
Silvia De Conca. 2023. The present looks nothing like the Jetsons: Deceptive design in virtual assistants and the protection of the rights of users. Computer Law & Security Review 51 (2023), 105866.
[20]
Commission Nationale de l’Informatique et des Libertés. 2019. Shaping Choices in the Digital World.From dark patterns to data protection: the influence of ux/ui design on user empowerment. https://www.cnil.fr/sites/default/files/atoms/files/cnil_ip_report_06_shaping_choices_in_the_digital_world.pdf Last accessed 9 January 2023.
[21]
Linda Di Geronimo, Larissa Braz, Enrico Fregnan, Fabio Palomba, and Alberto Bacchelli. 2020. UI dark patterns and where to find them: a study on mobile applications and user perception. In Proceedings of the 2020 CHI conference on human factors in computing systems. 1–14.
[22]
Tiffany D Do, Ryan P McMahan, and Pamela J Wisniewski. 2022. A New Uncanny Valley? The Effects of Speech Fidelity and Human Listener Gender on Social Perceptions of a Virtual-Human Speaker. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. 1–11.
[23]
Mateusz Dubiel, Kerstin Bongard-Blanchy, Luis A Leiva, and Anastasia Sergeeva. 2023. Are you sure you want to order that? On Appropriateness of Voice-only Proactive Feedback Strategies. In Proceedings of the 5th International Conference on Conversational User Interfaces. 1–6.
[24]
Mateusz Dubiel, Sylvain Daronnat, and Luis A Leiva. 2022. Conversational Agents Trust Calibration: A User-Centred Perspective to Design. In Proceedings of the 4th Conference on Conversational User Interfaces. 1–6.
[25]
Mateusz Dubiel, Martin Halvey, Pilar Oplustil Gallegos, and Simon King. 2020. Persuasive synthetic speech: Voice perception and user behaviour. In Proceedings of the 2nd Conference on Conversational User Interfaces. 1–9.
[26]
Elizabeth Dula, Andres Rosero, and Elizabeth Phillips. 2023. Identifying Dark Patterns in Social Robot Behavior. In 2023 Systems and Information Engineering Design Symposium (SIEDS). IEEE, 7–12.
[27]
Bureau européen des unions de consommateurs. 2022. “Dark patterns” and the EU consumer law acquisition. https://www.beuc.eu/sites/default/files/publications/beuc-x-2022-013_dark_patters_paper.pdf Last accessed 9 January 2023.
[28]
Tanis Furst, Margaret Connors, Carole A Bisogni, Jeffery Sobal, and Laura Winter Falk. 1996. Food choice: a conceptual model of the process. Appetite 26, 3 (1996), 247–266.
[29]
Marcel Gohsen, Johannes Kiesel, Mariam Korashi, Jan Ehlers, and Benno Stein. 2023. Guiding Oral Conversations: How to Nudge Users Towards Asking Questions?. In Proceedings of the 2023 Conference on Human Information Interaction and Retrieval. 34–42.
[30]
Colin M Gray, Yubo Kou, Bryan Battles, Joseph Hoggatt, and Austin L Toombs. 2018. The dark (patterns) side of UX design. In Proceedings of the 2018 CHI conference on human factors in computing systems. 1–14.
[31]
Colin M Gray, Lorena Sanchez Chamorro, Ike Obi, and Ja-Nae Duane. 2023. Mapping the Landscape of Dark Patterns Scholarship: A Systematic Literature Review. In Designing Interactive Systems Conference. 188–193.
[32]
Colin M Gray, Cristiana Santos, and Nataliia Bielova. 2023. Towards a Preliminary Ontology of Dark Patterns Knowledge. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–9.
[33]
Johanna Gunawan, Amogh Pradeep, David Choffnes, Woodrow Hartzog, and Christo Wilson. 2021. A comparative study of dark patterns across web and mobile modalities. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–29.
[34]
Pelle Guldborg Hansen and Andreas Maaløe Jespersen. 2013. Nudge and the manipulation of choice: A framework for the responsible use of the nudge approach to behaviour change in public policy. European Journal of Risk Regulation 4, 1 (2013), 3–28.
[35]
Eddie Harmon-Jones and Judson Mills. 2019. An introduction to cognitive dissonance theory and an overview of current perspectives on the theory. (2019).
[36]
Tianzhi He and Farrokh Jazizadeh. 2022. Nudging Occupants for Energy-Saving through Voice-Based Proactive Virtual Assistants. In Construction Research Congress 2022. 402–411.
[37]
Lars Hornuf and Sonja Mangold. 2022. Digital dark nudges. In Diginomics Research Perspectives: The Role of Digitalization in Business and Society. Springer, 89–104.
[38]
Luiza Jarovsky. 2022. Dark Patterns in Personal Data Collection: Definition, Taxonomy and Lawfulness. Taxonomy and Lawfulness (March 1, 2022) (2022).
[39]
Daniel Kahneman, Olivier Sibony, and Cass R Sunstein. 2021. Noise: a flaw in human judgment. Hachette UK.
[40]
Mandy B Korpusik. 2019. Deep learning for spoken dialogue systems: application to nutrition. Ph. D. Dissertation. Massachusetts Institute of Technology.
[41]
Monica Kowalczyk, Johanna T Gunawan, David Choffnes, Daniel J Dubois, Woodrow Hartzog, and Christo Wilson. 2023. Understanding Dark Patterns in Home IoT Devices. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–27.
[42]
Philippe Laban, Elicia Ye, Srujay Korlakunta, John Canny, and Marti Hearst. 2022. Newspod: Automatic and interactive news podcasts. In 27th International Conference on Intelligent User Interfaces. 691–706.
[43]
Cherie Lacey and Catherine Caudwell. 2019. Cuteness as a ‘dark pattern’in home robots. In 2019 14th ACM/IEEE International Conference on Human-Robot Interaction (HRI). IEEE, 374–381.
[44]
Lorraine L Landais, Olga C Damman, Linda J Schoonmade, Danielle RM Timmermans, Evert ALM Verhagen, and Judith GM Jelsma. 2020. Choice architecture interventions to change physical activity and sedentary behavior: a systematic review of effects on intention, behavior and health outcomes during and after intervention. International Journal of Behavioral Nutrition and Physical Activity 17, 1 (2020), 1–37.
[45]
Jamie Luguri and Lior Jacob Strahilevitz. 2021. Shining a light on dark patterns. Journal of Legal Analysis 13, 1 (2021), 43–109.
[46]
Raju Maharjan, Kevin Doherty, Darius Adam Rohani, Per Bækgaard, and Jakob E Bardram. 2022. What’s Up With These Conversational Health Agents? From Users’ Critiques to Implications for Design. Frontiers in Digital Health 4 (2022), 840232.
[47]
Arunesh Mathur, Gunes Acar, Michael J Friedman, Eli Lucherini, Jonathan Mayer, Marshini Chetty, and Arvind Narayanan. 2019. Dark patterns at scale: Findings from a crawl of 11K shopping websites. Proceedings of the ACM on Human-Computer Interaction 3, CSCW (2019), 1–32.
[48]
Arunesh Mathur, Mihir Kshirsagar, and Jonathan Mayer. 2021. What makes a dark pattern... dark? Design attributes, normative considerations, and measurement methods. In Proceedings of the 2021 CHI conference on human factors in computing systems. 1–18.
[49]
Phil McAleer, Alexander Todorov, and Pascal Belin. 2014. How do you say ‘Hello’? Personality impressions from brief novel voices. PloS one 9, 3 (2014), e90779.
[50]
National Public Media. 2022. The Smart Audio Report | National Public Media — nationalpublicmedia.com. https://www.nationalpublicmedia.com/insights/reports/smart-audio-report/. [Accessed 05-Sep-2023].
[51]
Alberto Monge Roffarello, Kai Lukoff, and Luigi De Russis. 2023. Defining and Identifying Attention Capture Deceptive Designs in Digital Interfaces. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–19.
[52]
Oksana Mont, Matthias Lehner, and Eva Heiskanen. 2014. Nudging: A tool for sustainable behaviour?Naturvårdsverket.
[53]
Emi Moriuchi. 2019. Okay, Google!: An empirical study on voice assistants on consumer engagement and loyalty. Psychology & Marketing 36, 5 (2019), 489–501.
[54]
Nasim Motalebi, Eugene Cho, S Shyam Sundar, and Saeed Abdullah. 2019. Can alexa be your therapist? how back-channeling transforms smart-speakers to be active listeners. In Conference Companion Publication of the 2019 on Computer Supported Cooperative Work and Social Computing. 309–313.
[55]
John W Mullennix, Steven E Stern, Stephen J Wilson, and Corrie-lynn Dyson. 2003. Social perception of male and female computer synthesized speech. Computers in Human Behavior 19, 4 (2003), 407–424.
[56]
Anna Oleszkiewicz, Katarzyna Pisanski, Kinga Lachowicz-Tabaczek, and Agnieszka Sorokowska. 2017. Voice-based assessments of trustworthiness, competence, and warmth in blind and sighted adults. Psychonomic bulletin & review 24 (2017), 856–862.
[57]
Kentrell Owens, Johanna Gunawan, David Choffnes, Pardis Emami-Naeini, Tadayoshi Kohno, and Franziska Roesner. 2022. Exploring deceptive design patterns in voice interfaces. In Proceedings of the 2022 European Symposium on Usable Security. 64–78.
[58]
David A Puts, Alexander K Hill, Drew H Bailey, Robert S Walker, Drew Rendall, John R Wheatley, Lisa LM Welling, Khytam Dawood, Rodrigo Cárdenas, Robert P Burriss, 2016. Sexual selection on male vocal fundamental frequency in humans and other anthropoids. Proceedings of the Royal Society B: Biological Sciences 283, 1829 (2016), 20152830.
[59]
Emma Rodero. 2016. Influence of speech rate and information density on recognition: The moderate dynamic mechanism. Media Psychology 19, 2 (2016), 224–242.
[60]
Emma Rodero, Olatz Larrea, Isabel Rodríguez-de Dios, and Ignacio Lucas. 2022. The expressive balance effect: perception and physiological responses of prosody and gestures. Journal of language and social psychology 41, 6 (2022), 659–684.
[61]
Klaus R Scherer, Tom Johnstone, and Gundrun Klasmeyer. 2003. Vocal expression of emotion. Handbook of affective sciences (2003), 433–456.
[62]
Christoph Schild, Julia Stern, and Ingo Zettler. 2020. Linking men’s voice pitch to actual and perceived trustworthiness across domains. Behavioral Ecology 31, 1 (2020), 164–175.
[63]
Ji Youn Shin and Jina Huh-Yoo. 2020. Designing everyday conversational agents for managing health and wellness: a study of alexa skills reviews. In Proceedings of the 14th EAI International Conference on Pervasive Computing Technologies for Healthcare. 50–61.
[64]
Paul Slovic, Melissa L Finucane, Ellen Peters, and Donald G MacGregor. 2007. The affect heuristic. European journal of operational research 177, 3 (2007), 1333–1352.
[65]
Nisan Stiennon, Long Ouyang, Jeffrey Wu, Daniel Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul F Christiano. 2020. Learning to summarize with human feedback. Advances in Neural Information Processing Systems 33 (2020), 3008–3021.
[66]
Selina Jeanne Sutton, Paul Foulkes, David Kirk, and Shaun Lawson. 2019. Voice as a design material: Sociophonetic inspired design strategies in human-computer interaction. In Proceedings of the 2019 CHI conference on human factors in computing systems. 1–14.
[67]
Barnabas Szaszi, Anna Palinkas, Bence Palfi, Aba Szollosi, and Balazs Aczel. 2018. A systematic scoping review of the choice architecture movement: Toward understanding when and why nudges work. Journal of Behavioral Decision Making 31, 3 (2018), 355–366.
[68]
Anssi Tarkiainen and Sanna Sundqvist. 2009. Product involvement in organic food consumption: does ideology meet practice?Psychology & Marketing 26, 9 (2009), 844–863.
[69]
Vito Tassiello, Jack S Tillotson, and Alexandra S Rome. 2021. “Alexa, order me a pizza!”: The mediating role of psychological power in the consumer–voice assistant interaction. Psychology & Marketing 38, 7 (2021), 1069–1080.
[70]
Paul Taylor. 2009. Text-to-speech synthesis. Cambridge university press.
[71]
Richard H Thaler and Cass R Sunstein. 2009. Nudge: Improving decisions about health, wealth, and happiness. Penguin.
[72]
Ingo R Titze and Daniel W Martin. 1998. Principles of voice production.
[73]
Reha Tuncer, Anastasia Sergeeva, Kerstin Bongard-Blanchy, Verena Distler, Sophie Doublet, and Vincent Koenig. 2023. Running out of time (rs): effects of scarcity cues on perceived task load, perceived benevolence and user experience on e-commerce sites. Behaviour & Information Technology (2023), 1–19.
[74]
Iis Tussyadiah and Graham Miller. 2019. Nudged by a robot: Responses to agency and feedback. Annals of Tourism Research 78 (2019), 102752.
[75]
Anisha L Varghese and Elizabeth S Nilsen. 2020. Is that how you should talk to her? Using appropriate prosody affects adults’, but not children’s, judgments of communicators’ competence. Journal of Language and Social Psychology 39, 5-6 (2020), 738–750.
[76]
Sarah Theres Völkel, Daniel Buschek, Malin Eiband, Benjamin R Cowan, and Heinrich Hussmann. 2021. Eliciting and analysing users’ envisioned dialogues with perfect voice assistants. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–15.
[77]
Jovana Vukovic, Benedict C Jones, David R Feinberg, Lisa M DeBruine, Finlay G Smith, Lisa LM Welling, and Anthony C Little. 2011. Variation in perceptions of physical dominance and trustworthiness predicts individual differences in the effect of relationship context on women’s preferences for masculine pitch in men’s voices. British Journal of Psychology 102, 1 (2011), 37–48.
[78]
Aarron Walter and Jared M Spool. 2011. Designing for emotion. Vol. 9. A book apart New York.
[79]
Xian Wang, Lik-Hang Lee, Carlos Bermejo Fernandez, and Pan Hui. 2023. The Dark Side of Augmented Reality: Exploring Manipulative Designs in AR. International Journal of Human–Computer Interaction (2023), 1–16.
[80]
Vincent Cheow Sern Yeo, See-Kwong Goh, and Sajad Rezaei. 2017. Consumer experiences, attitude and behavioral intention toward online food delivery (OFD) services. Journal of Retailing and Consumer services 35 (2017), 150–162.
[81]
José P Zagal, Staffan Björk, and Chris Lewis. 2013. Dark patterns in the design of games. In Foundations of Digital Games 2013.
[82]
Nima Zargham, Michael Bonfert, Robert Porzel, Tanja Doring, and Rainer Malaka. 2021. Multi-Agent Voice Assistants: An Investigation of User Experience. In Proceedings of the 20th International Conference on Mobile and Ubiquitous Multimedia. 98–107.

Cited By

View all
  • (2024)Voicecraft: Designing Task-specific Voice Assistant PersonasProceedings of the 6th ACM Conference on Conversational User Interfaces10.1145/3640794.3670000(1-3)Online publication date: 8-Jul-2024
  • (2024)Examining Humanness as a Metaphor to Design Voice User InterfacesACM Conversational User Interfaces 202410.1145/3640794.3665535(1-15)Online publication date: 8-Jul-2024

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences
IUI '24: Proceedings of the 29th International Conference on Intelligent User Interfaces
March 2024
955 pages
ISBN:9798400705083
DOI:10.1145/3640543
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives International 4.0 License.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 April 2024

Check for updates

Author Tags

  1. Conversational Agents
  2. Dark Patterns
  3. Design Ethics
  4. Synthetic Speech

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Funding Sources

Conference

IUI '24
Sponsor:

Acceptance Rates

Overall Acceptance Rate 746 of 2,811 submissions, 27%

Upcoming Conference

IUI '25

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)379
  • Downloads (Last 6 weeks)102
Reflects downloads up to 24 Sep 2024

Other Metrics

Citations

Cited By

View all
  • (2024)Voicecraft: Designing Task-specific Voice Assistant PersonasProceedings of the 6th ACM Conference on Conversational User Interfaces10.1145/3640794.3670000(1-3)Online publication date: 8-Jul-2024
  • (2024)Examining Humanness as a Metaphor to Design Voice User InterfacesACM Conversational User Interfaces 202410.1145/3640794.3665535(1-15)Online publication date: 8-Jul-2024

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Get Access

Login options

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media