research-article

Speaking with My Screen Reader: Using Audio Fictions to Explore Conversational Access to Interfaces

Authors:

Mahika Phutane,

Crescentia Jung,

Shiri AzenkotAuthors Info & Claims

ASSETS '23: Proceedings of the 25th International ACM SIGACCESS Conference on Computers and Accessibility

Article No.: 53, Pages 1 - 18

https://doi.org/10.1145/3597638.3608404

Published: 22 October 2023 Publication History

Abstract

Conversational assistants, an inherently accessible mode of interaction for blind and low vision (BLV) individuals, offer opportunities to support nonvisual access in new ways. In this paper, we explore whether and how human-like conversations can support access to interfaces, advancing the impersonal linear access provided by screen readers today. We first interviewed 10 BLV participants about this approach, but found it difficult to situate conversations around a future technology. So we turned to a speculative design approach and created four audio fictions: pre-recorded dialogues between users and their hypothetical screen reader assistants wherein assistants assumed distinct roles: a friend, butler, expert, and caregiver. We presented the audio fictions to 14 BLV participants and found that personable conversations can meaningfully extend the screen reader experience. We observed a tension between AI adaptation and screen reader customization. Participants further expressed a need to maintain control at three distinct levels: granular cursor movement, screen representation, and task assistance. Through the lens of assistant roles, we address fundamental questions about anthropomorphizing CAs and assistive technology broadly.

Supplemental Material

MP3 File

Presentation Audio: Introduction, Creation, and Original Snippets of Audio Fictions Presentation Audio Transcript: Written Script of Audio Presentation

Download
6.91 MB

PDF File

Presentation Audio: Introduction, Creation, and Original Snippets of Audio Fictions Presentation Audio Transcript: Written Script of Audio Presentation

Download
42.28 KB

References

[1]

[n. d.]. Apple Accessibility - Vision. https://www.apple.com/accessibility/vision/

[2]

[n. d.]. Introducing ChatGPT. https://openai.com/blog/chatgpt

[3]

[n. d.]. JAWS® – Freedom Scientific. https://www.freedomscientific.com/products/software/jaws/

[4]

[n. d.]. AI Voice Generator with Realistic Text-to-Speech. https://play.ht/

[5]

[n. d.]. WebAIM: Screen Reader User Survey #8 Results. https://webaim.org/projects/screenreadersurvey8/

[6]

[n. d.]. WebGPT: Improving the factual accuracy of language models through web browsing. https://openai.com/research/webgpt

[7]

2017. Alexa will seem more human with breathing pauses and learning skills. https://www.engadget.com/amazon-2020-alexa-breathing-teach-voice-profiles-for-kids-172918631.html

[8]

2017. NVDA. https://www.nvaccess.org/download/

[9]

2021. Apple Voice Control. https://support.apple.com/en-us/HT210417

[10]

2023. Chatgpt plugins. https://openai.com/blog/chatgpt-plugins#browsing

[11]

Ali Abdolrahmani, Ravi Kuber, and Stacy M. Branham. 2018. “Siri Talks at You": An Empirical Investigation of Voice-Activated Personal Assistant (VAPA) Usage by Individuals Who Are Blind. In Proceedings of the 20th International ACM SIGACCESS Conference on Computers and Accessibility. ACM, Galway Ireland, 249–258. https://doi.org/10.1145/3234695.3236344

Digital Library

[12]

Ali Abdolrahmani, Rishin Mukkath, Ravi Kuber, and Stacy M Branham. 2020. Blind People Are Power Users: An Argument for Centering Blind Users in Design of Voice Interfaces. UMBC Student Collection (2020).

[13]

Faisal Ahmed, Yevgen Borodin, Yury Puzis, and I. V. Ramakrishnan. 2012. Why read if you can skim: towards enabling faster screen reading. In Proceedings of the International Cross-Disciplinary Conference on Web Accessibility - W4A ’12. ACM Press, Lyon, France, 1. https://doi.org/10.1145/2207016.2207052

Digital Library

[14]

Faisal Ahmed, Yevgen Borodin, Andrii Soviak, Muhammad Islam, IV Ramakrishnan, and Terri Hedgpeth. 2012. Accessible skimming: faster screen reading of web pages. In Proceedings of the 25th annual ACM symposium on User interface software and technology. 367–378.

Digital Library

[15]

Saleema Amershi, Dan Weld, Mihaela Vorvoreanu, Adam Fourney, Besmira Nushi, Penny Collisson, Jina Suh, Shamsi Iqbal, Paul N. Bennett, Kori Inkpen, Jaime Teevan, Ruth Kikin-Gil, and Eric Horvitz. 2019. Guidelines for Human-AI Interaction. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. ACM, Glasgow Scotland Uk, 1–13. https://doi.org/10.1145/3290605.3300233

Digital Library

[16]

Tawfiq Ammari, Jofish Kaye, Janice Y. Tsai, and Frank Bentley. 2019. Music, Search, and IoT: How People (Really) Use Voice Assistants. ACM Trans. Comput.-Hum. Interact. 26, 3, Article 17 (apr 2019), 28 pages. https://doi.org/10.1145/3311956

Digital Library

[17]

Aboubakr Aqle, Dena Al-Thani, and Ali Jaoua. 2022. Can search result summaries enhance the web search efficiency and experiences of the visually impaired users?Universal Access in the Information Society 21, 1 (March 2022), 171–192. https://doi.org/10.1007/s10209-020-00777-w

Digital Library

[18]

Vikas Ashok, Syed Masum Billah, Yevgen Borodin, and IV Ramakrishnan. 2019. Auto-Suggesting Browsing Actions for Personalized Web Screen Reading. In Proceedings of the 27th ACM Conference on User Modeling, Adaptation and Personalization (Larnaca, Cyprus) (UMAP ’19). Association for Computing Machinery, New York, NY, USA, 252–260. https://doi.org/10.1145/3320435.3320460

Digital Library

[19]

Vikas Ashok, Yevgen Borodin, Yury Puzis, and I. V. Ramakrishnan. 2015. Capti-speak: a speech-enabled web screen reader. In Proceedings of the 12th International Web for All Conference(W4A ’15). Association for Computing Machinery, New York, NY, USA, 1–10. https://doi.org/10.1145/2745555.2746660

Digital Library

[20]

Marcos Baez, Florian Daniel, and Fabio Casati. 2020. Conversational web interaction: proposal of a dialog-based natural language interaction paradigm for the web., 94–110 pages.

[21]

Cynthia L. Bennett, Erin Brady, and Stacy M. Branham. 2018. Interdependence as a Frame for Assistive Technology Research and Design. In Proceedings of the 20th International ACM SIGACCESS Conference on Computers and Accessibility(ASSETS ’18). Association for Computing Machinery, New York, NY, USA, 161–173. https://doi.org/10.1145/3234695.3236348

Digital Library

[22]

Jeffrey P. Bigham, Anna C. Cavender, Jeremy T. Brudvik, Jacob O. Wobbrock, and Richard E. Lander. 2007. WebinSitu: a comparative analysis of blind and sighted browsing behavior. In Proceedings of the 9th international ACM SIGACCESS conference on Computers and accessibility - Assets ’07. ACM Press, Tempe, Arizona, USA, 51. https://doi.org/10.1145/1296843.1296854

Digital Library

[23]

Jeffrey P. Bigham, Irene Lin, and Saiph Savage. 2017. The Effects of "Not Knowing What You Don’t Know" on Web Accessibility for Blind Web Users. In Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility (Baltimore, Maryland, USA) (ASSETS ’17). Association for Computing Machinery, New York, NY, USA, 101–109. https://doi.org/10.1145/3132525.3132533

Digital Library

[24]

Julian Bleecker. 2022. Design Fiction. In Machine Learning and the City. John Wiley & Sons, Ltd, Chapter 24, 561–578. https://doi.org/10.1002/9781119815075.ch47

[25]

Yevgen Borodin, Jeffrey P. Bigham, Glenn Dausch, and I. V. Ramakrishnan. 2010. More than meets the eye: a survey of screen-reader browsing strategies. In Proceedings of the 2010 International Cross Disciplinary Conference on Web Accessibility (W4A) - W4A ’10. ACM Press, Raleigh, North Carolina, 1. https://doi.org/10.1145/1805986.1806005

Digital Library

[26]

Michael Braun, Anja Mainz, Ronee Chadowitz, Bastian Pfleging, and Florian Alt. 2019. At Your Service: Designing Voice Assistant Personalities to Improve Automotive User Interfaces. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI ’19). Association for Computing Machinery, New York, NY, USA, 1–11. https://doi.org/10.1145/3290605.3300270

Digital Library

[27]

Virginia Braun and Victoria Clarke. 2006. Using Thematic Analysis in Psychology. Qualitative Research in Psychology 3, 2 (Jan. 2006), 77–101. https://doi.org/10.1191/1478088706qp063oa

[28]

Virginia Braun and Victoria Clarke. 2019. Reflecting on Reflexive Thematic Analysis. Qualitative Research in Sport, Exercise and Health 11, 4 (Aug. 2019), 589–597. https://doi.org/10.1080/2159676X.2019.1628806

[29]

David J. Brin. 2023. 50% of U.S. consumers use voice search daily. https://upcity.com/experts/consumers-and-voice-search-study/

[30]

Zoraida Callejas, Ramón López-Cózar, Nieves Ábalos, and David Griol. 2011. Affective Conversational Agents: The Role of Personality and Emotion in Spoken Interactions.

[31]

Julia Cambre, Alex C Williams, Afsaneh Razi, Ian Bicking, Abraham Wallin, Janice Tsai, Chinmay Kulkarni, and Jofish Kaye. 2021. Firefox Voice: An Open and Extensible Voice Assistant Built Upon the Web. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. ACM, Yokohama Japan, 1–18. https://doi.org/10.1145/3411764.3445409

Digital Library

[32]

Dasom Choi, Daehyun Kwak, Minji Cho, and Sangsu Lee. 2020. "Nobody Speaks That Fast!" An Empirical Study of Speech Rate in Conversational Agents for People with Vision Impairments. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3313831.3376569

Digital Library

[33]

Saheli Roy Choudhury. [n. d.]. Microsoft researchers are trying to humanize virtual assistants by studying multilingual speech. https://www.cnbc.com/2017/05/29/microsoft-is-studying-multilingual-speech-to-humanize-virtual-assistants-like-cortana.html

[34]

Wei-Ming Chung and Rung-Huei Liang. 2021. Listening Is Believing: Exploring the Value of Sounds in an Audio Drama Board Game for Shaping Technology Futures. International Journal of Dsign 15, 2 (2021), 55–68. http://www.ijdesign.org/index.php/IJDesign/article/view/4013

[35]

Nazli Cila. 2022. Designing Human-Agent Collaborations: Commitment, Responsiveness, and Support. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (New Orleans, LA, USA) (CHI ’22). Association for Computing Machinery, New York, NY, USA, Article 420, 18 pages. https://doi.org/10.1145/3491102.3517500

Digital Library

[36]

Sylvain Daronnat, Leif Azzopardi, Martin Halvey, and Mateusz Dubiel. 2020. Impact of Agent Reliability and Predictability on Trust in Real Time Human-Agent Collaboration. In Proceedings of the 8th International Conference on Human-Agent Interaction(HAI ’20). Association for Computing Machinery, New York, NY, USA, 131–139. https://doi.org/10.1145/3406499.3415063

Digital Library

[37]

Audrey Desjardins, Cayla Key, Heidi R. Biggs, and Kelsey Aschenbeck. 2019. Bespoke Booklets: A Method for Situated Co-Speculation. In Proceedings of the 2019 on Designing Interactive Systems Conference(DIS ’19). Association for Computing Machinery, New York, NY, USA, 697–709. https://doi.org/10.1145/3322276.3322311

Digital Library

[38]

Lynn Dombrowski, Ellie Harmon, and Sarah Fox. 2016. Social Justice-Oriented Interaction Design: Outlining Key Design Strategies and Commitments. In Proceedings of the 2016 ACM Conference on Designing Interactive Systems(DIS ’16). Association for Computing Machinery, New York, NY, USA, 656–671. https://doi.org/10.1145/2901790.2901861

Digital Library

[39]

Kees Dorst. 2011. The Core of ‘Design Thinking’ and Its Application. Design Studies 32, 6 (Nov. 2011), 521–532. https://doi.org/10.1016/j.destud.2011.07.006

[40]

Philip R. Doyle, Justin Edwards, Odile Dumbleton, Leigh Clark, and Benjamin R. Cowan. 2019. Mapping Perceptions of Humanness in Intelligent Personal Assistant Interaction(MobileHCI ’19). Association for Computing Machinery, New York, NY, USA, Article 5, 12 pages. https://doi.org/10.1145/3338286.3340116

Digital Library

[41]

Anthony Dunne and Fiona Raby. 2013. Speculative Everything: Design, Fiction, and Social Dreaming. MIT Press.

Digital Library

[42]

Danyang Fan, Alexa F. Siu, Hrishikesh V. Rao, Gene S-H Kim, Xavier Vazquez, Lucy Greco, Sile O’Modhrain, and Sean Follmer. 2022. The Accessibility of Data Visualizations on the Web for Screen Reader Users: Practices and Experiences During COVID-19. ACM Transactions on Accessible Computing (Aug. 2022). https://doi.org/10.1145/3557899 Just Accepted.

Digital Library

[43]

Luciano Floridi, Josh Cowls, Monica Beltrametti, Raja Chatila, Patrice Chazerand, Virginia Dignum, Christoph Luetge, Robert Madelin, Ugo Pagallo, Francesca Rossi, Burkhard Schafer, Peggy Valcke, and Effy Vayena. 2018. AI4People—An Ethical Framework for a Good AI Society: Opportunities, Risks, Principles, and Recommendations. Minds and Machines 28, 4 (Dec. 2018), 689–707. https://doi.org/10.1007/s11023-018-9482-5

Digital Library

[44]

Asbjørn Følstad, Marita Skjuve, and Petter Bae Brandtzaeg. 2019. Different Chatbots for Different Purposes: Towards a Typology of Chatbots to Understand Interaction Design. In Internet Science, Svetlana S. Bodrunova, Olessia Koltsova, Asbjørn Følstad, Harry Halpin, Polina Kolozaridi, Leonid Yuldashev, Anna Smoliarova, and Heiko Niedermayer (Eds.). Springer International Publishing, Cham, 145–156.

[45]

Prathik Gadde and Davide Bolchini. 2014. From screen reading to aural glancing: towards instant access to key page sections. In Proceedings of the 16th international ACM SIGACCESS conference on Computers & accessibility - ASSETS ’14. ACM Press, Rochester, New York, USA, 67–74. https://doi.org/10.1145/2661334.2661363

Digital Library

[46]

J. Goetz, S. Kiesler, and A. Powers. 2003. Matching robot appearance and behavior to tasks to improve human-robot cooperation. In The 12th IEEE International Workshop on Robot and Human Interactive Communication, 2003. Proceedings. ROMAN 2003.55–60. https://doi.org/10.1109/ROMAN.2003.1251796

[47]

Kristen Graham, Abby LaChance, and Jolie B. Wormwood. 2022. Bystander Intervention in Intimate Partner Violence: An Audio Vignette Study of Heterosexual, Gay, and Lesbian Dating Partners. Journal of Interpersonal Violence (July 2022), 08862605221111412. https://doi.org/10.1177/08862605221111412

[48]

Jonathan Grudin and Richard Jacques. 2019. Chatbots, Humbots, and the Quest for Artificial General Intelligence. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI ’19). Association for Computing Machinery, New York, NY, USA, 1–11. https://doi.org/10.1145/3290605.3300439

Digital Library

[49]

João Guerreiro and Daniel Gonçalves. 2015. Faster Text-to-Speeches: Enhancing Blind People’s Information Scanning with Faster Concurrent Speech. In Proceedings of the 17th International ACM SIGACCESS Conference on Computers & Accessibility - ASSETS ’15. ACM Press, Lisbon, Portugal, 3–11. https://doi.org/10.1145/2700648.2809840

Digital Library

[50]

Sheon Han. 2022. The hidden history of screen readers. https://www.theverge.com/23203911/screen-readers-history-blind-henter-curran-teh-nvda

[51]

Simon Harper and Neha Patel. 2005. Gist summaries for visually impaired surfers. In Proceedings of the 7th international ACM SIGACCESS conference on Computers and accessibility - Assets ’05. ACM Press, Baltimore, MD, USA, 90. https://doi.org/10.1145/1090785.1090804

Digital Library

[52]

Fan He and Catherine M. Burns. 2022. A Battle of Voices: A Study of the Relationship between Driving Experience, Driving Style, and In-Vehicle Voice Assistant Character. In Proceedings of the 14th International Conference on Automotive User Interfaces and Interactive Vehicular Applications(AutomotiveUI ’22). Association for Computing Machinery, New York, NY, USA, 236–242. https://doi.org/10.1145/3543174.3546845

Digital Library

[53]

W3C Web Accessibility Initiative (WAI). [n. d.]. WCAG 2 Overview. https://www.w3.org/WAI/standards-guidelines/wcag/

[54]

Muhammad Asiful Islam, Faisal Ahmed, Yevgen Borodin, and I. V. Ramakrishnan. 2011. Tightly Coupling Visual and Linguistic Features for Enriching Audio-Based Web Browsing Experience. In Proceedings of the 20th ACM International Conference on Information and Knowledge Management (Glasgow, Scotland, UK) (CIKM ’11). Association for Computing Machinery, New York, NY, USA, 2085–2088. https://doi.org/10.1145/2063576.2063896

Digital Library

[55]

Crescentia Jung, Shubham Mehta, Atharva Kulkarni, Yuhang Zhao, and Yea-Seul Kim. 2021. Communicating Visualizations without Visuals: Investigation of Visualization Alternative Text for People with Visual Impairments. https://doi.org/10.48550/arXiv.2108.03657 arXiv:2108.03657 [cs].

[56]

Pranav Khadpe, Ranjay Krishna, Li Fei-Fei, Jeffrey T. Hancock, and Michael S. Bernstein. 2020. Conceptual Metaphors Impact Perceptions of Human-AI Collaboration. Proceedings of the ACM on Human-Computer Interaction 4, CSCW2 (Oct. 2020), 163:1–163:26. https://doi.org/10.1145/3415234

Digital Library

[57]

Rushil Khurana, Duncan McIsaac, Elliot Lockerman, and Jennifer Mankoff. 2018. Nonvisual Interaction Techniques at the Keyboard Surface. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, Montreal QC Canada, 1–12. https://doi.org/10.1145/3173574.3173585

Digital Library

[58]

Hankyung Kim, Dong Yoon Koh, Gaeun Lee, Jung-Mi Park, and Youn-kyung Lim. 2019. Designing Personalities of Conversational Agents. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI EA ’19). Association for Computing Machinery, New York, NY, USA, 1–6. https://doi.org/10.1145/3290607.3312887

Digital Library

[59]

Lucy Kimbell. 2011. Rethinking Design Thinking: Part I. Design and Culture 3, 3 (Nov. 2011), 285–306. https://doi.org/10.2752/175470811X13071166525216

[60]

Jonathan Lazar, Aaron Allen, Jason Kleinman, and Chris Malarkey. 2007. What Frustrates Screen Reader Users on the Web: A Study of 100 Blind Users. International Journal of HumanComputer Interaction (2007), 247–269.

[61]

Calvin A. Liang, Sean A. Munson, and Julie A. Kientz. 2021. Embracing Four Tensions in Human-Computer Interaction Research with Marginalized People. ACM Transactions on Computer-Human Interaction 28, 2 (April 2021), 14:1–14:47. https://doi.org/10.1145/3443686

Digital Library

[62]

E Liberman-Pincu and T. Oron-Gilad. 2021. Impacting the Perception of Socially Assistive Robots- Evaluating the Effect of Visual Qualities among Children. In 2021 30th IEEE International Conference on Robot & Human Interactive Communication (RO-MAN) (Vancouver, BC, Canada). IEEE Press, 612–618. https://doi.org/10.1109/RO-MAN50785.2021.9515458

Digital Library

[63]

Ela Liberman-Pincu, Yisrael Parmet, and Tal Oron-Gilad. 2023. Judging a Socially Assistive Robot by Its Cover: The Effect of Body Structure, Outline, and Color on Users’ Perception. J. Hum.-Robot Interact. 12, 2, Article 23 (apr 2023), 26 pages. https://doi.org/10.1145/3571717

Digital Library

[64]

Siân E. Lindley, Sam Meek, Abigail Sellen, and Richard Harper. 2012. "It’s Simply Integral to What I Do": Enquiries into How the Web is Weaved into Everyday Life. In Proceedings of the 21st International Conference on World Wide Web (Lyon, France) (WWW ’12). Association for Computing Machinery, New York, NY, USA, 1067–1076. https://doi.org/10.1145/2187836.2187979

Digital Library

[65]

Wing-Yue Geoffrey Louie, Derek McColl, and Goldie Nejat. 2014. Acceptance and Attitudes Toward a Human-like Socially Assistive Robot by Older Adults. Assistive Technology 26, 3 (2014), 140–150. https://doi.org/10.1080/10400435.2013.869703 26131794.

[66]

Darren Lunn, Simon Harper, and Sean Bechhofer. 2011. Identifying Behavioral Strategies of Visually Impaired Users to Improve Access to Web Content. ACM Transactions on Accessible Computing 3, 4 (April 2011), 13:1–13:35. https://doi.org/10.1145/1952388.1952390

Digital Library

[67]

Margaret McRorie, Ian Sneddon, Gary McKeown, Elisabetta Bevacqua, Etienne de Sevin, and Catherine Pelachaud. 2012. Evaluation of Four Designed Virtual Agent Personalities. IEEE Transactions on Affective Computing 3, 3 (2012), 311–322. https://doi.org/10.1109/T-AFFC.2011.38

Digital Library

[68]

Rishabh Mehrotra, A Hassan Awadallah, Ahmed E Kholy, and Imed Zitouni. 2017. Hey Cortana! Exploring the use cases of a Desktop based Digital Assistant.

[69]

Hisashi Miyashita, Daisuke Sato, Hironobu Takagi, and Chieko Asakawa. 2007. Aibrowser for multimedia: introducing multimedia content accessibility for visually impaired users. In Proceedings of the 9th international ACM SIGACCESS conference on Computers and accessibility(Assets ’07). Association for Computing Machinery, New York, NY, USA, 91–98. https://doi.org/10.1145/1296843.1296860

Digital Library

[70]

Kate Moran. [n. d.]. The Four Dimensions of Tone of Voice. https://www.nngroup.com/articles/tone-of-voice-dimensions/

[71]

Giulio Mori, Maria Claudia Buzzi, Marina Buzzi, Barbara Leporini, and Victor MR Penichet. 2011. Collaborative editing for all: the google docs example. In Universal Access in Human-Computer Interaction. Applications and Services: 6th International Conference, UAHCI 2011, Held as Part of HCI International 2011, Orlando, FL, USA, July 9-14, 2011, Proceedings, Part IV 6. Springer, 165–174.

[72]

Emma Murphy, Ravi Kuber, Graham McAllister, Philip Strain, and Wai Yu. 2008. An empirical investigation into the difficulties experienced by visually impaired Internet users. Universal Access in the Information Society 7, 1 (April 2008), 79–91. https://doi.org/10.1007/s10209-007-0098-4

Digital Library

[73]

Elizabeth D Mynatt and Gerhard Weber. 1994. Nonvisual presentation of graphical user interfaces: contrasting two approaches. In Proceedings of the SIGCHI conference on Human factors in computing systems. 166–172.

Digital Library

[74]

Clifford Nass and Scott Brave. 2007. How Voice Activates and Advances the Human-Computer Relationship. In Wired for Speech.

[75]

Clifford Nass, Jonathan Steuer, and Ellen R. Tauber. 1994. Computers are social actors. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems(CHI ’94). Association for Computing Machinery, New York, NY, USA, 72–78. https://doi.org/10.1145/191666.191703

Digital Library

[76]

Danielle Marie Olson and Yan Xu. 2021. Building Trust Over Time in Human-Agent Relationships. In Proceedings of the 9th International Conference on Human-Agent Interaction(HAI ’21). Association for Computing Machinery, New York, NY, USA, 193–201. https://doi.org/10.1145/3472307.3484178

Digital Library

[77]

David Pierce. [n. d.]. How Apple Finally Made Siri Sound More Human. Wired ([n. d.]). https://www.wired.com/story/how-apple-finally-made-siri-sound-more-human/ Section: tags.

[78]

E. Pontelli, D. Gillan, W. Xiong, E. Saad, G. Gupta, and A. I. Karshmer. 2002. Navigation of HTML tables, frames, and XML fragments. In Proceedings of the fifth international ACM conference on Assistive technologies(Assets ’02). Association for Computing Machinery, New York, NY, USA, 25–32. https://doi.org/10.1145/638249.638256

Digital Library

[79]

Atieh Poushneh. 2021. Humanizing voice assistant: The impact of voice assistant personality on consumers’ attitudes and behaviors. Journal of Retailing and Consumer Services 58 (Jan. 2021), 102283. https://doi.org/10.1016/j.jretconser.2020.102283

[80]

Christopher Power. 2012. Guidelines are only half of the story: accessibility problems encountered by blind users on the web. (2012), 10.

[81]

Alisha Pradhan, Leah Findlater, and Amanda Lazar. 2019. “Phantom Friend" or “Just a Box with Information": Personification and Ontological Categorization of Smart Speaker-Based Voice Assistants by Older Adults. Proc. ACM Hum.-Comput. Interact. 3, CSCW, Article 214 (nov 2019), 21 pages. https://doi.org/10.1145/3359316

Digital Library

[82]

Alisha Pradhan, Kanika Mehta, and Leah Findlater. 2018. "Accessibility Came by Accident": Use of Voice-Controlled Intelligent Personal Assistants by People with Disabilities. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems(CHI ’18). Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3173574.3174033

Digital Library

[83]

Amanda Purington, Jessie G. Taft, Shruti Sannon, Natalya N. Bazarova, and Samuel Hardman Taylor. 2017. "Alexa is my new BFF": Social Roles, User Satisfaction, and Personification of the Amazon Echo. In Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems. ACM, Denver Colorado USA, 2853–2859. https://doi.org/10.1145/3027063.3053246

Digital Library

[84]

Ronda Ringfort-Felner, Matthias Laschke, Shadan Sadeghian, and Marc Hassenzahl. 2022. Kiro: A Design Fiction to Explore Social Conversation with Voice Assistants. Proc. ACM Hum.-Comput. Interact., Article 33 (jan 2022), 21 pages. https://doi.org/10.1145/3492852

Digital Library

[85]

Nuzhah Gooda Sahib, Anastasios Tombros, and Tony Stockman. 2012. A comparative analysis of the information-seeking behavior of visually impaired and sighted searchers. Journal of the American Society for Information Science and Technology 63, 2 (2012), 377–391. https://doi.org/10.1002/asi.21696

Digital Library

[86]

S Sandhya and K.A. Sumithra Devi. 2011. Accessibility evaluation of websites using screen reader. In 2011 7th International Conference on Next Generation Web Services Practices. 338–341. https://doi.org/10.1109/NWeSP.2011.6088201

[87]

Ari Schlesinger, W. Keith Edwards, and Rebecca E. Grinter. 2017. Intersectional HCI: Engaging Identity through Gender, Race, and Class. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems(CHI ’17). Association for Computing Machinery, New York, NY, USA, 5412–5427. https://doi.org/10.1145/3025453.3025766

Digital Library

[88]

John G. Schoeberlein and Yuanqiong Wang. 2012. Accessible Collaborative Writing for Persons Who Are Blind: A Usability Study. In Proceedings of the 14th International ACM SIGACCESS Conference on Computers and Accessibility (Boulder, Colorado, USA) (ASSETS ’12). Association for Computing Machinery, New York, NY, USA, 267–268. https://doi.org/10.1145/2384916.2384986

Digital Library

[89]

Richard S Schwerdtfeger. 1991. Making the GUI talk. Byte (1991), 118–128.

[90]

William Seymour and Max Van Kleek. 2020. Does Siri Have a Soul? Exploring Voice Assistants Through Shinto Design Fictions. https://doi.org/10.1145/3334480.3381809

Digital Library

[91]

Kristen Shinohara and Jacob O. Wobbrock. 2016. Self-Conscious or Self-Confident? A Diary Study Conceptualizing the Social Accessibility of Assistive Technology. ACM Trans. Access. Comput. 8, 2, Article 5 (jan 2016), 31 pages. https://doi.org/10.1145/2827857

Digital Library

[92]

Alexa F. Siu, Danyang Fan, Gene S-H Kim, Hrishikesh V. Rao, Xavier Vazquez, Sile O’Modhrain, and Sean Follmer. 2021. COVID-19 Highlights the Issues Facing Blind and Visually Impaired People in Accessing Data on the Web(W4A ’21). Association for Computing Machinery, New York, NY, USA, Article 11, 15 pages. https://doi.org/10.1145/3430263.3452432

Digital Library

[93]

Marie Louise Juul Søndergaard and Lone Koefoed Hansen. 2016. PeriodShare: A Bloody Design Fiction. In Proceedings of the 9th Nordic Conference on Human-Computer Interaction(NordiCHI ’16). Association for Computing Machinery, New York, NY, USA, 1–6. https://doi.org/10.1145/2971485.2996748

Digital Library

[94]

Abigale Stangl, Meredith Ringel Morris, and Danna Gurari. 2020. "Person, Shoes, Tree. Is the Person Naked?" What People with Vision Impairments Want in Image Descriptions. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. ACM, Honolulu HI USA, 1–13. https://doi.org/10.1145/3313831.3376404

Digital Library

[95]

Abigale Stangl, Emma Sadjo, Pardis Emami-Naeini, Yang Wang, Danna Gurari, and Leah Findlater. 2023. “Dump it, Destroy it, Send it to Data Heaven”: Blind People’s Expectations for Visual Privacy in Visual Assistance Technologies. In Proceedings of the 20th International Web for All Conference(W4A ’23). Association for Computing Machinery, New York, NY, USA, 134–147. https://doi.org/10.1145/3587281.3587296

Digital Library

[96]

Tony Stockman and Oussama Metatla. 2008. The Influence of Screen-Readers on Web Cognition. (01 2008).

[97]

Hironobu Takagi, Shin Saito, Kentarou Fukuda, and Chieko Asakawa. 2007. Analysis of Navigability of Web Applications for Improving Blind Usability. ACM Trans. Comput.-Hum. Interact. 14, 3 (sep 2007), 13–es. https://doi.org/10.1145/1279700.1279703

Digital Library

[98]

Adriana Tapus and Maja Mataric. 2008. Socially Assistive Robots: The Link between Personality, Empathy, Physiological Signals, and Task Performance.AAAI Spring Symposium - Technical Report, 133–140.

[99]

Lorainne Tudor Car, Dhakshenya Ardhithy Dhinagaran, Bhone Myint Kyaw, Tobias Kowatsch, Shafiq Joty, Yin-Leng Theng, and Rifat Atun. 2020. Conversational agents in health care: scoping review and conceptual analysis. Journal of medical Internet research 22, 8 (2020), e17158.

[100]

Pooja Upadhyay. 2020. Comparing Non-Visual and Visual Information Foraging on The Web. In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI EA ’20). Association for Computing Machinery, New York, NY, USA, 1–8. https://doi.org/10.1145/3334480.3383025

Digital Library

[101]

Markel Vigo and Simon Harper. 2013. Coping tactics employed by visually disabled users on the web. International Journal of Human-Computer Studies 71, 11 (Nov. 2013), 1013–1025. https://doi.org/10.1016/j.ijhcs.2013.08.002

Digital Library

[102]

James Vlahos. 2019. Talk to Me: How Voice Computing Will Transform the Way We Live, Work, and Think. Houghton Mifflin Harcourt. Google-Books-ID: HuaLDwAAQBAJ.

[103]

Sarah Theres Völkel, Daniel Buschek, Malin Eiband, Benjamin R. Cowan, and Heinrich Hussmann. 2021. Eliciting and Analysing Users’ Envisioned Dialogues with Perfect Voice Assistants. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (Yokohama, Japan) (CHI ’21). Association for Computing Machinery, New York, NY, USA, Article 254, 15 pages. https://doi.org/10.1145/3411764.3445536

Digital Library

[104]

Sarah Theres Völkel, Ramona Schödel, Daniel Buschek, Clemens Stachl, Verena Winterhalter, Markus Bühner, and Heinrich Hussmann. 2020. Developing a Personality Model for Speech-Based Conversational Agents Using the Psycholexical Approach. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–14. https://doi.org/10.1145/3313831.3376210

Digital Library

[105]

Alexandra Vtyurina, Adam Fourney, Meredith Ringel Morris, Leah Findlater, and Ryen W. White. 2019. VERSE: Bridging Screen Readers and Voice Assistants for Enhanced Eyes-Free Web Search. In The 21st International ACM SIGACCESS Conference on Computers and Accessibility. ACM, Pittsburgh PA USA, 414–426. https://doi.org/10.1145/3308561.3353773

Digital Library

[106]

Ruolin Wang, Zixuan Chen, Mingrui Ray Zhang, Zhaoheng Li, Zhixiu Liu, Zihan Dang, Chun Yu, and Xiang ’Anthony’ Chen. 2021. Revamp: Enhancing Accessible Information Seeking Experience of Online Shopping for Blind or Low Vision Users. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (Yokohama, Japan) (CHI ’21). Association for Computing Machinery, New York, NY, USA, Article 494, 14 pages. https://doi.org/10.1145/3411764.3445547

Digital Library

[107]

Richmond Y. Wong and Vera Khovanskaya. 2018. Speculative Design in HCI: From Corporate Imaginations to Critical Orientations. In New Directions in Third Wave Human-Computer Interaction: Volume 2 - Methodologies, Michael Filimowicz and Veronika Tzankova (Eds.). Springer International Publishing, Cham, 175–202. https://doi.org/10.1007/978-3-319-73374-6_10

[108]

Lotus Zhang, Jingyao Shao, Augustina Ao Liu, Lucy Jiang, Abigale Stangl, Adam Fourney, Meredith Ringel Morris, and Leah Findlater. 2022. Exploring Interactive Sound Design for Auditory Websites. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (New Orleans, LA, USA) (CHI ’22). Association for Computing Machinery, New York, NY, USA, Article 222, 16 pages. https://doi.org/10.1145/3491102.3517695

Digital Library

[109]

Qiaoning Zhang, Matthew L Lee, and Scott Carter. 2022. You Complete Me: Human-AI Teams and Complementary Expertise. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems(CHI ’22). Association for Computing Machinery, New York, NY, USA, 1–28. https://doi.org/10.1145/3491102.3517791

Digital Library

[110]

Yu Zhong, T. V. Raman, Casey Burkhardt, Fadi Biadsy, and Jeffrey P. Bigham. 2014. JustSpeak: enabling universal voice control on Android. In Proceedings of the 11th Web for All Conference(W4A ’14). Association for Computing Machinery, New York, NY, USA, 1–4. https://doi.org/10.1145/2596695.2596720

Digital Library

[111]

Shaojian Zhu, Daisuke Sato, Hironobu Takagi, and Chieko Asakawa. 2010. Sasayaki: an augmented voice-based web browsing experience. In Proceedings of the 12th international ACM SIGACCESS conference on Computers and accessibility(ASSETS ’10). Association for Computing Machinery, New York, NY, USA, 279–280. https://doi.org/10.1145/1878803.1878870

Digital Library

Cited By

Kodandaram SUckun UBi XRamakrishnan IAshok V(2024)Enabling Uniform Computer Interaction Experience for Blind Users through Large Language ModelsProceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3663548.3675605(1-14)Online publication date: 27-Oct-2024
https://dl.acm.org/doi/10.1145/3663548.3675605
Perera M(2024)Enhancing Productivity Applications for People who are Blind using AI AssistantsExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3638180(1-6)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3638180
Deriba FWubineh BTeka M(2024)Review of AI Technologies for Enhancing the Lives of Visually Impaired Individuals: Applications, Outcomes, and Future Directions2024 International Conference on Information and Communication Technology for Development for Africa (ICT4DA)10.1109/ICT4DA62874.2024.10777268(241-246)Online publication date: 18-Nov-2024
https://doi.org/10.1109/ICT4DA62874.2024.10777268

Index Terms

Speaking with My Screen Reader: Using Audio Fictions to Explore Conversational Access to Interfaces
1. Human-centered computing
  1. Accessibility
  2. Human computer interaction (HCI)
    1. HCI design and evaluation methods
      1. User studies
    2. Interaction paradigms
      1. Natural language interfaces

Recommendations

Chart Reader: Accessible Visualization Experiences Designed with Screen Reader Users
CHI '23: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems

Even though screen readers are a core accessibility tool for blind and low vision individuals (BLVIs), most visualizations are incompatible with screen readers. To improve accessible visualization experiences, we partnered with 10 BLV screen reader users ...
Conversational Web Interaction: Proposal of a Dialog-Based Natural Language Interaction Paradigm for the Web
Chatbot Research and Design
Abstract
This paper lays the foundation for a new delivery paradigm for web-accessible content and functionality, i.e., conversational interaction. Instead of asking users to read text, click through links and type on the keyboard, the vision is to enable ...
Conversational Agents: Acting on the Wave of Research and Development
CHI EA '19: Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems

In the last five years, work on software that interacts with people via typed or spoken natural language, called chatbots, intelligent assistants, social bots, virtual companions, non-human players, and so on, increased dramatically. Chatbots burst into ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ASSETS '23: Proceedings of the 25th International ACM SIGACCESS Conference on Computers and Accessibility

October 2023

1163 pages

ISBN:9798400702204

DOI:10.1145/3597638

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGACCESS: ACM Special Interest Group on Accessible Computing

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 22 October 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

ASSETS '23

Sponsor:

SIGACCESS

ASSETS '23: The 25th International ACM SIGACCESS Conference on Computers and Accessibility

October 22 - 25, 2023

NY, New York, USA

Acceptance Rates

ASSETS '23 Paper Acceptance Rate 55 of 182 submissions, 30%;

Overall Acceptance Rate 436 of 1,556 submissions, 28%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
373
Total Downloads

Downloads (Last 12 months)251
Downloads (Last 6 weeks)21

Reflects downloads up to 17 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Kodandaram SUckun UBi XRamakrishnan IAshok V(2024)Enabling Uniform Computer Interaction Experience for Blind Users through Large Language ModelsProceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3663548.3675605(1-14)Online publication date: 27-Oct-2024
https://dl.acm.org/doi/10.1145/3663548.3675605
Perera M(2024)Enhancing Productivity Applications for People who are Blind using AI AssistantsExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3638180(1-6)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3638180
Deriba FWubineh BTeka M(2024)Review of AI Technologies for Enhancing the Lives of Visually Impaired Individuals: Applications, Outcomes, and Future Directions2024 International Conference on Information and Communication Technology for Development for Africa (ICT4DA)10.1109/ICT4DA62874.2024.10777268(241-246)Online publication date: 18-Nov-2024
https://doi.org/10.1109/ICT4DA62874.2024.10777268

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View Table of Conten