research-article

Voice User Interfaces in Schools: Co-designing for Inclusion with Visually-Impaired and Sighted Pupils

Authors:

Oussama Metatla,

Alison Oldfield,

Antonis Vafeas,

Sunny MiglaniAuthors Info & Claims

CHI '19: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems

Paper No.: 378, Pages 1 - 15

https://doi.org/10.1145/3290605.3300608

Published: 02 May 2019 Publication History

Abstract

Voice user interfaces (VUIs) are increasingly popular, particularly in homes. However, little research has investigated their potential in other settings, such as schools. We investigated how VUIs could support inclusive education, particularly for pupils with visual impairments (VIs). We organised focused discussions with educators at a school, with support staff from local authorities and, through bodystorming, with a class of 27 pupils. We then ran a series of co-design workshops with participants with mixed-visual abilities to design an educational VUI application. This provided insights into challenges faced by pupils with VIs in mainstream schools, and opened a space for educators, sighted and visually impaired pupils to reflect on and design for their shared learning experiences through VUIs. We present scenarios, a design space and an example application that show novel ways of using VUIs for inclusive education. We also reflect on co-designing with mixed-visual-ability groups in this space.

Supplementary Material

ZIP File (pn7444.zip)

Two PDF files; 1) a description of the Lego design language developed as part of the co-design work reported in this paper; 2) a technical specification of the Voxtopus application prototype that resulted from the co-design process

Download
4.58 MB

MP4 File (paper378.mp4)

Supplemental video

Download
19.36 MB

MP4 File (paper378p.mp4)

Preview video

Download
2.10 MB

References

[1]

Roberto Aldunate and Miguel Nussbaum. 2013. Teacher adoption of technology. Computers in Human Behavior 29, 3 (2013), 519--524.

Digital Library

[2]

Mohammed Ibrahim Alhojailan. 2012. Thematic analysis: A critical review of its process and evaluation. West East Journal of Social Sciences 1, 1 (2012), 39--47.

[3]

M Astrid, Nicole C Krämer, Jonathan Gratch, and Sin-Hwa Kang. 2010. "It doesn't matter what you are!" Explaining social effects of agents and avatars. Computers in Human Behavior 26, 6 (2010), 1641--1650.

Digital Library

[4]

Julie A Bardin and Sandra Lewis. 2008. A survey of the academic engagement of students with visual impairments in general education classes. Journal of Visual Impairment & Blindness 102, 8 (2008), 472.

[5]

Laura Benton and Hilary Johnson. 2015. Widening participation in technology design: A review of the involvement of children with special educational needs and disabilities. International Journal of Child-Computer Interaction 3 (2015), 23--40.

[6]

Stacy M Branham and Shaun K Kane. 2015. Collaborative accessibility: How blind and sighted companions co-create accessible home spaces. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. ACM, 2373--2382.

Digital Library

[7]

Emeline Brulé and Gilles Bailly. 2018. Taking into Account Sensory Knowledge: The Case of Geo-techologies for Children with Visual Impairments. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, 236.

Digital Library

[8]

Colin Burns, Eric Dishman, William Verplank, and Bud Lassiter. 1994. Actors, Hairdos &Amp; Videotape&Mdash;Informance Design. In Conference Companion on Human Factors in Computing Systems (CHI '94). ACM, New York, NY, USA, 119--120.

Digital Library

[9]

Andreea Danielescu and Gwen Christian. 2018. A Bot is Not a Polyglot: Designing Personalities for Multi-Lingual Conversational Agents. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, CS01.

Digital Library

[10]

David DeVault, Ron Artstein, Grace Benn, Teresa Dey, Ed Fast, Alesia Gainer, Kallirroi Georgila, Jon Gratch, Arno Hartholt, Margaux Lhommet, et al. 2014. SimSensei Kiosk: A virtual human interviewer for healthcare decision support. In Proceedings of the 2014 international conference on Autonomous agents and multi-agent systems. International Foundation for Autonomous Agents and Multiagent Systems, 1061--1068.

Digital Library

[11]

Peter Farrell, Alison Alborz, Andy Howes, and Diana Pearson. 2010. The impact of teaching assistants on improving pupils' academic achievement in mainstream schools: A review of the literature. Educational Review 62, 4 (2010), 435--448.

[12]

Daniel Fitton, Janet C Read, Gavin Sim, and Brendan Cassidy. 2018. Co-designing voice user interfaces with teenagers in the context of smart homes. In Proceedings of the 17th ACM Conference on Interaction Design and Children. ACM, 55--66.

Digital Library

[13]

German Flores and Roberto Manduchi. 2018. Easy Return: An App for Indoor Backtracking Assistance. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, 17. CHI 2019, May 4--9, 2019, Glasgow, Scotland UK O. Metatla et al.

Digital Library

[14]

Ashley Flower, Matthew K. Burns, and Nicole A. Bottsford - Miller. 2007. Meta-analysis of disability simulation research. Remedial and Special Education 28, 2 (2007), 72--79.

[15]

Christopher Frauenberger, Julia Makhaeva, and Katharina Spiel. 2017. Blending Methods: Developing Participatory Design Sessions for Autistic Children. In Proceedings of the 2017 Conference on Interaction Design and Children. ACM, 39--49.

Digital Library

[16]

Euan Freeman, Graham Wilson, Stephen Brewster, Gabriel Baud-Bovy, Charlotte Magnusson, and Hector Caltenco. 2017. Audible Beacons and Wearables in Schools: Helping Young Visually Impaired Children Play and Move Independently. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. ACM, 4146--4157.

Digital Library

[17]

Michael F. Giangreco. 2013. Teacher assistant supports in inclusive schools: Research, practices and alternatives. Australasian Journal of Special Education 37, 2 (2013), 93--106.

[18]

Marion Hersh. 2017. Classification framework for ICT-based learning technologies for disabled people. British Journal of Educational Technology 48, 3 (2017), 768--788.

[19]

Hernisa Kacorri, Eshed Ohn-Bar, Kris M Kitani, and Chieko Asakawa. 2018. Environmental Factors in Indoor Navigation Based on RealWorld Trajectories of Blind Users. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, 56.

Digital Library

[20]

Joseph'Jofish' Kaye, Joel Fischer, Jason Hong, Frank R Bentley, Cosmin Munteanu, Alexis Hiniker, Janice Y Tsai, and Tawfiq Ammari. 2018. Panel: Voice Assistants, UX Design and Research. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, panel01.

Digital Library

[21]

Alice Kerly, Richard Ellis, and Susan Bull. 2008. CALMsystem: a conversational agent for learner modelling. In Applications and Innovations in Intelligent Systems XV. Springer, 89--102.

[22]

Ewa Luger and Abigail Sellen. 2016. Like having a really bad PA: the gulf between user expectation and experience of conversational agents. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. ACM, 5286--5297.

Digital Library

[23]

Charlotte Magnusson, Héctor Caltenco, Sara Finocchietti, Giulia Cappagli, Graham Wilson, and Monica Gori. 2015. What do you like? early design explorations of sound and haptic preferences. In Proceedings of the 17th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct. ACM, 766--773.

Digital Library

[24]

Charlotte Magnusson, Per-Olof Hedvall, and Héctor Caltenco. 2018. Co-designing together with Persons with Visual Impairments. In Mobility of Visually Impaired People. Springer, 411--434.

[25]

Laura Malinverni, Joan Mora-Guiard, Vanesa Padillo, MariaAngeles Mairena, Amaia Hervás, and Narcis Pares. 2014. Participatory design strategies to enhance the creative contribution of children with special needs. In Proceedings of the 2014 conference on Interaction design and children. ACM, 85--94.

Digital Library

[26]

Joanne McElligott and Lieselotte Van Leeuwen. 2004. Designing sound tools and toys for blind and visually impaired children. In Proceedings of the 2004 conference on Interaction design and children: building a community. ACM, 65--72.

Digital Library

[27]

Michael McTear, Zoraida Callejas, and David Griol. 2016. The conversational interface: Talking to smart devices. Springer.

Digital Library

[28]

Oussama Metatla, Nick Bryan-Kinns, and Tony Stockman. 2018. "I Hear You": Understanding Awareness Information Exchange in an Audio-only Workspace. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, 546.

Digital Library

[29]

Oussama Metatla, Nick Bryan-Kinns, Tony Stockman, and Fiore Martin. 2015. Designing with and for people living with visual impairments: audio-tactile mock-ups, audio diaries and participatory prototyping. CoDesign 11, 1 (2015), 35--48.

[30]

Oussama Metatla and Clare Cullen. 2018. "Bursting the Assistance Bubble": Designing Inclusive Technology with Children with Mixed Visual Abilities. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, 346.

Digital Library

[31]

Oussama Metatla, Fiore Martin, Adam Parkinson, Nick Bryan-Kinns, Tony Stockman, and Atau Tanaka. 2016. Audio-haptic interfaces for digital audio workstations. Journal on Multimodal User Interfaces 10, 3 (01 Sep 2016), 247--258.

[32]

Oussama Metatla, Marcos Serrano, Christophe Jouffrais, Anja Thieme, Shaun Kane, Stacy Branham, Émeline Brulé, and Cynthia L Bennett. 2018. Inclusive Education Technologies: Emerging Opportunities for People with Visual Impairments. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, W13.

Digital Library

[33]

Yash Mittal, Paridhi Toshniwal, Sonal Sharma, Deepika Singhal, Ruchi Gupta, and Vinay Kumar Mittal. 2015. A voice-controlled multifunctional smart home automation system. In India Conference (INDICON), 2015 Annual IEEE. IEEE, 1--6.

[34]

Michelle R. Nario-Redmond, Dobromir Gospodinov, and Angela Cobb. 2017. Crip for a day: The unintended negative consequences of disability simulations. Rehabilitation Psychology 62, 3 (2017), 324--333.

[35]

Antti Oulasvirta, Esko Kurvinen, and Tomi Kankainen. 2003. Understanding contexts by being there: case studies in bodystorming. Personal and ubiquitous computing 7, 2 (2003), 125--134.

Digital Library

[36]

Rebecca Palmer, Pam Enderby, and Mark Hawley. 2007. Addressing the needs of speakers with longstanding dysarthria: computerized and traditional therapy compared. International journal of language & communication disorders 42, sup1 (2007), 61--79.

[37]

Martin Porcheron, Joel E Fischer, Stuart Reeves, and Sarah Sharples. 2018. Voice Interfaces in Everyday Life. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, 640.

Digital Library

[38]

François Portet, Michel Vacher, Caroline Golanski, Camille Roux, and Brigitte Meillon. 2013. Design and evaluation of a smart home voice interface for the elderly: acceptability and objection aspects. Personal and Ubiquitous Computing 17, 1 (2013), 127--144.

Digital Library

[39]

Alisha Pradhan, Kanika Mehta, and Leah Findlater. 2018. Accessibility Came by Accident: Use of Voice-Controlled Intelligent Personal Assistants by People with Disabilities. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, 459.

Digital Library

[40]

Amanda Purington, Jessie G Taft, Shruti Sannon, Natalya N Bazarova, and Samuel Hardman Taylor. 2017. Alexa is my new BFF: social roles, user satisfaction, and personification of the amazon echo. In Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems. ACM, 2853--2859.

Digital Library

[41]

RNIB. 2013. Key statistics on the prevalence and population of children and young people with vision impairment. (2013).

[42]

Nuzhah Gooda Sahib, Tony Stockman, Anastasios Tombros, and Oussama Metatla. 2013. Participatory design with blind users: a scenariobased approach. In IFIP Conference on Human-Computer Interaction. Springer, 685--701.

[43]

Eva-Lotta Sallnas, Jonas Moll, and Kerstin Severinson-Eklundh. 2007. Group work about geometrical concepts among blind and sighted pupils using haptic interfaces. In EuroHaptics Conference, 2007 and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. World Haptics 2007. Second Joint. IEEE, 330--335.

Digital Library

[44]

U Sharma and SJ Salend. 2016. Teaching Assistants in Inclusive Classrooms: A Systematic Analysis of the International Research. Australian Journal of Teacher Education 41, 8 (2016), 118--134.

[45]

Arielle M. Silverman, Jennifer S. Pitonyak, Ian K. Nelson, Patricia N. Matsuda, Deborah Kartin, and Ivan R. Molton. 2017. Instilling positive beliefs about disabilities: pilot testing a novel experiential learning activity for rehabilitation students. Disability and Rehabilitation 0 (2017), 1--6. Voice User Interfaces in Schools CHI 2019, May 4--9, 2019, Glasgow, Scotland UK

[46]

Miriam D Skjørten. 2001. Towards inclusion and enrichment. Education Special Needs Education: An Introduction. Oslo: Unipub Forlag (2001).

[47]

Iris Soute, Susanne Lagerström, and Panos Markopoulos. 2013. Rapid prototyping of outdoor games for children in an iterative design process. In Proceedings of the 12th International Conference on Interaction Design and Children. ACM, 74--83.

Digital Library

[48]

Tony Stockman and Oussama Metatla. 2008. The influence of screenreaders on web cognition. In Proceeding of Accessible design in the digital world conference (ADDW 2008), York, UK.

[49]

Stergios Tegos, Stavros Demetriadis, and Anastasios Karakostas. 2015. Promoting academically productive talk with conversational agent interventions in collaborative learning settings. Computers & Education 87 (2015), 309--325.

Digital Library

[50]

Anja Thieme, Cecily Morrison, Nicolas Villar, Martin Grayson, and Siân Lindley. 2017. Enabling Collaboration in Learning Computer Programing Inclusive of Children with Vision Impairments. In Proceedings of the 2017 Conference on Designing Interactive Systems. ACM, 739--752.

Digital Library

[51]

Laura Pfeifer Vardoulakis, Lazlo Ring, Barbara Barry, Candace L Sidner, and Timothy Bickmore. 2012. Designing relational agents as long term social companions for older adults. In International Conference on Intelligent Virtual Agents. Springer, 289--302.

Digital Library

[52]

Giacomo Vivanti, Ed Duncan, Geraldine Dawson, and Sally J Rogers. 2017. Facilitating Learning Through Peer Interactions and Social Participation. In Implementing the Group-Based Early Start Denver Model for Preschoolers with Autism. Springer, 87--99.

[53]

Amber Wagner, Ramaraju Rudraraju, Srinivasa Datla, Avishek Banerjee, Mandar Sudame, and Jeff Gray. 2012. Programming by voice: a hands-free approach for motorically challenged children. In CHI'12 Extended Abstracts on Human Factors in Computing Systems. ACM, 2087--2092.

Digital Library

[54]

Ru Zarin and Daniel Fallman. 2011. Through the troll forest: exploring tabletop interaction design for children with special cognitive needs. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 3319--3322.

Digital Library

[55]

Li Zhou, Amy T Parker, Derrick W Smith, and Nora Griffin-Shirley. 2011. Assistive technology for students with visual impairments: Challenges and needs in teachers' preparation programs and practice. J. Visual Impairment & Blindness 105, 4 (2011), 197.

Cited By

Seaborn KSawa YWatanabe M(2024)Coimagining the Future of Voice Assistants with Cultural SensitivityHuman Behavior and Emerging Technologies10.1155/2024/32387372024(1-21)Online publication date: 25-Mar-2024
https://doi.org/10.1155/2024/3238737
Sobel KDas MBehbakht SKientz J(2024)Incloodle-Classroom: Technology for Inclusive Joint Media Engagement in a Neurodiverse Kindergarten ClassroomACM Transactions on Computer-Human Interaction10.1145/367450631:3(1-45)Online publication date: 28-Jun-2024
https://dl.acm.org/doi/10.1145/3674506
Kameswaran VRobinson JSambasivan NAggarwal GMorris M(2024)Help and The Social Construction of Access: A Case-Study from IndiaProceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3663548.3675606(1-12)Online publication date: 27-Oct-2024
https://dl.acm.org/doi/10.1145/3663548.3675606
Show More Cited By

Index Terms

Voice User Interfaces in Schools: Co-designing for Inclusion with Visually-Impaired and Sighted Pupils
1. Applied computing
  1. Education
    1. Collaborative learning
2. Human-centered computing
  1. Interaction design
    1. Interaction design process and methods
      1. Participatory design

Recommendations

Robots for Inclusive Play: Co-designing an Educational Game With Visually Impaired and sighted Children
CHI '20: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems

Despite being included in mainstream schools, visually impaired children still face barriers to social engagement and participation. Games could potentially help, but games that cater for both visually impaired and sighted players are scarce. We used a ...
“Bursting the Assistance Bubble”: Designing Inclusive Technology with Children with Mixed Visual Abilities
CHI '18: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems

Children living with visual impairments (VIs) are increasingly educated in mainstream rather than special schools. But knowledge about the challenges they face in inclusive schooling environments and how to design technology to overcome them remains ...
Using Experience-Based Participatory Approach to Design Interactive Voice User Interfaces for Delivering Physical Activity Programs with Older Adults
HAI '23: Proceedings of the 11th International Conference on Human-Agent Interaction

Voice User Interfaces (VUIs) are popular among older adults, who find them easy to use and perceive them as social companions. However, there is a lack of research on voice-based applications to support physical activities for older adults. To address ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '19: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems

May 2019

9077 pages

ISBN:9781450359702

DOI:10.1145/3290605

General Chairs:
Stephen Brewster
University of Glasgow, Scotland, UK
,
Geraldine Fitzpatrick
TU Wien, Austria
,
Program Chairs:
Anna Cox
University College London, UK
,
Vassilis Kostakos
University of Melbourne, Australia

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 02 May 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Badges

Best Paper

Author Tags

Qualifiers

Research-article

Funding Sources

Engineering and Physical Sciences Research Council
Brigstow institute

Conference

CHI '19

Sponsor:

SIGCHI

CHI '19: CHI Conference on Human Factors in Computing Systems

May 4 - 9, 2019

Glasgow, Scotland Uk

Acceptance Rates

CHI '19 Paper Acceptance Rate 703 of 2,958 submissions, 24%;

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI '25

Sponsor:
sigchi

CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

66
Total Citations
View Citations
2,094
Total Downloads

Downloads (Last 12 months)287
Downloads (Last 6 weeks)34

Reflects downloads up to 21 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Seaborn KSawa YWatanabe M(2024)Coimagining the Future of Voice Assistants with Cultural SensitivityHuman Behavior and Emerging Technologies10.1155/2024/32387372024(1-21)Online publication date: 25-Mar-2024
https://doi.org/10.1155/2024/3238737
Sobel KDas MBehbakht SKientz J(2024)Incloodle-Classroom: Technology for Inclusive Joint Media Engagement in a Neurodiverse Kindergarten ClassroomACM Transactions on Computer-Human Interaction10.1145/367450631:3(1-45)Online publication date: 28-Jun-2024
https://dl.acm.org/doi/10.1145/3674506
Kameswaran VRobinson JSambasivan NAggarwal GMorris M(2024)Help and The Social Construction of Access: A Case-Study from IndiaProceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3663548.3675606(1-12)Online publication date: 27-Oct-2024
https://dl.acm.org/doi/10.1145/3663548.3675606
Seaborn KUrakami JPennefather PMiyake N(2024)Qualitative Approaches to Voice UXACM Computing Surveys10.1145/365866656:12(1-34)Online publication date: 20-Apr-2024
https://dl.acm.org/doi/10.1145/3658666
Ahire SSimon BRohs M(2024)WorkFit: Designing Proactive Voice Assistance for the Health and Well-Being of Knowledge WorkersProceedings of the 6th ACM Conference on Conversational User Interfaces10.1145/3640794.3665561(1-14)Online publication date: 8-Jul-2024
https://dl.acm.org/doi/10.1145/3640794.3665561
Jones KLeonards UMetatla O(2024)“I Don't Really Get Involved In That Way”: Investigating Blind and Visually Impaired Individuals' Experiences of Joint Attention with Sighted PeopleProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642940(1-16)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642940
Wen SPing SWang JLiang HXu XYan Y(2024)AdaptiveVoice: Cognitively Adaptive Voice Interface for Driving AssistanceProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642876(1-18)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642876
Alcaraz Valencia PGaytán-Lugo LSalazar M(2023)Activity Design for Students With Visual Disabilities Through an Auditive Notification and Voice Response InterfaceHandbook of Research on Innovative Frameworks and Inclusive Models for Online Learning10.4018/978-1-6684-9072-3.ch011(215-237)Online publication date: 18-Aug-2023
https://doi.org/10.4018/978-1-6684-9072-3.ch011
Sharma AThakur KKapoor DSingh K(2023)Designing Inclusive Learning EnvironmentsThe Impact and Importance of Instructional Design in the Educational Landscape10.4018/978-1-6684-8208-7.ch002(24-61)Online publication date: 30-Jun-2023
https://doi.org/10.4018/978-1-6684-8208-7.ch002
Delgado FYang SMadaio MYang Q(2023)The Participatory Turn in AI Design: Theoretical Foundations and the Current State of PracticeProceedings of the 3rd ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization10.1145/3617694.3623261(1-23)Online publication date: 30-Oct-2023
https://dl.acm.org/doi/10.1145/3617694.3623261
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents