Investigating Sign Language Interpreter Rendering and Guiding Methods in Virtual Reality 360-Degree Content
Article No.: 89, Pages 1 - 6
Abstract
As sign languages are often the native languages for members of the Deaf community, text captioning can be inaccessible, highlighting the importance of sign language interpretation. This research explores sign language in virtual reality 360-degree three degrees of freedom videos, exploring two rendering modes, fixed-position and always-visible, and two visual guiding methods, arrows and radar. Findings from testing with eight participants indicates that fixed-position rendering provides participants with a greater sense of presence than always-visible rendering, whilst always-visible rendering produces less of a blocking effect. Arrows appear more usable than radar for visually guiding participants to active speakers, including providing a higher level of sign language understanding. Future research is needed to validate these findings with six degrees of freedom content.
References
[1]
Belén Agualló, Anna Matamala, and Pilar Orero. 2018. From Disabilities to Capabilities: testing subtitles in immersive environments with end users. Hikma, 17, 195-220. Retrieved 13 September, 2022 from http://hdl.handle.net/10396/19462
[2]
Belén Agulló, Mario Montagud, and Isaac Fraile. 2019. Making interaction with virtual reality accessible: rendering and guiding methods for subtitles. Artificial Intelligence for Engineering Design, Analysis and Manufacturing, 33, 4, 416-428. https://doi.org/10.1017/S0890060419000362
[3]
Danielle Bragg, Meredith Ringel Morris, Christian Vogler, Raja Kushalnagar, Matt Huenerfauth, and Hernisa Kacorri. 2020. Sign Language Interfaces: Discussing the Field's Biggest Challenges. Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems, 1-5. https://doi.org/10.1145/3334480.3381053
[4]
John Brooke. 2013. SUS: a retrospective. Journal of usability studies, 8, 2, 29-40. Retrieved 13 September, 2022 from https://www.researchgate.net/profile/John-Brooke-6/publication/285811057_SUS_a_retrospective/links/5ee5c4a792851ce9e7e38a75/SUS-a-retrospective.pdf
[5]
Andy Brown, Jayson Turner, Jake Patterson, Anastasia Schmitz, Mike Armstrong and Maxine Glancy. 2017. Subtitles in 360-degree Video. Adjunct Publication of the 2017 ACM International Conference on Interactive Experiences for TV and Online Video, 3-8. https://doi.org/10.1145/3084289.3089915
[6]
Andy Brown, Jayson Turner, Jake Patterson, Anastasia Schmitz, Mike Armstrong, and Maxine Glancy. 2018. Exploring Subtitle Behaviour for 360 Video (WHP 330). Retrieved 13 September, 2022 from http://downloads.bbc.co.uk/rd/pubs/whp/whp-pdf-files/WHP330.pdf
[7]
Eunhee Chang, Hyun Taek Kim, and Byounghyun Yoo. 2020. Virtual Reality Sickness: A Review of Causes and Measurements. International Journal of Human–Computer Interaction, 36, 17, 1658-1682. https://doi.org/10.1080/10447318.2020.1778351.
[8]
Albert M. Cook, Janice Miller Polgar, Pedro Encarnação. 2020. Assistive Technologies: Principles & Practice. 5th edn. St. Louis, Missouri: Elsevier
[9]
James J. Cummings and Jeremy N. Bailenson. 2016. How Immersive Is Enough? A Meta-Analysis of the Effect of Immersive Technology on User Presence. Media psychology, 19, 2, 272-309. https://doi.org/10.1080/15213269.2015.1015740
[10]
Matjaž Debevc, Primož Kosec, and Andreas Holzinger. 2011. Improving multimodal web accessibility for deaf people: sign language interpreter module. Multimedia Tools and Applications, 54, 1, 181-199. https://doi.org/10.1007/s11042-010-0529-8
[11]
Department for Work and Pensions. 2022. Government backs vital British Sign Language Bill. [press release] 28 January. Retrieved 13 September, 2022 from https://www.gov.uk/government/news/government-backs-vital-british-sign-language-bill
[12]
Equality Act 2010, c. 15. Retrieved 13 September, 2022 from https://www.legislation.gov.uk/ukpga/2010/15/contents
[13]
FLLegal. 2022. Early Win for Deaf Plaintiff in VR Captioning Lawsuit. Retrieved 13 September, 2022 from https://www.lflegal.com/2022/04/vr-caption-lawsuit/
[14]
Chris J. Hughes and Mario Montagud. 2020. Accessibility in 360° video players. Multimedia tools and applications, 80, 20, 30993-31020. https://doi.org/10.1007/s11042-020-10088-0
[15]
Chris Hughes, Mario Montagud Climent, and Peter tho Pesch. 2019. Disruptive Approaches for Subtitling in Immersive Environments. Proceedings of the 2019 ACM International Conference on Interactive Experiences for TV and Online Video, 216-229. https://doi.org/10.1145/3317697.3325123
[16]
Dhruv Jain, Leah Findlater, Jamie Gilkeson, Benjamin Holland, Ramani Duraiswami, Dmitry Zotkin, Christian Vogler, and Jon E. Froehlich. 2015. Head-Mounted Display Visualizations to Support Sound Awareness for the Deaf and Hard of Hearing. Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, 241-250. https://doi.org/10.1145/2702123.2702393
[17]
Michael Jones, M. Jeannette Lawler, Eric Hintz, Nathan Bench, Fred Mangrubang, and Mallory Trullender. 2014. Head Mounted Displays and Deaf Children: Facilitating Sign Language in Challenging Learning Environments. Proceedings of the 2014 conference on Interaction design and children, 317-320. https://doi.org/10.1145/2593968.2610481
[18]
Hyun K. Kim, Jaehyun Park, Yeongcheol Choi, and Mungyeong Choe. 2018 Virtual reality sickness questionnaire (VRSQ): Motion sickness measurement index in a virtual reality environment. Applied ergonomics, 69, 66-73. https://doi.org/10.1016/j.apergo.2017.12.016
[19]
Yun Ling, Harold T. Nefs, Nexhmedin Morina, Ingrid Heynderickx, and Willem-Paul Brinkman. 2014. A Meta-Analysis on the Relationship between Self-Reported Presence and Anxiety in Virtual Reality Exposure Therapy for Anxiety Disorders. PLoS ONE 9, 5, 96144. https://doi.org/10.1371/journal.pone.0096144.
[20]
X.Z. Liu & D. Yan. 2007. Ageing and hearing loss. The Journal of Pathology: A Journal of the Pathological Society of Great Britain and Ireland, 211, 2, 188-197. https://doi.org/10.1002/path.2102
[21]
Wai Han Lo and Ka Lun Benjamin Cheng. 2020. Does virtual reality attract visitors? The mediating effect of presence on consumer response in virtual reality tourism advertising. Information Technology & Tourism 22, 4, 537–562. https://doi.org/10.1007/s40558-020-00190-2
[22]
Stephen J. Macdonald and John Clayton, 2013, Back to the future, disability and the digital divide. Disability & Society, 28, 5, 702-718. https://doi.org/10.1080/09687599.2012.732538
[23]
The Makaton Charity (2021) About Makaton. Retrieved 13 September, 2022 from: https://makaton.org/TMC/AboutMakaton.aspx
[24]
Matthew McConnell and Mary Ellen Foster. 2020. Two Dimensional Sign Language Agent. Proceedings of the 20th ACM International Conference on Intelligent Virtual Agents. 1-3. Retrieved 13 September, 2022 from https://dl.acm.org/doi/pdf/10.1145/3383652.3423898?casa_token=f5dN4OR-4z4AAAAA:hI4M4Pxk2BWZOXzkHsJdQ6g2LmPkIvfuNuui7ff_Wis5YUfvBYNb5rkqb4bzIbtTLf_Y4Ch4UFl9
[25]
Mario Montagud, Pilar Orero, and Anna Matamala. 2020. Culture 4 all: accessibility-enabled cultural experiences through immersive VR360 content. Personal and Ubiquitous Computing, 24, 6, 887-905.
[26]
Jaziar Radiantia, Tim A. Majchrzaka, Jennifer Fromm, and Isabell Wohlgenannt. 2020. A systematic review of immersive virtual reality applications for higher education: Design elements, lessons learned, and research agenda. Computers & Education, 147, 103778.
[27]
Gary E. Riccio and Thomas A. Stoffregen. 1991. An Ecological Theory of Motion Sickness and Postural Instability. Ecological Psychology, 3, 3, 195-240. Retrieved from: https://www.researchgate.net/profile/Thomas-Stoffregen/publication/239393320_An_Ecological_Theory_of_Motion_Sickness_and_Postural_Instability/links/00b4952a787f250971000000/An-Ecological-Theory-of-Motion-Sickness-and-Postural-Instability.pdf
[28]
Sylvia Rothe, Kim Tran, and Heinrich Hußmann. 2018. Dynamic Subtitles in Cinematic Virtual Reality. Proceedings of the 2018 ACM International Conference on Interactive Experiences for TV and Online Video, 209-214.
[29]
Marianne Schulze. 2010. Understanding The UN Convention On The Rights Of Persons With Disabilities. Retrieved from https://accessible-techcomm.org/wp-content/uploads/Understanding_The_UN.pdf
[30]
Valentin Schwind, Pascal Knierim, Nico Haas, and Niels Henze. 2019. Using Presence Questionnaires in Virtual Reality. Proceedings of the 2019 CHI conference on human factors in computing systems, 1-12.
[31]
Peter tho Pesch, Romain Bouqueau, and Mario Montagud. 2020. White Paper: Recommendations for immersive accessibility services, in press. Retrieved from https://imacproject.eu/wp-content/uploads/2020/05/ImAc_WhitePaper-submitted.pdf
[32]
W3C. 2021. Sign Languages. Retrieved from https://www.w3.org/WAI/media/av/sign-languages/
[33]
World Health Organization (WHO) (2021) Ageing and health. Retrieved from https://www.who.int/news-room/fact-sheets/detail/ageing-and-health
[34]
Séamas Weech, Sophie Kenny, and Michael Barnett-Cowan. 2019. Presence and Cybersickness in Virtual Reality Are Negatively Related: A Review. Frontiers in psychology, 10, 158.
Index Terms
- Investigating Sign Language Interpreter Rendering and Guiding Methods in Virtual Reality 360-Degree Content
Recommendations
Teaching ASL Signs using Signing Avatars and Immersive Learning in Virtual Reality
ASSETS '20: Proceedings of the 22nd International ACM SIGACCESS Conference on Computers and AccessibilityWe present here a new system, in which signing avatars (computer-animated virtual humans built from motion capture recordings) teach introductory American Sign Language (ASL) in an immersive virtual environment. The system is called Signing Avatars & ...
Nonverbal behavior of interviewers influences the competence ratings of observers in recruitment interviews: a study investigating social influence using 360-degree videos with virtual reality and 2D screen displays
AbstractThis study examined whether an interviewer’s nonverbal behavior influences observers’ competence ratings in a recruitment interview using 360-degree videos experienced with immersive virtual reality (VR-cardboard) and 2D screen displays. ...
Sign Language Recognition: Learning American Sign Language in a Virtual Environment
CHI EA '19: Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing SystemsIn this paper, we propose an approach, for sign language recognition, that makes use of a virtual reality headset to create an immersive environment. We show how features from data acquired by the Leap Motion controller, using an egocentric view, can be ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
October 2022
902 pages
Copyright © 2022 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 22 October 2022
Check for updates
Author Tags
Qualifiers
- Short-paper
- Research
- Refereed limited
Conference
ASSETS '22
Sponsor:
ASSETS '22: The 24th International ACM SIGACCESS Conference on Computers and Accessibility
October 23 - 26, 2022
Athens, Greece
Acceptance Rates
ASSETS '22 Paper Acceptance Rate 35 of 132 submissions, 27%;
Overall Acceptance Rate 436 of 1,556 submissions, 28%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 225Total Downloads
- Downloads (Last 12 months)90
- Downloads (Last 6 weeks)10
Reflects downloads up to 16 Nov 2024
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format