AI-Enabled Smart Glasses for People with Severe Vision Impairments
Article No.: 3, Page 1
Abstract
Over the last decade, there has been significant research on how smart assistive devices with artificial intelligence (AI) built into them can assist people with severe vision impairments to comprehend their surroundings. These devices come in various forms such as smartphone applications, smart-glasses, and smart canes. Smart glasses have gained popularity lately due to recent technological advancements, as well as their natural position in front of the user's eyes. However, there has been limited research to understand how people with severe vision impairments would prefer to interact with them. The objective of this project is to investigate the use of AI-enabled smart-glasses to aid individuals with severe vision impairments. The research aims to comprehend the differences between the types of research conducted by researchers and the needs and desires of the community. The study will utilise a Design Thinking approach and involve vision-impaired users throughout the project by utilising co-design methods. The research will involve the development of smart-glass application prototypes through iterative case studies with individuals who are blind, have low-vision, and have cerebral vision impairment (CVI). The goal is to understand the users' preferred interaction model from their lived experiences. The outcome of the research will also be a software architecture that enables people with severe vision impairments to seamlessly access information about their surroundings.
References
[1]
2023. Empowering your vision. https://www.nueyes.com/
[2]
2023. Envision glasses. https://www.letsenvision.com/glasses
[3]
2023. Seeing AI app from Microsoft. https://www.microsoft.com/en-us/ai/seeing-ai
[4]
2023. Strap tech. https://strap.tech/home
[5]
2023. Vision buddy - television watching for visually impaired. https://visionbuddy.com/
[6]
Ali Abdolrahmani, Maya Howes Gupta, Mei-Lian Vader, Ravi Kuber, and Stacy Branham. 2021. Towards More Transactional Voice Assistants: Investigating the Potential for a Multimodal Voice-Activated Indoor Navigation Assistant for Blind and Sighted Travelers. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1--16.
[7]
Ali Abdolrahmani, Kevin M Storer, Antony Rishin Mukkath Roy, Ravi Kuber, and Stacy M Branham. 2020. Blind leading the sighted: drawing design insights from blind users towards more productivity-oriented voice interfaces. ACM Transactions on Accessible Computing (TACCESS) 12, 4 (2020), 1--35.
[8]
Younes Akbari, Hanadi Hassen, Nandhini Subramanian, Jayakanth Kunhoth, Somaya Al-Maadeed, and Wael Alhajyaseen. 2020. A vision-based zebra crossing detection method for people with visual impairments. In 2020 IEEE International Conference on Informatics, IoT, and Enabling Technologies (ICIoT). IEEE, 118--123.
[9]
Mojisola Rachael Akinsiku, Brendan Ubochi, and Waliu Apena. 2020. Smart Walking Stick for the Blind with Text Image Recognition. Journal of Electrical Engineering, Electronics, Control and Computer Science 6, 4 (2020), 17--22.
[10]
Erick Javier Argüello Prada and Lina María Santacruz Forero. 2022. A belt-like assistive device for visually impaired people: Toward a more collaborative approach. Cogent Engineering 9, 1 (2022), 2048440.
[11]
Christopher R Bennett, Emma S Bailin, Timothy K Gottlieb, Corinna M Bauer, Peter J Bex, and Lotfi B Merabet. 2018. Assessing visual search performance in ocular compared to cerebral visual impairment using a virtual reality simulation of human dynamic movement. In Proceedings of the Technology, Mind, and Society. 1--6.
[12]
Christopher R Bennett, Emma S Bailin, Timothy K Gottlieb, Corinna M Bauer, Peter J Bex, and Lotfi B Merabet. 2018. Virtual reality based assessment of static object visual search in ocular compared to cerebral visual impairment. In Universal Access in Human-Computer Interaction. Virtual, Augmented, and Intelligent Environments: 12th International Conference, UAHCI 2018, Held as Part of HCI International 2018, Las Vegas, NV, USA, July 15-20, 2018, Proceedings, Part II 12. Springer, 28--38.
[13]
Christopher R Bennett, Corinna M Bauer, Peter J Bex, Davide Bottari, and Lotfi B Merabet. 2021. Visual search performance in cerebral visual impairment is associated with altered alpha band oscillations. Neuropsychologia 161 (2021), 108011.
[14]
Carlos Bermejo, Tristan Braud, Ji Yang, Shayan Mirjafari, Bowen Shi, Yu Xiao, and Pan Hui. 2020. VIMES: A Wearable Memory Assistance System for Automatic Information Retrieval. In Proceedings of the 28th ACM International Conference on Multimedia (Seattle, WA, USA) (MM '20). Association for Computing Machinery, New York, NY, USA, 3191--3200.
[15]
Jay S Bhatia, Nimit K Vasavat, Manali U Maniyar, Neha N Doshi, and Ruhina Karani. 2022. Audio-Based Staircase Navigation System for Visually Impaired. In Machine Learning and Autonomous Systems: Proceedings of ICMLAS 2021. Springer, 411--424.
[16]
Faith A Birnbaum, Steven A Hackley, and Lenworth N Johnson. 2015. Enhancing visual performance in individuals with cortical visual impairment (homonymous hemianopsia): Tapping into blindsight. Journal of Medical Hypotheses and Ideas 9, 2 (2015), S8--S13.
[17]
Alain Boldini, John-Ross Rizzo, and Maurizio Porfiri. 2020. A piezoelectric-based advanced wearable: obstacle avoidance for the visually impaired built into a backpack. In Nano-, Bio-, Info-Tech Sensors, and 3D Systems IV, Vol. 11378. SPIE, 1137806.
[18]
Stacy E Butler, Adele Crudden, William K Sansing, and BJ LeJeune. 2002. Employment barriers: Access to assistive technology and research needs. Journal of Visual Impairment & Blindness 96, 9 (2002), 664--667.
[19]
Qingqing Cao, Prerna Khanna, Nicholas D. Lane, and Aruna Balasubramanian. 2022. MobiVQA: Efficient On-Device Visual Question Answering. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 6, 2, Article 44 (jul 2022), 23 pages.
[20]
Roee Cohen. 2023. Experience the power of assistive technology with Orcam's AI devices. https://www.orcam.com/en-us/home
[21]
Pei Du and Nirupama Bulusu. 2021. An automated AR-based annotation tool for indoor navigation for visually impaired people. In Proceedings of the 23rd International ACM SIGACCESS Conference on Computers and Accessibility. 1--4.
[22]
Paul S. Duckett and Rebekah Pratt. 2001. The Researched Opinions on Research: Visually impaired people and visual impairment research. Disability & Society 16, 6 (2001), 815--835. arXiv:https://doi.org/10.1080/09687590120083976
[23]
MR Everingham, BT Thomas, T Troscianko, et al. 1999. Head-mounted mobility aid for low vision using scene classification techniques. The International Journal of Virtual Reality 3, 4 (1999), 3.
[24]
Bhanuka Gamage, Thanh-Toan Do, Nicholas Seow Chiang Price, Arthur Lowery, and Kim Marriott. 2023. What do Blind and Low-Vision People Really Want from Assistive Smart Devices? Comparison of the Literature with a Focus Study. In Proceedings of the 25th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '23). Association for Computing Machinery, New York, NY, USA, Article 30, 21 pages.
[25]
Eugenie Golubova, Sandra D Starke, Michael D Crossland, and James S Wolffsohn. 2021. Design considerations for the ideal low vision aid: insights from de-brief interviews following a real-world recording study. Ophthalmic and Physiological Optics 41, 2 (March 2021), 266--280.
[26]
Ravi Gorapudi, Puvvula Priya Darsini, Uddanti Naga Kavya, and Oggu Jaswanthi. 2020. Product label, obstacle and sign boards detection for visually impaired people. In International Conference of Advance Research & Innovation (ICARI).
[27]
Renan Guarese, Franklin Bastidas, João Becker, Mariane Giambastiani, Yhonatan Iquiapaza, Lennon Macedo, Luciana Nedel, Anderson Maciel, Fabio Zambetta, and Ron van Schyndel. 2021. Cooking in the Dark: Exploring Spatial Audio as MR Assistive Technology for the Visually Impaired. In IFIP Conference on Human-Computer Interaction. Springer, 318--322.
[28]
Robert Harper, Louise Culham, and Christine Dickinson. 1999. Head mounted video magnification devices for low vision rehabilitation: a comparison with existing technology. British journal of ophthalmology 83, 4 (1999), 495--500.
[29]
Paul J Hershberger. 1992. Information loss: The primary psychological trauma of the loss of vision. Perceptual and motor skills 74, 2 (1992), 509--510.
[30]
MT Islam, MA Rashid, M Ahmad, A Kuwana, and H Kobayashi. 2020. Design and Implementation of Smart Guided Glass for Visually Impaired People. International Journal of Electrical and Computer Engineering (IJECE) 9, 4 (2020).
[31]
M Kathiravan et al. 2021. A Smart Hoodie-Assistive Technology for Blind People Using IoT. Turkish Journal of Computer and Mathematics Education (TURCOMAT) 12, 10 (2021), 7565--7569.
[32]
Oliver Beren Kaul, Kersten Behrens, and Michael Rohs. 2021. Mobile Recognition and Tracking of Objects in the Environment through Augmented Reality and 3D Audio Cues for People with Visual Impairments. In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems. 1--7.
[33]
Seita Kayukawa, Tatsuya Ishihara, Hironobu Takagi, Shigeo Morishima, and Chieko Asakawa. 2020. Guiding Blind Pedestrians in Public Spaces by Understanding Walking Behavior of Nearby Pedestrians. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 4, 3, Article 85 (sep 2020), 22 pages.
[34]
Jaejoon Kim. 2020. Application on character recognition system on road sign for visually impaired: Case study approach and future. International Journal of Electrical and Computer Engineering (IJECE) 10, 1 (2020), 778--785.
[35]
Matshehla Konaite, Pius A Owolawi, Temitope Mapayi, Vusi Malele, Kehinde Odeyemi, Gbolahan Aiyetoro, and Joseph S Ojo. 2021. Smart Hat for the blind with Real-Time Object Detection using Raspberry Pi and TensorFlow Lite. In Proceedings of the International Conference on Artificial Intelligence and its Applications. 1--6.
[36]
Y R Sanjay Kumar, T Nivethetha, P Priyadharshini, and U. Jayachandiran. 2022. SMART GLASSES FOR VISUALLY IMPAIRED PEOPLE WITH FACIAL RECOGNITION. In 2022 International Conference on Communication, Computing and Internet of Things (IC3IoT). 1--4. https://doi.org/10. 1109/IC3IOT53935.2022.9768012
[37]
Florian Lang, Albrecht Schmidt, and Tonja Machulla. 2020. Augmented reality for people with low vision: symbolic and alphanumeric representation of information. In Computers Helping People with Special Needs: 17th International Conference, ICCHP 2020, Lecco, Italy, September 9--11, 2020, Proceedings, Part I 17. Springer, 146--156.
[38]
Chan-Su Lee, Jae-Ik Lee, and Han Eol Seo. 2021. Deep Learning Based Mobile Assistive Device for Visually Impaired People. In 2021 IEEE International Conference on Consumer Electronics-Asia (ICCE-Asia). IEEE, 1--3.
[39]
Yifan Li, Kangsoo Kim, Austin Erickson, Nahal Norouzi, Jonathan Jules, Gerd Bruder, and Gregory F Welch. 2022. A Scoping Review of Assistance and Therapy with Head-Mounted Displays for People Who Are Visually Impaired. ACM Transactions on Accessible Computing (TACCESS) (2022).
[40]
Jyun-You Lin, Chi-Lin Chiang, Meng-Jin Wu, Chih-Chiung Yao, and Ming-Chiao Chen. 2020. Smart Glasses Application System for Visually Impaired People Based on Deep Learning. In 2020 Indo--Taiwan 2nd International Conference on Computing, Analytics and Networks (Indo-Taiwan ICAN). IEEE, 202--206.
[41]
Amanda Hall Lueck and Gordon Dutton. 2015. Vision and the brain: Understanding cerebral visual impairment in children. Vol. 2. AFB Press, American Foundation for the Blind Arlington, VA.
[42]
Claire E Manley, Christopher R Bennett, and Lotfi B Merabet. 2022. Assessing Higher-Order Visual Processing in Cerebral Visual Impairment Using Naturalistic Virtual-Reality-Based Visual Search Tasks. Children 9, 8 (2022), 1114.
[43]
Maria B. C. Martín, Alejandro Santos-Lozano, Juan Martín-Hernández, Alberto López-Miguel, Miguel Maldonado, Carlos Baladrón, Corinna M. Bauer, and Lotfi B. Merabet. 2016. Cerebral versus Ocular Visual Impairment: The Impact on Developmental Neuroplasticity. Frontiers in Psychology 7 (2016).
[44]
Zachary Munn, Micah DJ Peters, Cindy Stern, Catalin Tufanaru, Alexa McArthur, and Edoardo Aromataris. 2018. Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC medical research methodology 18 (2018), 1--7.
[45]
T Lavanya Narayani, M Sivapalanirajan, B Keerthika, M Ananthi, and M Arunarani. 2021. Design of Smart Cane with integrated camera module for visually impaired people. In 2021 International Conference on Artificial Intelligence and Smart Systems (ICAIS). IEEE, 999--1004.
[46]
Imene Ouali, Mohamed Ben Halima, and Ali Wali. 2022. Text detection and recognition using augmented reality and deep learning. In Advanced Information Networking and Applications: Proceedings of the 36th International Conference on Advanced Information Networking and Applications (AINA-2022), Volume 1. Springer, 13--23.
[47]
Eli Peli, Robert B Goldstein, George M Young, Clement L Trempe, and Sheldon M Buzney. 1991. Image enhancement for the visually impaired. Simulations and experimental results. Investigative ophthalmology & visual science 32, 8 (1991), 2337--2350.
[48]
Eli Peli, Estella Lee, Clement L Trempe, and Sheldon Buzney. 1994. Image enhancement for the visually impaired: the effects of enhancement on face recognition. JOSA A 11, 7 (1994), 1929--1939.
[49]
R. Prabha, M. Razmah, G. Saritha, RM Asha, Senthil G. A, and R. Gayathiri. 2022. Vivoice - Reading Assistant for the Blind using OCR and TTS. In 2022 International Conference on Computer Communication and Informatics (ICCCI). 01--07.
[50]
Tuhina Priya, Kotla Sai Sravya, and S Umamaheswari. 2020. Machine-learning-based device for visually impaired person. In Artificial Intelligence and Evolutionary Computations in Engineering Systems. Springer, 79--88.
[51]
Christine Roman-Lantzy. 2007. Cortical visual impairment: An approach to assessment and intervention. American Foundation for the Blind.
[52]
Elizabeth B-N Sanders and Pieter Jan Stappers. 2008. Co-creation and the new landscapes of design. Co-design 4, 1 (2008), 5--18.
[53]
Frode Eika Sandnes. 2016. What do low-vision users really want from smart glasses? Faces, text and perhaps no glasses at all. In Computers Helping People with Special Needs: 15th International Conference, ICCHP 2016, Linz, Austria, July 13-15, 2016, Proceedings, Part I 15. Springer, 187--194.
[54]
Yue-Ting Siu. 2019. Foundations and recommendations for research in access technology. In The Routledge handbook of visual impairment. Routledge, 205--219.
[55]
M Sreeraj, Jestin Joy, Alphonsa Kuriakose, MB Bhameesh, Anoop K Babu, and Merin Kunjumon. 2020. VIZIYON: Assistive handheld device for visually challenged. Procedia Computer Science 171 (2020), 2486--2492.
[56]
S Srija, P Kawya, T Akshara Reddy, and M Dhanalakshmi. 2020. Raspberry Pi Based Wearable Reader For Visually Impaired People with Haptic Feedback. In 2020 International Conference on Electronics and Sustainable Communication Systems (ICESC). IEEE, 257--260.
[57]
Marc Steen, Menno Manschot, and Nicole De Koning. 2011. Benefits of co-design in service design projects. International Journal of Design 5, 2 (2011).
[58]
Ricardo Tachiquin, Ramiro Velázquez, Carolina Del-Valle-Soto, Carlos A Gutiérrez, Miguel Carrasco, Roberto De Fazio, Andrés Trujillo-León, Paolo Visconti, and Fernando Vidal-Verdú. 2021. Wearable urban mobility assistive device for visually impaired pedestrians using a smartphone and a tactile-foot interface. Sensors 21, 16 (2021), 5274.
[59]
Diego Villamarín and José Manuel Menéndez. 2021. Haptic glove TV device for people with visual impairment. Sensors 21, 7 (2021), 2325.
[60]
World Health Organization (WHO). 2023. Vision impairment and blindness. https://www.who.int/news-room/fact-sheets/detail/blindness-and- visual-impairment
[61]
Saumya Yadav, Rakesh Chandra Joshi, Malay Kishore Dutta, Martin Kiac, and Pavel Sikora. 2020. Fusion of object recognition and obstacle detection approach for assisting visually challenged person. In 2020 43rd International Conference on Telecommunications and Signal Processing (TSP). IEEE, 537--540.
[62]
Yutaro Yamanaka, Seita Kayukawa, Hironobu Takagi, Yuichi Nagaoka, Yoshimune Hiratsuka, and Satoshi Kurihara. 2022. One-Shot Wayfinding Method for Blind People via OCR and Arrow Analysis with a 360-Degree Smartphone Camera. In Mobile and Ubiquitous Systems: Computing, Networking and Services: 18th EAI International Conference, MobiQuitous 2021, Virtual Event, November 8-11, 2021, Proceedings. Springer, 150--168.
[63]
Ciyuan Yang, Shuchang Xu, Tianyu Yu, Guanhong Liu, Chun Yu, and Yuanchun Shi. 2021. LightGuide: Directing Visually Impaired People along a Path Using Light Cues. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 5, 2 (2021), 1--27.
[64]
Shamima Yasmin. 2020. Haptic Selfies: Bold and Beautiful Living for the Blind and Visually Impaired. In Proceedings of the 26th ACM Symposium on Virtual Reality Software and Technology. 1--3.
[65]
Hanlu Ye, Meethu Malu, Uran Oh, and Leah Findlater. 2014. Current and Future Mobile and Wearable Device Use by People with Visual Impairments. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Toronto, Ontario, Canada) (CHI '14). Association for Computing Machinery, New York, NY, USA, 3123--3132.
[66]
Yuhang Zhao, Elizabeth Kupferstein, Brenda Veronica Castro, Steven Feiner, and Shiri Azenkot. 2019. Designing AR visualizations to facilitate stair navigation for people with low vision. In Proceedings of the 32nd annual ACM symposium on user interface software and technology. 387--402.
[67]
Yuhang Zhao, Elizabeth Kupferstein, Hathaitorn Rojnirun, Leah Findlater, and Shiri Azenkot. 2020. The effectiveness of visual and audio wayfinding guidance on smartglasses for people with low vision. In Proceedings of the 2020 CHI conference on human factors in computing systems. 1--14.
[68]
Yuhang Zhao, Sarit Szpiro, and Shiri Azenkot. 2015. Foresee: A customizable head-mounted vision enhancement system for people with low vision. In Proceedings of the 17th international ACM SIGACCESS conference on computers & accessibility. 239--249.
[69]
Yuhang Zhao, Sarit Szpiro, Jonathan Knighten, and Shiri Azenkot. 2016. CueSee: exploring visual cues for people with low vision to facilitate a visual search task. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing. 73--84.
Index Terms
- AI-Enabled Smart Glasses for People with Severe Vision Impairments
Index terms have been assigned to the content through auto-classification.
Recommendations
Navigating blind people with walking impairments using a smart walker
Navigation in complex and unknown environments is a major challenge for elderly blind people. Unfortunately, conventional navigation aids such as white canes and guide dogs provide only limited assistance to blind people with walking impairments as they ...
CityGuide: A Seamless Indoor-Outdoor Wayfinding System for People With Vision Impairments
ASSETS '19: Proceedings of the 21st International ACM SIGACCESS Conference on Computers and AccessibilityGPS accuracy is poor in indoor environments and around buildings. Thus, reading and following signs still remains the most common mechanism for providing and receiving wayfinding information in such spaces. This puts individuals who are blind or ...
CityGuide: a seamless indoor–outdoor wayfinding system for people with vision impairments
AbstractThe accuracy of satellite-based positioning systems is poor in indoor environments and around built environments. Reading and following visual cues still remain the most common mechanism for providing and receiving wayfinding information in such ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
Copyright © 2024 Copyright is held by the owner/author(s).
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 26 March 2024
Published in SIGACCESS , Issue 137
Check for updates
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 164Total Downloads
- Downloads (Last 12 months)164
- Downloads (Last 6 weeks)19
Reflects downloads up to 21 Nov 2024
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
HTML Format
View this article in HTML Format.
HTML Format