A Wearable Haptic Interface for Assisting Blind and Visually Impaired Students in Learning Algebraic Equations
Authors: 
Sunhee Kim, Josh Cherian, Samantha Ray, Amanda Lacy, 
Paul Taele, Jung In Koh, 
Tracy HammondAuthors Info & Claims
Article No.: 17, Pages 1 - 7
Abstract
BVI (blind and visually impaired) persons do not have the same degree of access to concepts and procedures in learning algebraic equations as sighted persons. As tactile resources are uncommon, most rely on auditory-based methods that place heavy stress on working memory. We propose a novel approach that leverages haptic technology to communicate mathematical symbols via vibrotactile patterns; this design addresses open challenges found in existing resources for BVI people and explores the potential utility of this medium for all users. We conducted three separate studies to evaluate the feasibility of our gloves that included BVI participants. The outcomes of our studies yielded insightful design discussions, including a blind participant who found the device immediately intuitive and exciting in potential.
Supplementary Material
References
[1]
Dragan Ahmetovic, Valeria Alampi, Cristian Bernareggi, Andrea Gerino, and Sergio Mascetti. 2017. Math Melodies: Supporting Visually Impaired Primary School Students in Learning Math. In Proceedings of the 14th International Web for All Conference. ACM, Perth, Australia, 2 pages.
[2]
Peter Albert. 2006. Math Class: An Application for Dynamic Tactile Graphics. In Computers Helping People with Special Needs, Klaus Miesenberger, Joachim Klaus, Wolfgang L. Zagler, and Arthur I. Karshmer (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 1118–1121.
[3]
Mrim Alnfiai and Srinivas Sampalli. 2017. BrailleEnter: A Touch Screen Braille Text Entry Method for the Blind. Procedia Computer Science 109 (Jan. 2017), 257–264. https://doi.org/10.1016/j.procs.2017.05.349
[4]
Thimoty Barbieri, Lorenzo Mosca, and Licia Sbattella. 2008. Learning Math for Visually Impaired Users. In Computers Helping People with Special Needs, Klaus Miesenberger, Joachim Klaus, Wolfgang Zagler, and Arthur Karshmer (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 907–914.
[5]
Thimoty Barbieri, Lorenzo Mosca, and Licia Sbattella. 2008. Learning Math for Visually Impaired Users. Klaus Miesenberger, Joachim Klaus, Wolfgang Zagler, and Arthur Karshmer, Springer Berlin Heidelberg, Berlin.
[6]
Jeffrey R Blum and Jeremy R Cooperstock. 2018. Exploring the Limits of Vibrotactile Numeric Information Delivery. In Proceedings of the 2018 ACM International Joint Conference and 2018 International Symposium on Pervasive and Ubiquitous Computing and Wearable Computers. ACM, Singapore, Singapore, 17–20.
[7]
Jeffrey R Blum and Jeremy R Cooperstock. 2019. Single-Actuator Vibrotactile Numeric Information Delivery in the Face of Distraction. In 2019 IEEE World Haptics Conference (WHC). IEEE, Tokyo, Japan, 461–466.
[8]
Lorna M Brown, Stephen A Brewster, and Helen C Purchase. 2006. Multidimensional tactons for non-visual information presentation in mobile devices. In Proceedings of the 8th conference on Human-computer interaction with mobile devices and services. ACM, Helsinki, Finland, 231–238.
[9]
Niccolò Cantù, Mattia Ducci, Dragan Ahmetovic, Cristian Bernareggi, and Sergio Mascetti. 2018. MathMelodies 2: A Mobile Assistive Application for People with Visual Impairments Developed with React Native. In Proceedings of the 20th International ACM SIGACCESS Conference on Computers and Accessibility (Galway, Ireland) (ASSETS ’18). Association for Computing Machinery, New York, NY, USA, 453–455. https://doi.org/10.1145/3234695.3241006
[10]
Marjorie Anne Darrah. 2013. Computer haptics: A new way of increasing access and understanding of math and science for students who are blind and visually impaired. Journal of Blindness Innovation and Research 3, 2 (2013), 3–47. https://nfb.org//images/nfb/publications/jbir/jbir13/jbir030202.html
[11]
Don Samitha Elvitigala, Denys JC Matthies, Vipula Dissanayaka, Chamod Weerasinghe, and Suranga Nanayakkara. 2019. 2bit-TactileHand: Evaluating Tactons for On-Body Vibrotactile Displays on the Hand and Wrist. In Proceedings of the 10th Augmented Human International Conference 2019. ACM, Reims, France, 3.
[12]
Don Samitha Elvitigala, Denys J. C. Matthies, Vipula Dissanayaka, Chamod Weerasinghe, and Suranga Nanayakkara. 2019. 2bit-TactileHand: Evaluating Tactons for On-Body Vibrotactile Displays on the Hand and Wrist. In Proceedings of the 10th Augmented Human International Conference 2019. ACM, Reims France, 1–8. https://doi.org/10.1145/3311823.3311832
[13]
Liv Sissel Grønmo. 2018. The Role of Algebra in School Mathematics. In Invited Lectures from the 13th International Congress on Mathematical Education, Gabriele Kaiser, Helen Forgasz, Mellony Graven, Alain Kuzniak, Elaine Simmt, and Binyan Xu (Eds.). Springer International Publishing, Cham, 175–193.
[14]
Ann P Gulley, Luke A Smith, Jordan A Price, Logan C Prickett, and Matthew F Ragland. 2017. Process-driven math: An auditory method of mathematics instruction and assessment for students who are blind or have low vision. Journal of visual impairment & blindness 111, 5 (2017), 465–471.
[15]
René Gutschmidt, Maria Schiewe, Francis Zinke, and Helmut Jürgensen. 2010. Haptic emulation of games: haptic Sudoku for the blind. In Proceedings of the 3rd International conference on Pervasive Technologies Related to Assistive environments. ACM, Samos, Greece, 1–8.
[16]
Richard Jackson. 2022. Audio-Supported Reading for Students Who are Blind or Visually Impaired. https://aem.cast.org/get-started/resources/2021/audio-supported-reading
[17]
Prajaks Jitngernmadan, Bernhard Stöger, Andrea Petz, and Klaus Miesenberger. 2017. IDMILE: An interactive didactic math inclusion learning environment for blind students. Technology and Disability 29, 1-2 (July 2017), 47–61. https://doi.org/10.3233/TAD-170173
[18]
Idin Karuei, Karon E MacLean, Zoltan Foley-Fisher, Russell MacKenzie, Sebastian Koch, and Mohamed El-Zohairy. 2011. Detecting vibrations across the body in mobile contexts. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, Vancouver, Canada, 3267–3276.
[19]
Hyung Nam Kim. 2009. Haptic User Interface Design for Students with Visual Impairments. In Proceedings of the 11th International ACM SIGACCESS Conference on Computers and Accessibility (Pittsburgh, Pennsylvania, USA) (Assets ’09). Association for Computing Machinery, New York, NY, USA, 267–268. https://doi.org/10.1145/1639642.1639709
[20]
Hyung Nam Kim, Tonya L. Smith-Jackson, and Brian M. Kleiner. 2014. Accessible Haptic User Interface Design Approach for Users with Visual Impairments. Univers. Access Inf. Soc. 13, 4 (Nov. 2014), 415–437. https://doi.org/10.1007/s10209-013-0325-0
[21]
Kiduk Kim, Ji-Hoon Jeong, Jeong-Hyun Cho, Sunghyun Kim, Jeonggoo Kang, Jeha Ryu, and Seong-Whan Lee. 2021. Development of a Human-Display Interface with Vibrotactile Feedback for Real-World Assistive Applications. Sensors 21, 2 (2021), 17 pages. https://doi.org/10.3390/s21020592
[22]
Oliv G. Klingenberg. 2012. Conceptual Understanding of Shape and Space by Braille-Reading Norwegian Students in Elementary School. Journal of Visual Impairment & Blindness 106, 8 (Aug. 2012), 453–465. https://doi.org/10.1177/0145482X1210600802
[23]
Eric Knuth, Ana Stephens, Maria Blanton, and Angela Gardiner. 2016. Build an early foundation for algebra success. Phi Delta Kappan 97, 6 (2016), 65–68.
[24]
Edna Laudenslager. 1972. Braille handbook for the Nemeth code of braille mathematics and scientific notation. American Printing House for the Blind, Louisville, KY.
[25]
Yi-Chi Liao, Yi-Ling Chen, Jo-Yu Lo, Rong-Hao Liang, Liwei Chan, and Bing-Yu Chen. 2016. Edgevib: Effective alphanumeric character output using a wrist-worn tactile display. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology. ACM, Tokyo, Japan, 595–601.
[26]
Granit Luzhnica and Eduardo Veas. 2019. Background perception and comprehension of symbols conveyed through vibrotactile wearable displays. In Proceedings of the 24th International Conference on Intelligent User Interfaces. ACM, Marina del Ray, California, USA, 57–64.
[27]
Granit Luzhnica and Eduardo Veas. 2019. Optimising Encoding for Vibrotactile Skin Reading. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. ACM, Glasgow, Scotland UK, 1–14.
[28]
Granit Luzhnica, Eduardo Veas, and Viktoria Pammer. 2016. Skin reading: Encoding text in a 6-channel haptic display. In Proceedings of the 2016 ACM International Symposium on Wearable Computers. ACM, Heidelberg, Germany, 148–155.
[29]
Jani Lylykangas, Veikko Surakka, Jussi Rantala, and Roope Raisamo. 2013. Intuitiveness of vibrotactile speed regulation cues. ACM Transactions on Applied Perception (TAP) 10, 4 (2013), 1–15.
[30]
Jonas Moll and Eva-Lotta Sallnäs Pysander. 2013. A haptic tool for group work on geometrical concepts engaging blind and sighted pupils. ACM Transactions on Accessible Computing (TACCESS) 4, 4 (2013), 1–37.
[31]
Tony Morelli, John Foley, Luis Columna, Lauren Lieberman, and Eelke Folmer. 2010. VI-Tennis: a vibrotactile/audio exergame for players who are visually impaired. In Proceedings of the Fifth International Conference on the Foundations of Digital Games. ACM, Monterey, California, 147–154.
[32]
Tony Morelli, John Foley, and Eelke Folmer. 2010. Vi-bowling: a tactile spatial exergame for individuals with visual impairments. In Proceedings of the 12th international ACM SIGACCESS conference on Computers and accessibility. ACM, Orlando, Florida, USA, 179–186.
[33]
Kristen Murphy and Marjorie Darrah. 2015. Haptics-Based Apps for Middle School Students with Visual Impairments. IEEE Transactions on Haptics 8, 3 (2015), 318–326. https://doi.org/10.1109/TOH.2015.2401832
[34]
Hugo Nicolau, Kyle Montague, Tiago Guerreiro, André Rodrigues, and Vicki L. Hanson. 2015. HoliBraille: Multipoint Vibrotactile Feedback on Mobile Devices. In Proceedings of the 12th International Web for All Conference (Florence, Italy) (W4A ’15). Association for Computing Machinery, New York, NY, USA, Article 30, 4 pages. https://doi.org/10.1145/2745555.2746643
[35]
Lucie Pantera, Charles Hudin, and Sabrina Panëels. 2021. LotusBraille: Localised Multifinger Feedback on a Surface for Reading Braille Letters. In 2021 IEEE World Haptics Conference (WHC). IEEE, New York, NY, USA, 973–978. https://doi.org/10.1109/WHC49131.2021.9517225
[36]
Jerome Pasquero, Scott J Stobbe, and Noel Stonehouse. 2011. A haptic wristwatch for eyes-free interactions. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, Vancouver, Canada, 3257–3266.
[37]
Manoj Prasad, Murat Russell, and Tracy Hammond. 2014. Designing Vibrotactile Codes to Communicate Verb Phrases. In ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM), Vol. 11-1. ACM, New York, New York, USA, Article 11: pp.1–21.
[38]
Manoj Prasad, Murat Russell, and Tracy Hammond. 2014. A User Centric Model to Design Tactile Codes with Shapes and Waveforms. In Haptics Symposium (HAPTICS). IEEE, Houston, TX, USA, 597–602. ISBN: 978-1-4799-3131-6.
[39]
Manoj Prasad, Paul Taele, Ayobami Olubeku, and Tracy Hammond. 2014. HaptiGo: A Navigational ‘Tap on the Shoulder’. In Haptics Symposium (HAPTICS). IEEE, Houston, TX, USA, 339–345. ISBN: 978-1-4799-3131-6.
[40]
Thijs Roumen, Simon T. Perrault, and Shengdong Zhao. 2015. NotiRing: A Comparative Study of Notification Channels for Wearable Interactive Rings. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. ACM, Seoul Republic of Korea, 2497–2500. https://doi.org/10.1145/2702123.2702350
[41]
Derrick W. Smith and Sinikka M. Smothers. 2012. The Role and Characteristics of Tactile Graphics in Secondary Mathematics and Science Textbooks in Braille. Journal of Visual Impairment & Blindness 106, 9 (Sept. 2012), 543–554. https://doi.org/10.1177/0145482X1210600905
[42]
Burkay Sucu and Eelke Folmer. 2014. The blind driver challenge: steering using haptic cues. In Proceedings of the 16th international ACM SIGACCESS conference on Computers & accessibility. ACM, Rochester, New York, USA, 3–10.
[43]
Sampo Töyssy, Jukka Raisamo, and Roope Raisamo. 2008. Telling time by vibration. In International Conference on Human Haptic Sensing and Touch Enabled Computer Applications. Springer, Madrid, Spain, 924–929.
[44]
Marlon Twyman, Joe Mullenbach, Craig Shultz, J. Edward Colgate, and Anne Marie Piper. 2015. Designing Wearable Haptic Information Displays for People with Vision Impairments. In Proceedings of the Ninth International Conference on Tangible, Embedded, and Embodied Interaction (Stanford, California, USA) (TEI ’15). Association for Computing Machinery, New York, NY, USA, 341–344. https://doi.org/10.1145/2677199.2680578
[45]
A.B. Vallbo and Roland Johansson. 1984. Properties of cutaneous mechanoreceptors in the human hand related to touch sensation. Human neurobiology 3 (02 1984), 3–14.
[46]
Jens Vetter. 2019. Tangible Signals - Physical Representation of Sound and Haptic Control Feedback. In Proceedings of the Thirteenth International Conference on Tangible, Embedded, and Embodied Interaction (Tempe, Arizona, USA) (TEI ’19). Association for Computing Machinery, New York, NY, USA, 741–744. https://doi.org/10.1145/3294109.3302958
[47]
Carl Vogel. 2008. Algebra: Changing the Equation. District Administration 44, 6 (2008), 34–38.
[48]
Yulin Wang, Barbara Millet, and James L Smith. 2014. Informing the use of vibrotactile feedback for information communication: an analysis of user performance across different vibrotactile designs. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 58 (2014), 1859–1863.
[49]
Yulin Wang, Barbara Millet, and James L Smith. 2016. Designing wearable vibrotactile notifications for information communication. International Journal of Human-Computer Studies 89 (2016), 24–34.
[50]
Bei Yuan and Eelke Folmer. 2008. Blind hero: enabling guitar hero for the visually impaired. In Proceedings of the 10th international ACM SIGACCESS conference on Computers and accessibility. ACM, Halifax, Nova Scotia, Canada, 169–176.
Index Terms
- A Wearable Haptic Interface for Assisting Blind and Visually Impaired Students in Learning Algebraic Equations
Recommendations
HELF (Haptic Encoded Language Framework): a digital script for deaf-blind and visually impaired
AbstractPurposeDigital media has brought a revolution, making the world a global village. For people who are visually impaired and people with visual and hearing impairment, navigating through the digital world can be as precarious as moving through the ...
Feasibility of using haptic directions through maps with a tablet and smart watch for people who are blind and visually impaired
MobileHCI '16: Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and ServicesIn order to navigate through the world, people who are blind and visually impaired typically use maps through either textual directions or tactile printouts. However, visual maps on a touchscreen are not accessible to this population. Two prototypes ...
Designing Haptic Assistive Technology for Individuals Who Are Blind or Visually Impaired
This paper considers issues relevant for the design and use of haptic technology for assistive devices for individuals who are blind or visually impaired in some of the major areas of importance: Braille reading, tactile graphics, orientation and ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
April 2023
3914 pages
ISBN:9781450394222
DOI:10.1145/3544549
Copyright © 2023 Owner/Author.
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 19 April 2023
Check for updates
Author Tags
Qualifiers
- Work in progress
- Research
- Refereed limited
Conference
CHI '23
Sponsor:
Acceptance Rates
Overall Acceptance Rate 6,164 of 23,696 submissions, 26%
Upcoming Conference
CHI '25
- Sponsor:
- sigchi
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 548Total Downloads
- Downloads (Last 12 months)347
- Downloads (Last 6 weeks)41
Reflects downloads up to 13 Nov 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderFull Text
View this article in Full Text.
Full TextHTML Format
View this article in HTML Format.
HTML Format