research-article

Visually Augmented Audio-Tactile Graphics for Visually Impaired People

Author:

T. GötzelmannAuthors Info & Claims

ACM Transactions on Accessible Computing (TACCESS), Volume 11, Issue 2

Article No.: 8, Pages 1 - 31

https://doi.org/10.1145/3186894

Published: 08 June 2018 Publication History

Abstract

Tactile graphics play an essential role in knowledge transfer for blind people. The tactile exploration of these graphics is often challenging because of the cognitive load caused by physiological constraints and their complexity. The coupling of physical tactile graphics with electronic devices offers to support the tactile exploration by auditory feedback. Often, these systems have strict constraints regarding their mobility or the process of coupling both components. Additionally, visually impaired people cannot appropriately benefit from their residual vision. This article presents a concept for 3D printed tactile graphics, which offers to use audio-tactile graphics with usual smartphones or tablet-computers. By using capacitive markers, the coupling of the tactile graphics with the mobile device is simplified. These tactile graphics integrating these markers can be printed in one turn by off-the-shelf 3D printers without any post-processing and allows us to use multiple elevation levels for graphical elements. Based on the developed generic concept on visually augmented audio-tactile graphics, we presented a case study for maps. A prototypical implementation was tested by a user study with visually impaired people. All the participants were able to interact with the 3D printed tactile maps using a standard tablet computer. To study the effect of visual augmentation of graphical elements, we conducted another comprehensive user study. We tested multiple types of graphics and obtained evidence that visual augmentation may offer clear advantages for the exploration of tactile graphics. Even participants with a minor residual vision could solve the tasks with visual augmentation more quickly and accurately.

References

[1]

Catherine M. Baker, Lauren R. Milne, Ryan Drapeau, Jeffrey Scofield, Cynthia L. Bennett, and Richard E. Ladner. 2016. Tactile graphics with a voice. ACM Trans. Access. Comput. 8, 1, 1--22. Retrieved from

Digital Library

[2]

Blender Foundation. blender.org—Home of the Blender project—Free and open 3D creation software. Retrieved from http://www.blender.org/.

[3]

Braille Authority of North America. 2017. Braille Authority of North America: Guidelines and standards for tactile graphics. Retrieved from http://www.brailleauthority.org/tg/web-manual/index.html.

[4]

Anke M. Brock. 2013. Interactive maps for visually impaired people: Design, usability, and spatial cognition. Université Toulouse 3 Paul Sabatier. Retrieved from http://tel.archives-ouvertes.fr/tel-00934643/.

[5]

Anke M. Brock, Philippe Truillet, Bernard Oriola, Delphine Picard, and Christophe Jouffrais. 2015. Interactivity improves usability of geographic maps for visually impaired people. Hum.--Comput. Interact. 30, 2, 156--194.

Digital Library

[6]

Craig Brown and Amy Hurst. 2012. VizTouch: Automatically generated tactile visualizations of coordinate spaces. In Proceedings of the 6th International Conference on Tangible, Embedded and Embodied Interaction, 131--138. Retrieved from http://dl.acm.org/citation.cfm?id=2148160.

Digital Library

[7]

Emeline Brulé, Gilles Bailly, Anke M. Brock, Frédéric Valentin, Grégoire Denis, and Christophe Jouffrais. 2016. MapSense: Multi-sensory interactive maps for children living with visual impairments. In Proceedings of the ACM Conference on Computer-Human Interaction (CHI’16).

Digital Library

[8]

Andrew Crossan and Stephen Brewster. 2008. Multimodal trajectory playback for teaching shape information and trajectories to visually impaired computer users. ACM Trans. Access. Comput. 1, 2, 12.

Digital Library

[9]

M. Espinosa, Simon Ungar, Esperanza Ochaı́ta, Mark Blades, and Christopher Spencer. 1998. Comparing methods for introducing blind and visually impaired people to unfamiliar urban environments. J. Environ. Psychol. 18, 3: 277--287.

[10]

Tyler J. Ferro and Dianne T. V. Pawluk. 2013. Automatic image conversion to tactile graphic. In Proceedings of the 15th International ACM SIGACCESS Conference on Computers and Accessibility. 39.

Digital Library

[11]

Timo Götzelmann. 2016. LucentMaps: 3D printed audiovisual tactile maps for blind and visually impaired people. In Proceedings of 18th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS’16). 81--90. Retrieved from

Digital Library

[12]

Timo Götzelmann. 2016. CapMaps. In Proceedings of the 15th International Conference on Computers Helping People with Special Needs (ICCHP’16). Springer, 146--152.

[13]

Timo Götzelmann and Laura Eichler. 2016. BlindWeb maps—An interactive web service for the selection and generation of personalized audio-tactile maps. In Computers Helping People with Special Needs, Klaus Miesenberger, Christian Bühler and Petr Penaz (eds.). Springer International Publishing, Cham, 139--145.

[14]

Timo Götzelmann and Aleksander Pavkovic. 2014. Towards automatically generated tactile detail maps by 3D printers for blind persons. In Proceedings of the 14th International Conference on Computers Helping People with Special Needs (ICCHP’14). 1--7.

[15]

Timo Götzelmann and Daniel Schneider. 2016. CapCodes: Capacitive 3D printable identification and on-screen tracking for tangible interaction. In Proceedings of 9th Nordic Conference on Human-Computer Interaction (NordiCHI’16), Article No. 32. Retrieved from

Digital Library

[16]

Timo Götzelmann and Klaus Winkler. 2015. SmartTactMaps: A smartphone-based approach to support blind persons in exploring tactile maps. In Proceedings of the 8th International Conference on Pervasive Technologies Related to Assistive Environments (PETRAE’15). 1--8. Retrieved from

Digital Library

[17]

Jaume Gual, Marina Puyuelo, and Joaquim Lloveras. 2015. The effect of volumetric (3D) tactile symbols within inclusive tactile maps. Appl. Ergonom. 48, 1--10.

[18]

Ronald A. L. Hinton. 1993. Tactile and audio-tactile images as vehicles for learning. In Proceedings of the INSERM-SETAA Conference 228 (1993), 169--179.

[19]

David Huggins-Daines, Mohit Kumar, Arthur Chan, Alan W. Black, Mosur Ravishankar, and Alex I. Rudnicky. 2006. Pocketsphinx: A free, real-time continuous speech recognition system for hand-held devices. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing. 185--188.

[20]

Rubén Iglesias, Sara Casado, T. Gutierrez, J. I. Barbero, C. A. Avizzano, S. Marcheschi, and M. Bergamasco. 2004. Computer graphics access for blind people through a haptic and audio virtual environment. In Proceedings of the 3rd IEEE International Workshop on Haptic, Audio and Visual Environments and Their Applications (HAVE’04). 13--18.

[21]

R. Dan Jacobson. 1998. Navigating maps with little or no sight: An audio-tactile approach. In Proceedings of the Workshop on Content Visualization and Intermedia Representations (CVIR’98). 95--102.

[22]

Chandrika Jayant, Matt Renzelmann, Dana Wen, Satria Krisnandi, Richard Ladner, and Dan Comden. 2007. Automated tactile graphics translation: In the field. In Proceedings of the 9th International ACM SIGACCESS Conference on Computers and Accessibility. 75--82.

Digital Library

[23]

Sandra Jehoel, Snir Dinar, Don McCallum, Jonathan Rowell, and Simon Ungar. 2005. A scientific approach to tactile map design: Minimum elevation of tactile map symbols. In Proceedings of 22nd International Cartographic Conference a Coruña (CD’05).

[24]

Sandra Jehoel, Don McCallum, Jonathan Rowell, and Simon Ungar. 2006. An empirical approach on the design of tactile maps and diagrams: The cognitive tactualization approach. Brit. J. Visual Impair.t 24, 2, 67--75.

[25]

Shaun K. Kane, Brian Frey, and Jacob O. Wobbrock. 2013. Access lens: A gesture-based screen reader for real-world documents. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 347--350.

Digital Library

[26]

Shaun K. Kane, Meredith Ringel Morris, and Jacob O. Wobbrock. 2013. Touchplates: Low-cost tactile overlays for visually impaired touch screen users. In Proceedings of the 15th International ACM SIGACCESS Conference on Computers and Accessibility. 22.

Digital Library

[27]

Jeeeun Kim and Tom Yeh. 2015. Toward 3D-printed movable tactile pictures for children with visual impairments. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. 2815--2824.

Digital Library

[28]

Marius Kintel and Clifford Wolf. OpenSCAD—The Programmers Solid 3D CAD Modeller. Retrieved from http://www.openscad.org/.

[29]

Sven Kratz, Tilo Westermann, Michael Rohs, and Georg Essl. 2011. CapWidgets: Tangible widgets versus multi-touch controls on mobile devices. In Proceedings of the ACM Conference on Computer-Human Interaction (CHI’11). 1351--1356.

Digital Library

[30]

Stephen E. Krufka and Kenneth E. Barner. 2005. Automatic production of tactile graphics from scalable vector graphics. In Proceedings of the 7th International ACM SIGACCESS Conference on Computers and Accessibility. 166--172.

Digital Library

[31]

Robert H. LaMotte and Mandayam A. Srinivasan. 1991. Surface microgeometry: Tactile perception and neural encoding. In Information Processing in the Somatosensory System. Macmillan Education, UK, 49--58.

[32]

Steven Landau and Lesley Wells. 2003. Merging tactile sensory input and audio data by means of the talking tactile tablet. In Proceedings of the European Conference on Haptics (EuroHaptics’03). 414--418.

[33]

Nan Li, Zheshen Wang, Jesus Yuriar, and Baoxin Li. 2011. Tactileface: A system for enabling access to face photos by visually-impaired people. In Proceedings of the 16th International Conference on Intelligent User Interfaces. 445--446.

Digital Library

[34]

M. Rotard and T. Ertl. 2006. Tactile 3D-Graphics for blind people. In Proceedings of the Workshop on Accessible Media. Retrieved from http://cumbia.informatik.uni-stuttgart.de/ger/research/pub/pub2006/Accessible-Media-2006-Rotard.pdf.

[35]

David McGookin, Stephen Brewster, and WeiWei Jiang. 2008. Investigating touchscreen accessibility for people with visual impairments. In Proceedings of the 5th Nordic Conference on Human-computer Interaction: Building Bridges. 298--307.

Digital Library

[36]

Joshua. A. Miele, Steven Landau, and Deborah Gilden. 2006. Talking TMAP: Automated generation of audio-tactile maps using smith-kettlewell's TMAP software. Brit. J. Visual Impair. 24, 2: 93--100. Retrieved from

[37]

Don Parkes. 1988. NOMAD: An audio-tactile tool for the acquisition, use and management of spatially distributed information by partially sighted and blind persons. In Proceedings of 2nd International Conference on Maps and Graphics for Visually Disabled People. 24--29.

[38]

Grégory Petit, Aude Dufresne, Vincent Levesque, Vincent Hayward, and Nicole Trudeau. 2008. Refreshable tactile graphics applied to schoolbook illustrations for students with visual impairment. In Proceedings of the 10th International ACM SIGACCESS Conference on Computers and Accessibility. 89--96.

Digital Library

[39]

Benjamin Poppinga, Charlotte Magnusson, Martin Pielot, and Kirsten Rassmus-Gröhn. 2011. TouchOver map: Audio-tactile exploration of interactive maps. In Proceedings of the 13th International Conference on Human Computer Interaction with Mobile Devices and Services. 545--550. Retrieved from http://dl.acm.org/citation.cfm?id=2037458.

Digital Library

[40]

Bernhard Schmitz and Thomas Ertl. 2012. Interactively displaying maps on a tactile graphics display. In Proceedings of the Conference on Spatial Knowledge Acquisition with Limited Information Displays (SKALID’12). 13.

[41]

Caterina Senette, Maria Claudia Buzzi, Marina Buzzi, Barbara Leporini, and Loredana Martusciello. 2013. Enriching graphic maps to enable multimodal interaction by blind people. In Universal Access in Human-Computer Interaction. Design Methods, Tools, and Interaction Techniques for eInclusion. Springer, 576--583. Retrieved from http://link.springer.com/chapter/10.1007/978-3-642-39188-0_62.

Digital Library

[42]

Lei Shi, Idan Zelzer, Catherine Feng, and Shiri Azenkot. 2016. Tickers and talker: An accessible labeling toolkit for 3D printed models. 4896--4907. Retrieved from

Digital Library

[43]

Jing Su, Alyssa Rosenzweig, Ashvin Goel, Eyal de Lara, and Khai N. Truong. 2010. Timbremap: Enabling the visually-impaired to use maps on touch-enabled devices. In Proceedings of the 12th International Conference on Human Computer Interaction with Mobile Devices and Services. 17--26.

Digital Library

[44]

Saiganesh Swaminathan, Thijs Roumen, Robert Kovacs, David Stangl, Stefanie Mueller, and Patrick Baudisch. 2016. Linespace: A sensemaking platform for the blind. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. 2175--2185. Retrieved from

Digital Library

[45]

A. F. Tatham. 1991. The design of tactile maps: Theoretical and practical considerations. Proceedings of the International Cartographic Association: Mapping the Nations. 157--166.

[46]

Simon Voelker, Kosuke Nakajima, Christian Thoresen, Yuichi Itoh, Kjell Ivar Øvergård, and Jan Borchers. 2013. PUCs: Detecting transparent, passive untouched capacitive widgets on unmodified multi-touch displays. In Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces. 101--104.

Digital Library

[47]

Vít Voženílek, Magdaléna Kozáková, Zuzana Št’ávová, Libuše Ludíková, Veronika Rŭžičková, and Dita Finková. 2009. 3D Printing technology in tactile maps compiling. In Proceedings of 24th International Cartographic Conference, International Cartographic Association.

[48]

Steven Wall and Stephen Brewster. 2006. Feeling what you hear: Tactile feedback for navigation of audio graphs. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 1123--1132.

Digital Library

[49]

Zheshen Wang, Baoxin Li, Terri Hedgpeth, and Teresa Haven. 2009. Instant tactile-audio map: enabling access to digital maps for people with visual impairment. In Proceedings of the 11th International ACM SIGACCESS Conference on Computers & Accessibility. 43--50. Retrieved from http://dl.acm.org/citation.cfm?id=1639652.

Digital Library

[50]

Tetsuya Watanabe and Susumu Oouchi. 2003. A study on legible braille pattern on capsule paper: Diameters of braille dots and their interspaces on the original ink-printed paper. Bull. Natl. Inst. Special Edu. 30, 1--8.

[51]

Thomas P. Way and Kenneth E. Barner. 1997. Automatic visual to tactile translation. II. evaluation of the TACTile image creation system. IEEE Trans. Rehabil. Engineer. 5, 1, 95--105.

[52]

Thomas P. Way and Kenneth E. Barner. 1997. Automatic visual to tactile translation. i. Human factors, access methods, and image manipulation. IEEE Trans. Rehabil. Engineer. 5, 1, 81--94.

[53]

Alexander Wiethoff, Hanna Schneider, Michael Rohs, Andreas Butz, and Saul Greenberg. 2012. Sketch-a-TUI: Low cost prototyping of tangible interactions using cardboard and conductive ink. In Proceedings of the Conference on Tangible, Embedded, and Embodied Interaction (TEI’12). 309--312.

Digital Library

[54]

Frank Wilcoxon. 1945. Individual comparisons by ranking methods. Biometr. Bull. 1, 6, 80--83.

[55]

World Health Organization. 2016. Visual impairment and blindness. Retrieved from http://www.who.int/mediacentre/factsheets/fs282/en/.

[56]

World Health Organization. 2017. Change the definition of blindness. Retrieved from http://www.who.int/blindness/Change%20the%20Definition%20of%20Blindness.pdf?ua=1.

[57]

Robert Xiao, Julia Schwarz, and Chris Harrison. 2015. Estimating 3D finger angle on commodity touchscreens. 47--50. Retrieved from

Digital Library

[58]

Limin Zeng, Mei Miao, and Gerhard Weber. 2015. Interactive audio-haptic map explorer on a tactile display. Interact. Comput. 27, 4, 413--429. Retrieved from

[59]

Limin Zeng and Gerhard Weber. 2011. Accessible maps for the visually impaired. In Proceedings of the IFIP INTERACT Workshop on Accessible Design in the Digital World. 54--60.

[60]

Limin Zeng and Gerhard Weber. 2012. ATMap: Annotated tactile maps for the visually impaired. In Cognitive Behavioural Systems. Springer, 290--298.

Digital Library

Cited By

Jiang CKuang EFan M(2025)How Can Haptic Feedback Assist People with Blind and Low Vision (BLV): A Systematic Literature ReviewACM Transactions on Accessible Computing10.1145/371193118:1(1-57)Online publication date: 13-Jan-2025
https://dl.acm.org/doi/10.1145/3711931
Rong AHansopaheluwakan-Edward N(2024)A bibliometric analysis of research trends and advanced technologies in tactile graphics designBlucher Design Proceedings10.5151/ead2023-1BIL-01Full-06Rong(78-89)Online publication date: May-2024
https://doi.org/10.5151/ead2023-1BIL-01Full-06Rong
Cui MYuan CLiu YZheng YKostakos VKay JHoang T(2024)More Than Shapes: Exploring the Tactile Parameters of Art Appreciation for the Visually ImpairedCompanion of the 2024 on ACM International Joint Conference on Pervasive and Ubiquitous Computing10.1145/3675094.3678391(786-790)Online publication date: 5-Oct-2024
https://dl.acm.org/doi/10.1145/3675094.3678391
Show More Cited By

Index Terms

Visually Augmented Audio-Tactile Graphics for Visually Impaired People
1. Hardware
  1. Communication hardware, interfaces and storage
    1. Tactile and hand-based interfaces
      1. Touch screens
2. Human-centered computing

Recommendations

LucentMaps: 3D Printed Audiovisual Tactile Maps for Blind and Visually Impaired People
ASSETS '16: Proceedings of the 18th International ACM SIGACCESS Conference on Computers and Accessibility

Tactile maps support blind and visually impaired people in orientation and to familiarize with unfamiliar environments. Interactive approaches complement these maps with auditory feedback. However, commonly these approaches focus on blind people. We ...
Interactive audio-tactile maps for visually impaired people

Visually impaired people face important challenges related to orientation and mobility. Indeed, 56% of visually impaired people in France declared having problems concerning autonomous mobility [10]. These problems often mean that visually impaired ...
Multimodal Tactile Graphics Using T-TATIL, A Mobile Application for Tactile Exploration by Visually Impaired People
Universal Access in Human-Computer Interaction. Design Methods and User Experience
Abstract
Graphic contents are important resources to communicate information and their use is fundamental in the teaching process. Visually impaired people do not have easy access to this resource owing to their impairment. They mainly use tactile graphics ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Transactions on Accessible Computing

ACM Transactions on Accessible Computing Volume 11, Issue 2

Paper from ASSETS 2016 and Regular Papers

June 2018

112 pages

ISSN:1936-7228

EISSN:1936-7236

DOI:10.1145/3229065

Editors:
Matt Huenerfauth
Rochester Institute of Technology, USA
,
Kathleen F. McCoy
University of Delaware, USA

Issue’s Table of Contents

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 June 2018

Accepted: 01 February 2018

Revised: 01 February 2018

Received: 01 April 2017

Published in TACCESS Volume 11, Issue 2

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

41
Total Citations
View Citations
1,156
Total Downloads

Downloads (Last 12 months)187
Downloads (Last 6 weeks)19

Reflects downloads up to 18 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Jiang CKuang EFan M(2025)How Can Haptic Feedback Assist People with Blind and Low Vision (BLV): A Systematic Literature ReviewACM Transactions on Accessible Computing10.1145/371193118:1(1-57)Online publication date: 13-Jan-2025
https://dl.acm.org/doi/10.1145/3711931
Rong AHansopaheluwakan-Edward N(2024)A bibliometric analysis of research trends and advanced technologies in tactile graphics designBlucher Design Proceedings10.5151/ead2023-1BIL-01Full-06Rong(78-89)Online publication date: May-2024
https://doi.org/10.5151/ead2023-1BIL-01Full-06Rong
Cui MYuan CLiu YZheng YKostakos VKay JHoang T(2024)More Than Shapes: Exploring the Tactile Parameters of Art Appreciation for the Visually ImpairedCompanion of the 2024 on ACM International Joint Conference on Pervasive and Ubiquitous Computing10.1145/3675094.3678391(786-790)Online publication date: 5-Oct-2024
https://dl.acm.org/doi/10.1145/3675094.3678391
Seo JXia YLee BMccurry SYam Y(2024)MAIDR: Making Statistical Visualizations Accessible with Multimodal Data RepresentationProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642730(1-22)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642730
Wimer BSouth LWu KSzafir DBorkin MMetoyer R(2024)Beyond Vision Impairments: Redefining the Scope of Accessible Data RepresentationsIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2024.335656630:12(7619-7636)Online publication date: 1-Dec-2024
https://dl.acm.org/doi/10.1109/TVCG.2024.3356566
Gu QLi XCai QZhu KDai CZhang YSun LTao YWang G(2024)MyWay: a 3D and audio-enhanced transportation learning kit for the visually impaired teenagersCCF Transactions on Pervasive Computing and Interaction10.1007/s42486-024-00163-yOnline publication date: 23-Jul-2024
https://doi.org/10.1007/s42486-024-00163-y
Regal GMaidhof CPuthenkalam JKhermayer ŽPavšek KSackl A(2023)Feel the Art: Digital, Tangible Postcards for Accessible Cultural ExperiencesProceedings of the 22nd International Conference on Mobile and Ubiquitous Multimedia10.1145/3626705.3631877(559-561)Online publication date: 3-Dec-2023
https://dl.acm.org/doi/10.1145/3626705.3631877
Telesinska M(2023)Multimodal 3D printed urban maps for blind people. Evaluations and scientific investigationsProceedings of the 25th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3597638.3614503(1-7)Online publication date: 22-Oct-2023
https://dl.acm.org/doi/10.1145/3597638.3614503
Trinh VManduchi RGiudice N(2023)Experimental Evaluation of Multi-scale Tactile Maps Created with SIM, a Web App for Indoor Map AuthoringACM Transactions on Accessible Computing10.1145/359077516:2(1-26)Online publication date: 11-May-2023
https://dl.acm.org/doi/10.1145/3590775
Maćkowski MBrzoza PKawulok MMeisel RSpinczyk D(2023)Multimodal Presentation of Interactive Audio-Tactile Graphics Supporting the Perception of Visual Information by Blind PeopleACM Transactions on Multimedia Computing, Communications, and Applications10.1145/358607619:5s(1-22)Online publication date: 7-Jun-2023
https://dl.acm.org/doi/10.1145/3586076
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Issue’s Table of Contents