research-article

Investigating techniques for gaining depth perception using visual-to-auditory sensory substitution*

Authors:

J.L. WessonAuthors Info & Claims

SAICSIT '20: Conference of the South African Institute of Computer Scientists and Information Technologists 2020

Pages 141 - 148

https://doi.org/10.1145/3410886.3410916

Published: 14 September 2020 Publication History

Abstract

The aim of this paper is to investigate techniques for gaining depth perception for the visually impaired using visual-to-auditory sensory substitution. The intent being to improve on existing algorithms, specifically using the existing MeloSee algorithm as a baseline prototype (). This paper presents two new visual-to-auditory sensory substitution prototypes ( and ), which investigate different techniques for gaining depth perception. The primary manner in which the prototypes achieve this is through sound localisation techniques. Comparative studies were conducted against the researcher's implementation of the MeloSee algorithm. For the first study, and were compared. For the second study, and were compared. Each study had 8 unique participants, for a total of 16 participants across the two studies. It was found that the use of the various techniques had a measurable effect on improving the performance of visual-to-auditory sensory substitution for depth perception in certain scenarios. Specifically, performance was improved when identifying the quadrant in which an object was in, and when navigating a narrow corridor.

References

[1]

“WHO | Visual impairment and blindness,” Visual impairment and blindness. World Health Organization, 2014.

[2]

D. Dakopoulos and N. G. Bourbakis, “Wearable Obstacle Avoidance Electronic Travel Aids for Blind: A Survey,” IEEE Trans. Syst. Man, Cybern. Part C (Applications Rev., vol. 40, no. 1, pp. 25–35, Jan. 2010.

[3]

L. Renier and A. G. De Volder, “Cognitive and brain mechanisms in sensory substitution of vision: a contribution to the study of human perception.,” J. Integr. Neurosci., vol. 4, no. 4, pp. 489–503, Dec. 2005.

[4]

H. C. Stronks, A. C. Nau, M. R. Ibbotson, and N. Barnes, “The role of visual deprivation and experience on the performance of sensory substitution devices,” Brain Res., vol. 1624, pp. 140–152, 2015.

[5]

P. Meijer, “An experimental system for auditory image representations,” IEEE Trans. Biomed. Eng., vol. 39, no. 2, pp. 112–121, 1992.

[6]

J. Ward and T. D. Wright, “Sensory Substitution Devices as Advanced Sensory Tools,” 2014. [Online]. Available: https://www.researchgate.net/publication/266315050_Sensory_Substitution_Devices_as_Advanced_Sensory_Tools. [Accessed: 08-Oct-2018].

[7]

C. Capelle, C. Trullemans, P. Arno, and C. Veraart, “A real-time experimental prototype for enhancement of vision rehabilitation using auditory substitution,” IEEE Trans. Biomed. Eng., vol. 45, no. 10, pp. 1279–1293, 1998.

[8]

S. Abboud, S. Hanassy, S. Levy-Tzedek, S. Maidenbaum, and A. Amedi, “EyeMusic: Introducing a ‘visual’ colorful experience for the blind using auditory sensory substitution,” Restor. Neurol. Neurosci., vol. 32, no. 2, pp. 247–257, Jan. 2014.

[9]

C. Stoll, R. Palluel-Germain, V. Fristot, D. Pellerin, D. Alleysson, and C. Graff, “Navigating from a Depth Image Converted into Sound,” Appl. Bionics Biomech., vol. 2015, pp. 1–9, 2015.

[10]

V. Fristot, J. Boucheteil, L. Granjon, D. Pellerin, and D. Alleysson, “Depth - Melody substitution,” pp. 1–5, Aug. 2012.

[11]

E. B. Goldstein, Sensation and Perception 8th Edition. Wadsworth, Cengage Learning, 2009.

[12]

S. K. Roffler and R. A. Butler, “Factors That Influence the Localization of Sound in the Vertical Plane,” J. Acoust. Soc. Am., vol. 43, no. 6, pp. 1255–1259, 1968.

[13]

S. K. Roffler and R. A. Butler, “Localization of tonal stimuli in the vertical plane.,” J. Acoust. Soc. Am., vol. 43, no. 6, pp. 1260–6, Jun. 1968.

[14]

S. Spagnol, “Are spectral elevation cues in head-related transfer functions distance-independent?,” Proc. 19th CIM, Trieste, Novemb. 21-24, 2012, pp. 166–171, 2012.

[15]

D. R. Begault,3-D Sound for Virtual Reality and Multimedia. National Aeronautics and Space Administration, 2000.

[16]

M. von Appen, “PyAL Documentation Release 0.2.0,” 2017.

[17]

OpenAL, “OpenAL,” 2017. [Online]. Available: https://openal.org/. [Accessed: 14-May-2017].

Cited By

Maimon AWald ISnir ABen Oz MAmedi A(2024)Perceiving depth beyond sight: Evaluating intrinsic and learned cues via a proof of concept sensory substitution method in the visually impaired and sightedPLOS ONE10.1371/journal.pone.031003319:9(e0310033)Online publication date: 25-Sep-2024
https://doi.org/10.1371/journal.pone.0310033
Kim MPark YMoon KJeong C(2024)Impact of Device and Environment on Visual-Auditory Sensory Substitution: A Comprehensive Behavioral Analysis Using the vOICe AlgorithmIEEE Access10.1109/ACCESS.2024.341910212(90501-90510)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3419102

Recommendations

Visual perception substitution by the auditory sense
ICCSA'11: Proceedings of the 2011 international conference on Computational science and its applications - Volume Part II

This paper presents the methodology to develop a sensory substitution system. We deal with the absence of visual inputs in humans, and we substitute the blindness by the auditory sense. Visual to auditory substitution involves delivering information ...
A depth perception and visual comfort guided computational model for stereoscopic 3D visual saliency

With the emerging development of three-dimensional (3D) related technologies, 3D visual saliency modeling is becoming particularly important and challenging. This paper presents a new depth perception and visual comfort guided saliency computational ...
Depth of Field Affects Perceived Depth in Stereographs

Although it has been reported that depth of field influences depth perception in nonstereo photographs, it remains unclear how depth of field affects depth perception under stereo viewing conditions. We showed participants stereo photographs with ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

SAICSIT '20: Conference of the South African Institute of Computer Scientists and Information Technologists 2020

September 2020

258 pages

ISBN:9781450388474

DOI:10.1145/3410886

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 14 September 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

SAICSIT '20

SAICSIT '20: Conference of the South African Institute of Computer Scientists and Information Technologists 2020

September 14 - 16, 2020

Cape Town, South Africa

Acceptance Rates

Overall Acceptance Rate 187 of 439 submissions, 43%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
67
Total Downloads

Downloads (Last 12 months)29
Downloads (Last 6 weeks)1

Reflects downloads up to 29 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Maimon AWald ISnir ABen Oz MAmedi A(2024)Perceiving depth beyond sight: Evaluating intrinsic and learned cues via a proof of concept sensory substitution method in the visually impaired and sightedPLOS ONE10.1371/journal.pone.031003319:9(e0310033)Online publication date: 25-Sep-2024
https://doi.org/10.1371/journal.pone.0310033
Kim MPark YMoon KJeong C(2024)Impact of Device and Environment on Visual-Auditory Sensory Substitution: A Comprehensive Behavioral Analysis Using the vOICe AlgorithmIEEE Access10.1109/ACCESS.2024.341910212(90501-90510)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3419102

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents