research-article

Using Audio Reverberation to Compensate Distance Compression in Virtual Reality

Authors:

Yi-Hsuan Huang,

Roshan Venkatakrishnan,

Rohith Venkatakrishnan,

Sabarish V. Babu,

Wen-Chieh LinAuthors Info & Claims

SAP '21: ACM Symposium on Applied Perception 2021

Article No.: 13, Pages 1 - 10

https://doi.org/10.1145/3474451.3476236

Published: 16 September 2021 Publication History

Abstract

Virtual Reality (VR) technologies are increasingly being applied to various contexts like those gaming, therapy, training, and education. Several of these applications require high degrees of accuracy in spatial and depth perception. Contemporary VR experiences continue to be confronted by the issue of distance compression wherein people systematically underestimate distances in the virtual world, leading to impoverished experiences. Consequently, a number of studies have focused on extensively understanding and exploring factors that influence this phenomenon to address the challenges it poses. Inspired by previous work that has sought to compensate distance compression effects in VR, we examined the potential of manipulating an auditory stimulus’ reverberation time (RT) to alter how users perceive depth. To this end, we conducted a two action forced choice study in which participants repeatedly made relative depth judgements between a pair of scenes featuring a virtual character placed at different distances with varying RTs. Results revealed that RT influences how users perceive depth with this influence being more pronounced in the near field. We found that users tend to associate longer RTs with farther distances and vice versa, indicating the potential to alter RT towards compensating distance underestimation in VR. However, it must be noted that excessively increasing RT (especially in the near field) could come at the cost of sensory segregation, wherein users are unable to unify visual and auditory sensory stimuli in their perceptions of depth. Researchers must hence strive to find the optimal amount of RT to add to a stimulus to ensure seamless virtual experiences.

References

[1]

Scott M. Andrus, Graham Gaylor, and Bobby Bodenheimer. 2014. Distance Estimation in Virtual Environments Using Different HMDs. In Proceedings of the ACM Symposium on Applied Perception. ACM, 130–130.

Digital Library

[2]

Will Bailey and Bruno M Fazenda. 2017. The effect of reverberation and audio spatialization on egocentric distance estimation of objects in stereoscopic virtual reality. The Journal of the Acoustical Society of America 141, 5 (2017), 3510–3510.

[3]

Leo Leroy Beranek and Tim Mellow. 2012. Acoustics: sound fields and transducers. Academic Press, 473.

[4]

Jonathan S Berry, David AT Roberts, and Nicolas S Holliman. 2014. 3D sound and 3D image interactions: a review of audio-visual depth perception. Proc.SPIE 9014(2014), 1–16.

[5]

Stephanie Claire Boyle. 2018. Investigating the neural mechanisms underlying audio-visual perception using electroencephalography (EEG). Ph.D. Dissertation. University of Glasgow.

[6]

P Breedveld, HG Stassen, DW Meijer, and LPS Stassen. 1999. Theoretical background and conceptual solution for depth perception and eye-hand coordination problems in laparoscopic surgery. Minimally invasive therapy & allied technologies 8, 4(1999), 227–234.

[7]

Adelbert W Bronkhorst and Tammo Houtgast. 1999. Auditory distance perception in rooms. Nature 397, 6719 (1999), 517.

[8]

Gerd Bruder, Fernando Argelaguet Sanz, Anne-Hélène Olivier, and Anatole Lécuyer. 2015. Distance estimation in large immersive projection systems, revisited. In 2015 IEEE Virtual Reality (VR). IEEE, 27–32.

[9]

Lauren E. Buck, Mary K. Young, and Bobby Bodenheimer. 2018. A Comparison of Distance Estimation in HMD-Based Virtual Environments with Different HMD-Based Conditions. ACM Trans. Appl. Percept. 15, 3 (July 2018), 21:1–21:15.

Digital Library

[10]

Yi-Ting Chen, Chi-Hsuan Hsu, Chih-Han Chung, Yu-Shuen Wang, and Sabarish V Babu. 2019. iVRNote: Design, creation and evaluation of an interactive note-taking interface for study and reflection in VR learning environments. In 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). IEEE, 172–180.

[11]

Marina A Cidota, Rory MS Clifford, Stephan G Lukosch, and Mark Billinghurst. 2016. Using Visual Effects to Facilitate Depth Perception for Spatial Tasks in Virtual and Augmented Reality. In 2016 IEEE International Symposium on Mixed and Augmented Reality (ISMAR-Adjunct). IEEE, 172–177.

[12]

Philip Coleman, Andreas Franck, P Jackson, R Hughes, Luca Remaggi, and Frank Melchior. 2017. Object-based reverberation for spatial audio. Journal of the Audio Engineering Society 65, 1/2 (2017), 66–77.

[13]

Brian Cullen, Karen Collins, Andrew Hogue, and Bill Kapralos. 2016. Sound and stereoscopic 3D: Examining the effects of sound on depth perception in stereoscopic 3D. In 2016 7th International Conference on Information, Intelligence, Systems Applications (IISA). IEEE, 1–6.

[14]

Alexandre Gomes de Siqueira, Rohith Venkatakrishnan, Roshan Venkatakrishnan, Ayush Bharqava, Kathryn Lucaites, Hannah Solini, Moloud Nasiri, Andrew Robb, Christopher Pagano, Brygg Ullmer, and Sabarish V. Babu. 2021. Empirically Evaluating the Effects of Perceptual Information Channels on the Size Perception of Tangibles in Near-Field Virtual Reality. In 2021 IEEE Virtual Reality and 3D User Interfaces (VR). 1–10. https://doi.org/10.1109/VR50410.2021.00086

[15]

Elham Ebrahimi, Bliss M Altenhoff, Christopher C Pagano, and Sabarish V Babu. 2015. Carryover effects of calibration to visual and proprioceptive information on near field distance judgments in 3D user interaction. In 2015 IEEE Symposium on 3D User Interfaces (3DUI). IEEE, 97–104.

[16]

Elham Ebrahimi, Leah S Hartman, Andrew Robb, Christopher C Pagano, and Sabarish V Babu. 2018. Investigating the effects of anthropomorphic fidelity of self-avatars on near field depth perception in immersive virtual environments. In 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). IEEE, 1–8.

[17]

Marc O Ernst and Martin S Banks. 2002. Humans integrate visual and haptic information in a statistically optimal fashion. Nature 415, 6870 (2002), 429.

[18]

Pablo Etchemendy, Ezequiel Abregú, Esteban Calcagno, Manuel Eguía, Nilda Vechiatti, Federico Iasi, and Ramiro Vergara. 2017. Auditory environmental context affects visual distance perception. Scientific Reports 7(2017).

[19]

Daniel Finnegan. 2017. Compensating for distance compression in virtual audiovisual environments. Ph.D. Dissertation. University of Bath.

[20]

Daniel J. Finnegan, Eamonn O’Neill, and Michael J. Proulx. 2016. Compensating for Distance Compression in Audiovisual Virtual Environments Using Incongruence. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. ACM, 200–212.

Digital Library

[21]

Donald L Fisher, AP Pollatsek, and Abhishek Pradhan. 2006. Can novice drivers be trained to scan for information that will reduce their likelihood of a crash?Injury Prevention 12, suppl 1 (2006), i25–i29.

[22]

Timofey Y. Grechkin, Tien Dat Nguyen, Jodie M. Plumert, James F. Cremer, and Joseph K. Kearney. 2010. How Does Presentation Method and Measurement Protocol Affect Distance Estimation in Real and Virtual Environments?ACM Trans. Appl. Percept. 7, 4 (July 2010), 26:1–26:18.

[23]

William LeRoy Heinrichs, Patricia Youngblood, Phillip M Harter, and Parvati Dev. 2008. Simulation for team training and assessment: case studies of online training with virtual worlds. World journal of surgery 32, 2 (2008), 161–170.

[24]

Fernando Iazzetta, Fabio Kon, Marcelo Gomes de Queiroz, Flávio Soares, Correa da Silva, and Marcio de Avelar Gomes. 2004. AcMus: Computational tools for measurement, analysis and simulation of room acoustics. In Proceedings of the European Acoustics Symposium.

[25]

Victoria Interrante, Brian Ries, and Lee Anderson. 2006. Distance perception in immersive virtual environments, revisited. In IEEE virtual reality conference (VR 2006). IEEE, 3–10.

Digital Library

[26]

Neofytos Kaplanis, Søren Bech, Søren Holdt Jensen, and Toon van Waterschoot. 2014. Perception of Reverberation in Small Rooms: A Literature Study. In Audio Engineering Society Conference: 55th International Conference: Spatial Audio.

[27]

Jonathan W. Kelly, Lucia A. Cherep, Brenna Klesel, Zachary D. Siegel, and Seth George. 2018. Comparison of Two Methods for Improving Distance Perception in Virtual Reality. ACM Trans. Appl. Percept. 15, 2 (March 2018), 11:1–11:11.

Digital Library

[28]

Scott A. Kuhl, William B. Thompson, and Sarah H. Creem-Regehr. 2006. Minification Influences Spatial Judgments in Virtual Environments. In Proceedings of the 3rd Symposium on Applied Perception in Graphics and Visualization. ACM, 15–19.

Digital Library

[29]

Eike Langbehn, Tino Raupp, Gerd Bruder, Frank Steinicke, Benjamin Bolte, and Markus Lappe. 2016. Visual Blur in Immersive Virtual Environments: Does Depth of Field or Motion Blur Affect Distance and Speed Estimation?. In Proceedings of the 22Nd ACM Conference on Virtual Reality Software and Technology. ACM, 241–250.

Digital Library

[30]

Markus Leyrer, Sally A Linkenauger, Heinrich H Bülthoff, Uwe Kloos, and Betty Mohler. 2011. The influence of eye height and avatars on egocentric distance estimates in immersive virtual environments. In Proceedings of the ACM SIGGRAPH symposium on applied perception in graphics and visualization. 67–74.

Digital Library

[31]

Markus Leyrer, Sally A. Linkenauger, Heinrich H. Bülthoff, and Betty J. Mohler. 2015. Eye Height Manipulations: A Possible Solution to Reduce Underestimation of Egocentric Distances in Head-Mounted Displays. ACM Trans. Appl. Percept. 12, 1 (Feb. 2015), 1:1–1:23.

Digital Library

[32]

Bochao Li, Ruimin Zhang, and Scott Kuhl. 2014. Minication affects action-based distance judgments in oculus rift HMDs. In Proceedings of the ACM Symposium on Applied Perception. ACM, 91–94.

Digital Library

[33]

Chiuhsiang Joe Lin, Bereket Haile Woldegiorgis, Dino Caesaron, and Lai-Yu Cheng. 2015. Distance estimation with mixed real and virtual targets in stereoscopic displays. Displays 36(2015), 41–48.

[34]

Jack M Loomis, Joshua M Knapp, 2003. Visual perception of egocentric distance in real and virtual environments. Virtual and adaptive environments 11 (2003), 21–46.

[35]

Donald H Mershon and L Edward King. 1975. Intensity and reverberation as factors in the auditory perception of egocentric distance. Perception & Psychophysics 18, 6 (Nov. 1975), 409–415.

[36]

Ross Messing and Frank H Durgin. 2005. Distance perception and the visual horizon in head-mounted displays. ACM Transactions on Applied Perception (TAP) 2, 3 (2005), 234–250.

Digital Library

[37]

Damian T. Murphy and Simon Shelley. 2010. OpenAIR: An Interactive Auralization Web Resource and Database. In Audio Engineering Society Convention 129. Audio Engineering Society.

[38]

Adrian KT Ng, Leith KY Chan, and Henry YK Lau. 2017. Corrective feedback for depth perception in CAVE-like systems. In 2017 IEEE Virtual Reality (VR). IEEE, 293–294.

[39]

Alex Peer and Kevin Ponto. 2016. Perceptual space warping: Preliminary exploration. In 2016 IEEE Virtual Reality (VR). IEEE, 261–262.

[40]

Kevin Ponto, Michael Gleicher, Robert G Radwin, and Hyun Joon Shin. 2013. Perceptual calibration for immersive display environments. IEEE Transactions on Visualization and Computer Graphics 19, 4(2013), 691–700.

Digital Library

[41]

Rebekka S. Renner, Boris M. Velichkovsky, and Jens R. Helmert. 2013. The Perception of Egocentric Distances in Virtual Environments - A Review. ACM Comput. Surv. 46, 2 (Dec. 2013), 23:1–23:40.

Digital Library

[42]

Adam R Richardson and David Waller. 2005. The effect of feedback training on distance estimation in virtual environments. Applied Cognitive Psychology: The Official Journal of the Society for Applied Research in Memory and Cognition 19, 8 (2005), 1089–1108.

[43]

Jannick P Rolland, William Gibson, and Dan Ariely. 1995. Towards quantifying depth and size perception in virtual environments. Presence: Teleoperators & Virtual Environments 4, 1(1995), 24–49.

Digital Library

[44]

Cynthia S Sahm, Sarah H Creem-Regehr, William B Thompson, and Peter Willemsen. 2005. Throwing versus walking as indicators of distance perception in similar real and virtual environments. ACM Transactions on Applied Perception (TAP) 2, 1 (2005), 35–45.

Digital Library

[45]

Ladan Shams and Robyn Kim. 2010. Crossmodal influences on visual perception. Physics of life reviews 7, 3 (2010), 269–284.

[46]

Amy Turner, Jonathan Berry, and Nick Holliman. 2011. Can the perception of depth in stereoscopic images be influenced by 3D sound?Proc.SPIE 7863(2011), 1–10.

[47]

Vesa Valimaki, Julian D Parker, Lauri Savioja, Julius O Smith, and Jonathan S Abel. 2012. Fifty years of artificial reverberation. IEEE Transactions on Audio, Speech, and Language Processing 20, 5(2012), 1421–1448.

Digital Library

[48]

Koorosh Vaziri, Peng Liu, Sahar Aseeri, and Victoria Interrante. 2017. Impact of Visual and Experiential Realism on Distance Perception in VR Using a Custom Video See-through System. In Proceedings of the ACM Symposium on Applied Perception. ACM, 8:1–8:8.

Digital Library

[49]

Ekaterina Volkova, Stephan de la Rosa, Heinrich H. Bülthoff, and Betty Mohler. 2014a. The MPI Emotional Body Expressions Database for Narrative Scenarios. PLOS ONE 9, 12 (Dec. 2014), 1–28.

[50]

Ekaterina P Volkova, Betty J Mohler, Trevor J Dodds, Joachim Tesch, and Heinrich H Bülthoff. 2014b. Emotion categorization of body expressions in narrative scenarios. Frontiers in psychology 5 (2014), 623.

[51]

Jaye Wald and Steven Taylor. 2000. Efficacy of virtual reality exposure therapy to treat driving phobia: a case report. Journal of behavior therapy and experimental psychiatry 31, 3-4(2000), 249–257.

[52]

Peter Willemsen, Mark B Colton, Sarah H Creem-Regehr, and William B Thompson. 2009a. The effects of head-mounted display mechanical properties and field of view on distance judgments in virtual environments. ACM Transactions on Applied Perception (TAP) 6, 2 (2009), 1–14.

Digital Library

[53]

Peter Willemsen, Mark B. Colton, Sarah H. Creem-Regehr, and William B. Thompson. 2009b. The Effects of Head-mounted Display Mechanical Properties and Field of View on Distance Judgments in Virtual Environments. ACM Trans. Appl. Percept. 6, 2 (March 2009), 8:1–8:14.

Digital Library

[54]

Bob G Witmer and Paul B Kline. 1998. Judging perceived and traversed distance in virtual environments. Presence 7, 2 (1998), 144–167.

Digital Library

[55]

Yang Xi, Ning Gao, Mengchao Zhang, Lin Liu, and Qi Li. 2018. The Bayesian Causal Inference in Multisensory Information Processing: A Narrative Review. In International Conference on Intelligent Information Hiding and Multimedia Signal Processing. Springer, 151–161.

[56]

Seraphina Yong and Hao-Chuan Wang. 2018. Using Spatialized Audio to Improve Human Spatial Knowledge Acquisition in Virtual Reality. In Proceedings of the 23rd International Conference on Intelligent User Interfaces Companion. ACM, 51:1–51:2.

Digital Library

[57]

Mary K. Young, Graham B. Gaylor, Scott M. Andrus, and Bobby Bodenheimer. 2014. A Comparison of Two Cost-differentiated Virtual Reality Systems for Perception and Action Tasks. In Proceedings of the ACM Symposium on Applied Perception. ACM, 83–90.

Digital Library

[58]

Zhiying Zhou, Adrian David Cheok, Xubo Yang, and Yan Qiu. 2004. An experimental study on the role of software synthesized 3D sound in augmented reality environments. Interacting with Computers 16, 5 (2004), 989–1016.

[59]

Christine J Ziemer, Jodie M Plumert, James F Cremer, and Joseph K Kearney. 2009. Estimating distance in real and virtual environments: Does order make a difference?Attention, Perception, & Psychophysics 71, 5 (2009), 1095–1106.

Cited By

Lin WVenkatakrishnan RVenkatakrishnan RBabu SPagano CLin W(2024)An Empirical Evaluation of the Calibration of Auditory Distance Perception under Different Levels of Virtual Environment Visibilities2024 IEEE Conference Virtual Reality and 3D User Interfaces (VR)10.1109/VR58804.2024.00089(690-700)Online publication date: 16-Mar-2024
https://doi.org/10.1109/VR58804.2024.00089
Venkatakrishnan RVenkatakrishnan RRaveendranath BSarno DRobb ALin WBabu S(2024)The Effects of Auditory, Visual, and Cognitive Distractions on Cybersickness in Virtual RealityIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.329340530:8(5350-5369)Online publication date: Aug-2024
https://doi.org/10.1109/TVCG.2023.3293405
Paterson JKadel O(2023)Audio for extended realities: A case study informed expositionConvergence: The International Journal of Research into New Media Technologies10.1177/13548565231169723Online publication date: 5-Jun-2023
https://doi.org/10.1177/13548565231169723
Show More Cited By

Using Audio Reverberation to Compensate Distance Compression in Virtual Reality
1. Computing methodologies
  1. Computer graphics
    1. Graphics systems and interfaces
2. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction paradigms

Recommendations

Near-field distance perception in real and virtual environments using both verbal and action responses

Few experiments have been performed to investigate near-field egocentric distance estimation in an Immersive Virtual Environment (IVE) as compared to the Real World (RW). This article investigates near-field distance estimation in IVEs and RW conditions ...
The effects of virtual reality, augmented reality, and motion parallax on egocentric depth perception
APGV '08: Proceedings of the 5th symposium on Applied perception in graphics and visualization

As the use of virtual and augmented reality applications becomes more common, the need to fully understand how observers perceive spatial relationships grows more critical. One of the key requirements in engineering a practical virtual or augmented ...
Extending Virtual Reality Display Wall Environments Using Augmented Reality
SUI '19: Symposium on Spatial User Interaction

Two major form factors for virtual reality are head-mounted displays and large display environments such as CAVE®and the LCD-based successor CAVE2®. Each of these has distinct advantages and limitations based on how they’re used. This work explores ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

SAP '21: ACM Symposium on Applied Perception 2021

September 2021

109 pages

ISBN:9781450386630

DOI:10.1145/3474451

Editors:
Eakta Jain
University of Florida, USA
,
Anne-Hélène Olivier
University of Rennes 2, France
,
Massimiliano Di Luca
University of Birmingham, UK
,
Katja Zibrek
Inria Rennes, France
,
Rebecca Fribourg
Trinity College Dublin, Ireland

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGGRAPH: ACM Special Interest Group on Computer Graphics and Interactive Techniques

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 16 September 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

SAP '21

Sponsor:

SIGGRAPH

SAP '21: ACM Symposium on Applied Perception 2021

September 16 - 17, 2021

Virtual Event, France

Acceptance Rates

Overall Acceptance Rate 43 of 94 submissions, 46%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
340
Total Downloads

Downloads (Last 12 months)80
Downloads (Last 6 weeks)10

Reflects downloads up to 01 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Lin WVenkatakrishnan RVenkatakrishnan RBabu SPagano CLin W(2024)An Empirical Evaluation of the Calibration of Auditory Distance Perception under Different Levels of Virtual Environment Visibilities2024 IEEE Conference Virtual Reality and 3D User Interfaces (VR)10.1109/VR58804.2024.00089(690-700)Online publication date: 16-Mar-2024
https://doi.org/10.1109/VR58804.2024.00089
Venkatakrishnan RVenkatakrishnan RRaveendranath BSarno DRobb ALin WBabu S(2024)The Effects of Auditory, Visual, and Cognitive Distractions on Cybersickness in Virtual RealityIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.329340530:8(5350-5369)Online publication date: Aug-2024
https://doi.org/10.1109/TVCG.2023.3293405
Paterson JKadel O(2023)Audio for extended realities: A case study informed expositionConvergence: The International Journal of Research into New Media Technologies10.1177/13548565231169723Online publication date: 5-Jun-2023
https://doi.org/10.1177/13548565231169723
Tolchinsky MVenkatakrishnan RVenkatakrishnan RPagano CBabu S(2023)Empirical Evaluation of the Effects of Visuo-Auditory Perceptual Information on Head Oriented Tracking of Dynamic Objects in VR2023 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)10.1109/ISMAR59233.2023.00124(1074-1083)Online publication date: 16-Oct-2023
https://doi.org/10.1109/ISMAR59233.2023.00124
Buck LVargas MMcDonnell R(2022)The Effect of Spatial Audio on the Virtual Representation of Personal Space2022 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW)10.1109/VRW55335.2022.00079(354-356)Online publication date: Mar-2022
https://doi.org/10.1109/VRW55335.2022.00079
Lin WWang YWu DVenkatakrishnan RVenkatakrishnan REbrahimi EPagano CBabu SLin W(2022)Empirical Evaluation of Calibration and Long-term Carryover Effects of Reverberation on Egocentric Auditory Depth Perception in VR2022 IEEE Conference on Virtual Reality and 3D User Interfaces (VR)10.1109/VR51125.2022.00042(232-240)Online publication date: Mar-2022
https://doi.org/10.1109/VR51125.2022.00042

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents