Nothing Special   »   [go: up one dir, main page]

skip to main content
research-article

Moving sounds enhance the visually-induced self-motion illusion (circular vection) in virtual reality

Published: 10 March 2009 Publication History

Abstract

While rotating visual and auditory stimuli have long been known to elicit self-motion illusions (“circular vection”), audiovisual interactions have hardly been investigated. Here, two experiments investigated whether visually induced circular vection can be enhanced by concurrently rotating auditory cues that match visual landmarks (e.g., a fountain sound). Participants sat behind a curved projection screen displaying rotating panoramic renderings of a market place. Apart from a no-sound condition, headphone-based auditory stimuli consisted of mono sound, ambient sound, or low-/high-spatial resolution auralizations using generic head-related transfer functions (HRTFs). While merely adding nonrotating (mono or ambient) sound showed no effects, moving sound stimuli facilitated both vection and presence in the virtual environment. This spatialization benefit was maximal for a medium (20° × 15°) FOV, reduced for a larger (54° × 45°) FOV and unexpectedly absent for the smallest (10° × 7.5°) FOV. Increasing auralization spatial fidelity (from low, comparable to five-channel home theatre systems, to high, 5° resolution) provided no further benefit, suggesting a ceiling effect. In conclusion, both self-motion perception and presence can benefit from adding moving auditory stimuli. This has important implications both for multimodal cue integration theories and the applied challenge of building affordable yet effective motion simulators.

References

[1]
Algazi, V. R., Duda, R. O., Thompson, D. M., and Avendano, C. 2001. The CIPIC HRTF database. In Proceedings of the IEEE Workshop on Applications of Signal Processing to Audio and Acoustics. IEEE, Los Alamitos, CA, 99--102.
[2]
Best, V., Van Schaik, A., Jin, C., and Carlile, S. 2005. Auditory spatial perception with sources overlapping in frequency and time. Acta Acustica 91, 421--428.
[3]
Boer, E. R., Girshik, A. R., Yamamura, T., and Kuge, N. 2000. Experiencing the same road twice: A driver-centred comparison between simulation and reality. In Proceedings of the Driving Simulation Conference (DSC'00). Elsevier, New York.
[4]
Brandt, T., Dichgans, J., and Held, R. 1973. Optokinesis affects body posture and subjective visual vertical. Pflugers Archiv-European J. Physiol. 339, 97--97.
[5]
Brandt, T., Dichgans, J., and Koenig, E. 1973. Differential effects of central versus peripheral vision on egocentric and exocentric motion perception. Exp. Brain Res. 16, 476--491.
[6]
Brungart, D., Simpson, B., and Kordik, A. 2005. Localization in the presence of multiple simultaneous sounds. Acta Acustica 91, 3, 471--479.
[7]
Burki-Cohen, J., Go, T. H., Chung, W. Y., Schroeder, J., Jacobs, S., and Longridge, T. 2003. Simulator fidelity requirements for airline pilot training and evaluation continued: An update on motion requirements research. In Proceedings of the 12th International Symposium on Aviation Psychology. 182--189.
[8]
Cater, K., Chalmers, A., and Ward, G. 2003. Detail to attention: Exploiting visual tasks for selective rendering. In Proceedings of the 14th Eurographics Workshop on Rendering (EGRW '03). ACM, New York, 270--280.
[9]
Chance, S. S., Gaunet, F., Beall, A. C., and Loomis, J. M. 1998. Locomotion mode affects the updating of objects encountered during travel: The contribution of vestibular and proprioceptive inputs to path integration. Presence Teleoperators Virtual Environ. 7, 2, 168--178.
[10]
Dichgans, J. and Brandt, T. 1978. Visual-vestibular interaction: Effects on self-motion perception and postural control. In Perception. Handbook of Sensory Physiology, vol. VIII. Springer, New York, 756--804.
[11]
Dodge, R. 1923. Thresholds of rotation. J. Exp. Psychol. 6, 107--137.
[12]
Durgin, F. H., Pelah, A., Fox, L. F., Lewis, J. Y., Kane, R., and Walley, K. A. 2005. Self-motion perception during locomotor recalibration: More than meets the eye. J. Exp. Psychol. 31, 3, 398--419.
[13]
Durlach, N. I. and Mavor, A. S., Eds. 1995. Virtual Reality: Scientific and Technological Challenges. National Academy Press, Washington, DC.
[14]
Ernst, M. O. and Bülthoff, H. H. 2004. Merging the senses into a robust percept. Trends Cog. Sci. 8, 4, 162--169.
[15]
Fischer, M. H. and Kornmüller, A. E. 1930. Optokinetisch ausgelöste Bewegungswahrnehmung und optokinetischer Nystagmus {Optokinetically induced motion perception and optokinetic nystagmus}. Journal für Psychologie und Neurologie, 273--308.
[16]
Flanagan, M. B., May, J. G., and Dobie, T. G. 2004. The role of vection, eye movements and postural instability in the etiology of motion sickness. J. Vestibular Res. Equilibrium Orientation 14, 4, 335--346.
[17]
Gardner, W. and Martin, K. 1995. Hrtf measurements of a KEMAR. J. Acoust. Soc. Am. 97, 3907--3908.
[18]
Gekhman, B. 1991. Audiokinetic nystagmus. Sensornye Sistemy 5, 2, 71--78. (In Russian.)
[19]
Hendrix, C. and Barfield, W. 1996. The sense of presence within auditory virtual environments. Presence Teleoperators Virtual Environ. 5, 3, 290--301.
[20]
Hennebert, P. E. 1960. Audiokinetic nystagmus. J. Auditory Res. 1, 1, 84--87.
[21]
Hettinger, L. J. 2002. Illusory self-motion in virtual environments. In Handbook of Virtual Environments, K. M. Stanney, Ed. Lawrence Erlbaum, Mahwah, NJ, 471--492.
[22]
Hollerbach, J. M. 2002. Locomotion interfaces. In Handbook of Virtual Environments, K. M. Stanney, Ed. Lawrence Erlbaum, Mahwah, NJ, 239--254.
[23]
Howard, I. P. 1982. Human Visual Orientation. J. Wiley, Hoboken, NJ.
[24]
Howard, I. P. 1986. The perception of posture, self motion, and the visual vertical. In Handbook of Human Perception and Performance, vol. 1, K. R. Boff, L. Kaufman, and J. P. Thomas, Eds. Wiley, Hoboken, NJ, 18.1--18.62.
[25]
Kapralos, B., Zikovitz, D., Jenkin, M., and Harris, L. 2004. Auditory cues in the perception of self-motion. In Proceedings of the 116th AES Convention.
[26]
Kleiner, M. and Dalenbäck, B-I. Andsvensson, P. 1993. Auralization -- an overview. J. Audio Engin. Soc. 41, 11, 861--875.
[27]
Lackner, J. R. 1977. Induction of illusory self-rotation and nystagmus by a rotating sound-field. Aviat. Space Environ. Med. 48, 2, 129--131.
[28]
Langendijk, E. H. A. and Bronkhorst, A. W. 2000. Fidelity of three-dimensional-sound reproduction using a virtual auditory display. Acoustical Soc. Am. J. 107, 528--537.
[29]
Larsson, P., Väljamäe, A., Västfjäll, D., Tajadura-Jiménez, A., and Kleiner, M. 2009. Auditory-induced presence in mixed presence reality environments and related technology. In The Engineering of Mixed Reality Systems. E. Dubois, L. Nigay, and P. Gray, Eds. Springer.
[30]
Larsson, P., Västfjäll, D., Tajadura and Kleiner, M. 2004. Perception of self-motion and presence in auditory virtual environments. In Proceedings of 7th Annual Workshop of Presence. 252--258. http://www.kyb.mpg.de/publication.html?publ=2953.
[31]
Lepecq, J. C., Giannopulu, I., and Baudonniere, P. M. 1995. Cognitive effects on visually induced body motion in children. Perception 24, 4, 435--449.
[32]
Lombard, M. and Ditton, T. 1999. At the heart of it all: The concept of presence. J. Comput. Mediated Comm. 3, 2.
[33]
Luciani, A. 2004. Dynamics as a common criterion to enhance the sense of presence in virtual environments. In Proceedings of 7th Annual Workshop Presence 2004. 96--103.
[34]
Mach, E. 1875. Grundlinien der Lehre von der Bewegungsempfindung. Engelmann, Leipzig, Germany.
[35]
Marmekarelse, A. M. and Bles, W. 1977. Circular vection and human posture ii: Does the auditory-system play a role. Agressologie 18, 6, 329--333.
[36]
McFarland, W. and Weber, B. 1969. An investigation of ocular response to various forms of sound field auditory stimulation. J. Auditory Res. 9, 3, 236--239.
[37]
Moeck, T., Bonneel, N., Tsingos, N., Drettakis, G., Viaud-Delmon, I., and Alloza, D. 2007. Progressive perceptual audio rendering of complex scenes. In Proceedings of the Symposium on Interactive 3D Graphics and Games (I3D '07). ACM, New York, 189--196.
[38]
Mulder, M., Van Paassen, M. M., and Boer, E. R. 2004. Exploring the roles of information in the control of vehicular locomotion—from kinematics and dynamics to cybernetics. Presence Teleoperators Virtual Environ. 13, 535--548.
[39]
Ozawa, K., Chujo, Y., Suzuki, Y., and Sone, T. 2004. Psychological factors involved in auditory presence. Acoustical Sci. Tech. 24, 42--44.
[40]
Pope, J. and Chalmers, A. 1999. Multi-sensory rendering: Combining graphics and acoustics. In Proceedings of the 7th International Conference in Central Europe on Computer Graphics. 233--242.
[41]
Riecke, B. E., Feuereissen, D., and Rieser, J. J. 2009. Auditory self-motion simulation is facilitated by haptic and vibrational cues suggesting the possibility of actual motion. ACM Trans. Appl. Percept. Submitted.
[42]
Riecke, B. E., Feuereissen, D., and Rieser, J. J. 2008a. Auditory self-motion illusions (“circular vection”) can be facilitated by vibrations and the potential for actual motion. In Proceedings of the ACM SIGGRAPH Symposium on Applied Perception in Graphics and Visualization (APGV '08). ACM, New York, 147--154.
[43]
Riecke, B. E., Feuereissen, D., and Rieser, J. J. 2008b. Contribution and interaction of auditory and biomechanical cues for self-motion illusions (“circular vection”). In Proceedings of the CyberWalk Workshop.
[44]
Riecke, B. E., Schulte-Pelkum, J., Avraamides, M. N., and Bülthoff, H. H. 2004. Enhancing the visually induced self-motion illusion (vection) under natural viewing conditions in virtual reality. In Proceedings of 7th Annual Workshop Presence 2004. 125--132. www.kyb.mpg.de/publication.html?publ=2864.
[45]
Riecke, B. E., Schulte-Pelkum, J., Avraamides, M. N., von der Heyde, M., and Bülthoff, H. H. 2006. Cognitive factors can influence self-motion perception (vection) in virtual reality. ACM Trans. Appl. Percep. 3, 3, 194--216.
[46]
Riecke, B. E., Schulte-Pelkum, J., and Bülthoff, H. H. 2005. Perceiving simulated ego-motions in virtual reality: Comparing large screen displays with HMDs. Proc. SPIE, Vol. 5666. SPIE, Bellingham, WA, 344--355.
[47]
Riecke, B. E., Schulte-Pelkum, J., Caniard, F., and Bülthoff, H. H. 2005a. Influence of auditory cues on the visually-induced self-motion illusion (circular vection) in virtual reality. In Proceedings of 8th Annual Workshop Presence 2005. ACM, New York, 49--57.
[48]
Riecke, B. E., Schulte-Pelkum, J., Caniard, F., and Bülthoff, H. H. 2005b. Towards lean and elegant self-motion simulation in virtual reality. In Proceedings of IEEE Virtual Reality. IEEE, Los Alamitos, CA, 131--138.
[49]
Riecke, B. E., Västfjäll, D., Larsson, P., and Schulte-Pelkum, J. 2005. Top-down and multi-modal influences on self-motion perception in virtual reality. In Proceedings of the Internation Human-Computer Interaction Conference. HCI, Toronto, Ontario, 1--10.
[50]
Riecke, B. E. and Wiener, J. M. 2007. Can people not tell left from right in vr? point-to-origin studies revealed qualitative errors in visual path integration. In Proceedings of IEEE Virtual Reality 2007. IEEE, Los Alamitos, CA, 3--10.
[51]
Rumsey, F. 2002. Spatial quality evaluation for reproduced sound: Terminology, meaning, and a scene-based paradigm. J. Audio Engin. Soc. 50, 9, 651.
[52]
Sakamoto, S., Osada, Y., Suzuki, Y., and Gyoba, J. 2004. The effects of linearly moving sound images on selfmotion perception. Acoustical Sci. Tech. 25, 100--102.
[53]
Schinauer, T., Hellmann, A., and Höger, R. 1993. Dynamic acoustical stimulation affects self-motion perception. In Contributions to Psychological Acoustics. Results of the 6th Oldenburg Symposium on Psychological Acoustics, A. Schick, Ed. BIS, Oldenburg, Germany, 373--385.
[54]
Schubert, T., Friedmann, F., and Regenbrecht, H. 2001. The experience of presence: Factor analytic insights. Presence Teleop. Virtual Environ. 10, 3, 266--281.
[55]
Schulte-Pelkum, J. and Riecke, B. E. 2009. An integrative approach to presence and self-motion perception research. http://www.sfu.ca/~ber1/web/Shuite-PelkumRiecke_HOP_chapter.pdf.
[56]
Schulte-Pelkum, J., Riecke, B. E., and Bülthoff, H. H. 2004. Vibrational cues enhance believability of ego-motion simulation. In International Multisensory Research Forum (IMRF). http://www.kyb.mpg.de/publication.html?publ=2766.
[57]
Schulte-Pelkum, J., Riecke, B. E., von der Heyde, M., and Bülthoff, H. H. 2004. Influence of display device and screen curvature on perceiving and controlling simulated ego-rotations from optic flow. Tech. rep. 122, MPI for Biological Cybernetics. http://www.kyb.mpg.de/publication.html?publ=2574.
[58]
Soto-Faraco, S., Kingstone, A., and Spence, C. 2003. Multisensory contributions to the perception of motion. Neuropsychologia 41, 13, 1847--1862.
[59]
Sundstedt, V., Debattista, K., and Chalmers, A. 2004. Selective rendering using task-importance maps. In Proceedings of the 1st Symposium on Applied Perception in Graphics and Visualization (APGV'04). ACM, New York, 175.
[60]
Tan, D. S., Gergle, D., Scupelli, P., and Pausch, R. 2006. Physically large displays improve performance on spatial tasks. ACM Trans. Comput. Hum. Interact. 13, 1, 71--99.
[61]
Tan, D. S., Gergle, D., Scupelli, P. G., and Pausch, R. 2004. Physically large displays improve path integration in 3D virtual navigation tasks. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '04). ACM Press, New York, 439--446.
[62]
Väljamäe, A., Larsson, P., Tajadura-Jiménez, A., Västfjäll, D., and Kleiner, M. 2008. Auditory cues for multisensory enhancement of self-motion perception. IEEE Multimedia.
[63]
Väljamäe, A., Larsson, P., Västfjäll, D., and Kleiner, M. 2004. Auditory presence, individualized head-related transfer functions, and illusory ego-motion in virtual environments. In Proceedings of 7th Annual Workshop of Presence. 141--147.
[64]
Väljamäe, A., Larsson, P., Västfjäll, D., and Kleiner, M. 2005. Travelling without moving: Auditory scene cues for translational self-motion. In Proceedings of the 11th Meeting of International Conference on Auditory Display (ICAD 2005).
[65]
Väljamäe, A., Larsson, P., Västfjäll, D., and Kleiner, M. 2006. Vibrotactile enhancement of auditory induced self-motion and spatial presence. J. Acoustic Engin. Soc. 54, 10, 954--963.
[66]
Väljamäe, A., Larsson, P., Västfjäll, D., and Kleiner, M. 2008a. Auditory landmarks enhance multimodal circular vection simulations in virtual reality. J. Acoustic Engin. Soc. Submitted.
[67]
Väljamäe, A., Larsson, P., Västfjäll, D., and Kleiner, M. 2008b. Sound representing self-motion in virtual environments enhances linear vection. Presence: Teleoperators Virtual Environ. 17, 1, 43--56.
[68]
Väljamäe, A., Västfjäll, D., Larsson, P., and Kleiner, M. 2008. Perceived sound in mediated environments. In Immersed in Media Experiences: Presence Psychology and Design (Handbook of Presence). Lawrence Erlbaum, Mahwah, NJ.
[69]
Van Der Steen, F. A. M. and Brockhoff, P. T. M. 2000. Induction and impairment of saturated yaw and surge vection. Percep. Psychophys. 62, 1, 89--99.
[70]
Von Helmholtz, H. 1896. Handbuch der physiologischen Optik. L. Voss, Hamburg-Leipzig.
[71]
Warren, R. and Wertheim, A. H., Eds. 1990. Perception & Control of Self-Motion. Erlbaum, Mahwah, NJ.
[72]
Watson, J. E. 1968. Evaluation of audiokinetic nystagmus as as test of auditory sensitivity. J. Auditory Res. 8, 161--165.
[73]
Wright, W. G., Dizio, P., and Lackner, J. R. 2006. Perceived self-motion in two visual contexts: dissociable mechanisms underlie perception. J. Vestib. Res. 16, 1--2, 23--28.

Cited By

View all
  • (2024)Classifying Presence Scores: Insights and Analysis from Two Decades of the Igroup Presence Questionnaire (IPQ)ACM Transactions on Computer-Human Interaction10.1145/368904631:5(1-26)Online publication date: 17-Aug-2024
  • (2024)The Effect of Directional Airflow toward Vection and Cybersickness2024 IEEE Conference Virtual Reality and 3D User Interfaces (VR)10.1109/VR58804.2024.00103(839-848)Online publication date: 16-Mar-2024
  • (2023)Measuring vection: a review and critical evaluation of different methods for quantifying illusory self-motionBehavior Research Methods10.3758/s13428-023-02148-856:3(2292-2310)Online publication date: 27-Jun-2023
  • Show More Cited By

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Transactions on Applied Perception
ACM Transactions on Applied Perception  Volume 6, Issue 2
February 2009
111 pages
ISSN:1544-3558
EISSN:1544-3965
DOI:10.1145/1498700
Issue’s Table of Contents
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 10 March 2009
Accepted: 01 May 2008
Revised: 01 March 2008
Received: 01 August 2007
Published in TAP Volume 6, Issue 2

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Audiovisual interactions
  2. presence
  3. psychophysics
  4. self-motion simulation
  5. spatial sound
  6. vection
  7. virtual reality

Qualifiers

  • Research-article
  • Research
  • Refereed

Funding Sources

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)61
  • Downloads (Last 6 weeks)10
Reflects downloads up to 23 Nov 2024

Other Metrics

Citations

Cited By

View all
  • (2024)Classifying Presence Scores: Insights and Analysis from Two Decades of the Igroup Presence Questionnaire (IPQ)ACM Transactions on Computer-Human Interaction10.1145/368904631:5(1-26)Online publication date: 17-Aug-2024
  • (2024)The Effect of Directional Airflow toward Vection and Cybersickness2024 IEEE Conference Virtual Reality and 3D User Interfaces (VR)10.1109/VR58804.2024.00103(839-848)Online publication date: 16-Mar-2024
  • (2023)Measuring vection: a review and critical evaluation of different methods for quantifying illusory self-motionBehavior Research Methods10.3758/s13428-023-02148-856:3(2292-2310)Online publication date: 27-Jun-2023
  • (2023)You spin me right round, baby, right round: Examining the Impact of Multi-Sensory Self-Motion Cues on Motion Sickness During a VR Reading TaskProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3580966(1-16)Online publication date: 19-Apr-2023
  • (2023)Inducing Self-Motion Sensations With Haptic Feedback: State-of-The-Art and Perspectives on “Haptic Motion”IEEE Transactions on Haptics10.1109/TOH.2023.327926716:2(171-181)Online publication date: 1-Apr-2023
  • (2023)Where is the Sound: User Sound Source Perception in Virtual Reality Environment2023 IEEE Smart World Congress (SWC)10.1109/SWC57546.2023.10449009(1-8)Online publication date: 28-Aug-2023
  • (2023)Design Guidelines for an Immersive Auditory ExperienceDesign in the Era of Industry 4.0, Volume 110.1007/978-981-99-0293-4_51(641-647)Online publication date: 25-Jul-2023
  • (2023)Perceived LocationThe Perceptual Structure of Sound10.1007/978-3-031-25566-3_9(449-558)Online publication date: 11-Jun-2023
  • (2022)Multimodality in VR: A SurveyACM Computing Surveys10.1145/350836154:10s(1-36)Online publication date: 13-Sep-2022
  • (2022)Does the Vividness of Imagination Influence Illusory Self-Motion in Virtual Reality?2022 IEEE International Conference on Systems, Man, and Cybernetics (SMC)10.1109/SMC53654.2022.9945359(1065-1071)Online publication date: 9-Oct-2022
  • Show More Cited By

View Options

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media