research-article

Performance Is Not Everything: Audio Feedback Preferred Over Visual Feedback for Grasping Task in Virtual Reality

Authors:

Sophie JörgAuthors Info & Claims

MIG '20: Proceedings of the 13th ACM SIGGRAPH Conference on Motion, Interaction and Games

Article No.: 4, Pages 1 - 6

https://doi.org/10.1145/3424636.3426897

Published: 22 November 2020 Publication History

Abstract

In this work, we investigate the influence that audio and visual feedback have on a manipulation task in virtual reality (VR). Without the tactile feedback of a controller, grasping virtual objects using one’s hands can result in slower interactions because it may be unclear to the user that a grasp has occurred. Providing alternative feedback, such as visual or audio cues, may lead to faster and more precise interactions, but might also affect user preference and perceived ownership of the virtual hands. In this study, we test four feedback conditions for virtual grasping. Three of the conditions provide feedback for when a grasp or release occurs, either visual, audio, or both, and one provides no feedback for these occurrences. We analyze the effect each feedback condition has on interaction performance, measure their effect on the perceived ownership of the virtual hands, and gauge user preference. In an experiment, users perform a pick-and-place task with each feedback condition. We found that audio feedback for grasping is preferred over visual feedback even though it seems to decrease grasping performance, and found that there were little to no differences in ownership between our conditions.

References

[1]

Motoyuki Akamatsu, I. Scott Mackenzie, and Thierry Hasbroucq. 1995. A comparison of tactile, auditory, and visual feedback in a pointing task using a mouse-type device. Ergonomics 38, 4 (1995), 816–827. https://doi.org/10.1080/00140139508925152

[2]

Ferran Argelaguet, Ludovic Hoyet, Michaël Trico, and Anatole Lecuyer. 2016. The role of interaction in virtual embodiment: Effects of the virtual hand representation. In 2016 IEEE Virtual Reality. 3–10. https://doi.org/10.1109/VR.2016.7504682

[3]

Oscar Ariza, Gerd Bruder, Nicholas Katzakis, and Frank Steinicke. 2018. Analysis of Proximity-Based Multimodal Feedback for 3D Selection in Immersive Virtual Environments. 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR) (March 2018), 327–334. https://doi.org/10.1109/VR.2018.8446317

[4]

Frank Biocca, Jin Kim, and Yung Kyun Choi. 2001. Visual Touch in Virtual Environments: An Exploratory Study of Presence, Multimodal Interfaces, and Cross-Modal Sensory Illusions. Presence 10 (06 2001), 247–265. https://doi.org/10.1162/105474601300343595

Digital Library

[5]

Meera M. Blattner, Denise A. Sumikawa, and Robert M. Greenberg. 1989. Earcons and Icons: Their Structure and Common Design Principles (Abstract Only). SIGCHI Bull. 21, 1 (Aug. 1989), 123–124. https://doi.org/10.1145/67880.1046599

Digital Library

[6]

Matthew Botvinick and Jonathan Cohen. 1998. Rubber hands ’feel’ touch that eyes see. Nature 391 (Feb. 1998), 756. https://doi.org/10.1038/35784

[7]

Ryan Canales, Aline Normoyle, Yu Sun, Yuting Ye, Massimiliano Di Luca, and Sophie Jörg. 2019. Virtual Grasping Feedback and Virtual Hand Ownership. In ACM Symposium on Applied Perception 2019(SAP ’19). ACM, Article 4, 9 pages. https://doi.org/10.1145/3343036.3343132

Digital Library

[8]

Andy Cockburn and Stephen Brewster. 2005. Multimodal feedback for the acquisition of small targets. Ergonomics 48, 9 (2005), 1129–1150. https://doi.org/10.1080/00140130500197260

[9]

Natalia Cooper, Ferdinando Milella, Carlo Pinto, Iain Cant, Mark White, and Georg Meyer. 2018. The effects of substitute multisensory feedback on task performance and the sense of presence in a virtual reality environment. PLOS ONE 13, 2 (02 2018), 1–25. https://doi.org/10.1371/journal.pone.0191846

[10]

Guillaume Cortes, Ferran Argelaguet, Eric Marchand, and Anatole Lécuyer. 2018. Virtual Shadows for Real Humans in a CAVE: Influence on Virtual Embodiment and 3D Interaction. In Proceedings of the 15th ACM Symposium on Applied Perception(SAP ’18). ACM, Article 1, 8 pages. https://doi.org/10.1145/3225153.3225165

Digital Library

[11]

Inger Ekman. 2013. On the desire to not kill your players: Rethinking sound in pervasive and mixed reality games. In Proceedings of the 8th International Conference on the Foundations of Digital Games 2013. Society for the Advancement of the Science of Digital Games, 142–149.

[12]

Andreas Geiger, Imke Bewersdorf, Elisabeth Brandenburg, and Rainer Stark. 2018. Visual Feedback for Grasping in Virtual Reality Environments for an Interface to Instruct Digital Human Models. In Advances in Usability and User Experience. Springer International Publishing, 228–239. https://doi.org/10.1007/978-3-319-60492-3_22

[13]

Shangchen Han, Beibei Liu, Robert Wang, Yuting Ye, Christopher D. Twigg, and Kenrick Kin. 2018. Online Optical Marker-based Hand Tracking with Deep Labels. ACM Trans. Graph. 37, 4, Article 166 (July 2018), 10 pages. https://doi.org/10.1145/3197517.3201399

Digital Library

[14]

Lorraine Lin and Sophie Jörg. 2016. Need a Hand?: How Appearance Affects the Virtual Hand Illusion. In Proceedings of the ACM Symposium on Applied Perception(SAP ’16). ACM, 69–76. https://doi.org/10.1145/2931002.2931006

Digital Library

[15]

Matthew Longo, Friederike Schüür, Marjolein P M Kammers, Manos Tsakiris, and Patrick Haggard. 2008. What is embodiment? A psychometric approach. Cognition 107 (July 2008), 978–98. https://doi.org/10.1016/j.cognition.2007.12.004

[16]

Jean-Luc Lugrin, David Obremski, Daniel Roth, and Marc Erich Latoschik. 2016. Audio Feedback and Illusion of Virtual Body Ownership in Mixed Reality. In Proceedings of the 22nd ACM Conference on Virtual Reality Software and Technology(VRST ’16). ACM, 309–310. https://doi.org/10.1145/2993369.2996319

Digital Library

[17]

Jim R. Parker and John Heerma. 2008. Audio Interaction in Computer Mediated Games. International Journal of Computer Games Technology (2008). https://doi.org/10.1155/2008/178923

[18]

Mores Prachyabrued and Christoph W. Borst. 2014. Visual Feedback for Virtual Grasping. 2014 IEEE Symposium on 3D User Interfaces (3DUI) (March 2014), 19–26. https://doi.org/10.1109/3DUI.2014.6798835

[19]

Valerio Santangelo and Charles Spence. 2007. Multisensory Cues Capture Spatial Attention Regardless of Perceptual Load. Journal of Experimental Psychology. Human Perception and Performance 33 (Dec 2007), 1311–21. https://doi.org/10.1037/0096-1523.33.6.1311

[20]

Roland Sigrist, Georg Rauter, Robert Riener, and Peter Wolf. 2012. Augmented visual, auditory, haptic, and multimodal feedback in motor learning: A review. Psychonomic Bulletin & Review 20 (Nov 2012), 21–53. https://doi.org/10.3758/s13423-012-0333-8

[21]

Lode Vanacken, Tovi Grossman, and Karin Coninx. 2009. Multimodal Selection Techniques for Dense and Occluded 3D Virtual Environments. Int. J. Hum.-Comput. Stud. 67, 3 (March 2009), 237–255. https://doi.org/10.1016/j.ijhcs.2008.09.001

Digital Library

[22]

Spyros Vosinakis and Panayiotis Koutsabasis. 2018. Evaluation of visual feedback techniques for virtual grasping with bare hands using Leap Motion and Oculus Rift. Virtual Reality 22, 1 (01 Mar 2018), 47–62. https://doi.org/10.1007/s10055-017-0313-4

Digital Library

[23]

Mathias Wellner, Audrey Schaufelberger, Joachim v. Zitzewitz, and Robert Riener. 2008. Evaluation of Visual and Auditory Feedback in Virtual Obstacle Walking. Presence: Teleoperators and Virtual Environments 17, 5(2008), 512–524. https://doi.org/10.1162/pres.17.5.512

Digital Library

[24]

Ye Yuan and Anthony Steed. 2010. Is the Rubber Hand Illusion Induced by Immersive Virtual Reality?. In Proceedings of the 2010 IEEE Virtual Reality Conference(VR ’10). IEEE Computer Society, 95–102. https://doi.org/10.1109/VR.2010.5444807

Digital Library

[25]

Y. Zhang, T. Fernando, H. Xiao, and A. R. L. Travis. 2006. Evaluation of Auditory and Visual Feedback on Task Performance in a Virtual Assembly Environment. Presence 15, 6 (Dec 2006), 613–626. https://doi.org/10.1162/pres.15.6.613

Digital Library

Cited By

Karaosmanoglu SWegner MCmentowski SSteinicke F(2024)Move, React, Repeat! The Role of Continuous Cues in Immersive ExergamesProceedings of the ACM on Human-Computer Interaction10.1145/36770918:CHI PLAY(1-23)Online publication date: 15-Oct-2024
https://dl.acm.org/doi/10.1145/3677091
Bashar MBatmaz A(2024)Virtual Task Environments Factors Explored in 3D Selection StudiesProceedings of the 50th Graphics Interface Conference10.1145/3670947.3670983(1-16)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3670947.3670983
Adkins ACanales RJörg S(2024)Hands or Controllers? How Input Devices and Audio Impact Collaborative Virtual RealityProceedings of the 30th ACM Symposium on Virtual Reality Software and Technology10.1145/3641825.3687718(1-12)Online publication date: 9-Oct-2024
https://dl.acm.org/doi/10.1145/3641825.3687718
Show More Cited By

Recommendations

Virtual Grasping Feedback and Virtual Hand Ownership
SAP '19: ACM Symposium on Applied Perception 2019

In this work, we investigate the influence of different visualizations on a manipulation task in virtual reality (VR). Without the haptic feedback of the real world, grasping in VR might result in intersections with virtual objects. As people are highly ...
Effective Visual Feedback for Virtual Grasping
SIGGRAPH '24: ACM SIGGRAPH 2024 Posters

The purpose of this study is to examine effectiveness of visual feedback in virtual grasping with a bare hand. We propose two types of visual feedback that can generate a pseudo-haptics and compare them with five types of existing visual feedback. In an ...
Factors of Haptic Feedback in a VR Experience Using Virtual Tools: Evaluating the Impact of Visual and Force Presentation
Virtual, Augmented and Mixed Reality
Abstract
In this study, we explore the factors influencing the generation of haptic feedback during interactions with virtual objects using a rod-shaped tool within a virtual reality environment. Previously, we introduced a haptic feedback method involving ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

MIG '20: Proceedings of the 13th ACM SIGGRAPH Conference on Motion, Interaction and Games

October 2020

190 pages

ISBN:9781450381710

DOI:10.1145/3424636

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGGRAPH: ACM Special Interest Group on Computer Graphics and Interactive Techniques

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 22 November 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

National Science Foundation

Conference

MIG '20

Sponsor:

SIGGRAPH

MIG '20: Motion, Interaction and Games

October 16 - 18, 2020

SC, Virtual Event, USA

Acceptance Rates

Overall Acceptance Rate -9 of -9 submissions, 100%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

9
Total Citations
View Citations
351
Total Downloads

Downloads (Last 12 months)89
Downloads (Last 6 weeks)17

Reflects downloads up to 23 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Karaosmanoglu SWegner MCmentowski SSteinicke F(2024)Move, React, Repeat! The Role of Continuous Cues in Immersive ExergamesProceedings of the ACM on Human-Computer Interaction10.1145/36770918:CHI PLAY(1-23)Online publication date: 15-Oct-2024
https://dl.acm.org/doi/10.1145/3677091
Bashar MBatmaz A(2024)Virtual Task Environments Factors Explored in 3D Selection StudiesProceedings of the 50th Graphics Interface Conference10.1145/3670947.3670983(1-16)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3670947.3670983
Adkins ACanales RJörg S(2024)Hands or Controllers? How Input Devices and Audio Impact Collaborative Virtual RealityProceedings of the 30th ACM Symposium on Virtual Reality Software and Technology10.1145/3641825.3687718(1-12)Online publication date: 9-Oct-2024
https://dl.acm.org/doi/10.1145/3641825.3687718
Canales R(2024)[DC] Animating Interactive Virtual Humans2024 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW)10.1109/VRW62533.2024.00368(1148-1149)Online publication date: 16-Mar-2024
https://doi.org/10.1109/VRW62533.2024.00368
Venkatakrishnan RVenkatakrishnan RCanales RRaveendranath BPagano CRobb ALin WBabu S(2024)Investigating the Effects of Avatarization and Interaction Techniques on Near-field Mixed Reality Interactions with Physical ComponentsIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2024.337205030:5(2756-2766)Online publication date: 4-Mar-2024
https://dl.acm.org/doi/10.1109/TVCG.2024.3372050
Fukui TInoue KKomatsu TSato N(2024)Kinematic Properties of Haptic-Free Reach-to-Grasp Movements in Virtual Reality: A Comparison with Natural Prehension in Real Spaces and Pantomimed OnesInternational Journal of Human–Computer Interaction10.1080/10447318.2024.2399874(1-15)Online publication date: 16-Sep-2024
https://doi.org/10.1080/10447318.2024.2399874
Lafuma LBouyer GGoguel ODidier J(2023)Influence of hand representation on a grasping task in augmented realityProceedings of the 25th International Conference on Multimodal Interaction10.1145/3577190.3614128(364-372)Online publication date: 9-Oct-2023
https://dl.acm.org/doi/10.1145/3577190.3614128
Misztal SSchild J(2022)Visual Delegate Generalization Frame – Evaluating Impact of Visual Effects and Elements on Player and User Experiences in Video Games and Interactive Virtual EnvironmentsProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3501885(1-20)Online publication date: 29-Apr-2022
https://dl.acm.org/doi/10.1145/3491102.3501885
Furmanek MMangalam MLockwood KSmith AYarossi MTunik E(2021)Effects of Sensory Feedback and Collider Size on Reach-to-Grasp Coordination in Haptic-Free Virtual RealityFrontiers in Virtual Reality10.3389/frvir.2021.6485292Online publication date: 19-Aug-2021
https://doi.org/10.3389/frvir.2021.648529

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents