Work in Progress

GaitWay: Gait Data-Based VR Locomotion Prediction System Robust to Visual Distraction

Authors:

Seokhyun Hwang,

Seungjun KimAuthors Info & Claims

CHI EA '24: Extended Abstracts of the CHI Conference on Human Factors in Computing Systems

Article No.: 173, Pages 1 - 8

https://doi.org/10.1145/3613905.3651073

Published: 11 May 2024 Publication History

Abstract

In VR environments, user’s sense of presence is enhanced through natural locomotion. Redirected Walking (RDW) technology can provide a wider walking area by manipulating the trajectory of the user. Considering that the user’s future position enables a broader application of RDW, research has utilized gaze data combined with past positions to reduce prediction errors. However, in VR content that are replete with creatures and decorations, gaze dispersion may deteriorate the data quality. Thus, we propose an alternative system that utilizes gait data, GaitWay, which correlates directly to user locomotion. This study involved 11 participants navigating a visually distracting three-tiered VR environment while performing designated tasks. We employed a long short-term memory network for GaitWay to forecast positions two seconds ahead and evaluated the prediction accuracy. The findings demonstrated that incorporating gaze data significantly increased errors in highly-distracted settings, whereas GaitWay consistently reduced errors, regardless of the environmental complexity.

Supplemental Material

MP4 File - Video Preview

Video Preview

Transcript for: Video Preview

MP4 File

Talk Video

Transcript for: Talk Video

MP4 File

Video Figure

Transcript for: Video Figure

References

[1]

Hironori Akiduki, Suetaka Nishiike, Hiroshi Watanabe, Katsunori Matsuoka, Takeshi Kubo, and Noriaki Takeda. 2003. Visual-vestibular conflict induced by virtual reality in humans. Neuroscience letters 340, 3 (2003), 197–200.

[2]

Gustavo Arechavaleta, Jean-Paul Laumond, Halim Hicheur, and Alain Berthoz. 2008. An Optimality Principle Governing Human Walking. IEEE Transactions on Robotics 24, 1 (2008), 5–14. https://doi.org/10.1109/TRO.2008.915449

Digital Library

[3]

Luke Bölling, Niklas Stein, Frank Steinicke, and Markus Lappe. 2019. Shrinking circles: Adaptation to increased curvature gain in redirected walking. IEEE transactions on visualization and computer graphics 25, 5 (2019), 2032–2039.

[4]

Benjamin Bolte and Markus Lappe. 2015. Subliminal reorientation and repositioning in immersive virtual environments using saccadic suppression. IEEE transactions on visualization and computer graphics 21, 4 (2015), 545–552.

Digital Library

[5]

Evren Bozgeyikli, Andrew Raij, Srinivas Katkoori, and Rajiv Dubey. 2016. Point & teleport locomotion technique for virtual reality. In Proceedings of the 2016 annual symposium on computer-human interaction in play. 205–216.

Digital Library

[6]

Gianni Bremer, Niklas Stein, and Markus Lappe. 2021. Predicting Future Position From Natural Walking and Eye Movements with Machine Learning. In 2021 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR). 19–28. https://doi.org/10.1109/AIVR52153.2021.00013

[7]

Hugo Brument, Iana Podkosova, Hannes Kaufmann, Anne Hélène Olivier, and Ferran Argelaguet. 2019. Virtual vs. Physical Navigation in VR: Study of Gaze and Body Segments Temporal Reorientation Behaviour. In 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). 680–689. https://doi.org/10.1109/VR.2019.8797721

[8]

Dirk Calow and Markus Lappe. 2008. Efficient encoding of natural optic flow. Network: Computation in Neural Systems 19, 3 (2008), 183–212. https://doi.org/10.1080/09548980802368764 arXiv:https://doi.org/10.1080/09548980802368764PMID: 18946836.

[9]

Polona Caserman, Augusto Garcia-Agundez, Alvar Gámez Zerban, and Stefan Göbel. 2021. Cybersickness in current-generation virtual reality head-mounted displays: systematic review and outlook. Virtual Reality 25, 4 (2021), 1153–1170.

Digital Library

[10]

Baojun Chen, Enhao Zheng, and Qining Wang. 2014. A Locomotion Intent Prediction System Based on Multi-Sensor Fusion. Sensors 14, 7 (July 2014), 12349–12369. https://doi.org/10.3390/s140712349

[11]

Yong-Hun Cho, Dong-Yong Lee, and In-Kwon Lee. 2018. Path Prediction Using LSTM Network for Redirected Walking. In 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). 527–528. https://doi.org/10.1109/VR.2018.8446442

[12]

Szonya Durant and Johannes M. Zanker. 2020. The combined effect of eye movements improve head centred local motion information during walking. PLOS ONE 15, 1 (01 2020), 1–17. https://doi.org/10.1371/journal.pone.0228345

[13]

Richard C Fitzpatrick, Daniel L Wardman, and Janet L Taylor. 1999. Effects of galvanic vestibular stimulation during human walking. The Journal of Physiology 517, Pt 3 (1999), 931.

[14]

Jonathan Gandrud and Victoria Interrante. 2016. Predicting Destination Using Head Orientation and Gaze Direction during Locomotion in VR. In Proceedings of the ACM Symposium on Applied Perception (Anaheim, California) (SAP ’16). Association for Computing Machinery, New York, NY, USA, 31–38. https://doi.org/10.1145/2931002.2931010

Digital Library

[15]

Mary Hayhoe and Dana Ballard. 2005. Eye movements in natural behavior. Trends in Cognitive Sciences 9, 4 (2005), 188–194. https://doi.org/10.1016/j.tics.2005.02.009

[16]

Mark Hollands and Dilwyn Marple-Horvat. 1996. Visually guided stepping under conditions of step cycle-related denial of visual information. Experimental brain research. Experimentelle Hirnforschung. Expérimentation cérébrale 109 (06 1996), 343–56. https://doi.org/10.1007/BF00231792

[17]

M A Hollands, AftabE. Patla, and Joan N. Vickers. 2002. “Look where you’re going!”: gaze behaviour associated with maintaining and changing the direction of locomotion. Experimental Brain Research 143 (2002), 221–230. https://api.semanticscholar.org/CorpusID:29510260

[18]

Courtney Hutton and Evan Suma. 2016. A realistic walking model for enhancing redirection in virtual reality. In 2016 IEEE Virtual Reality (VR). 183–184. https://doi.org/10.1109/VR.2016.7504714

[19]

S. Hwang, Y. Kim, Y. Seo, and S. Kim. 2023. Enhancing Seamless Walking in Virtual Reality: Application of Bone-Conduction Vibration in Redirected Walking. In 2023 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). IEEE Computer Society, Los Alamitos, CA, USA, 1181–1190. https://doi.org/10.1109/ISMAR59233.2023.00135

[20]

Seokhyun Hwang, Jieun Lee, YoungIn Kim, and SeungJun Kim. 2022. REVES: Redirection Enhancement Using Four-Pole Vestibular Electrode Stimulation. In Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems (New Orleans, LA, USA) (CHI EA ’22). Association for Computing Machinery, New York, NY, USA, Article 267, 7 pages. https://doi.org/10.1145/3491101.3519626

Digital Library

[21]

Seokhyun Hwang, Jieun Lee, Youngin Kim, Youngseok Seo, and Seungjun Kim. 2023. Electrical, Vibrational, and Cooling Stimuli-Based Redirected Walking: Comparison of Various Vestibular Stimulation-Based Redirected Walking Systems. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (Hamburg, Germany) (CHI ’23). Association for Computing Machinery, New York, NY, USA, Article 767, 18 pages. https://doi.org/10.1145/3544548.3580862

Digital Library

[22]

Su-Bin Joo, Seung Eel Oh, Taeyong Sim, Hyunggun Kim, Chang Hyun Choi, Hyeran Koo, and Joung Hwan Mun. 2014. Prediction of gait speed from plantar pressure using artificial neural networks. Expert Systems with Applications 41, 16 (2014), 7398–7405. https://doi.org/10.1016/j.eswa.2014.06.002

Digital Library

[23]

Anuradha Kar and Peter Corcoran. 2017. A Review and Analysis of Eye-Gaze Estimation Systems, Algorithms and Performance Evaluation Methods in Consumer Platforms. IEEE Access 5 (2017), 16495–16519. https://doi.org/10.1109/ACCESS.2017.2735633

[24]

Lucie Kruse, Eike Langbehn, and Frank Steinicke. 2018. I can see on my feet while walking: Sensitivity to translation gains with visible feet. In 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). IEEE, 305–312.

[25]

Michael F. Land and Mary Hayhoe. 2001. In what ways do eye movements contribute to everyday activities?Vision Research 41, 25 (2001), 3559–3565. https://doi.org/10.1016/S0042-6989(01)00102-X

[26]

Michael F. Land and Benjamin W. Tatler. 2009. 100101Locomotion on foot. In Looking and Acting: Vision and eye movements in natural behaviour. Oxford University Press. https://doi.org/10.1093/acprof:oso/9780198570943.003.0006 arXiv:https://academic.oup.com/book/0/chapter/144253574/chapter-ag-pdf/45015183/book_3269_section_144253574.ag.pdf

[27]

Eike Langbehn, Tobias Eichler, Sobin Ghose, Kai von Luck, Gerd Bruder, and Frank Steinicke. 2015. Evaluation of an Omnidirectional Walking-in- Place User Interface with Virtual Locomotion Speed Scaled by Forward Leaning Angle. Proceedings of the GI Workshop on Virtual and Augmented Reality (GI VR/AR) (2015), 149–160. https://basilic.informatik.uni-hamburg.de/Publications/2015/LEGVBS15/

[28]

Eike Langbehn, Paul Lubos, and Frank Steinicke. 2018. Evaluation of Locomotion Techniques for Room-Scale VR: Joystick, Teleportation, and Redirected Walking. In Proceedings of the Virtual Reality International Conference - Laval Virtual (Laval, France) (VRIC ’18). Association for Computing Machinery, New York, NY, USA, Article 4, 9 pages. https://doi.org/10.1145/3234253.3234291

Digital Library

[29]

Eike Langbehn, Frank Steinicke, Ping Koo-Poeggel, Lisa Marshall, and Gerd Bruder. 2019. Stimulating the brain in VR: Effects of transcranial direct-current stimulation on redirected walking. In ACM Symposium on Applied Perception 2019. 1–9.

Digital Library

[30]

Eike Langbehn, Frank Steinicke, Markus Lappe, Gregory F Welch, and Gerd Bruder. 2018. In the blink of an eye: leveraging blink-induced suppression for imperceptible position and orientation redirection in virtual reality. ACM Transactions on Graphics (TOG) 37, 4 (2018), 1–11.

Digital Library

[31]

Jieun Lee, Seokhyun Hwang, Aya Ataya, and SeungJun Kim. 2024. Effect of optical flow and user VR familiarity on curvature gain thresholds for redirected walking. Virtual Reality 28, 1 (29 Jan 2024), 35. https://doi.org/10.1007/s10055-023-00935-4

Digital Library

[32]

Jieun Lee, Seokhyun Hwang, Kyunghwan Kim, and SeungJun Kim. 2022. Auditory and Olfactory Stimuli-Based Attractors to Induce Reorientation in Virtual Reality Forward Redirected Walking. In Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems (New Orleans, LA, USA) (CHI EA ’22). Association for Computing Machinery, New York, NY, USA, Article 446, 7 pages. https://doi.org/10.1145/3491101.3519719

Digital Library

[33]

Jonathan Samir Matthis, Jacob L. Yates, and Mary M. Hayhoe. 2018. Gaze and the Control of Foot Placement When Walking in Natural Terrain. Current Biology 28, 8 (2018), 1224–1233.e5. https://doi.org/10.1016/j.cub.2018.03.008

[34]

Thomas Nescher, Ying-Yin Huang, and Andreas Kunz. 2014. Planning redirection techniques for optimal free walking experience using model predictive control. In 2014 IEEE Symposium on 3D User Interfaces (3DUI). 111–118. https://doi.org/10.1109/3DUI.2014.6798851

[35]

Niels Christian Nilsson, Tabitha Peck, Gerd Bruder, Eri Hodgson, Stefania Serafin, Mary Whitton, Frank Steinicke, and Evan Suma Rosenberg. 2018. 15 years of research on redirected walking in immersive virtual environments. IEEE computer graphics and applications 38, 2 (2018), 44–56.

Digital Library

[36]

Sharif Razzaque, Zachariah Kohn, and Mary C. Whitton. 2001. Redirected Walking. In Eurographics 2001 - Short Presentations. Eurographics Association. https://doi.org/10.2312/egs.20011036

[37]

Alexandra Sipatchin, Siegfried Wahl, and Katharina Rifai. 2021. Eye-Tracking for Clinical Ophthalmology with Virtual Reality (VR): A Case Study of the HTC Vive Pro Eye’s Usability. Healthcare 9, 2 (2021). https://doi.org/10.3390/healthcare9020180

[38]

Niklas Stein, Gianni Bremer, and Markus Lappe. 2022. Eye Tracking-based LSTM for Locomotion Prediction in VR. In 2022 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). 493–503. https://doi.org/10.1109/VR51125.2022.00069

[39]

Niklas Stein, Diederick Niehorster, Tamara Watson, Frank Steinicke, Katharina Rifai, Siegfried Wahl, and Markus Lappe. 2021. A Comparison of Eye Tracking Latencies Among Several Commercial Head-Mounted Displays. i-Perception 12 (02 2021), 204166952098333. https://doi.org/10.1177/2041669520983338

[40]

Binbin Su, Yi-Xing Liu, and Elena M. Gutierrez-Farewik. 2021. Locomotion Mode Transition Prediction Based on Gait-Event Identification Using Wearable Sensors and Multilayer Perceptrons. Sensors 21, 22 (Nov. 2021), 7473. https://doi.org/10.3390/s21227473

[41]

Evan A. Suma, Gerd Bruder, Frank Steinicke, David M. Krum, and Mark Bolas. 2012. A taxonomy for deploying redirection techniques in immersive virtual environments. In 2012 IEEE Virtual Reality Workshops (VRW). 43–46. https://doi.org/10.1109/VR.2012.6180877

Digital Library

[42]

Evan A Suma, Seth Clark, David Krum, Samantha Finkelstein, Mark Bolas, and Zachary Warte. 2011. Leveraging change blindness for redirection in virtual environments. In 2011 IEEE Virtual Reality Conference. IEEE, 159–166.

[43]

Qi Sun, Anjul Patney, Li-Yi Wei, Omer Shapira, Jingwan Lu, Paul Asente, Suwen Zhu, Morgan McGuire, David Luebke, and Arie Kaufman. 2018. Towards virtual reality infinite walking: dynamic saccadic redirection. ACM Transactions on Graphics (TOG) 37, 4 (2018), 1–13.

Digital Library

[44]

Bernard t Hart and Wolfgang Einhäuser. 2012. Mind the step: Complementary effects of an implicit task on eye and head movements in real-life gaze allocation. Experimental Brain Research 223 (11 2012). https://doi.org/10.1007/s00221-012-3254-x

[45]

Benjamin W. Tatler, Mary M. Hayhoe, Michael F. Land, and Dana H. Ballard. 2011. Eye guidance in natural vision: Reinterpreting salience. Journal of Vision 11, 5 (05 2011), 5–5. https://doi.org/10.1167/11.5.5 arXiv:https://arvojournals.org/arvo/content_public/journal/jov/933487/jov-11-5-5.pdf

[46]

Benjamin W. Tatler and Sarah L Tatler. 2013. The influence of instructions on object memory in a real-world setting.Journal of vision 13 2 (2013), 5. https://api.semanticscholar.org/CorpusID:20754143

[47]

Phuc Truong, Jinwook Lee, Ae-Ran Kwon, and Gu-Min Jeong. 2016. Stride Counting in Human Walking and Walking Distance Estimation Using Insole Sensors. Sensors 16, 6 (June 2016), 823. https://doi.org/10.3390/s16060823

[48]

Yolanda Vazquez-Alvarez and Stephen A. Brewster. 2011. Eyes-Free Multitasking: The Effect of Cognitive Load on Mobile Spatial Audio Interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Vancouver, BC, Canada) (CHI ’11). Association for Computing Machinery, New York, NY, USA, 2173–2176. https://doi.org/10.1145/1978942.1979258

Digital Library

[49]

Shu Wei, Desmond Bloemers, and Aitor Rovira. 2023. A Preliminary Study of the Eye Tracker in the Meta Quest Pro. In Proceedings of the 2023 ACM International Conference on Interactive Media Experiences (Nantes, France) (IMX ’23). Association for Computing Machinery, New York, NY, USA, 216–221. https://doi.org/10.1145/3573381.3596467

Digital Library

[50]

Jan Malte Wiener, Olivier De Condappa, and Christoph Hölscher. 2011. Do you have to look where you go? Gaze behaviour during spatial decision making. Cognitive Science 33 (2011). https://api.semanticscholar.org/CorpusID:14148078

[51]

Huacai Xian, Lisheng Jin, Haijing Hou, Qingning Niu, and Huanhuan Lv. 2014. Analyzing Effects of Pressing Radio Button on Driver’s Visual Cognition. Vol. 215. 69–78. https://doi.org/10.1007/978-3-642-37835-5_7

[52]

Markus Zank and Andreas Kunz. 2016. Eye tracking for locomotion prediction in redirected walking. In 2016 IEEE Symposium on 3D User Interfaces (3DUI). 49–58. https://doi.org/10.1109/3DUI.2016.7460030

[53]

Markus Zank and Andreas Kunz. 2016. Where Are You Going? Using Human Locomotion Models for Target Estimation. Vis. Comput. 32, 10 (oct 2016), 1323–1335. https://doi.org/10.1007/s00371-016-1229-9

Digital Library

[54]

Markus Zank and Andreas Kunz. 2017. Optimized graph extraction and locomotion prediction for redirected walking. In 2017 IEEE Symposium on 3D User Interfaces (3DUI). 120–129. https://doi.org/10.1109/3DUI.2017.7893328

[55]

Michael A. Zmuda, Joshua L. Wonser, Eric R. Bachmann, and Eric Hodgson. 2013. Optimizing Constrained-Environment Redirected Walking Instructions Using Search Techniques. IEEE Transactions on Visualization and Computer Graphics 19, 11 (2013), 1872–1884. https://doi.org/10.1109/TVCG.2013.88

Digital Library

Cited By

Hwang SKang SOh JPark JShin SLuo YDelPreto JMatusik WRus DKim SKostakos VKay JHoang T(2024)Proposal of a Framework for Enhancing Teleoperation Experience with Biomechanical Simulation-Based Electrical Muscle Stimulation in Virtual RealityCompanion of the 2024 on ACM International Joint Conference on Pervasive and Ubiquitous Computing10.1145/3675094.3678380(826-831)Online publication date: 5-Oct-2024
https://dl.acm.org/doi/10.1145/3675094.3678380

Index Terms

GaitWay: Gait Data-Based VR Locomotion Prediction System Robust to Visual Distraction
1. Computing methodologies
  1. Computer graphics
    1. Graphics systems and interfaces
2. Social and professional topics
  1. Professional topics

Index terms have been assigned to the content through auto-classification.

Recommendations

Electrical, Vibrational, and Cooling Stimuli-Based Redirected Walking: Comparison of Various Vestibular Stimulation-Based Redirected Walking Systems
CHI '23: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems

Redirected walking (RDW) is a technology that enables users to walk seamlessly in an enormous virtual space within a narrow real space while avoiding collisions with physical elements. Although RDW provides accurate proprioceptive sensations, ...
Evaluation of Locomotion Techniques for Room-Scale VR: Joystick, Teleportation, and Redirected Walking
VRIC '18: Proceedings of the Virtual Reality International Conference - Laval Virtual

Due to its multimodal nature virtual reality technology imposes new challenges, for example, when it comes to navigating through a virtual environment. Joystick-based controls and teleportation techniques support only limited self-motion experiences, ...
VR locomotion: walking > walking in place > arm swinging
VRCAI '16: Proceedings of the 15th ACM SIGGRAPH Conference on Virtual-Reality Continuum and Its Applications in Industry - Volume 1

There are many methods of exploring an HMD-based virtual environment such as using a game controller, physically walking, walking in place, teleporting, flying, leaning, etc. The purpose of this work is to introduce a simple method of implementing "...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI EA '24: Extended Abstracts of the CHI Conference on Human Factors in Computing Systems

May 2024

4761 pages

ISBN:9798400703317

DOI:10.1145/3613905

Editors:
Florian Floyd Mueller
Monash University
,
Penny Kyburz
The Australian National University
,
Julie R. Williamson
University of Glasgow
,
Corina Sas
Lancaster University

Copyright © 2024 Owner/Author.

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 May 2024

Check for updates

Author Tags

Qualifiers

Work in progress
Research
Refereed limited

Conference

CHI '24

Sponsor:

CHI '24: CHI Conference on Human Factors in Computing Systems

May 11 - 16, 2024

HI, Honolulu, USA

Acceptance Rates

Overall Acceptance Rate 6,164 of 23,696 submissions, 26%

Upcoming Conference

CHI '25

Sponsor:
sigchi

CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
292
Total Downloads

Downloads (Last 12 months)292
Downloads (Last 6 weeks)38

Reflects downloads up to 22 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Hwang SKang SOh JPark JShin SLuo YDelPreto JMatusik WRus DKim SKostakos VKay JHoang T(2024)Proposal of a Framework for Enhancing Teleoperation Experience with Biomechanical Simulation-Based Electrical Muscle Stimulation in Virtual RealityCompanion of the 2024 on ACM International Joint Conference on Pervasive and Ubiquitous Computing10.1145/3675094.3678380(826-831)Online publication date: 5-Oct-2024
https://dl.acm.org/doi/10.1145/3675094.3678380

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Full Text

View this article in Full Text.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View full text|Download PDF

View Table of Contents