research-article

Public Access

Synthesizing scene-aware virtual reality teleport graphs

Authors:

Lap-Fai YuAuthors Info & Claims

ACM Transactions on Graphics (TOG), Volume 40, Issue 6

Article No.: 229, Pages 1 - 15

https://doi.org/10.1145/3478513.3480478

Published: 10 December 2021 Publication History

Abstract

We present a novel approach for synthesizing scene-aware virtual reality teleport graphs, which facilitate navigation in indoor virtual environments by suggesting desirable teleport positions. Our approach analyzes panoramic views at candidate teleport positions by extracting scene perception graphs, which encode scene perception relationships between the observer and the surrounding objects, and predict how desirable the views at these positions are. We train a graph convolutional model to predict the scene perception scores of different teleport positions. Based on such predictions, we apply an optimization approach to sample a set of desirable teleport positions while considering other navigation properties such as coverage and connectivity to synthesize a teleport graph. Using teleport graphs, users can navigate virtual environments efficaciously. We demonstrate our approach for synthesizing teleport graphs for common indoor scenes. By conducting a user study, we validate the efficacy and desirability of navigating virtual environments via the synthesized teleport graphs. We also extend our approach to cope with different constraints, user preferences, and practical scenarios.

Supplementary Material

ZIP File (a229-li.zip)

Supplemental files.

Download
11.23 MB

MP4 File (a229-li.mp4)

Download
189.00 MB

References

[1]

Majed Al Zayer, Paul MacNeilage, and Eelke Folmer. 2018. Virtual locomotion: a survey. IEEE transactions on visualization and computer graphics (2018).

[2]

Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473 (2014).

[3]

Jiwan Bhandari, Paul R MacNeilage, and Eelke Folmer. 2018. Teleportation without Spatial Disorientation Using Optical Flow Cues. In Graphics interface. 162--167.

Digital Library

[4]

Costas Boletsis. 2017. The new era of virtual reality locomotion: A systematic literature review of techniques and a proposed typology. Multimodal Technologies and Interaction 1, 4 (2017), 24.

[5]

Evren Bozgeyikli, Andrew Raij, Srinivas Katkoori, and Rajiv Dubey. 2016. Point & teleport locomotion technique for virtual reality. In Proceedings of the 2016 Annual Symposium on Computer-Human Interaction in Play. ACM, 205--216.

Digital Library

[6]

Fabio Buttussi and Luca Chittaro. 2019. Locomotion in Place in Virtual Reality: A Comparative Evaluation of Joystick, Teleport, and Leaning. IEEE transactions on visualization and computer graphics (2019).

[7]

Angel Chang, Angela Dai, Thomas Funkhouser, Maciej Halber, Matthias Niessner, Manolis Savva, Shuran Song, Andy Zeng, and Yinda Zhang. 2017. Matterport3D: Learning from RGB-D Data in Indoor Environments. International Conference on 3D Vision (3DV) (2017).

[8]

Shih-Han Chou, Cheng Sun, Wen-Yen Chang, Wan-Ting Hsu, Min Sun, and Jianlong Fu. 2020. 360-Indoor: Towards Learning Real-World Objects in 360deg Indoor Equirectangular Images. In The IEEE Winter Conference on Applications of Computer Vision. 845--853.

[9]

Benjamin Coors, Alexandru Paul Condurache, and Andreas Geiger. 2018. Spherenet: Learning spherical representations for detection and classificationin omnidirectional images. In Proceedings of the European Conference on Computer Vision. 518--533.

[10]

Zhi-Chao Dong, Xiao-Ming Fu, Zeshi Yang, and Ligang Liu. 2019. Redirected smooth mappings for multiuser real walking in virtual reality. ACM Transactions on Graphics (TOG) 38, 5 (2019), 1--17.

Digital Library

[11]

Li Fei-Fei. 2007. Recognizing and learning object categories. CVPR Short Course, 2007 (2007).

[12]

Tian Feng, Lap-Fai Yu, Sai-Kit Yeung, KangKang Yin, and Kun Zhou. 2016. Crowd-driven mid-scale layout design. ACM Trans. Graph. 35, 4 (2016), 132--1.

Digital Library

[13]

Sebastian Freitag, Benjamin Weyers, and Torsten W Kuhlen. 2016. Automatic speed adjustment for travel through immersive virtual environments based on viewpoint quality. In 2016 IEEE Symposium on 3D User Interfaces (3DUI). IEEE, 67--70.

[14]

Sebastian Freitag, Benjamin Weyers, and Torsten W Kuhlen. 2017. Efficient approximate computation of scene visibility based on navigation meshes and applications for navigation and scene analysis. In 2017 IEEE Symposium on 3D User Interfaces (3DUI). IEEE, 134--143.

[15]

Sebastian Freitag, Benjamin Weyers, and Torsten W Kuhlen. 2018. Interactive exploration assistance for immersive virtual environments based on object visibility and viewpoint quality. In 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). IEEE, 355--362.

[16]

Markus Funk, Florian Müller, Marco Fendrich, Megan Shene, Moritz Kolvenbach, Niclas Dobbertin, Sebastian Günther, and Max Mühlhäuser. 2019. Assessing the Accuracy of Point & Teleport Locomotion with Orientation Indication for Virtual Reality using Curved Trajectories. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1--12.

Digital Library

[17]

Thomas Gärtner, Peter Flach, and Stefan Wrobel. 2003. On graph kernels: Hardness results and efficient alternatives. In Learning theory and kernel machines. Springer, 129--143.

[18]

Wilson S Geisler. 2008. Visual perception and the statistical properties of natural scenes. Annu. Rev. Psychol. 59 (2008), 167--192.

[19]

Peter J Green. 1995. Reversible jump Markov chain Monte Carlo computation and Bayesian model determination. Biometrika 82, 4 (1995), 711--732.

[20]

MP Jacob Habgood, David Moore, David Wilson, and Sergio Alapont. 2018. Rapid, continuous movement between nodes as an accessible virtual reality locomotion technique. In 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). IEEE, 371--378.

[21]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770--778.

[22]

Peter Hedström and Charlotta Stern. 2008. Rational choice and sociology. The new Palgrave dictionary of economics 2 (2008).

[23]

Dichao Hu. 2019. An introductory survey on attention mechanisms in NLP problems. In Proceedings of SAI Intelligent Systems Conference. Springer, 432--448.

[24]

Laurent Itti, Christof Koch, and Ernst Niebur. 1998. A model of saliency-based visual attention for rapid scene analysis. IEEE Transactions on pattern analysis and machine intelligence 20, 11 (1998), 1254--1259.

Digital Library

[25]

Scott Kirkpatrick, C Daniel Gelatt, and Mario P Vecchi. 1983. Optimization by simulated annealing. science 220, 4598 (1983), 671--680.

[26]

Eike Langbehn, Paul Lubos, and Frank Steinicke. 2018. Evaluation of locomotion techniques for room-scale vr: Joystick, teleportation, and redirected walking. In Proceedings of the Virtual Reality International Conference-Laval Virtual. 1--9.

Digital Library

[27]

Yujia Li, Oriol Vinyals, Chris Dyer, Razvan Pascanu, and Peter Battaglia. 2018. Learning deep generative models of graphs. arXiv preprint arXiv:1803.03324 (2018).

[28]

Yong-Lu Li, Siyuan Zhou, Xijie Huang, Liang Xu, Ze Ma, Hao-Shu Fang, Yanfeng Wang, and Cewu Lu. 2019. Transferable interactiveness knowledge for human-object interaction detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 3585--3594.

[29]

Tsung-Yi Lin, Piotr Dollár, Ross Girshick, Kaiming He, Bharath Hariharan, and Serge Belongie. 2017. Feature pyramid networks for object detection. In Proceedings of the IEEE conference on computer vision and pattern recognition. 2117--2125.

[30]

Imran Mahalil, Azmi Mohd Yusof, Nazrita Ibrahim, Eze Manzura Mohd Mahidin, and Mohd Ezanee Rusli. 2019. Virtual Reality Mini Map Presentation Techniques: Lessons and experience learned. In 2019 IEEE Conference on Graphics and Media (GAME). IEEE, 26--31.

[31]

Siyuan Qi, Wenguan Wang, Baoxiong Jia, Jianbing Shen, and Song-Chun Zhu. 2018a. Learning human-object interactions by graph parsing neural networks. In Proceedings of the European Conference on Computer Vision (ECCV). 401--417.

Digital Library

[32]

Siyuan Qi, Yixin Zhu, Siyuan Huang, Chenfanfu Jiang, and Song-Chun Zhu. 2018b. Human-centric indoor scene synthesis using stochastic grammar. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 5899--5908.

[33]

Sharif Razzaque, Zachariah Kohn, and Mary C. Whitton. 2001. Redirected Walking. In Eurographics 2001 - Short Presentations. Eurographics Association.

[34]

Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. 2015. Faster r-cnn: Towards real-time object detection with region proposal networks. In Advances in neural information processing systems. 91--99.

Digital Library

[35]

Ronald A Rensink. 2000. Scene perception. Encyclopedia of psychology 7 (2000), 151--155.

[36]

Jeffrey N Rouder, Paul L Speckman, Dongchu Sun, Richard D Morey, and Geoffrey Iverson. 2009. Bayesian t tests for accepting and rejecting the null hypothesis. Psychonomic bulletin & review 16, 2 (2009), 225--237.

[37]

Shlomo S Sawilowsky. 2009. New effect size rules of thumb. Journal of modern applied statistical methods 8, 2 (2009), 26.

[38]

Ehsan Sayyad, Misha Sra, and Tobias Höllerer. 2020. Walking and Teleportation in Wide-area Virtual Reality Experiences. In 2020 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). IEEE, 608--617.

[39]

Nino Shervashidze, Pascal Schweitzer, Erik Jan Van Leeuwen, Kurt Mehlhorn, and Karsten M Borgwardt. 2011. Weisfeiler-lehman graph kernels. Journal of Machine Learning Research 12, 9 (2011).

Digital Library

[40]

Mel Slater, Martin Usoh, and Anthony Steed. 1995. Taking steps: the influence of a walking technique on presence in virtual reality. ACM Transactions on Computer-Human Interaction (TOCHI) 2, 3 (1995), 201--219.

Digital Library

[41]

Richard Stoakley, Matthew J Conway, and Randy Pausch. 1995. Virtual reality on a WIM: interactive worlds in miniature. In Proceedings of the SIGCHI conference on Human factors in computing systems. 265--272.

Digital Library

[42]

Qi Sun, Anjul Patney, Li-Yi Wei, Omer Shapira, Jingwan Lu, Paul Asente, Suwen Zhu, Morgan Mcguire, David Luebke, and Arie Kaufman. 2018. Towards virtual reality infinite walking: dynamic saccadic redirection. ACM Transactions on Graphics (TOG) 37, 4 (2018), 67.

Digital Library

[43]

Oytun Ulutan, ASM Iftekhar, and Bangalore S Manjunath. 2020. VSGNet: Spatial Attention Network for Detecting Human Object Interactions Using Graph Convolutions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 13617--13626.

[44]

Martin Usoh, Kevin Arthur, Mary C Whitton, Rui Bastos, Anthony Steed, Mel Slater, and Frederick P Brooks Jr. 1999. Walking> walking-in-place> flying, in virtual environments. In Proceedings of the 26th annual conference on Computer graphics and interactive techniques. ACM Press/Addison-Wesley Publishing Co., 359--364.

Digital Library

[45]

Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. 2017. Graph attention networks. arXiv preprint arXiv:1710.10903 (2017).

[46]

S Vichy N Vishwanathan, Nicol N Schraudolph, Risi Kondor, and Karsten M Borgwardt. 2010. Graph kernels. The Journal of Machine Learning Research 11 (2010), 1201--1242.

Digital Library

[47]

Kai Wang, Yu-An Lin, Ben Weissmann, Manolis Savva, Angel X Chang, and Daniel Ritchie. 2019. Planit: Planning and instantiating indoor scenes with relation graph and spatial prior networks. ACM Transactions on Graphics (TOG) 38, 4 (2019), 1--15.

Digital Library

[48]

Fei Xia, Amir R. Zamir, Zhi-Yang He, Alexander Sax, Jitendra Malik, and Silvio Savarese. 2018. Gibson env: real-world perception for embodied agents. In Proceedings of the IEEE conference on computer vision and pattern recognition.

[49]

Si Zhang, Hanghang Tong, Jiejun Xu, and Ross Maciejewski. 2019. Graph convolutional networks: a comprehensive review. Computational Social Networks 6, 1 (2019), 11.

[50]

Yin Zhang, Rong Jin, and Zhi-Hua Zhou. 2010. Understanding bag-of-words model: a statistical framework. International Journal of Machine Learning and Cybernetics 1, 1-4 (2010), 43--52.

[51]

Yixin Zhu, Chenfanfu Jiang, Yibiao Zhao, Demetri Terzopoulos, and Song-Chun Zhu. 2016. Inferring forces and learning human utilities from videos. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 3823--3833.

Cited By

Lyu YQin PXu TZhu CChen E(2024)InteractNet: Social Interaction Recognition for Semantic-rich VideosACM Transactions on Multimedia Computing, Communications, and Applications10.1145/366366820:8(1-21)Online publication date: 12-Jun-2024
https://dl.acm.org/doi/10.1145/3663668
Sharma KLee YNambi SSalian AShah SKim SKumar S(2024)A Survey of Graph Neural Networks for Social Recommender SystemsACM Computing Surveys10.1145/366182156:10(1-34)Online publication date: 22-Jun-2024
https://dl.acm.org/doi/10.1145/3661821
Aseniero BLee MWang YZhou QShahmansouri NGoldstein R(2024)Experiential Views: Towards Human Experience Evaluation of Designed Spaces using Vision-Language ModelsExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3650815(1-7)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3650815
Show More Cited By

Index Terms

Synthesizing scene-aware virtual reality teleport graphs
1. Computing methodologies
  1. Computer graphics
    1. Graphics systems and interfaces
      1. Virtual reality

Recommendations

Point & Teleport Locomotion Technique for Virtual Reality
CHI PLAY '16: Proceedings of the 2016 Annual Symposium on Computer-Human Interaction in Play

With the increasing popularity of virtual reality (VR) and new devices getting available with relatively lower costs, more and more video games have been developed recently. Most of these games use first person interaction techniques since it is more ...
Tailored Reality: Perception-aware Scene Restructuring for Adaptive VR Navigation
In virtual reality (VR), the virtual scenes are pre-designed by creators. Our physical surroundings, however, comprise significantly varied sizes, layouts, and components. To bridge the gap and further enable natural navigation, recent solutions have been ...
Assessing the Accuracy of Point & Teleport Locomotion with Orientation Indication for Virtual Reality using Curved Trajectories
CHI '19: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems

Room-scale Virtual Reality (VR) systems have arrived in users' homes where tracked environments are set up in limited physical spaces. As most Virtual Environments (VEs) are larger than the tracked physical space, locomotion techniques are used to ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics

ACM Transactions on Graphics Volume 40, Issue 6

December 2021

1351 pages

ISSN:0730-0301

EISSN:1557-7368

DOI:10.1145/3478513

Issue’s Table of Contents

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 10 December 2021

Published in TOG Volume 40, Issue 6

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Science Foundation

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

33
Total Citations
View Citations
607
Total Downloads

Downloads (Last 12 months)246
Downloads (Last 6 weeks)30

Reflects downloads up to 22 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Lyu YQin PXu TZhu CChen E(2024)InteractNet: Social Interaction Recognition for Semantic-rich VideosACM Transactions on Multimedia Computing, Communications, and Applications10.1145/366366820:8(1-21)Online publication date: 12-Jun-2024
https://dl.acm.org/doi/10.1145/3663668
Sharma KLee YNambi SSalian AShah SKim SKumar S(2024)A Survey of Graph Neural Networks for Social Recommender SystemsACM Computing Surveys10.1145/366182156:10(1-34)Online publication date: 22-Jun-2024
https://dl.acm.org/doi/10.1145/3661821
Aseniero BLee MWang YZhou QShahmansouri NGoldstein R(2024)Experiential Views: Towards Human Experience Evaluation of Designed Spaces using Vision-Language ModelsExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3650815(1-7)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3650815
Medlar ALehtikari MGlowacka D(2024)Behind the Scenes: Adapting Cinematography and Editing Concepts to Navigation in Virtual RealityProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642412(1-12)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642412
Wang MLi YShi JSteinicke F(2024)SceneFusion: Room-Scale Environmental Fusion for Efficient Traveling Between Separate Virtual EnvironmentsIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.327170930:8(4615-4630)Online publication date: Aug-2024
https://doi.org/10.1109/TVCG.2023.3271709
Hu YWang KLiu MTang HNie L(2023)Semantic Collaborative Learning for Cross-Modal Moment LocalizationACM Transactions on Information Systems10.1145/362066942:2(1-26)Online publication date: 7-Nov-2023
https://dl.acm.org/doi/10.1145/3620669
Moore CLages W(2023)Planning Locomotion Techniques for Virtual Reality GamesProceedings of the 29th ACM Symposium on Virtual Reality Software and Technology10.1145/3611659.3615711(1-10)Online publication date: 9-Oct-2023
https://dl.acm.org/doi/10.1145/3611659.3615711
Razon AChen YHan YGagniere STupek MTeran J(2023)A Linear and Angular Momentum Conserving Hybrid Particle/Grid Iteration for Volumetric Elastic ContactProceedings of the ACM on Computer Graphics and Interactive Techniques10.1145/36069246:3(1-25)Online publication date: 24-Aug-2023
https://dl.acm.org/doi/10.1145/3606924
Liu HBommes D(2023)Locally Meshable Frame FieldsACM Transactions on Graphics10.1145/359245742:4(1-20)Online publication date: 26-Jul-2023
https://dl.acm.org/doi/10.1145/3592457
Vidulis MRen YPanetta JGrinspun EPauly M(2023)Computational Exploration of Multistable Elastic KnotsACM Transactions on Graphics10.1145/359239942:4(1-16)Online publication date: 26-Jul-2023
https://dl.acm.org/doi/10.1145/3592399
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

Media

Figures

Other

Tables

View Issue’s Table of Contents