research-article

Public Access

Evaluating snapping-to-photos virtual travel interfaces for 3D reconstructed visual reality

Authors:

Benjamin Nuernberger,

Tobias HöllererAuthors Info & Claims

VRST '17: Proceedings of the 23rd ACM Symposium on Virtual Reality Software and Technology

Article No.: 22, Pages 1 - 11

https://doi.org/10.1145/3139131.3139138

Published: 08 November 2017 Publication History

Abstract

Navigating through a virtual, 3D reconstructed scene has recently become very important in many applications. A popular approach is to virtually travel to the photos used in reconstructing the scene; such an approach may be generally termed a "snapping-to-photos" virtual travel interface. While previous work has either used fully constrained interfaces (always at the photos) or minimally constrained interfaces (free-flight navigation), in this paper we introduce new snapping-to-photos interfaces that lie in between these two extremes. Our snapping-to-photos interfaces snap the view to a photo in 3D based on viewpoint similarity and optionally the user's mouse cursor or finger-tap position. Experimental results, with both indoor and outdoor scene reconstructions, found that our snapping-to-photos interfaces are preferred over the baseline fully constrained-to-photos interface, that there exist differences between indoor and outdoor scenes, and that users preferred and were able to reach target photos better with click-to-snap point-of-interest snapping compared to automatic point-of-view snapping.

References

[1]

Dragomir Anguelov, Carole Dulong, Daniel Filip, Christian Frueh, Stphane Lafon, Richard Lyon, Abhijit Ogale, Luc Vincent, and Josh Weaver. 2010. Google street view: Capturing the world at street level. Computer 43, 6 (2010), 32--38.

Digital Library

[2]

Luca Ballan, Gabriel J. Brostow, Jens Puwein, and Marc Pollefeys. 2010. Unstructured video-based rendering. ACM Transactions on Graphics 29, 4 (July 2010), 1.

Digital Library

[3]

Steffi Beckhaus, Felix Ritter, and Thomas Strothotte. 2001. Guided exploration with dynamic potential fields: The CubicalPath system. Computer Graphics Forum 20, 4 (Dec. 2001), 201--210.

[4]

Marco Di Benedetto, Fabio Ganovelli, Marcos Balsa Rodriguez, Alberto Jaspe Villanueva, Roberto Scopigno, and Enrico Gobbetti. 2014. ExploreMaps: Efficient construction and ubiquitous exploration of panoramic view graphs of complex 3D environments. Computer Graphics Forum 33, 2 (May 2014), 459--468.

Digital Library

[5]

Doug A. Bowman, Ernst Kruijff, Joseph J. LaViola, and Ivan Poupyrev. 2004. 3D User Interfaces: Theory and Practice. Addison Wesley Longman Publishing Co., Inc.

Digital Library

[6]

Paolo Brivio, Luca Benedetti, Marco Tarini, Federico Ponchio, Paolo Cignoni, and Roberto Scopigno. 2013. PhotoCloud: Interactive remote exploration of joint 2D and 3D datasets. IEEE CG&A 33, 2 (2013), 86--97.

Digital Library

[7]

Shenchang Eric Chen. 1995. QuickTime VR: An Image-based Approach to Virtual Environment Navigation. In ACM SIGGRAPH. ACM, New York, NY, USA, 29--38.

Digital Library

[8]

Marc Christie, Patrick Olivier, and Jean Marie Normand. 2008. Camera control in computer graphics. Computer Graphics Forum 27, 8 (Dec. 2008), 2197--2218.

[9]

Angela Dai, Matthias Nießner, Michael Zollhöfer, Shahram Izadi, and Christian Theobalt. 2017. BundleFusion: Real-Time Globally Consistent 3D Reconstruction Using On-the-Fly Surface Reintegration. ACM Trans. Graph. 36, 3, Article 24 (May 2017), 18 pages.

Digital Library

[10]

T. Todd Elvins, David R. Nadeau, Rina Schul, and David Kirsh. 2001. Worldlets: 3-D Thumbnails for Wayfinding in Large Virtual Worlds. Presence: Teleoper. Virtual Environ. 10, 6 (Dec. 2001), 565--582.

Digital Library

[11]

Mark Everingham, Luc Van Gool, Christopher K. I. Williams, John Winn, and Andrew Zisserman. 2010. The Pascal Visual Object Classes (VOC) Challenge. International Journal of Computer Vision 88, 2 (2010), 303--338.

Digital Library

[12]

Thomas Forgione, Axel Carlier, Géraldine Morin, Wei Tsang Ooi, and Vincent Charvillat. 2016. Impact of 3D Bookmarks on Navigation and Streaming in a Networked Virtual Environment. In ACM MMSys. ACM, New York, NY, USA, Article9, 10 pages.

Digital Library

[13]

Sebastian Freitag, Benjamin Weyers, Andrea Bönsch, and Torsten W. Kuhlen. 2015. Comparison and Evaluation of Viewpoint Quality Estimation Algorithms for Immersive Virtual Environments. In ICAT - EGVE '15. Eurographics Association, Aire-la-Ville, Switzerland, Switzerland, 53--60.

[14]

Simon Fuhrmann, Fabian Langguth, Nils Moehrle, Michael Waechter, and Michael Goesele. 2015. MVE---An image-based reconstruction environment. Computers & Graphics 53, Part A (Dec. 2015), 44--53.

Digital Library

[15]

Yasutaka Furukawa, Brian Curless, Steven M. Seitz, and Richard Szeliski. 2009. Reconstructing building interiors from images. In 2009 IEEE 12th International Conference on Computer Vision. 80--87.

[16]

Steffen Gauglitz, Benjamin Nuernberger, Matthew Turk, and Tobias Höllerer. 2014a. In Touch with the Remote World: Remote Collaboration with Augmented Reality Drawings and Virtual Navigation. In ACM VRST. ACM, New York, NY, USA, 197--205.

Digital Library

[17]

Steffen Gauglitz, Benjamin Nuernberger, Matthew Turk, and Tobias Höllerer. 2014b. World-stabilized Annotations and Virtual Scene Navigation for Remote Collaboration. In ACM UIST. ACM, New York, NY, USA, 449--459.

Digital Library

[18]

Michael Goesele, Jens Ackermann, Simon Fuhrmann, Carsten Haubold, Ronny Klowsky, Drew Steedly, and Richard Szeliski. 2010. Ambient Point Clouds for View Interpolation. ACM Trans. Graph. 29, 4, Article 95 (July 2010), 6 pages.

Digital Library

[19]

Sandra G. Hart and Lowell E. Staveland. 1988. Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research. Adv. in Psyc. 52 (1988), 139--183.

[20]

Jacek Jankowski and Martin Hachet. 2015. Advances in interaction with 3D environments. Computer Graphics Forum 34, 1 (2015), 152--190.

Digital Library

[21]

Johannes Kopf, Billy Chen, Richard Szeliski, and Michael Cohen. 2010. Street Slide: Browsing Street Level Imagery. ACM Trans. Graph. 29, 4, Article 96 (July 2010), 8 pages.

Digital Library

[22]

Johannes Kopf, Michael F. Cohen, and Richard Szeliski. 2014. First-person Hyperlapse Videos. ACM Trans. Graph. 33, 4, Article 78 (July 2014), 10 pages.

Digital Library

[23]

Avanish Kushal, Ben Self, Yasutaka Furukawa, David Gallup, Carlos Hernandez, Brian Curless, and Steven M. Seitz. 2012. Photo tours. 3DIM/3DPVT Conf. (Oct. 2012), 57--64.

Digital Library

[24]

Andrew Lippman. 1980. Movie-maps: An Application of the Optical Videodisc to Computer Graphics. SIGGRAPH Comput. Graph. 14, 3 (July 1980), 32--42.

Digital Library

[25]

Clement Moerman, Damien Marchal, and Laurent Grisoni. 2012. Drag'n Go: Simple and fast navigation in virtual environment. In 2012 IEEE Symposium on 3D User Interfaces (3DUI). 15--18.

[26]

Benjamin Nuernberger, Kuo-Chin Lien, Lennon Grinta, Chris Sweeney, Matthew Turk, and Tobias Höllerer. 2016. Multi-view Gesture Annotations in Image-based 3D Reconstructed Scenes. In Proceedings of the 22nd ACM VRST. ACM, New York, NY, USA, 129--138.

Digital Library

[27]

Said Pertuz, Domenec Puig, and Miguel Angel Garcia. 2013. Analysis of Focus Measure Operators for Shape-from-focus. Pattern Recogn. 46, 5 (May 2013), 1415--1432.

Digital Library

[28]

Adrian Secord, Jingwan Lu, Adam Finkelstein, Manish Singh, and Andrew Nealen. 2011. Perceptual Models of Viewpoint Preference. ACM Trans. Graph. 30, 5, Article 109 (Oct. 2011), 12 pages.

Digital Library

[29]

Noah Snavely, Rahul Garg, Steven M. Seitz, and Richard Szeliski. 2008. Finding Paths Through the World's Photos. ACM Trans. Graph. 27, 3, Article 15 (Aug. 2008), 11 pages.

Digital Library

[30]

Noah Snavely, Steven M. Seitz, and Richard Szeliski. 2006. Photo tourism: Exploring Photo Collections in 3D. ACM Transactions on Graphics 25, 3 (July 2006), 835.

Digital Library

[31]

Wolfgang Stuerzlinger and Chadwick A. Wingrave. 2011. The Value of Constraints for 3D User Interfaces. Springer Vienna, Vienna, 203--223.

[32]

Christopher Sweeney, Tobias Höllerer, and Matthew Turk. 2015. Theia: A Fast and Scalable Structure-from-Motion Library. Proceedings of the 23rd ACM International Conference on Multimedia (2015), 693--696.

Digital Library

[33]

Ryohei Tanaka, Takuji Narumi, Tomohiro Tanikawa, and Michitaka Hirose. 2016. Guidance field: Potential field to guide users to target locations in virtual environments. In IEEE Symposium on 3D User Interfaces. 39--48.

[34]

James Tompkin, Kwang In Kim, Jan Kautz, and Christian Theobalt. 2012. Videoscapes: Exploring Sparse, Unstructured Video Collections. ACM Trans. Graph. 31, 4, Article 68 (July 2012), 12 pages.

Digital Library

[35]

Kathleen Tuite, Noah Snavely, and Nadine Tabing. 2011. PhotoCity : Training Experts at Large-scale Image Acquisition Through a Competitive Game. ACM SIGCHI (2011), 1383--1392.

Digital Library

[36]

Michael Waechter, Nils Moehrle, and Michael Goesele. 2014. Let There Be Color! Large-Scale Texturing of 3D Reconstructions. Springer International Publishing, Cham, 836--850.

[37]

Jacob O. Wobbrock, Leah Findlater, Darren Gergle, and James J. Higgins. 2011. The Aligned Rank Transform for Nonparametric Factorial Analyses Using Only Anova Procedures. In ACM SIGCHI. ACM, New York, NY, USA, 143--146.

Digital Library

[38]

Jianxiong Xiao and Yasutaka Furukawa. 2014. Reconstructing the World's Museums. IJCV 110, 3 (2014), 243--258.

Digital Library

[39]

Shanghong Zhao and Wei Tsang Ooi. 2016. Modeling 3D synthetic view dissimilarity. The Visual Computer 32, 4 (01 Apr 2016), 429--443.

Digital Library

[40]

Minhui Zhu, Sebastien Mondet, Géraldine Morin, Wei Tsang Ooi, and Wei Cheng. 2011. Towards Peer-assisted Rendering in Networked Virtual Environments. In Proceedings of the 19th ACM International Conference on Multimedia (MM '11). ACM, New York, NY, USA, 183--192.

Digital Library

Cited By

Klein GChardonnet JPlouzeau JMerienne F(2023)Evaluating Telecollaboration Modalities for the Realization of an Industrial Maintenance Operation in a Constrained EnvironmentExtended Reality10.1007/978-3-031-43404-4_33(477-490)Online publication date: 5-Sep-2023
https://doi.org/10.1007/978-3-031-43404-4_33
Abtahi PHough SLanday JFollmer S(2022)Beyond Being Real: A Sensorimotor Control Perspective on Interactions in Virtual RealityProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3517706(1-17)Online publication date: 29-Apr-2022
https://dl.acm.org/doi/10.1145/3491102.3517706
Zhang XBai XZhang SHe WWang PWang ZYan YYu Q(2022)Real-time 3D video-based MR remote collaboration using gesture cues and virtual replicasThe International Journal of Advanced Manufacturing Technology10.1007/s00170-022-09654-7121:11-12(7697-7719)Online publication date: 2-Aug-2022
https://doi.org/10.1007/s00170-022-09654-7
Show More Cited By

Index Terms

Evaluating snapping-to-photos virtual travel interfaces for 3D reconstructed visual reality
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction paradigms

Recommendations

Hybrid orbiting-to-photos in 3D reconstructed visual reality
VRST '18: Proceedings of the 24th ACM Symposium on Virtual Reality Software and Technology

Virtually navigating through photos from a 3D image-based reconstruction has recently become very popular in many applications. In this paper, we consider a particular virtual travel maneuver that is important for this type of virtual navigation---...
3D User Interfaces: New Directions and Perspectives

Three-dimensional user interfaces (3D UIs) let users interact with virtual objects, environments, or information using direct 3D input in the physical and/or virtual space. In this article, the founders and organizers of the IEEE Symposium on 3D User ...
Virtual Reality Software and Technology

This special issue contains an introductory survey on 3D user interfaces by leading VR authorities and significantly expanded versions of the four best short papers from the proceedings of the 14th ACM Symposium on Virtual Reality Software and ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

VRST '17: Proceedings of the 23rd ACM Symposium on Virtual Reality Software and Technology

November 2017

437 pages

ISBN:9781450355483

DOI:10.1145/3139131

General Chairs:
Morten Fjeld
Chalmers University of Technology
,
Marco Fratarcangeli
Chalmers University of Technology
,
Daniel Sjölie
University of Gothenburg
,
Program Chairs:
Oliver Staadt
University of Rostock
,
Jonas Unger
Linköping University

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 November 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Conference

VRST '17

Sponsor:

VRST '17: 23rd ACM Symposium on Virtual Reality Software and Technology

November 8 - 10, 2017

Gothenburg, Sweden

Acceptance Rates

Overall Acceptance Rate 66 of 254 submissions, 26%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
403
Total Downloads

Downloads (Last 12 months)67
Downloads (Last 6 weeks)11

Reflects downloads up to 22 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Klein GChardonnet JPlouzeau JMerienne F(2023)Evaluating Telecollaboration Modalities for the Realization of an Industrial Maintenance Operation in a Constrained EnvironmentExtended Reality10.1007/978-3-031-43404-4_33(477-490)Online publication date: 5-Sep-2023
https://doi.org/10.1007/978-3-031-43404-4_33
Abtahi PHough SLanday JFollmer S(2022)Beyond Being Real: A Sensorimotor Control Perspective on Interactions in Virtual RealityProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3517706(1-17)Online publication date: 29-Apr-2022
https://dl.acm.org/doi/10.1145/3491102.3517706
Zhang XBai XZhang SHe WWang PWang ZYan YYu Q(2022)Real-time 3D video-based MR remote collaboration using gesture cues and virtual replicasThe International Journal of Advanced Manufacturing Technology10.1007/s00170-022-09654-7121:11-12(7697-7719)Online publication date: 2-Aug-2022
https://doi.org/10.1007/s00170-022-09654-7
Piumsomboon TLee GIrlitti AEns BThomas BBillinghurst MBrewster SFitzpatrick GCox AKostakos V(2019)On the Shoulder of the GiantProceedings of the 2019 CHI Conference on Human Factors in Computing Systems10.1145/3290605.3300458(1-17)Online publication date: 2-May-2019
https://dl.acm.org/doi/10.1145/3290605.3300458
Teo TLawrence LLee GBillinghurst MAdcock MBrewster SFitzpatrick GCox AKostakos V(2019)Mixed Reality Remote Collaboration Combining 360 Video and 3D ReconstructionProceedings of the 2019 CHI Conference on Human Factors in Computing Systems10.1145/3290605.3300431(1-14)Online publication date: 2-May-2019
https://dl.acm.org/doi/10.1145/3290605.3300431
Lee GTeo TKim SBillinghurst M(2018)A User Study on MR Remote Collaboration Using Live 360 Video2018 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)10.1109/ISMAR.2018.00051(153-164)Online publication date: Oct-2018
https://doi.org/10.1109/ISMAR.2018.00051

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents