research-article

In touch with the remote world: remote collaboration with augmented reality drawings and virtual navigation

Authors:

Steffen Gauglitz,

Benjamin Nuernberger,

Tobias HöllererAuthors Info & Claims

VRST '14: Proceedings of the 20th ACM Symposium on Virtual Reality Software and Technology

Pages 197 - 205

https://doi.org/10.1145/2671015.2671016

Published: 11 November 2014 Publication History

Abstract

Augmented reality annotations and virtual scene navigation add new dimensions to remote collaboration. In this paper, we present a touchscreen interface for creating freehand drawings as world-stabilized annotations and for virtually navigating a scene reconstructed live in 3D, all in the context of live remote collaboration. Two main focuses of this work are (1) automatically inferring depth for 2D drawings in 3D space, for which we evaluate four possible alternatives, and (2) gesture-based virtual navigation designed specifically to incorporate constraints arising from partially modeled remote scenes. We evaluate these elements via qualitative user studies, which in addition provide insights regarding the design of individual visual feedback elements and the need to visualize the direction of drawings.

Supplementary Material

JPG File (p197-gauglitz.jpg)

Download
15.15 KB

MP4 File (p197-gauglitz.mp4)

Download
17.12 MB

References

[1]

Bauer, M., Kortuem, G., and Segall, Z. 1999. "Where are you pointing at?" A study of remote collaboration in a wearable videoconference system. In Proceedings of the 3rd International Symposium on Wearable Computers (ISWC), 151--158.

Digital Library

[2]

Cabral, B., and Leedom, L. C. 1993. Imaging vector fields using line integral convolution. In Proceedings of the 20th ACM SIGGRAPH Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH), 263--270.

Digital Library

[3]

Chen, S., Chen, M., Kunz, A., Yantaç, A. E., Bergmark, M., Sundin, A., and Fjeld, M. 2013. SEMarbeta: Mobile sketch-gesture-video remote support for car drivers. In Proceedings of the 4th Augmented Human International Conference, 69--76.

Digital Library

[4]

Christie, M., and Olivier, P. 2009. Camera control in computer graphics: Models, techniques and applications. In ACM SIGGRAPH ASIA 2009 Courses, 3:1--3:197.

Digital Library

[5]

Fussell, S. R., Setlock, L. D., Yang, J., Ou, J., Mauer, E., and Kramer, A. D. I. 2004. Gestures over video streams to support remote collaboration on physical tasks. Hum.-Comput. Interact. 19 (September), 273--309.

Digital Library

[6]

Gauglitz, S., Lee, C., Turk, M., and Höllerer, T. 2012. Integrating the physical environment into mobile remote collaboration. In Proceedings of the ACM SIGCHI International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI).

Digital Library

[7]

Gauglitz, S., Nuernberger, B., Turk, M., and Höllerer, T. 2014. World-stabilized annotations and virtual scene navigation for remote collaboration. In Proceedings of the 27th ACM Symposium on User Interfaces Software and Technology (UIST).

Digital Library

[8]

Gurevich, P., Lanir, J., Cohen, B., and Stone, R. 2012. TeleAdvisor: A versatile augmented reality tool for remote assistance. In Proceedings of the ACM SIGCHI Annual Conference on Human Factors in Computing Systems (CHI), 619--622.

Digital Library

[9]

Hachet, M., Declec, F., Knodel, S., and Guitton, P. 2008. Navidget for easy 3D camera positioning from 2D inputs. In Proceedings of the IEEE Symposium on 3D User Interfaces (3DUI), 83--89.

Digital Library

[10]

Hanson, A. J., and Wernert, E. A. 1997. Constrained 3D navigation with 2D controllers. In Proceedings of the 8th Conference on Visualization.

Digital Library

[11]

Huang, W., and Alem, L. 2013. HandsInAir: A wearable system for remote collaboration on physical tasks. In Proceedings of the Conference on Computer Supported Cooperative Work Companion (CSCW), 153--156.

Digital Library

[12]

Igarashi, T., Matsuoka, S., and Tanaka, H. 2007. Teddy: A sketching interface for 3D freeform design. In ACM SIGGRAPH 2007 Courses.

Digital Library

[13]

Jankowski, J., and Hachet, M. 2013. A survey of interaction techniques for interactive 3D environments. In Proc. Eurographics--STAR.

[14]

Jo, H., and Hwang, S. 2013. Chili: Viewpoint control and on-video drawing for mobile video calls. In CHI '13 Extended Abstracts on Human Factors in Computing Systems, 1425--1430.

Digital Library

[15]

Kasahara, S., Heun, V., Lee, A. S., and Ishii, H. 2012. Second surface: Multi-user spatial collaboration system based on augmented reality. In ACM SIGGRAPH Asia 2012 Emerging Technologies, 20:1--20:4.

Digital Library

[16]

Kim, S., Lee, G. A., Sakata, N., Vartiainen, E., and Billinghurst, M. 2013. Comparing pointing and drawing for remote collaboration. In Proceedings of the IEEE International Symposium on Mixed and Augmeted Reality (ISMAR) Extended Abstracts, 1--6.

[17]

Kirk, D., and Fraser, D. S. 2006. Comparing remote gesture technologies for supporting collaborative physical tasks. In Proceedings of the ACM SIGCHI Annual Conference on Human Factors in Computing Systems (CHI), 1191--1200.

Digital Library

[18]

Marchal, D., Moerman, C., Casiez, G., and Roussel, N. 2013. Designing intuitive multi-touch 3D navigation techniques. In Human-Computer Interaction -- INTERACT 2013, P. Kotzé, G. Marsden, G. Lindgaard, J. Wesson, and M. Winckler, Eds., vol. 8117 of Lecture Notes in Computer Science. Springer Berlin/Heidelberg, 19--36.

[19]

Oda, O., Sukan, M., Feiner, S., and Tversky, B. 2013. Poster: 3D referencing for remote task assistance in augmented reality. In Proceedings of the IEEE Symposium on 3D User Interfaces (3DUI), 179--180.

[20]

Ou, J., Fussell, S. R., Chen, X., Setlock, L. D., and Yang, J. 2003. Gestural communication over video stream: supporting multimodal interaction for remote collaborative physical tasks. In Proceedings of the 5th International Conference on Multimodal interfaces (ICMI), 242--249.

Digital Library

[21]

Snavely, N., Seitz, S., and Szeliski, R. 2006. Photo tourism: exploring photo collections in 3D. ACM Trans. Graph. 25, 3, 835--846.

Digital Library

[22]

Snavely, N., Garg, R., Seitz, S. M., and Szeliski, R. 2008. Finding paths through the world's photos. In Proceedings of the ACM SIGGRAPH Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH), 15:1--15:11.

Digital Library

[23]

Sodhi, R. S., Jones, B. R., Forsyth, D., Bailey, B. P., and Maciocci, G. 2013. BeThere: 3D mobile collaboration with spatial input. In Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems (CHI), 179--188.

Digital Library

[24]

Tan, D. S., Robertson, G. G., and Czerwinski, M. 2001. Exploring 3D navigation: combining speed-coupled flying with orbiting. In Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems (CHI), 418--425.

Digital Library

[25]

Tolba, O., Dorsey, J., and McMillan, L. 1999. Sketching with projective 2D strokes. In Proceedings of the 12th Annual ACM Symposium on User Interface Software and Technology (UIST), 149--157.

Digital Library

[26]

Uyttendaele, M., Criminisi, A., Kang, S. B., Winder, S., Szeliski, R., and Hartley, R. 2004. Image-based interactive exploration of real-world environments. IEEE Comput. Graph. Appl. 24, 3, 52--63.

Digital Library

[27]

van den Hengel, A., Dick, A., Thormählen, T., Ward, B., and Torr, P. H. S. 2007. Videotrace: Rapid interactive scene modelling from video. ACM Trans. Graph. 26, 3.

Digital Library

[28]

Xin, M., Sharlin, E., and Sousa, M. C. 2008. Napkin Sketch: Handheld mixed reality 3D sketching. In Proceedings of the ACM Symposium on Virtual Reality Software and Technology (VRST), 223--226.

Digital Library

[29]

Zeleznik, R., and Forsberg, A. 1999. Unicam -- 2D gestural camera controls for 3D environments. In Proceedings of the ACM Symposium on Interactive 3D Graphics, 169--173.

Digital Library

[30]

Zeleznik, R. C., Herndon, K. P., and Hughes, J. F. 2007. SKETCH: An interface for sketching 3D scenes. In ACM SIGGRAPH 2007 Courses.

Digital Library

Cited By

Liao JVan KXia ZSuzuki R(2024)RealityEffects: Augmenting 3D Volumetric Videos with Object-Centric Annotation and Dynamic Visual EffectsProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661631(1248-1261)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3643834.3661631
Stefanidi H(2024)Meaningful Interaction with Digital Data in MotionAdjunct Proceedings of the 26th International Conference on Mobile Human-Computer Interaction10.1145/3640471.3686643(1-3)Online publication date: 21-Sep-2024
https://dl.acm.org/doi/10.1145/3640471.3686643
Borhani ZSharma POrtega F(2024)Survey of Annotations in Extended Reality SystemsIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.328886930:8(5074-5096)Online publication date: Aug-2024
https://doi.org/10.1109/TVCG.2023.3288869
Show More Cited By

Index Terms

In touch with the remote world: remote collaboration with augmented reality drawings and virtual navigation

Recommendations

World-stabilized annotations and virtual scene navigation for remote collaboration
UIST '14: Proceedings of the 27th annual ACM symposium on User interface software and technology

We present a system that supports an augmented shared visual space for live mobile remote collaboration on physical tasks. The remote user can explore the scene independently of the local user's current camera position and can communicate via spatial ...
Belonging There: VROOM-ing into the Uncanny Valley of XR Telepresence
CSCW

The world is entering a new normal of hybrid organisations, in which it will be common that some members are co-located and others are remote. Hybridity is rife with asymmetries that affect our sense of belonging in an organisational space. This paper ...
MagicBook: transitioning between reality and virtuality
CHI EA '01: CHI '01 Extended Abstracts on Human Factors in Computing Systems

The MagicBook explores how interfaces can be developed that allow for seamless transition between Physical Reality, Augmented Reality (AR), and immersive Virtual Reality (VR) in a collaborative setting. The MagicBook is a normal book and can be read ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

VRST '14: Proceedings of the 20th ACM Symposium on Virtual Reality Software and Technology

November 2014

238 pages

ISBN:9781450332538

DOI:10.1145/2671015

Conference Chairs:
Rynson Lau
City University of Hong Kong
,
Dinesh Manocha
University of North Carolina, Chapel Hill
,
Program Chairs:
Taku Komura
University of Edinburgh
,
Aditi Majumder
University of California, Irvine
,
Weiwei Xu
Hangzhou Normal University

Copyright © 2014 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGGRAPH: ACM Special Interest Group on Computer Graphics and Interactive Techniques
Disney Research: Disney Research
NVIDIA
SIGCHI: ACM Special Interest Group on Computer-Human Interaction
UEDINI: Division of Informatics, University of Edinburgh
SICSA: The Scottish Informatics and Computer Science Alliance
Presence: Presence: Teleoperators and Virtual Environments

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 November 2014

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Conference

VRST '14

Sponsor:

SIGGRAPH
Disney Research
SIGCHI
UEDINI
SICSA
Presence

VRST '14: The 20th ACM Symposium on Virtual Reality Software and Technology

November 11 - 13, 2014

Edinburgh, Scotland

Acceptance Rates

Overall Acceptance Rate 66 of 254 submissions, 26%

Upcoming Conference

VRST '24

Sponsor:
sigchi
sigchi

30th ACM Symposium on Virtual Reality Software and Technology

October 9 - 11, 2024

Trier , Germany

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

101
Total Citations
View Citations
1,734
Total Downloads

Downloads (Last 12 months)119
Downloads (Last 6 weeks)6

Reflects downloads up to 03 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Liao JVan KXia ZSuzuki R(2024)RealityEffects: Augmenting 3D Volumetric Videos with Object-Centric Annotation and Dynamic Visual EffectsProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661631(1248-1261)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3643834.3661631
Stefanidi H(2024)Meaningful Interaction with Digital Data in MotionAdjunct Proceedings of the 26th International Conference on Mobile Human-Computer Interaction10.1145/3640471.3686643(1-3)Online publication date: 21-Sep-2024
https://dl.acm.org/doi/10.1145/3640471.3686643
Borhani ZSharma POrtega F(2024)Survey of Annotations in Extended Reality SystemsIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.328886930:8(5074-5096)Online publication date: Aug-2024
https://doi.org/10.1109/TVCG.2023.3288869
Numata TOgi YMitani KTajima KNakamura YIkeda NShimada K(2024)Metaverse‐based remote support system with smooth combination of free viewpoint observation and hand gesture instructionJournal of the Society for Information Display10.1002/jsid.133932:9(653-664)Online publication date: 14-Jul-2024
https://doi.org/10.1002/jsid.1339
Huang WKim SBillinghurst MAlem LHuang WBillinghurst MAlem LXiao CRasmussen T(2024)Sharing Hand Gesture and Sketch Cues with a Touch User InterfaceComputer‐Supported Collaboration10.1002/9781119719830.ch9(215-242)Online publication date: 24-May-2024
https://doi.org/10.1002/9781119719830.ch9
Huang WBillinghurst MAlem LXiao CRasmussen THuang WBillinghurst MAlem LXiao CRasmussen T(2024)Evaluating Augmented Reality Remote Guidance SystemsComputer‐Supported Collaboration10.1002/9781119719830.ch6(143-171)Online publication date: 24-May-2024
https://doi.org/10.1002/9781119719830.ch6
Huang WWakefield MRasmussen TKim SBillinghurst MHuang WBillinghurst MAlem LXiao CRasmussen T(2024)Communication Cues for Remote GuidanceComputer‐Supported Collaboration10.1002/9781119719830.ch4(81-114)Online publication date: 24-May-2024
https://doi.org/10.1002/9781119719830.ch4
Praveena PWang YSenft EGleicher MMutlu B(2023)Periscope: A Robotic Camera System to Support Remote Physical CollaborationProceedings of the ACM on Human-Computer Interaction10.1145/36101997:CSCW2(1-39)Online publication date: 4-Oct-2023
https://dl.acm.org/doi/10.1145/3610199
Sakashita MKim HWoodard BZhang RGuimbretière F(2023)VRoxy: Wide-Area Collaboration From an Office Using a VR-Driven Robotic ProxyProceedings of the 36th Annual ACM Symposium on User Interface Software and Technology10.1145/3586183.3606743(1-13)Online publication date: 29-Oct-2023
https://dl.acm.org/doi/10.1145/3586183.3606743
Bailly CCastet J(2023)WIM-Drone: Combining a World-In-Miniature and Micro-Gestures for Teleoperation in Augmented RealityProceedings of the 34th Conference on l'Interaction Humain-Machine10.1145/3583961.3583972(1-11)Online publication date: 3-Apr-2023
https://dl.acm.org/doi/10.1145/3583961.3583972
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents