research-article

With a little help from a holographic friend: the OpenIMPRESS mixed reality telepresence toolkit for remote collaboration systems

Authors:

Sander Giesselink,

Dennis Reidsma,

Mariët Theune,

Dirk HeylenAuthors Info & Claims

VRST '18: Proceedings of the 24th ACM Symposium on Virtual Reality Software and Technology

Article No.: 26, Pages 1 - 11

https://doi.org/10.1145/3281505.3281542

Published: 28 November 2018 Publication History

Abstract

Remote mixed reality (MR) collaboration systems allow for multimodal, real-time support from remote experts. We present our open toolkit that provides a flexible end-to-end solution for building such systems using off-the-shelf hardware. From related work, three core design aspects have been identified: 1) the independence of the viewpoint that the visitor (the remote expert) can take in relation to position and viewpoint of the visitee, 2) the immersiveness of the presentation technology for visitor and visitee, and 3) the extent to which the visitor's body is represented in the visitee's environment. This paper describes the implementation of our system, which includes these aspects. In a study aimed at validating whether we implemented these core aspects to good effect, conducted with a collaborative puzzle application built with our toolkit, we examine how variations of these aspects contribute to usability, performance and social presence related metrics.

Supplementary Material

MP4 File (a26-kolkmeier.mp4)

Download
36.89 MB

References

[1]

M. Adcock, S. Anderson, and B. Thomas. 2013. RemoteFusion: real time depth camera fusion for remote collaboration on physical tasks. In Proceedings of the 12th ACM SIGGRAPH International Conference on Virtual-Reality Continuum and Its Applications in Industry (VRCAI '13). ACM, New York, NY, USA, 235--242.

Digital Library

[2]

L. Alem and J. Li. 2011. A study of gestures in a video-mediated collaborative assembly task. Advances in Human-Computer Interaction (2011), 1.

Digital Library

[3]

J. Amores, X. Benavides, and P. Maes. 2015. Showme: A remote collaboration system that supports immersive gestural communication. In Extended Abstracts of the 33rd Conference on Human Factors in Computing Systems. 1343--1348.

Digital Library

[4]

R.M. Baños, C. Botella, M. Alcañiz, V. Liaño, B. Guerrero, and B. Rey. 2004. Immersion and emotion: their impact on the sense of presence. CyberPsychology & Behavior 7, 6 (2004), 734--741.

[5]

J. Brooke and Others. 1996. SUS-A quick and dirty usability scale. Usability evaluation in industry 189, 194 (1996), 4--7.

[6]

M. Le Chénéchal, T. Duval, V. Gouranton, J. Royan, and B. Arnaldi. 2016. Vishnu: virtual immersive support for HelpiNg users: an interaction paradigm for collaborative remote guiding in mixed reality. In Proceedings of the IEEE Third VR International Workshop on Collaborative Virtual Environments (3DCVE). IEEE, 9--12.

[7]

A. Fairchild, S. Campion, A. GarcÃą, R. Wolff, T. Fernando, and D. Roberts. 2017. A Mixed Reality Telepresence System for Collaborative Space Operation. IEEE Transactions on Circuits and Systems for Video Technology 27, 4 (2017), 814--827.

Digital Library

[8]

B. Ford, P. Srisuresh, and D. Kegel. 2005. Peer-to-Peer Communication Across Network Address Translators. http://www.brynosaurus.com/pub/net/p2pnat/

[9]

S.R. Fussell, L.D Setlock, and R.E. Kraut. 2003. Effects of head-mounted and scene-oriented video systems on remote collaboration on physical tasks. In Proc. of the SIGCHI conference on Human factors in computing systems. ACM, 513--520.

Digital Library

[10]

L. Gao, H. Bai, G. Lee, and M. Billinghurst. 2016. An oriented point-cloud view for MR remote collaboration. In SIGGRAPH ASIA 2016 Mobile Graphics and Interactive Applications. ACM, 8.

Digital Library

[11]

E.L. Groen and J.E. Bos. 2008. Simulator sickness depends on frequency of the simulator motion mismatch: An observation. Presence 17, 6 (2008), 584--593.

Digital Library

[12]

C. Harms and F. Biocca. 2004. Internal consistency and reliability of the networked minds measure of social presence. In Seventh Annual International Workshop: Presence 2004.

[13]

W. Huang and L. Alem. 2011. Supporting hand gestures in mobile remote collaboration: a usability evaluation. In Proceedings of the 25th BCS Conference on Human-Computer Interaction. British Computer Society, 211--216.

Digital Library

[14]

S. Kasahara and J. Rekimoto. 2014. JackIn: integrating first-person view with out-of-body vision generation for human-human augmentation. In Proceedings of the 5th Augmented Human International Conference. ACM, 46.

Digital Library

[15]

S. Kasahara and J. Rekimoto. 2015. JackIn head: immersive visual telepresence system with omnidirectional wearable camera for remote collaboration. In Proceedings of the 21st ACM Symposium on Virtual Reality Software and Technology. ACM, 217--225.

Digital Library

[16]

R. Komiyama, T. Miyaki, and J. Rekimoto. 2017. JackIn space: designing a seamless transition between first and third person view for effective telepresence collaborations. In Proceedings of the 8th Augmented Human International Conference. ACM, 14.

Digital Library

[17]

M. Kowalski, J. Naruniec, and M. Daniluk. 2015. Live Scan3D: A Fast and Inexpensive 3D Data Acquisition System for Multiple Kinect v2 Sensors. In 3D Vision (3DV), 2015 International Conference on. IEEE, 318--325.

Digital Library

[18]

A. Kristoffersson, S. Coradeschi, and A. Loutfi. 2013. A review of mobile robotic telepresence. Advances in Human-Computer Interaction (2013), 3.

Digital Library

[19]

G. Lee, T. Teo, S. Kim, and M. Billinghurst. 2017. Mixed reality collaboration through sharing a live panorama. In SIGGRAPH Asia 2017 Mobile Graphics Interactive Applications. ACM, New York, NY, USA. paper 14.

Digital Library

[20]

S. Nicholson. 2016. The State of Escape: Escape Room Design and Facilities. Meaningful Play 2016. Lansing, Michigan. (2016).

[21]

O. Oda, C. Elvezio, M. Sukan, S. Feiner, and B. Tversky. 2015. Virtual Replicas for Remote Assistance in Virtual and Augmented Reality. In Proceedings of the 28th Annual ACM Symposium on User Interface Software Technology. ACM, New York, NY, USA, 405--415.

Digital Library

[22]

O. Oda and S. Feiner. 2012. 3D referencing techniques for physical objects in shared augmented reality. In Proceedings of the 2012 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). 207--215.

Digital Library

[23]

S. Orts-Escolano, C. Rhemann, S. Fanello, W. Chang, A. Kowdle, Y. Degtyarev, D. Kim, P.L. Davidson, S. Khamis, M. Dou, and Others. 2016. Holoportation: Virtual 3d teleportation in real-time. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology. ACM, 741--754.

Digital Library

[24]

T. Piumsomboon, A. Dey, B. Ens, Y. Lee, G. Lee, and M. Billinghurst. 2017. Exploring enhancements for remote mixed reality collaboration. In SIGGRAPH Asia 2017 Mobile Graphics Interactive Applications. ACM, New York, NY, USA. paper 16.

Digital Library

[25]

T. Piumsomboon, G. Lee, J. Hart, B. Ens, R. Lindeman, B. Thomas, and M. Billinghurst. 2018. Mini-Me: An Adaptive Avatar for Mixed Reality Remote Collaboration. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, New York, NY, USA. paper 46.

Digital Library

[26]

R. Poelman, O. Akman, S. Lukosch, and P. Jonker. 2012. As if being there: mediated reality for crime scene investigation. In Proceedings of the ACM 2012 Conference on Computer Supported Cooperative Work. ACM, 1267--1276.

Digital Library

[27]

I. Rae, B. Mutlu, and L. Takayama. 2014. Bodies in motion: mobility, presence, and task awareness in telepresence. In Proceedings of the 32nd annual ACM conference on Human factors in computing systems. ACM, 2153--2162.

Digital Library

[28]

B. Rogers and M. Graham. 1979. Motion parallax as an independent cue for depth perception. Perception 8, 2 (1979), 125--134.

[29]

T. Schubert, F. Friedmann, and H. Regenbrecht. 2001. The experience of presence: Factor analytic insights. Presence: Teleoperators & Virtual Environments 10, 3 (2001), 266--281.

Digital Library

[30]

R.S. Sodhi, B.R. Jones, D. Forsyth, B.P. Bailey, and G. Maciocci. 2013. BeThere: 3D mobile collaboration with spatial input. In Proc. of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 179--188.

Digital Library

[31]

A. Steed, W. Steptoe, W. Oyekoya, F. Pece, T. Weyrich, J. Kautz, D. Friedman, A. Peer, M. Solazzi, F. Tecchia, and Others. 2012. Beaming: an asymmetric telepresence system. IEEE computer graphics and applications 32, 6 (2012), 10--17.

Digital Library

[32]

M. Tait and M. Billinghurst. 2015. The effect of view independence in a collaborative AR system. Computer Supported Cooperative Work 24, 6 (2015), 563--589.

Digital Library

[33]

F. Tecchia, L. Alem, and W. Huang. 2012. {3D} helping hands: a gesture based MR system for remote collaboration. In Proceedings of the 11th ACM SIGGRAPH International Conference on Virtual-Reality Continuum and its Applications in Industry. ACM, 323--328.

Digital Library

Cited By

Dawarwadikar MRao M(2024)3D Avatars in Virtual Reality ExperienceEncyclopedia of Computer Graphics and Games10.1007/978-3-031-23161-2_458(13-20)Online publication date: 5-Jan-2024
https://doi.org/10.1007/978-3-031-23161-2_458
Coupry CRichard PBigaud DNoblecourt SBaudry D(2023)The Value of Extended Reality Techniques to Improve Remote Collaborative Maintenance Operations: A User StudyCONVR 2023 - Proceedings of the 23rd International Conference on Construction Applications of Virtual Reality10.36253/979-12-215-0289-3.03(23-33)Online publication date: 2023
https://doi.org/10.36253/979-12-215-0289-3.03
Coupry CRichard PBigaud DNoblecourt SBaudry D(2023)The Value of Extended Reality Techniques to Improve Remote Collaborative Maintenance Operations: A User StudyCONVR 2023 - Proceedings of the 23rd International Conference on Construction Applications of Virtual Reality10.36253/10.36253/979-12-215-0289-3.03(23-33)Online publication date: 2023
https://doi.org/10.36253/10.36253/979-12-215-0289-3.03
Show More Cited By

Index Terms

With a little help from a holographic friend: the OpenIMPRESS mixed reality telepresence toolkit for remote collaboration systems
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interactive systems and tools
      1. User interface toolkits

Recommendations

Real-time 3D video avatar in mixed reality: an implementation for immersive telecommunication
Symposium: virtual reality simulation

This article presents an implementation of a real-time dynamic 3D avatar from multiview cameras for immersive telecommunication. Immersive telecommunication is a new challenging field that enables a user to share a virtual space with remote participants,...
VROOM: Virtual Robot Overlay for Online Meetings
CHI EA '20: Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems

Telepresence robots allow users to freely explore a remote space and provide a physical embodiment in that space. However, they lack a compelling representation of the remote user in the local space. We present VROOM (Virtual Robot Overlay for Online ...
A mixed reality telepresence system with limited DOF motion base and immersive display
ACE '09: Proceedings of the International Conference on Advances in Computer Entertainment Technology

This paper describes a mixed reality (MR) telepresence system for a ride to provide users with a highly realistic sensation. To make a realistic scene in a virtual environment, it is necessary to combine visual information with a reproduction of the ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

VRST '18: Proceedings of the 24th ACM Symposium on Virtual Reality Software and Technology

November 2018

570 pages

ISBN:9781450360869

DOI:10.1145/3281505

Editor:
Stephen N. Spencer
University of Washington
,
General Chair:
Shigeo Morishima
Waseda University
,
Program Chairs:
Yuichi Itoh
Osaka University
,
Takaaki Shiratori
Facebook Reality Labs
,
Yonghao Yue
The University of Tokyo
,
Rob Lindeman
University of Canterbury

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 28 November 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

VRST '18

Sponsor:

VRST '18: 24th ACM Symposium on Virtual Reality Software and Technology

November 28 - December 1, 2018

Tokyo, Japan

Acceptance Rates

Overall Acceptance Rate 66 of 254 submissions, 26%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

32
Total Citations
View Citations
777
Total Downloads

Downloads (Last 12 months)65
Downloads (Last 6 weeks)1

Reflects downloads up to 23 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Dawarwadikar MRao M(2024)3D Avatars in Virtual Reality ExperienceEncyclopedia of Computer Graphics and Games10.1007/978-3-031-23161-2_458(13-20)Online publication date: 5-Jan-2024
https://doi.org/10.1007/978-3-031-23161-2_458
Coupry CRichard PBigaud DNoblecourt SBaudry D(2023)The Value of Extended Reality Techniques to Improve Remote Collaborative Maintenance Operations: A User StudyCONVR 2023 - Proceedings of the 23rd International Conference on Construction Applications of Virtual Reality10.36253/979-12-215-0289-3.03(23-33)Online publication date: 2023
https://doi.org/10.36253/979-12-215-0289-3.03
Coupry CRichard PBigaud DNoblecourt SBaudry D(2023)The Value of Extended Reality Techniques to Improve Remote Collaborative Maintenance Operations: A User StudyCONVR 2023 - Proceedings of the 23rd International Conference on Construction Applications of Virtual Reality10.36253/10.36253/979-12-215-0289-3.03(23-33)Online publication date: 2023
https://doi.org/10.36253/10.36253/979-12-215-0289-3.03
Johnson JSharkey TButarbutar IXiong DHuang RSy LWeibel N(2023)UnMapped: Leveraging Experts’ Situated Experiences to Ease Remote Guidance in Collaborative Mixed RealityProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581444(1-20)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581444
Fidalgo CYan YCho HSousa MLindlbauer DJorge J(2023)A Survey on Remote Assistance and Training in Mixed Reality EnvironmentsIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.324708129:5(2291-2303)Online publication date: May-2023
https://doi.org/10.1109/TVCG.2023.3247081
Ghamandi RHmaiti YNguyen TGhasemaghaei AKattoju RTaranta ELaViola J(2023)What And How Together: A Taxonomy On 30 Years Of Collaborative Human-Centered XR Tasks2023 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)10.1109/ISMAR59233.2023.00047(322-335)Online publication date: 16-Oct-2023
https://doi.org/10.1109/ISMAR59233.2023.00047
Albeedan MKolivand HHammady R(2023)Effect of Augmented Reality and Virtual Reality in Crime Scene Investigations2023 16th International Conference on Developments in eSystems Engineering (DeSE)10.1109/DeSE60595.2023.10469037(653-657)Online publication date: 18-Dec-2023
https://doi.org/10.1109/DeSE60595.2023.10469037
Abd Karim Ishigaki SIsmail A(2023)Real-Time Full Body Tracking for Life-Size TelepresenceAdvances and Applications of Artificial Intelligence & Machine Learning10.1007/978-981-99-5974-7_22(261-273)Online publication date: 15-Nov-2023
https://doi.org/10.1007/978-981-99-5974-7_22
Brown GPrilla M(2023)ARBAY – Wissenschaftliche Erkenntnisse über eine Augmented-Reality-basierte Beratungs- und Verkaufsplattform für hochvariante und individualisierbare GüterPersonennahe Dienstleistungen der Zukunft10.1007/978-3-658-38813-3_5(69-82)Online publication date: 26-May-2023
https://doi.org/10.1007/978-3-658-38813-3_5
Jun HShaik HDeVeaux CLewek MFuchs HBailenson J(2022)An Evaluation Study of 2D and 3D Teleconferencing for Remote Physical TherapyPRESENCE: Virtual and Augmented Reality10.1162/pres_a_0037931(47-67)Online publication date: 1-Dec-2022
https://doi.org/10.1162/pres_a_00379
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents