research-article

User Behaviour Analysis of Mixed Reality Remote Collaboration with a Hybrid View Interface

Authors:

Mark Billinghurst,

Robert W. LindemanAuthors Info & Claims

OzCHI '20: Proceedings of the 32nd Australian Conference on Human-Computer Interaction

Pages 629 - 638

https://doi.org/10.1145/3441000.3441038

Published: 15 February 2021 Publication History

Abstract

In this paper, we present a Mixed Reality (MR) remote collaboration system that enables hybrid view sharing from the local worker to the remote expert for real-time remote guidance. The local worker can share either a 2D first-person view, a 360° live view with a fixed viewpoint, or a 3D free view within a point-cloud reconstruction of the local environment. The remote expert can access these views and freely switch between them in Virtual Reality (VR) for better awareness of the local worker’s surroundings. We investigate the remote experts’ behaviours while using the hybrid view interface during typical pick-and-place remote guiding tasks. We found that the remote experts prefer to learn the local physical layout and search for the targets with a global perspective from the 3D free view. The results also showed that the experts chose to use the 360° live view with independent view control rather than the 2D first-person view with high-resolution to control the task procedures and check the local worker’s actions. Our study informs the viewpoint interface design for a MR remote collaboration system in various guiding scenarios.

References

[1]

Matt Adcock, Stuart Anderson, and Bruce Thomas. 2013. RemoteFusion: real time depth camera fusion for remote collaboration on physical tasks. In Proceedings of the 12th ACM SIGGRAPH International Conference on Virtual-Reality Continuum and Its Applications in Industry. ACM, 235–242.

Digital Library

[2]

Leila Alem and Jane Li. 2011. A study of gestures in a video-mediated collaborative assembly task. Advances in Human-Computer Interaction 2011 (2011), 1.

Digital Library

[3]

Judith Amores, Xavier Benavides, and Pattie Maes. 2015. Showme: A remote collaboration system that supports immersive gestural communication. In Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems. ACM, 1343–1348.

Digital Library

[4]

Mark Billinghurst, Alaeddin Nassani, and Carolin Reichherzer. 2014. Social panoramas: using wearable computers to share experiences. In SIGGRAPH Asia 2014 Mobile Graphics and Interactive Applications. ACM, 25.

[5]

John Brooke. 1996. SUS: a “quick and dirty’usability. Usability evaluation in industry(1996), 189.

[6]

Thierry Duval, Thi Thuong Huyen Nguyen, Cédric Fleury, Alain Chauffaut, Georges Dumont, and Valérie Gouranton. 2014. Improving awareness for 3D virtual collaboration by embedding the features of users’ physical environments and by augmenting interaction tools with cognitive feedback cues. Journal on Multimodal User Interfaces 8, 2 (2014), 187–197.

[7]

Susan R Fussell, Robert E Kraut, and Jane Siegel. 2000. Coordination of communication: Effects of shared visual context on collaborative work. In Proceedings of the 2000 ACM conference on Computer supported cooperative work. ACM, 21–30.

Digital Library

[8]

Susan R Fussell, Leslie D Setlock, and Robert E Kraut. 2003. Effects of head-mounted and scene-oriented video systems on remote collaboration on physical tasks. In Proceedings of the SIGCHI conference on Human factors in computing systems. ACM, 513–520.

Digital Library

[9]

Lei Gao, Huidong Bai, Gun Lee, and Mark Billinghurst. 2016. An oriented point-cloud view for MR remote collaboration. In SIGGRAPH ASIA 2016 Mobile Graphics and Interactive Applications. ACM, 8.

Digital Library

[10]

Leonardo Giusti, Kotval Xerxes, Amelia Schladow, Nicholas Wallen, Francis Zane, and Federico Casalegno. 2012. Workspace configurations: setting the stage for remote collaboration on physical tasks. In Proceedings of the 7th Nordic Conference on Human-Computer Interaction: Making Sense Through Design. ACM, 351–360.

Digital Library

[11]

Saul Greenberg, Carl Gutwin, and Mark Roseman. 1996. Semantic telepointers for groupware. In Computer-Human Interaction, 1996. Proceedings., Sixth Australian Conference on. IEEE, 54–61.

[12]

Kunal Gupta, Gun A Lee, and Mark Billinghurst. 2016. Do you see what i see? the effect of gaze tracking on task space remote collaboration. IEEE transactions on visualization and computer graphics 22, 11(2016), 2413–2422.

Digital Library

[13]

Chad Harms and Frank Biocca. 2004. Internal Consistency and Reliability of the Networked Minds Measure of Social Presence. Seventh Annual International Workshop: Presence 2004 (2004), 246–251. http://cogprints.org/7026/

[14]

Shunichi Kasahara, Shohei Nagai, and Jun Rekimoto. 2014. LiveSphere: immersive experience sharing with 360 degrees head-mounted cameras. In Proceedings of the adjunct publication of the 27th annual ACM symposium on User interface software and technology. ACM, 61–62.

Digital Library

[15]

Robert S Kennedy, Norman E Lane, Kevin S Berbaum, and Michael G Lilienthal. 1993. Simulator sickness questionnaire: An enhanced method for quantifying simulator sickness. The international journal of aviation psychology 3, 3 (1993), 203–220.

[16]

Seungwon Kim, Gun Lee, Nobuchika Sakata, and Mark Billinghurst. 2014. Improving co-presence with augmented visual communication cues for sharing experience through video conference. In Mixed and Augmented Reality (ISMAR), 2014 IEEE International Symposium on. IEEE, 83–92.

[17]

David Kirk, Tom Rodden, and Danaë Stanton Fraser. 2007. Turn it this way: grounding collaborative action with remote gestures. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems. ACM, 1039–1048.

Digital Library

[18]

Joel Lanir, Ran Stone, Benjamin Cohen, and Pavel Gurevich. 2013. Ownership and control of point of view in remote assistance. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 2243–2252.

Digital Library

[19]

Joel Lanir, Ran Stone, Benjamin Cohen, and Pavel Gurevich. 2013. Ownership and control of point of view in remote assistance. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems - CHI ’13 (2013), 2243. https://doi.org/10.1145/2470654.2481309

Digital Library

[20]

Morgan Le Chénéchal, Thierry Duval, Valérie Gouranton, Jérôme Royan, and Bruno Arnaldi. 2016. Vishnu: virtual immersive support for helping users an interaction paradigm for collaborative remote guiding in mixed reality. In Collaborative Virtual Environments (3DCVE), 2016 IEEE Third VR International Workshop on. IEEE, 9–12.

[21]

Gun A Lee, Theophilus Teo, Seungwon Kim, and Mark Billinghurst. 2018. A user study on mr remote collaboration using live 360 video. In 2018 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). IEEE, 153–164.

[22]

Paul Milgram and Fumio Kishino. 1994. A taxonomy of mixed reality visual displays. IEICE TRANSACTIONS on Information and Systems 77, 12 (1994), 1321–1329.

[23]

Jiazhi Ou, Lui Min Oh, Susan R Fussell, Tal Blum, and Jie Yang. 2008. Predicting visual focus of attention from intention in remote collaborative tasks. IEEE Transactions on Multimedia 10, 6 (2008), 1034–1045.

Digital Library

[24]

Thammathip Piumsomboon, Arindam Day, Barrett Ens, Youngho Lee, Gun Lee, and Mark Billinghurst. 2017. Exploring enhancements for remote mixed reality collaboration. In SIGGRAPH Asia 2017 Mobile Graphics & Interactive Applications. ACM, 16.

[25]

Thammathip Piumsomboon, Gun A Lee, Jonathon D Hart, Barrett Ens, Robert W Lindeman, Bruce H Thomas, and Mark Billinghurst. 2018. Mini-Me: An Adaptive Avatar for Mixed Reality Remote Collaboration. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, 46.

Digital Library

[26]

Abhishek Ranjan, Jeremy P Birnholtz, and Ravin Balakrishnan. 2007. Dynamic shared visual spaces: experimenting with automatic camera control in a remote repair task. In Proceedings of the SIGCHI conference on Human factors in computing systems. ACM, 1177–1186.

Digital Library

[27]

Jeff Sauro. 2011. Measuring usability with the system usability scale. URL: http://www. measuringusability. com/sus. php and procedure (9.4. 2012.) (2011).

[28]

Jeff Sauro and Joseph S. Dumas. 2009. Comparison of three one-question, post-task usability questionnaires. Conference on Human Factors in Computing Systems - ProceedingsApril 2009 (2009), 1599–1608. https://doi.org/10.1145/1518701.1518946

Digital Library

[29]

Dongmahn Seo, Suhyun Kim, JaeWook Yoo, Hogun Park, and Heedong Ko. 2012. Immersive panorama TV service system. In Consumer Electronics (ICCE), 2012 IEEE International Conference on. IEEE, 201–202.

[30]

Rodrigo Silva, Bruno Feijó, Pablo B Gomes, Thiago Frensh, and Daniel Monteiro. 2016. Real time 360° video stitching and streaming. In ACM SIGGRAPH 2016 Posters. ACM, 70.

Digital Library

[31]

Rajinder S Sodhi, Brett R Jones, David Forsyth, Brian P Bailey, and Giuliano Maciocci. 2013. BeThere: 3D mobile collaboration with spatial input. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 179–188.

Digital Library

[32]

Franco Tecchia, Leila Alem, and Weidong Huang. 2012. 3D helping hands: a gesture based MR system for remote collaboration. In Proceedings of the 11th ACM SIGGRAPH International Conference on Virtual-Reality Continuum and its Applications in Industry. ACM, 323–328.

Digital Library

[33]

Theophilus Teo, Louise Lawrence, Gun A. Lee, Mark Billinghurst, and Matt Adcock. 2019. Mixed reality remote collaboration combining 360 video and 3D reconstruction. Conference on Human Factors in Computing Systems - Proceedings (2019), 1–14. https://doi.org/10.1145/3290605.3300431

Digital Library

[34]

Peter Vorderer, Werner Wirth, Feliz Ribeiro Gouveia, Frank Biocca, Timo Saari, Lutz Jäncke, Saskia Böcking, Holger Schramm, Andre Gysbers, Tilo Hartmann, Christoph Klimmt, Niklas Ravaja, Ana Sacau, Thomas Baumgartner, and Petra Jäncke. 2004. MEC Spatial Presence Questionnaire (MEC-SPQ). Short Documentation and Instructions for Application. Report to the European Community, Project Presence: MEC (IST-2001-37661). 18, August 2017(2004), 2015. arxiv:IST-2001-37661

[35]

Xiangyu Wang, Peter ED Love, Mi Jeong Kim, and Wei Wang. 2014. Mutual awareness in collaborative design: An Augmented Reality integrated telepresence system. Computers in Industry 65, 2 (2014), 314–324.

Digital Library

Cited By

Huang XYin MXia ZXiao R(2024)VirtualNexus: Enhancing 360-Degree Video AR/VR Collaboration with Environment Cutouts and Virtual ReplicasProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676377(1-12)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3654777.3676377
Chang ELee YBillinghurst MYoo B(2024)Efficient VR-AR communication method using virtual replicas in XR remote collaborationInternational Journal of Human-Computer Studies10.1016/j.ijhcs.2024.103304190:COnline publication date: 1-Oct-2024
https://dl.acm.org/doi/10.1016/j.ijhcs.2024.103304
Teo TSakurada KSugimoto M(2023)Exploring Enhancements towards Gaze Oriented Parallel Views in Immersive Tasks2023 IEEE Conference Virtual Reality and 3D User Interfaces (VR)10.1109/VR55154.2023.00077(620-630)Online publication date: Mar-2023
https://doi.org/10.1109/VR55154.2023.00077
Show More Cited By

User Behaviour Analysis of Mixed Reality Remote Collaboration with a Hybrid View Interface
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction paradigms

Recommendations

Mixed Reality Remote Collaboration Combining 360 Video and 3D Reconstruction
CHI '19: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems

Remote Collaboration using Virtual Reality (VR) and Augmented Reality (AR) has recently become a popular way for people from different places to work together. Local workers can collaborate with remote helpers by sharing 360-degree live video or 3D ...
Real-time visual representations for mobile mixed reality remote collaboration
SA '18: SIGGRAPH Asia 2018 Virtual & Augmented Reality

In this study we present a Mixed-Reality based mobile remote collaboration system that enables an expert providing real-time assistance over a physical distance. By using the Google ARCore position tracking, we can integrate the keyframes captured with ...
An oriented point-cloud view for MR remote collaboration
SA '16: SIGGRAPH ASIA 2016 Mobile Graphics and Interactive Applications

We present a Mixed Reality system for remote collaboration using Virtual Reality (VR) headsets with external depth cameras attached. By wirelessly sharing a 3D point-cloud data of a local workers' workspace with a remote helper, and sharing the remote ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

OzCHI '20: Proceedings of the 32nd Australian Conference on Human-Computer Interaction

December 2020

764 pages

ISBN:9781450389754

DOI:10.1145/3441000

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 February 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

OzCHI '20

OzCHI '20: 32nd Australian Conference on Human-Computer-Interaction

December 2 - 4, 2020

NSW, Sydney, Australia

Acceptance Rates

Overall Acceptance Rate 362 of 729 submissions, 50%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

10
Total Citations
View Citations
439
Total Downloads

Downloads (Last 12 months)113
Downloads (Last 6 weeks)19

Reflects downloads up to 14 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Huang XYin MXia ZXiao R(2024)VirtualNexus: Enhancing 360-Degree Video AR/VR Collaboration with Environment Cutouts and Virtual ReplicasProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676377(1-12)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3654777.3676377
Chang ELee YBillinghurst MYoo B(2024)Efficient VR-AR communication method using virtual replicas in XR remote collaborationInternational Journal of Human-Computer Studies10.1016/j.ijhcs.2024.103304190:COnline publication date: 1-Oct-2024
https://dl.acm.org/doi/10.1016/j.ijhcs.2024.103304
Teo TSakurada KSugimoto M(2023)Exploring Enhancements towards Gaze Oriented Parallel Views in Immersive Tasks2023 IEEE Conference Virtual Reality and 3D User Interfaces (VR)10.1109/VR55154.2023.00077(620-630)Online publication date: Mar-2023
https://doi.org/10.1109/VR55154.2023.00077
Fidalgo CYan YCho HSousa MLindlbauer DJorge J(2023)A Survey on Remote Assistance and Training in Mixed Reality EnvironmentsIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.324708129:5(2291-2303)Online publication date: 1-May-2023
https://dl.acm.org/doi/10.1109/TVCG.2023.3247081
Ghamandi RHmaiti YNguyen TGhasemaghaei AKattoju RTaranta ELaViola J(2023)What And How Together: A Taxonomy On 30 Years Of Collaborative Human-Centered XR Tasks2023 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)10.1109/ISMAR59233.2023.00047(322-335)Online publication date: 16-Oct-2023
https://doi.org/10.1109/ISMAR59233.2023.00047
Chang ELee YYoo B(2023)A User Study on the Comparison of View Interfaces for VR-AR Communication in XR Remote CollaborationInternational Journal of Human–Computer Interaction10.1080/10447318.2023.224129440:19(5794-5809)Online publication date: 17-Aug-2023
https://doi.org/10.1080/10447318.2023.2241294
Otsuki MWang TKuzuoka H(2022)Assessment of Instructor’s Capacity in One-to-Many AR Remote Instruction GivingProceedings of the 28th ACM Symposium on Virtual Reality Software and Technology10.1145/3562939.3565631(1-5)Online publication date: 29-Nov-2022
https://dl.acm.org/doi/10.1145/3562939.3565631
Zhang LHe WBai HZou QWang SBillinghurst M(2022)A hybrid 2D–3D tangible interface combining a smartphone and controller for virtual realityVirtual Reality10.1007/s10055-022-00735-227:2(1273-1291)Online publication date: 18-Dec-2022
https://dl.acm.org/doi/10.1007/s10055-022-00735-2
Druta RDruta CNegirla PSilea I(2021)A Review on Methods and Systems for Remote CollaborationApplied Sciences10.3390/app11211003511:21(10035)Online publication date: 26-Oct-2021
https://doi.org/10.3390/app112110035
Lee YYoo BLee S(2021)Sharing Ambient Objects Using Real-time Point Cloud Streaming in Web-based XR Remote CollaborationProceedings of the 26th International Conference on 3D Web Technology10.1145/3485444.3487642(1-9)Online publication date: 8-Nov-2021
https://dl.acm.org/doi/10.1145/3485444.3487642

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents