research-article

Subjective evaluation of group user QoE in collaborative virtual environment (CVE)

Authors:

Bhagyabati Moharana,

Conor Keighrey,

Niall MurrayAuthors Info & Claims

MMVE '22: Proceedings of the 14th International Workshop on Immersive Mixed and Virtual Environment Systems

Pages 23 - 29

https://doi.org/10.1145/3534086.3534333

Published: 11 July 2022 Publication History

Abstract

Interest in the applications of Extended Reality is growing across many different domains. Collaborative or shared experiences are seen as a primary use case. However, there is a surprisingly little research efforts on collaborative design tasks not considering social experiences using Virtual Reality (VR). In addition, there are very few research studies that have focused on the Quality of Experience (QoE) of small user groups working together on collaborative tasks. In this paper, the authors present the results of an experimental study conducted to understand user experience of collaborative tasks using Virtual Reality. The paper presents some initial analysis from self-reported questionnaire data. Two users allocated different roles (Describer and Finder) join remotely to perform the design task collaboratively in immersive VR. The results presented compare user QoE between the two groups (Describer Group vs Finder Group) and considers how different roles and position produces different levels of immersion, interaction, collaboration, post-usage acceptability and system-related consequences. Self-reported measures via post-test questionnaire (15-questions) shows statistically-significant differences in terms of the perceived QoE aspects between the two groups.

References

[1]

A Axelsson, Åsa Abelin, Ilona Heldal, Alexander Nilsson, Ralph Schroeder, and Josef Wideström. 1999. Collaboration and communication in multiuser virtual environments: A comparison of desktop and immersive virtual reality systems for molecular visualization. In Proceedings of the Sixth UKVRSIG Conference, 107--117.

[2]

Doug A Bowman and Ryan P McMahan. 2007. Virtual reality: how much immersion is enough? Computer 40, 7: 36--43.

Digital Library

[3]

P Le Callet, S Moller, and A Perkis. 2013. Qualinet white paper on definitions of quality of experience (QoE). In Output from the Fifth Qualinet Meeting.

[4]

Elizabeth F Churchill and Dave Snowdon. 1998. Collaborative virtual environments: an introductory review of issues and systems. virtual reality 3, 1: 3--15.

[5]

J H Clark. 1924. The Ishihara test for color blindness. American Journal of Physiological Optics.

[6]

David Concannon. 2020. Evaluating the Impact of Network Delay on user Quality of Experience of an Interactive Virtual Reality Industry 4 . 0 Application. September.

[7]

David Concannon, Ronan Flynn, and Niall Murray. 2019. A quality of experience evaluation system and research challenges for networked virtual reality-based teleoperation applications. Proceedings of the 11th ACM Workshop on Immersive Mixed and Virtual Environment Systems, MMVE 2019: 10--12.

Digital Library

[8]

Elena Dzardanova, Vlasios Kasapakis, Damianos Gavalas, and Stella Sylaiou. 2021. Virtual reality as a communication medium: a comparative study of forced compliance in virtual reality versus physical world. Virtual Reality.

Digital Library

[9]

Darragh Egan, Sean Brennan, John Barrett, Yuansong Qiao, Christian Timmerer, and Niall Murray. 2016. An evaluation of Heart Rate and ElectroDermal Activity as an objective QoE evaluation method for immersive virtual reality environments. 2016 8th International Conference on Quality of Multimedia Experience, QoMEX 2016: 3--8.

[10]

Yasin Farmani and Robert J Teather. 2020. Evaluating discrete viewpoint control to reduce cybersickness in virtual reality. Virtual Reality 24, 4: 645--664.

Digital Library

[11]

Anna Felnhofer, Oswald D Kothgassner, Nathalie Hauk, Leon Beutl, Helmut Hlavacs, and Ilse Kryspin-Exner. 2014. Physical and social presence in collaborative virtual environments: Exploring age and gender differences with respect to empathy. Computers in Human Behavior 31: 272--279.

Digital Library

[12]

Lawrence J Hettinger and Gary E Riccio. 1992. Visually induced motion sickness in virtual environments. Presence: Teleoperators & Virtual Environments 1, 3: 306--310.

Digital Library

[13]

E Hynes, R Flynn, ... B Lee - 2019 IEEE 21st, and undefined 2019. A Quality of Experience Evaluation Comparing Augmented Reality and Paper Based Instruction for Complex Task Assistance. ieeexplore.ieee.org. Retrieved from https://sci-hub.do/https://ieeexplore.ieee.org/abstract/document/8901705/?casa_token=BcUoEraMcC8AAAAA:k4uDLSCAGXlI9HeSHqd6g8fioe4_rI2NdpXkvEAC_5uALBAUit4cSQTDw-9LxemWTOLii1KghR4-CA

[14]

Roy S Kalawsky. 2000. The validity of presence as a reliable human performance metric in immersive environments. In 3rd International Workshop on Presence, 1--16.

[15]

Shunichi Kasahara, Shohei Nagai, and Jun Rekimoto. 2015. First person omnidirectional video: System design and implications for immersive experience. In Proceedings of the ACM International Conference on Interactive Experiences for TV and Online Video, 33--42.

Digital Library

[16]

Conor Keighrey, Ronan Flynn, Siobhan Murray, Sean Brennan, and Niall Murray. 2017. Comparing user QoE via physiological and interaction measurements of immersive AR and VR speech and language therapy applications. In Thematic Workshops 2017 - Proceedings of the Thematic Workshops of ACM Multimedia 2017, co-located with MM 2017, 485--492.

Digital Library

[17]

Conor Keighrey, Ronan Flynn, Siobhan Murray, and Niall Murray. 2017. A QoE evaluation of immersive augmented and virtual reality speech & language assessment applications. 2017 9th International Conference on Quality of Multimedia Experience, QoMEX 2017.

[18]

Seungwon Kim, Gun Lee, Weidong Huang, Hayun Kim, Woontack Woo, and Mark Billinghurst. 2019. Evaluating the Combination of Visual Communication Cues for HMD-based Mixed Reality Remote Collaboration. Conference on Human Factors in Computing Systems - Proceedings: 1--13.

Digital Library

[19]

Fernando Kuipers, Robert Kooij, Danny De Vleeschauwer, and Kjell Brunnström. 2010. Techniques for measuring quality of experience. In International Conference on Wired/Wireless Internet Communications, 216--227.

Digital Library

[20]

Michael E McCauley and Thomas J Sharkey. 1992. Cybersickness: Perception of self-motion in virtual environments. Presence: Teleoperators & Virtual Environments 1, 3: 311--318.

[21]

Paul McGraw, Barry Winn, and David Whitaker. 1995. Reliability of the Snellen chart. Bmj 310, 1481--1482.

[22]

Oliver Otto, Dave Roberts, and Robin Wolff. 2006. A review on effective closely-coupled collaboration using immersive CVE's. In Proceedings of the 2006 ACM international conference on Virtual reality continuum and its applications, 145--154.

Digital Library

[23]

Jinjia Ruan and Dongliang Xie. 2021. A survey on qoe-oriented vr video streaming: Some research issues and challenges. Electronics (Switzerland) 10, 17.

[24]

Holger Salzmann, Jan Jacobs, and Bernd Froehlich. 2009. Collaborative interaction in co-located two-user scenarios. In Proceedings of the 15th Joint virtual reality Eurographics conference on Virtual Environments, 85--92.

Digital Library

[25]

Marwin Schmitt, Judith Redi, Dick Bulterman, and Pablo S Cesar. 2017. Towards individual QoE for multiparty videoconferencing. IEEE Transactions on Multimedia 20, 7: 1781--1795.

[26]

Ralph Schroeder, Anthony Steed, Ann-Sofie Axelsson, Ilona Heldal, Åsa Abelin, Josef Wideström, Alexander Nilsson, and Mel Slater. 2001. Collaborating in networked immersive spaces: as good as being there together? Computers & Graphics 25, 5: 781--788.

[27]

Matias N Selzer, Nicolas F Gazcon, and Martin L Larrea. 2019. Effects of virtual presence and learning outcome using low-end virtual reality systems. Displays 59: 9--15.

[28]

Volkan Sevinc and Mehmet Ilker Berkman. 2020. Psychometric evaluation of Simulator Sickness Questionnaire and its variants as a measure of cybersickness in consumer virtual environments. Applied Ergonomics 82, September 2019.

[29]

Kay M Stanney, Robert S Kennedy, and Julie M Drexler. 1997. Cybersickness is not simulator sickness. In Proceedings of the Human Factors and Ergonomics Society annual meeting, 1138--1142.

[30]

Anthony Steed, Sebastian Frlston, Maria Murcia Lopez, Jason Drummond, Ye Pan, and David Swapp. 2016. An 'in the wild' experiment on presence and embodiment using consumer virtual reality equipment. IEEE transactions on visualization and computer graphics 22, 4: 1406--1414.

[31]

Katy Tcha-Tokey, Emilie Loup-Escande, Olivier Christmann, and Simon Richir. 2016. A questionnaire to measure the user experience in immersive virtual environments. In Proceedings of the 2016 virtual reality international conference, 1--5.

Digital Library

[32]

Huyen T T Tran, Nam Pham Ngoc, Anh T Pham, and Truong Cong Thang. 2016. A multi-factor QoE model for adaptive streaming over mobile networks. In 2016 IEEE Globecom Workshops (GC Wkshps), 1--6.

[33]

Peng Wang, Xiaoliang Bai, Mark Billinghurst, Shusheng Zhang, Sili Wei, Guangyao Xu, Weiping He, Xiangyu Zhang, and Jie Zhang. 2020. 3DGAM: using 3D gesture and CAD models for training on mixed reality remote collaboration. Multimedia Tools and Applications.

Digital Library

[34]

Caglar Yildirim. 2019. Cybersickness during VR gaming undermines game enjoyment: a mediation model. Displays 59: 35--43.

[35]

Unity Game Engine. Retrieved March 23, 2022 from https://unity.com/

[36]

Photon Unity 3D Networking Framework SDKs and Game Backend | Photon Engine. Retrieved from https://www.photonengine.com/en/PUN

[37]

ISO 8589:2007 Sensory analysis --- General guidance for the design of test rooms. Retrieved March 23, 2022 from https://www.iso.org/standard/36385.html

Cited By

Vijayakumar SFlynn RCorcoran PMurray N(2024)The Role of ECG and Respiration in Predicting Quality of Experience2024 16th International Conference on Quality of Multimedia Experience (QoMEX)10.1109/QoMEX61742.2024.10598275(118-124)Online publication date: 18-Jun-2024
https://doi.org/10.1109/QoMEX61742.2024.10598275
Moharana BKeighrey CMurray N(2024)Voices Unveiled: Quality of Experience in Collaborative VR via AssemblyAI and NASA-TLX Analysis2024 16th International Conference on Quality of Multimedia Experience (QoMEX)10.1109/QoMEX61742.2024.10598261(15-21)Online publication date: 18-Jun-2024
https://doi.org/10.1109/QoMEX61742.2024.10598261
Ghamandi RHmaiti YNguyen TGhasemaghaei AKattoju RTaranta ELaViola J(2023)What And How Together: A Taxonomy On 30 Years Of Collaborative Human-Centered XR Tasks2023 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)10.1109/ISMAR59233.2023.00047(322-335)Online publication date: 16-Oct-2023
https://doi.org/10.1109/ISMAR59233.2023.00047

Index Terms

Subjective evaluation of group user QoE in collaborative virtual environment (CVE)
1. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

Stepping off a ledge in an HMD-based immersive virtual environment
SAP '13: Proceedings of the ACM Symposium on Applied Perception

We explore whether a gender-matched, calibrated self-avatar affects the perception of the affordance of stepping off of a ledge, or visual cliff, in an immersive virtual environment. Visual cliffs form demonstrations in many immersive virtual ...
Comparative Study of HMD-based Virtual and Augmented Realities for Immersive Museums: User Acceptance, Medium, and Learning
Technologies like Head Mounted Display (HMD)-based Virtual Reality (VR) and Augmented Reality (AR) have made HMD-based immersive museums possible. To investigate the user acceptance, medium, and interaction experience for HMD-based immersive museums, an ...
The effect of avatar model in stepping off a ledge in an immersive virtual environment
SAP '15: Proceedings of the ACM SIGGRAPH Symposium on Applied Perception

Animated digital self-representations of the user in an immersive virtual environment, a self-avatar, have been shown to aid in perceptual judgments in the virtual environment and to provide critical information for people deciding what actions they can ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

MMVE '22: Proceedings of the 14th International Workshop on Immersive Mixed and Virtual Environment Systems

June 2022

59 pages

ISBN:9781450393829

DOI:10.1145/3534086

General Chair:
Conor Keighrey
Technology University of the Shannon: Midlands Midwest, Ireland

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 July 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Technological University of Shannon, Athlone Ireland

Conference

MMSys '22

Sponsor:

MMSys '22: 13th ACM Multimedia Systems Conference

June 14, 2022

Athlone, Ireland

Acceptance Rates

Overall Acceptance Rate 26 of 44 submissions, 59%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
148
Total Downloads

Downloads (Last 12 months)45
Downloads (Last 6 weeks)1

Reflects downloads up to 14 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Vijayakumar SFlynn RCorcoran PMurray N(2024)The Role of ECG and Respiration in Predicting Quality of Experience2024 16th International Conference on Quality of Multimedia Experience (QoMEX)10.1109/QoMEX61742.2024.10598275(118-124)Online publication date: 18-Jun-2024
https://doi.org/10.1109/QoMEX61742.2024.10598275
Moharana BKeighrey CMurray N(2024)Voices Unveiled: Quality of Experience in Collaborative VR via AssemblyAI and NASA-TLX Analysis2024 16th International Conference on Quality of Multimedia Experience (QoMEX)10.1109/QoMEX61742.2024.10598261(15-21)Online publication date: 18-Jun-2024
https://doi.org/10.1109/QoMEX61742.2024.10598261
Ghamandi RHmaiti YNguyen TGhasemaghaei AKattoju RTaranta ELaViola J(2023)What And How Together: A Taxonomy On 30 Years Of Collaborative Human-Centered XR Tasks2023 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)10.1109/ISMAR59233.2023.00047(322-335)Online publication date: 16-Oct-2023
https://doi.org/10.1109/ISMAR59233.2023.00047

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents