research-article

Designing Guided User Tasks in VR Embodied Experiences

Authors:

Florent Robert,

Brice Graulier,

Barthelemy Passin-Cauneau,

Lucile Sassatelli,

Marco WincklerAuthors Info & Claims

Proceedings of the ACM on Human-Computer Interaction, Volume 6, Issue EICS

Article No.: 158, Pages 1 - 24

https://doi.org/10.1145/3532208

Published: 17 June 2022 Publication History

Abstract

Virtual reality (VR) offers extraordinary opportunities in user behavior research to study and observe how people interact in immersive 3D environments. A major challenge of designing these 3D experiences and user tasks, however, lies in bridging the inter-relational gaps of perception between the designer, the user, and the 3D scene. Paul Dourish identified three gaps of perception: ontology between the scene representation and the user and designer interpretation, intersubjectivity of task communication between designer and user, and intentionality between the user's intentions and designer's interpretations.

We present the GUsT-3D framework for designing Guided User Tasks in embodied VR experiences, i.e., tasks that require the user to carry out a series of interactions guided by the constraints of the 3D scene. GUsT-3D is implemented as a set of tools that support a 4-step workflow to (1) annotate entities in the scene with navigation and interaction possibilities, (2) define user tasks with interactive and timing constraints, (3) manage interactions, task validation, and user logging in real-time, and (4) conduct post-scenario analysis through spatio-temporal queries using ontology definitions. To illustrate the diverse possibilities enabled by our framework, we present two case studies with an indoor scene and an outdoor scene, and conducted a formative evaluation involving six expert interviews to assess the framework and the implemented workflow. Analysis of the responses show that the GUsT-3D framework fits well into a designer's creative process, providing a necessary workflow to create, manage, and understand VR embodied experiences.

References

[1]

Abdul-Hadi G Abulrub, Alex N Attridge, and Mark A Williams. 2011. Virtual reality in engineering education: The future of creative learning. In 2011 IEEE global engineering education conference (EDUCON). IEEE, Amman, Jordan, 751--757.

[2]

Johannes Behr, Patrick D"ahne, and Marcus Roth. 2004. Utilizing X3D for Immersive Environments. In Proceedings of the Ninth International Conference on 3D Web Technology (Monterey, California) (Web3D '04). Association for Computing Machinery, New York, NY, USA, 71--78. https://doi.org/10.1145/985040.985051

Digital Library

[3]

Regina Bernhaupt, Célia Martinie, Philippe Palanque, and Günter Wallner. 2020. A Generic Visualization Approach Supporting Task-Based Evaluation of Usability and User Experience. In 8th International Conference on Human-Centered Software Engineering - IFIP WG 13.2 International Working Conference, HCSE 2020 (Lecture Notes in Computer Science book series (LNCS), Vol. 12481), IFIP : International Federation for Information Processing (Ed.). Springer, Eindhoven, Netherlands, 24--44. https://doi.org/10.1007/978--3-030--64266--2_2

[4]

Daniel Beßler, Robert Porzel, Mihai Pomarlan, Michael Beetz, Rainer Malaka, and John Bateman. 2020. A Formal Model of Affordances for Flexible Robotic Task Execution. In ECAI 2020. IOS Press, Santiago de Compostela, Spain, 2425--2432.

[5]

Paul Black and Dylan Wiliam. 2009. Developing the theory of formative assessment. Educational Assessment, Evaluation and Accountability (formerly: Journal of Personnel Evaluation in Education), Vol. 21, 1 (2009), 5--31.

[6]

Sebastian Blumenthal and Herman Bruyninckx. 2014. Towards a Domain Specific Language for a Scene Graph based Robotic World Model. arxiv: 1408.0200 [cs.RO]

[7]

Carol Boston. 2002. The concept of formative assessment. Practical Assessment, Research, and Evaluation, Vol. 8, 1 (2002), 9.

[8]

Rozenn Bouville, Valérie Gouranton, Thomas Boggini, Florian Nouviale, and Bruno Arnaldi. 2015. # FIVE: High-level components for developing collaborative and interactive virtual environments. In 2015 IEEE 8th Workshop on Software Engineering and Architectures for Realtime Interactive Systems (SEARIS). IEEE, 33--40.

[9]

Don Brutzman and Leonard Daly. 2010. X3D: extensible 3D graphics for Web authors .Elsevier, San Francisco, CA, USA.

Digital Library

[10]

Michel Buffa and J-C Lafon. 2000. 3D virtual warehouse on the WEB. In 2000 IEEE Conference on Information Visualization. An International Conference on Computer Visualization and Graphics. IEEE, London, UK, 479--484.

[11]

Michael Burmester, Marcus Mast, Kilian J"ager, and Hendrik Homans. 2010. Valence method for formative evaluation of user experience. In Proceedings of the 8th ACM conference on Designing Interactive Systems. ACM, Aarhus, Denmark, 364--367.

Digital Library

[12]

John Carroll, MARK SINGLEY, and Mary Beth Rosson. 1992. Integrating theory development with design evaluation. Behaviour & Information Technology - Behaviour & IT, Vol. 11 (Sept. 1992), 247--255. https://doi.org/10.1080/01449299208924345

[13]

Tiannan Chen and Stephen Guy. 2018. GIGL: A domain specific language for procedural content generation with grammatical representations. In Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, Vol. 14(1). AAAI, Edmonton, Canada, 8 pages.

[14]

Pierre Chevaillier, Thanh-Hai Trinh, Mukesh Barange, Pierre De Loor, Frédéric Devillers, Julien Soler, and Ronan Querrec. 2012. Semantic modeling of virtual environments using mascaret. In 2012 5th Workshop on Software Engineering and Architectures for Realtime Interactive Systems (SEARIS). IEEE, Costa Mesa, CA, USA, 1--8.

[15]

Paul Dourish. 2004. Where the action is: the foundations of embodied interaction .MIT press, USA.

[16]

Jakub Floty'nski and Paweł Soboci'nski. 2018. Logging Interactions in explorable immersive VR/AR applications. In 2018 International Conference on 3D Immersion (IC3D). IEEE, Brussels, Belgium, 1--8.

[17]

Jakub Floty'nski and Krzysztof Walczak. 2017. Ontology-Based Representation and Modelling of Synthetic 3D Content: A State-of-the-Art Review. In Computer Graphics Forum, Vol. 36(8). Wiley Online Library, Poznan, Poland, 329--353.

[18]

Daniel J Fremont, Tommaso Dreossi, Shromona Ghosh, Xiangyu Yue, Alberto L Sangiovanni-Vincentelli, and Sanjit A Seshia. 2019. Scenic: a language for scenario specification and scene generation. In Proceedings of the 40th ACM SIGPLAN Conference on Programming Language Design and Implementation. Association for Computing Machinery, New York, NY, USA, 63--78.

Digital Library

[19]

Filip Górski. 2017. Building virtual reality applications for engineering with knowledge-based approach. Management and Production Engineering Review, Vol. 8(4) (2017), 63--73. https://doi.org/10.1515/mper-2017-0037

[20]

Sebastian Gottschalk, Enes Yigitbas, Eugen Schmidt, and Gregor Engels. 2020. Model-based product configuration in augmented reality applications. In International Conference on Human-Centred Software Engineering. Springer, Eindhoven, The Netherlands, 84--104.

Digital Library

[21]

Deborah Hix and H. Rex Hartson. 1993. Developing user interfaces : ensuring usability through product & process .New York : J. Wiley. http://archive.org/details/developinguserin00hixd

[22]

Harry Hochheiser Jonathan Lazar, Jinjuan Feng. 2010. Research Methods in Human-Computer Interaction - 2nd Edition. Wiley, United States.

[23]

Wanwan Li, Javier Talavera, Amilcar Gomez Samayoa, Jyh-Ming Lien, and Lap-Fai Yu. 2020. Automatic Synthesis of Virtual Wheelchair Training Scenarios. In 2020 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). IEEE, Atlanta, GA, USA, 539--547.

[24]

Yunchao Liu, Jiajun Wu, Zheng Wu, Daniel Ritchie, William T. Freeman, and Joshua B. Tenenbaum. 2019. Learning to Describe Scenes with Programs. In International Conference on Learning Representations. ICLR, New Orleans, USA, 13 pages. https://openreview.net/forum?id=SyNPk2R9K7

[25]

Erik Meijer, Brian Beckman, and Gavin Bierman. 2006. Linq: reconciling object, relations and xml in the. net framework. In Proceedings of the 2006 ACM SIGMOD international conference on Management of data. ACM, NY, New York, 706--706.

Digital Library

[26]

David Navarre, Philippe A. Palanque, Ré mi Bastide, Amé lie Schyn, Marco Winckler, Luciana Porcher Nedel, and Carla Maria Dal Sasso Freitas. 2005. A Formal Description of Multimodal Interaction Techniques for Immersive Virtual Reality Applications. In Human-Computer Interaction - INTERACT 2005, IFIP TC13 International Conference (Lecture Notes in Computer Science, Vol. 3585), Maria Francesca Costabile and Fabio Paternò (Eds.). Springer, Rome, Italy, 170--183. https://doi.org/10.1007/11555261_17

Digital Library

[27]

Matthias Oberhauser and Daniel Dreyer. 2017. A virtual reality flight simulator for human factors engineering. Cognition, Technology & Work, Vol. 19, 2 (2017), 263--277.

Digital Library

[28]

Fabio Pittarello and Alessandro De Faveri. 2006. Semantic description of 3D environments: a proposal based on web standards. In Proceedings of the eleventh international conference on 3D web technology. ACM, New York, NY, USA, 85--95.

Digital Library

[29]

Thibault Raffaillac and Stéphane Huot. 2019. Polyphony: Programming Interfaces and Interactions with the Entity-Component-System Model. Proceedings of the ACM on Human-Computer Interaction, Vol. 3, EICS (2019), 1--22.

Digital Library

[30]

Rafael Sacks, Jennifer Whyte, Dana Swissa, Gabriel Raviv, Wei Zhou, and Aviad Shapira. 2015. Safety by design: dialogues between designers and builders using virtual reality. Construction Management and Economics, Vol. 33, 1 (2015), 55--72.

[31]

Julien Saunier, Mukesh Barange, Bernard Blandin, Ronan Querrec, and Joanna Taoum. 2016. Designing adaptable virtual reality learning environments. In Proceedings of the 2016 Virtual Reality International Conference. ACM, New York, NY, USA, 1--4.

Digital Library

[32]

Maximilian Speicher, Katy Lewis, and Michael Nebeling. 2021. Designers, the Stage Is Yours! Medium-Fidelity Prototyping of Augmented & Virtual Reality Interfaces with 360theater. Proceedings of ACM Human-Computer Interactions, Vol. 5, EICS, Article 205 (May 2021), 25 pages. https://doi.org/10.1145/3461727

Digital Library

[33]

Sandra Trösterer, Elke Beck, Fabiano Dalpiaz, Elda Paja, Paolo Giorgini, and Manfred Tscheligi. 2012. Formative user-centered evaluation of security modeling: Results from a case study. International Journal of Secure Software Engineering (IJSSE), Vol. 3, 1 (2012), 1--19.

[34]

Lode Vanacken, Chris Raymaekers, and Karin Coninx. 2007. Introducing semantic information during conceptual modelling of interaction for virtual environments. In Proceedings of the 2007 workshop on Multimodal interfaces in semantic interaction. ACM, New York, NY, USA, 17--24.

Digital Library

[35]

Diego Vergara, Manuel Pablo Rubio, and Miguel Lorenzo. 2017. On the design of virtual reality learning environments in engineering. Multimodal technologies and interaction, Vol. 1, 2 (2017), 11.

[36]

Kai Wang, Yu-An Lin, Ben Weissmann, Manolis Savva, Angel X Chang, and Daniel Ritchie. 2019. Planit: Planning and instantiating indoor scenes with relation graph and spatial prior networks. ACM Transactions on Graphics (TOG), Vol. 38, 4 (2019), 1--15. Publisher: ACM New York, NY, USA.

Digital Library

[37]

Haojie Wu, Daniel H Ashmead, Haley Adams, and Bobby Bodenheimer. 2018. Using Virtual Reality to Assess the Street Crossing Behavior of Pedestrians with Simulated Macular Degeneration at a Roundabout. Frontiers in ICT, Vol. 5 (2018), 27.

[38]

Hui-Yin Wu, Aurélie Calabrèse, and Pierre Kornprobst. 2021. Towards accessible news reading design in virtual reality for low vision. Multimedia Tools and Applications, Vol. 80 (2021), 1--20.

Digital Library

Cited By

Artizzu VLuyten KRuiz GSpano L(2024)ViRgilites: Multilevel Feedforward for Multimodal Interaction in VRProceedings of the ACM on Human-Computer Interaction10.1145/36586458:EICS(1-24)Online publication date: 17-Jun-2024
https://dl.acm.org/doi/10.1145/3658645
Javerliat CVillenave SRaimbaud PLavoué G(2024)PLUME: Record, Replay, Analyze and Share User Behavior in 6DoF XR ExperiencesIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2024.337210730:5(2087-2097)Online publication date: 4-Mar-2024
https://dl.acm.org/doi/10.1109/TVCG.2024.3372107
Artizzu V(2023)End User Development for Extended RealityEngineering Interactive Computer Systems. EICS 2023 International Workshops and Doctoral Consortium10.1007/978-3-031-59235-5_13(151-165)Online publication date: 27-Jun-2023
https://dl.acm.org/doi/10.1007/978-3-031-59235-5_13

Recommendations

An Integrated Framework for Understanding Multimodal Embodied Experiences in Interactive Virtual Reality
IMX '23: Proceedings of the 2023 ACM International Conference on Interactive Media Experiences

Virtual Reality (VR) technology enables “embodied interactions” in realistic environments where users can freely move and interact, with deep physical and emotional states. However, a comprehensive understanding of the embodied user experience is ...
Analyzing and understanding embodied interactions in virtual reality systems: research proposal
MMSys '22: Proceedings of the 13th ACM Multimedia Systems Conference

Virtual reality (VR) offers opportunities in human-computer interaction research, to embody users in immersive environments and observe how they interact with 3D scenarios under well-controlled environments. VR content has stronger influences on users ...
Analysing and Understanding Embodied Interactions in Virtual Reality Systems
IMX '23: Proceedings of the 2023 ACM International Conference on Interactive Media Experiences

Virtual reality (VR) offers opportunities in human-computer interaction research, to embody users in immersive environments and observe how they interact with 3D scenarios under well-controlled environments. VR content has stronger influences on users ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Human-Computer Interaction

Proceedings of the ACM on Human-Computer Interaction Volume 6, Issue EICS

EICS

June 2022

736 pages

EISSN:2573-0142

DOI:10.1145/3544787

Editor:
Jeff Nichols
Apple Inc., United States

Issue’s Table of Contents

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 June 2022

Published in PACMHCI Volume 6, Issue EICS

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Agence Nationale de la Recherche

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
295
Total Downloads

Downloads (Last 12 months)130
Downloads (Last 6 weeks)9

Reflects downloads up to 17 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Artizzu VLuyten KRuiz GSpano L(2024)ViRgilites: Multilevel Feedforward for Multimodal Interaction in VRProceedings of the ACM on Human-Computer Interaction10.1145/36586458:EICS(1-24)Online publication date: 17-Jun-2024
https://dl.acm.org/doi/10.1145/3658645
Javerliat CVillenave SRaimbaud PLavoué G(2024)PLUME: Record, Replay, Analyze and Share User Behavior in 6DoF XR ExperiencesIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2024.337210730:5(2087-2097)Online publication date: 4-Mar-2024
https://dl.acm.org/doi/10.1109/TVCG.2024.3372107
Artizzu V(2023)End User Development for Extended RealityEngineering Interactive Computer Systems. EICS 2023 International Workshops and Doctoral Consortium10.1007/978-3-031-59235-5_13(151-165)Online publication date: 27-Jun-2023
https://dl.acm.org/doi/10.1007/978-3-031-59235-5_13

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents