research-article

Agent: automatic generation of experimental protocol runtime

Authors:

Gwendal Le Moulec,

Ferran Argelaguet,

Valérie Gouranton,

Bruno ArnaldiAuthors Info & Claims

VRST '17: Proceedings of the 23rd ACM Symposium on Virtual Reality Software and Technology

Article No.: 10, Pages 1 - 10

https://doi.org/10.1145/3139131.3139152

Published: 08 November 2017 Publication History

Abstract

Due to the nature of Virtual Reality (VR) research, conducting experiments in order to validate the researcher's hypotheses is a must. However, the development of such experiments is a tedious and time-consuming task. In this work, we propose to make this task easier, more intuitive and faster with a method able to describe and generate the most tedious components of VR experiments. The main objective is to let experiment designers focus on their core tasks: designing, conducting, and reporting experiments. To that end, we propose the use of Domain-Specific Languages (DSLs) to ease the description and generation of VR experiments. An analysis of published VR experiments is used to identify the main properties that characterize VR experiments. This allowed us to design AGENT (Automatic Generation of ExperimeNtal proTocol runtime), a DSL for specifying and generating experimental protocol runtimes. We demonstrated the feasibility of our approach by using AGENT on two experiments published in the VRST'16 proceedings.

References

[1]

2016. VRST '16: Proceedings of the 22Nd ACM Conference on Virtual Reality Software and Technology. ACM, New York, NY, USA.

[2]

Doug A Bowman, Donald B Johnson, and Larry F Hodges. 1999. Testbed Evaluation of Virtual Environment Interaction Techniques. In VRST '99: Proceedings of the ACM symposium on Virtual reality software and technology. ACM, 26--33.

Digital Library

[3]

John Brooke et al. 1996. SUS-A quick and dirty usability scale. Usability evaluation in industry 189, 194 (1996), 4--7.

[4]

Andrew M Colman. 2015. A dictionary of psychology. Oxford University Press, USA.

[5]

John W Creswell. 2013. Research design: Qualitative, quantitative, and mixed methods approaches. Sage publications.

[6]

Frank T Durso, Andrew R Dattel, S Banbury, and Sebastien Tremblay. 2004. SPAM: The real-time assessment of SA. A cognitive approach to situation awareness: Theory and application 1 (2004), 137--154.

[7]

Andy Field. 2009. Discovering statistics using SPSS. Sage publications.

Digital Library

[8]

Andy Field and Graham Hole. 2002. How to design and report experiments. Sage.

[9]

Martin Fowler. 2010. Domain-specific languages. Pearson Education.

[10]

Joseph L Gabbard, Deborah Hix, and J Edward Swan. 1999. User-centered design and evaluation of virtual environments. IEEE computer Graphics and Applications 19, 6 (1999), 51--59.

Digital Library

[11]

Jascha Grübel, Raphael Weibel, Christoph Hölscher, and Victor R Schinazi. 2016. EVE: A Framework for Experiments in Virtual Environments. In Proceedings of Spatial Coginition 2016. https://www.researchgate.net/publication/307853873_Spatial_Coginition_2016_-_Poster_Session_1_EVE_Framework

[12]

Sandra G. Hart and Lowell E. Staveland. 1988. Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research. Advances in Psychology 52 (1988), 139 -- 183. Human Mental Workload.

[13]

Kasper Hornbæk. 2006. Current practice in measuring usability: Challenges to usability studies and research. International Journal of Human-Computer Studies 64, 2 (2006), 79 -- 102.

Digital Library

[14]

Weidong Huang, Leila Alem, and Mark A Livingston. 2012. Human factors in augmented reality environments. Springer Science & Business Media.

[15]

V. Interrante, B. Ries, and L. Anderson. 2006. Distance Perception in Immersive Virtual Environments, Revisited. In IEEE Virtual Reality. IEEE, 3--10.

Digital Library

[16]

Simon Julier, Steven Feiner, and Lawrence Rosenblum. 2001. Mobile augmented reality: a complex human-centered system. Springer Science & Business Media. 67 pages.

[17]

Marc Erich Latoschik, Jean-Luc Lugrin, and Daniel Roth. 2016. FakeMi: A Fake Mirror System for Avatar Embodiment Studies. In Proceedings of the 22Nd ACM Conference on Virtual Reality Software and Technology (VRST '16). ACM, New York, NY, USA, 73--76.

Digital Library

[18]

Gwendal Le Moulec, Ferran Argelaguet, Anatole Lécuyer, and Valérie Gouranton. 2016. Take-over Control Paradigms in Collaborative Virtual Environments for Training. In Proceedings of the 22Nd ACM Conference on Virtual Reality Software and Technology (VRST '16). ACM, New York, NY, USA, 65--68.

Digital Library

[19]

Daniel Medeiros, Maurício Sousa, Daniel Mendes, Alberto Raposo, and Joaquim Jorge. 2016. Perceiving Depth: Optical Versus Video See-through. In Proceedings of the 22Nd ACM Conference on Virtual Reality Software and Technology (VRST '16). ACM, New York, NY, USA, 237--240.

Digital Library

[20]

Anne Meier, Hans Spada, and Nikol Rummel. 2007. A rating scheme for assessing the quality of computer-supported collaboration processes. International Journal of Computer-Supported Collaborative Learning 2, 1 (01 Mar 2007), 63--86.

[21]

Marjan Mernik, Jan Heering, and Anthony M. Sloane. 2005. When and How to Develop Domain-specific Languages. ACM Comput. Surv. 37, 4 (Dec. 2005), 316--344.

Digital Library

[22]

Annette Mossel and Christian Koessler. 2016. Large Scale Cut Plane: An Occlusion Management Technique for Immersive Dense 3D Reconstructions. In Proceedings of the 22Nd ACM Conference on Virtual Reality Software and Technology (VRST '16). ACM, New York, NY, USA, 201--210.

Digital Library

[23]

Judea Pearl, Elias Bareinboim, et al. 2014. External validity: From do-calculus to transportability across populations. Statist. Sci. 29, 4 (2014), 579--595.

[24]

Douglas C Schmidt. 2006. Model-driven engineering. COMPUTER-IEEE COMPUTER SOCIETY 39, 2 (2006), 25.

Digital Library

[25]

Thomas Schubert, Frank Friedmann, and Holger Regenbrecht. 2001. The Experience of Presence: Factor Analytic Insights. Presence: Teleoperators and Virtual Environments 10, 3 (2001), 266--281. arXiv:http://dx.doi.org/10.1162/105474601300343603

Digital Library

[26]

William R. Shadish, Thomas D Cook, and Donald Thomas Campbell. 2002. Experimental and quasi-experimental designs for generalized causal inference. Wadsworth Cengage learning.

[27]

Mel Slater. 2009. Place illusion and plausibility can lead to realistic behaviour in immersive virtual environments. Philosophical transactions of the Royal Society of London. Series B, Biological sciences 364, 1535 (2009), 3549--3557.

[28]

Mel Slater, Martin Usoh, and Anthony Steed. 1994. Depth of Presence in Virtual Environments. Presence: Teleoperators and Virtual Environments 3, 2 (1994), 130--144. arXiv:http://dx.doi.org/10.1162/pres.1994.3.2.130

Digital Library

[29]

Kay M. Stanney, Mansooreh Mollaghasemi, Leah Reeves, Robert Breaux, and David A. Graeber. 2003. Usability engineering of virtual environments (VEs): identifying multiple criteria that drive effective VE system design. International Journal of Human-Computer Studies 58, 4 (2003), 447 -- 481.

Digital Library

[30]

Jolanda G. Tromp, Anthony Steed, and John R. Wilson. 2003. Systematic Usability Evaluation and Design Issues for Collaborative Virtual Environments. Presence: Teleoperators and Virtual Environments 12, 3 (2003), 241--267. arXiv:http://dx.doi.org/10.1162/105474603765879512

Digital Library

[31]

Arie van Deursen, Paul Klint, and Joost Visser. 2000. Domain-specific Languages: An Annotated Bibliography. SIGPLAN Not. 35, 6 (June 2000), 26--36.

Digital Library

[32]

David Wile. 2004. Lessons learned from real DSL experiments. Science of Computer Programming 51, 3 (2004), 265 -- 290.

Digital Library

[33]

Bob G. Witmer and Michael J. Singer. 1998. Measuring Presence in Virtual Environments: A Presence Questionnaire. Presence: Teleoperators and Virtual Environments 7, 3 (1998), 225--240. arXiv:http://dx.doi.org/10.1162/105474698565686

Digital Library

Cited By

Grübel J(2023)The design, experiment, analyse, and reproduce principle for experimentation in virtual realityFrontiers in Virtual Reality10.3389/frvir.2023.10694234Online publication date: 12-Apr-2023
https://doi.org/10.3389/frvir.2023.1069423
Ragnarsson AChakraborty SLiebel G(2021)ModRec: A Tool to Support Empirical Study Design for Papyrus and the Eclipse Modeling Framework2021 ACM/IEEE International Conference on Model Driven Engineering Languages and Systems Companion (MODELS-C)10.1109/MODELS-C53483.2021.00059(361-369)Online publication date: Oct-2021
https://doi.org/10.1109/MODELS-C53483.2021.00059
Lethbridge TPretschner AVoss SBurgueño L(2019)UmpleOnline as a testbed for modeling empirical studiesProceedings of the 22nd International Conference on Model Driven Engineering Languages and Systems10.1109/MODELS-C.2019.00064(413-414)Online publication date: 15-Sep-2019
https://dl.acm.org/doi/10.1109/MODELS-C.2019.00064

Recommendations

Declarative specifications for the development of multi-agent systems

The designing and implementation of a multi-agent system (MAS), where autonomous agents collaborate with other agents for solving problems, constitute complex tasks that may become even harder when agents work in new interactive environments such as the ...
Declaratively defining domain-specific language debuggers
GCPE '11

Tool support is vital to the effectiveness of domain-specific languages. With language workbenches, domain-specific languages and their tool support can be generated from a combined, high-level specification. This paper shows how such a specification ...
SESSL: A domain-specific language for simulation experiments

This article introduces SESSL (Simulation Experiment Specification via a Scala Layer), an embedded domain-specific language for simulation experiments. It serves as an additional software layer between users and simulation systems and is implemented in ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

VRST '17: Proceedings of the 23rd ACM Symposium on Virtual Reality Software and Technology

November 2017

437 pages

ISBN:9781450355483

DOI:10.1145/3139131

General Chairs:
Morten Fjeld
Chalmers University of Technology
,
Marco Fratarcangeli
Chalmers University of Technology
,
Daniel Sjölie
University of Gothenburg
,
Program Chairs:
Oliver Staadt
University of Rostock
,
Jonas Unger
Linköping University

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 November 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

VRST '17

Sponsor:

VRST '17: 23rd ACM Symposium on Virtual Reality Software and Technology

November 8 - 10, 2017

Gothenburg, Sweden

Acceptance Rates

Overall Acceptance Rate 66 of 254 submissions, 26%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
158
Total Downloads

Downloads (Last 12 months)9
Downloads (Last 6 weeks)0

Reflects downloads up to 12 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Grübel J(2023)The design, experiment, analyse, and reproduce principle for experimentation in virtual realityFrontiers in Virtual Reality10.3389/frvir.2023.10694234Online publication date: 12-Apr-2023
https://doi.org/10.3389/frvir.2023.1069423
Ragnarsson AChakraborty SLiebel G(2021)ModRec: A Tool to Support Empirical Study Design for Papyrus and the Eclipse Modeling Framework2021 ACM/IEEE International Conference on Model Driven Engineering Languages and Systems Companion (MODELS-C)10.1109/MODELS-C53483.2021.00059(361-369)Online publication date: Oct-2021
https://doi.org/10.1109/MODELS-C53483.2021.00059
Lethbridge TPretschner AVoss SBurgueño L(2019)UmpleOnline as a testbed for modeling empirical studiesProceedings of the 22nd International Conference on Model Driven Engineering Languages and Systems10.1109/MODELS-C.2019.00064(413-414)Online publication date: 15-Sep-2019
https://dl.acm.org/doi/10.1109/MODELS-C.2019.00064

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents