short-paper

Towards a formative instrument to evaluate user experience in virtual reality serious games

Authors:

Emilio Ormeño,

Claudio Alessio,

Sergio F. Ochoa,

César A. CollazosAuthors Info & Claims

REHAB '19: Proceedings of the 5th Workshop on ICTs for improving Patients Rehabilitation Research Techniques

Pages 53 - 57

https://doi.org/10.1145/3364138.3364152

Published: 29 January 2020 Publication History

Abstract

Regardless the large efforts done in assessing user experience in virtual reality games, few instruments have been proposed to conduct these evaluations on VR serious games. Most proposals focus on evaluating final products or address different subsets of design aspects, providing few or no feedback to designers during the development of a game. In order to help developers assess their prototypes and determine the soundness of their interim designs during the development of VR serious games, this article proposes a formative evaluation instrument based on a literature review. The instrument has been particularly designed to evaluate user experience and provide feedback to designers for improving their proposals when required. The usability and usefulness of this instrument was evaluated with experts in the design of serious games, and the preliminary results are highly positive.

References

[1]

H. K. Leng and D. Y. Pyun, "Development of a scale to measure gamer experiences in sport video games," IJSMM, vol. 18, no. 5, p. 417, 2018.

[2]

J. H. Brockmyer, C. M. Fox, K. A. Curtiss, E. McBroom, K. M. Burkhart, and J. N. Pidruzny, "The development of the Game Engagement Questionnaire: A measure of engagement in video game-playing," J. Exp. Soc. Psychol., vol. 45, no. 4, pp. 624--634, Jul. 2009.

[3]

H. Desurvire, M. Caplan, and J. A. Toth, "Using Heuristics to Evaluate the Playability of Games," in CHI '04 Extended Abstracts on Human Factors in Computing Systems, pp. 1509--1512, Vienna, Austria, 2004.

[4]

N. P. Zea, J. L. G. Sánchez, F. L. Gutiérrez, M. J. Cabrera, and P. Paderewski, "Design of educational multiplayer video games: A vision from collaborative learning," Adv. Eng. Softw., vol. 40, no. 12, pp. 1251--1260, Dec. 2009.

Digital Library

[5]

"ISO 9241-210:2010," ISO, 2018. Available: www.iso.org/standard/52075.html. [22-Jul.-2019]

[6]

M. H. Phan, J. R. Keebler, and B. S. Chaparro, "The Development and Validation of the Game User Experience Satisfaction Scale (GUESS)," Hum. Factors, vol. 58, no. 8, pp. 1217--1247, Dec. 2016.

[7]

W. J. Shelstad, D. C. Smith, and B. S. Chaparro, "Gaming on the Rift: How Virtual Reality Affects Game User Satisfaction," Proc. Hum. Fact. Ergon. Soc. Annu. Meet., vol. 61, no. 1, pp. 2072--2076, Sep. 2017.

[8]

S. C. B. Kitchenham, "Guidelines for performing Systematic Literature Reviews in Software Engineering," 2007. Available: citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.117.471. [22-Jul.-2019]

[9]

B. G. Witmer and M. J. Singer, "Measuring Presence in Virtual Environments: A Presence Questionnaire," Presence, vol. 7, no. 3, pp. 225--240, Jun. 1998.

Digital Library

[10]

W. Wirth et al., "A Process Model of the Formation of Spatial Presence Experiences," Media Psychol., vol. 9, no. 3, pp. 493--525, May 2007.

[11]

R. M. Baños, C. Botella, M. Alcañiz, V. Liaño, B. Guerrero, and B. Rey, "Immersion and emotion: their impact on the sense of presence," Cyberpsychol. Behav., vol. 7, no. 6, pp. 734--741, Dec. 2004.

[12]

N. C. Nilsson, R. Nordahl, and S. Serafin, "Immersion revisited: A review of existing definitions of immersion and their relation to different theories of presence," Human Technology, vol. 12, 2016.

[13]

I. van Hilvoorde and N. Pot, "Embodiment and fundamental motor skills in eSports," Sport, Ethics and Philosophy, vol. 10, no. 1, pp. 14--27, Jan. 2016.

[14]

R. Lindgren and M. Johnson-Glenberg, "Emboldened by Embodiment: Six Precepts for Research on Embodied Learning and Mixed Reality," Educ. Res., vol. 42, no. 8, pp. 445--452, Nov. 2013.

[15]

University of Granada, "Playability: What a video game needs to be successful," Science Daily, 10-Mar-2011 [Online]. Available: www.sciencedaily.com/releases/2011/03/110310082132.htm. [28-May-2019]

[16]

K. Isbister, "How Games Move Us: Emotion by Design | Books Gateway | MIT Press", 2016. [Online]. Available: https://direct.mit.edu/books/book/3115/how-games-move-usemotion-by-design. [Accessed: 30-May-2019]

[17]

M. P. J. Habgood, S. Ainsworth, S. Benford, "Endogenous fantasy and learning in digital games," Simul. Gaming, vol. 36, no. 4, pp. 483--498, Dec. 2005.

[18]

S. Patton, "The Definitive Guide to Playtest Questions," Schell Games, 28-Sep-2017. [Online]. Available: https://www.schellgames.com/blog/the-definitive-guide-to-playtest-questions. [07-Jan-2019]

[19]

T. Schubert, F. Friedmann, and H. Regenbrecht, "The Experience of Presence: Factor Analytic Insights," Presence: Teleoperators and Virtual Environments, vol. 10, no. 3, pp. 266--281, Jun. 2001.

[20]

J. E. Aspell, B. Lenggenhager, and O. Blanke, "Keeping in touch with one's self: multisensory mechanisms of self-consciousness," PLoS One, vol. 4, no. 8, p. e6488, Aug. 2009.

[21]

A. Kalckert and H. H. Ehrsson, "Moving a Rubber Hand that Feels Like Your Own: A Dissociation of Ownership and Agency," Front. Hum. Neurosci., vol. 6, p. 40, Mar. 2012.

[22]

K. Kilteni, R. Groten, and M. Slater, "The Sense of Embodiment in Virtual Reality," Presence: Teleoperators and Virtual Environments, vol. 21, no. 4, pp. 373--387, Nov. 2012.

Digital Library

[23]

A. Tristán-López, "Modification to the Lawshe's model to quantify the content validity of an objective instrument" (In Spanish: Modificación al modelo de Lawshe para el dictamen cuantitativo de la validez de contenido de un instrumento objetivo), Avances en medición, vol. 6, no. 1, pp. 37--48, 2008.

[24]

C. H. Lawshe, "A quantitative approach to content validity," Personnel Psychology, no. 28, pp. 563--575, Dec. 1975.

Cited By

Bareišytė LSlatman SAustin JRosema Mvan Sintemaartensdijk IWatson SBode C(2024)Questionnaires for Evaluating Virtual Reality: a Systematic Scoping ReviewComputers in Human Behavior Reports10.1016/j.chbr.2024.100505(100505)Online publication date: Oct-2024
https://doi.org/10.1016/j.chbr.2024.100505
Owoseni AKolade OEgbetokun AOwoseni AKolade OEgbetokun A(2024)Generative AI and Its Implications for Higher Education Students and EducatorsGenerative AI in Higher Education10.1007/978-3-031-60179-8_1(1-25)Online publication date: 19-Sep-2024
https://doi.org/10.1007/978-3-031-60179-8_1
Irshad SPerkis AMirza-Babaei PMcArthur VVanden Abeele VBirk M(2020)Serious Storytelling in Virtual RealityExtended Abstracts of the 2020 Annual Symposium on Computer-Human Interaction in Play10.1145/3383668.3419908(267-271)Online publication date: 2-Nov-2020
https://dl.acm.org/doi/10.1145/3383668.3419908

Index Terms

Towards a formative instrument to evaluate user experience in virtual reality serious games
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. HCI design and evaluation methods

Recommendations

Exploring User Experience While Playing Educational Games: Focus on Temporality and Attractiveness

The concept and methods of user experience UX are gaining momentum in the game industry. Designers and educational practitioners aim to provide rich and effective user experience through serious educational games. Nevertheless several phenomena that ...
Serious Economic Games: Designing a Simulation Game for an Economic Experiment
Proceedings of the Third International Conference on Design, User Experience, and Usability. User Experience Design for Diverse Interaction Platforms and Environments - Volume 8518

This project proposes the design of a Serious Game for an economic experiment. The analysis of existing games and research into current practices and requirements in economic experimentation are combined to create a Serious Economic Game. It is found ...
A Classification of Fantasy in Serious Games
CHI PLAY '19 Extended Abstracts: Extended Abstracts of the Annual Symposium on Computer-Human Interaction in Play Companion Extended Abstracts

Through a literature review, we explore fantasy in serious games, the function of fantasy, and what contributes to fantasy. This research firstly regards fantasy as a game characteristic, focusing on the definition of fantasy in games. We find two ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

REHAB '19: Proceedings of the 5th Workshop on ICTs for improving Patients Rehabilitation Research Techniques

September 2019

196 pages

ISBN:9781450371513

DOI:10.1145/3364138

Program Chairs:
Habib M. Fardoun
King Abdulaziz University (KAU), Saudi Arabia
,
Sergio Albiol Pérez
University of Zaragoza, Spain
,
Ahlam A. M. Hassan
Ahlia University, Bahrain
,
Elena de la Guia
University of Castilla-La Mancha, Spain

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 29 January 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Funding Sources

Fondo Nacional de Desarrollo Científico y Tecnológico

Conference

REHAB 2019

REHAB 2019: 5th Workshop on ICTs for improving Patients Rehabilitation Research Techniques

September 11 - 13, 2019

Popayan, Columbia

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
188
Total Downloads

Downloads (Last 12 months)21
Downloads (Last 6 weeks)1

Reflects downloads up to 16 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Bareišytė LSlatman SAustin JRosema Mvan Sintemaartensdijk IWatson SBode C(2024)Questionnaires for Evaluating Virtual Reality: a Systematic Scoping ReviewComputers in Human Behavior Reports10.1016/j.chbr.2024.100505(100505)Online publication date: Oct-2024
https://doi.org/10.1016/j.chbr.2024.100505
Owoseni AKolade OEgbetokun AOwoseni AKolade OEgbetokun A(2024)Generative AI and Its Implications for Higher Education Students and EducatorsGenerative AI in Higher Education10.1007/978-3-031-60179-8_1(1-25)Online publication date: 19-Sep-2024
https://doi.org/10.1007/978-3-031-60179-8_1
Irshad SPerkis AMirza-Babaei PMcArthur VVanden Abeele VBirk M(2020)Serious Storytelling in Virtual RealityExtended Abstracts of the 2020 Annual Symposium on Computer-Human Interaction in Play10.1145/3383668.3419908(267-271)Online publication date: 2-Nov-2020
https://dl.acm.org/doi/10.1145/3383668.3419908

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents