Training transfer validity of virtual reality simulator assessment
Abstract
This study utilises computer-based simulations to explore the transfer effects of competency training in maritime education, addressing the current lack of research on their transferability to real-world scenarios.
The research explores the accuracy of procedural knowledge assessment using virtual reality (VR), positing that head-mounted display (HMD) VR offers stronger concurrent validity through training transfer measures than 3D desktop VR. This is evaluated by regression on a training transfer condition. It also investigates motivation’s influence on training transfer and the regression model of this relationship.
Fifteen marine engineering students were divided into two experimental groups using 3D desktop VR and HMD VR systems, with eight experts in the control group. The students had previously received traditional lecture-based instruction and were given practical training using a 2D desktop simulator in the same scenario as in the VR treatment and in the training transfer condition.
The ANCOVA design experiment involved two levels of technical immersion before the operation of real-life equipment. Neither technical immersion nor expertise level as independent variables were found to have a significant effect in the relationship of the assessment predicting the training transfer. The direct relationship was significant (R2adj = 0.436) and further analysed with the influence of motivation, resulting in a moderation model with a decent effect size (R2 = 0.740). Based on these findings, we can infer that both types of VR simulations used for assessment demonstrate concurrent validity in predicting real-life performance before we discuss and define the characteristics of the observed transfer according to theory.
References
[1]
Aguinis H and Kraiger K Benefits of training and development for individuals and teams, organizations, and society [Review] Annu Rev Psychol 2009 60 451-474
[2]
Alvarez K, Salas E, and Garofano CM An integrated model of training evaluation and effectiveness [Article] Hum Resour Dev Rev 2016 3 4 385-416
[3]
Baldwin TT and Ford JK Transfer of training: a review and directions for future research Pers Psychol 1988 41 1 63-105
[4]
Baldwin TT, Kevin Ford J, and Blume BD The state of transfer of training research: moving toward more consumer-centric inquiry Hum Resour Dev Q 2017 28 1 17-28
[5]
Barnett SM and Ceci SJ When and where do we apply what we learn? A taxonomy for far transfer Psychol Bull 2002 128 4 612-637
[6]
Bell BS, Tannenbaum SI, Kevin Ford J, Noe RA, and Kraiger K 100 years of training and development research: what we know and where we should go J Appl Psychol 2017 102 3 305-323
[7]
Bloom BS (1956) Taxonomy of educational objectives: the classification of educational goals: book 1 cognitive domain (Vol. 16). Longman
[8]
Blume BD, Ford JK, Baldwin TT, and Huang JL Transfer of training: a meta-analytic review J Manag 2010 36 4 1065-1105
[9]
Colquitt JA, LePine JA, and Noe RA Toward an integrative theory of training motivation: a meta-analytic path analysis of 20 years of research J Appl Psychol 2000 85 5 678-707
[10]
Cummings JJ and Bailenson JN How immersive is enough? A meta-analysis of the effect of immersive technology on user presence Media Psychol 2015 19 2 272-309
[11]
Doozandeh P and Hedayati S The effect of simulation fidelity on transfer of training for troubleshooting professionals: a meta-analysis IISE Trans Occup Ergon Human Factors 2022 10 3 117-134
[12]
Duncan T, Pintrich P, Smith D, and Mckeachie W Motivated strategies for learning questionnaire (MSLQ) manual Nat Center Res Improve Postsecond Teach Learn 2015
[13]
Dweck CS and Leggett EL A social-cognitive approach to motivation and personality Psychol Rev 1988 95 2 256-273
[14]
Ford JK, Baldwin TT, and Prasad J Transfer of training: the known and the unknown [Review] Annu Rev Organ Psych Organ Behav 2018 5 1 201-225
[15]
Ford JK, Smith EM, Weissbein DA, Gully SM, and Salas E Relationships of goal orientation, metacognitive activity, and practice strategies with learning outcomes and transfer J Appl Psychol 1998 83 2 218-233
[16]
Gagné RM (1965) The conditions of learning. Holt, Rinehart and Winston. https://books.google.no/books?id=4lpIAAAAMAAJ
[17]
Grossman R and Salas E The transfer of training: what really matters [Article] Int J Train Dev 2011 15 2 103-120
[18]
Hayes AF Introduction to mediation, moderation, and conditional process analysis: a regression-based approach 2022 3 New York Guilford Publications
[19]
Hjellvik S and Mallam S Integrating motivated goal achievement in maritime simulator training WMU J Marit Aff 2023
[20]
Holland DF, Kraha A, Zientek LR, Nimon K, Fulmore JA, Johnson UY, Ponce HF, Aguilar MG, and Henson RK Reliability generalization of the motivated strategies for learning questionnaire: a meta-analytic view of reliability estimates SAGE Open 2018 8 3 215824401880233
[21]
Jensen L and Konradsen F A review of the use of virtual reality head-mounted displays in education and training Off J IFIP Tech Comm Educ 2018 23 4 1515-1529
[22]
Kvinge LMR, Engelsen KS (2016) Omsetjing og tilpassing av MSLQ: IMTE-rapport nr. 1/16. http://hdl.handle.net/11250/2399353
[23]
Lajoie SP Mayer RE and Fiorella L Multimedia learning with simulations The cambridge handbook of multimedia learning 2021 Cambridge University Press 461-471
[24]
Makransky G, Borre-Gude S, and Mayer RE Motivational and cognitive benefits of training in immersive virtual reality based on multiple assessments J Comput Assist Learn 2019 35 6 691-707
[25]
Makransky G and Petersen GB The cognitive affective model of immersive learning (CAMIL): a theoretical research-based model of learning in immersive virtual reality Educ Psychol Rev 2021 33 3 937-958
[26]
Makransky G, Terkildsen TS, and Mayer RE Adding immersive virtual reality to a science lab simulation causes more presence but less learning Learn Instr 2019 60 225-236
[27]
Mayer RE, Makransky G, and Parong J The promise and pitfalls of learning in immersive virtual reality Int J Human-Comput Interact 2022 39 11 2229-2238
[28]
Moglia A, Ferrari V, Morelli L, Ferrari M, Mosca F, and Cuschieri A A systematic review of virtual reality simulators for robot-assisted surgery Eur Urol 2016 69 6 1065-1080
[29]
Mulders M Confounding in educational research: an overview of research approaches investigating virtual and augmented reality Digital Psychol 2023 4 1S 9-12
[30]
Mulders M Learning about victims of holocaust in virtual reality: the main, mediating and moderating effects of technology, instructional method, flow, presence, and prior knowledge Multimodal Technol Interact 2023
[31]
Parks L and Guay RP Personality, values, and motivation Personality Individ Differ 2009 47 7 675-684
[32]
Pintrich PR, Smith DAF, Garcia T, McKeachie W (1991) A manual for the use of the motivated strategies for learning questionnaire (MSLQ)
[33]
Radianti J, Majchrzak TA, Fromm J, and Wohlgenannt I A systematic review of immersive virtual reality applications for higher education: design elements, lessons learned, and research agenda Comput Educ 2020
[34]
Rauter G, Sigrist R, Koch C, Crivelli F, van Raai M, Riener R, and Wolf P Transfer of complex skill learning from virtual to real rowing PLoS ONE 2013 8 12
[35]
Schmidt MW, Köppinger KF, Fan C, Kowalewski KF, Schmidt LP, Vey J, Proctor T, Probst P, Bintintan VV, Müller-Stich BP, and Nickel F Virtual reality simulation in robot-assisted surgery: meta-analysis of skill transfer and predictability of skill BJS Open 2021
[36]
Schubert T, Friedmann F, and Regenbrecht H The experience of presence: factor analytic insights Presence Teleoperators Virtual Environ 2001 10 3 266-281
[37]
Suh A and Prophet J The state of immersive technology research: A literature analysis Comput Hum Behav 2018 86 77-90
[38]
Sullman MJM, Dorn L, and Niemi P Eco-driving training of professional bus drivers—Does it work? Transp Res Part C Emerg Technol 2015 58 749-759
[39]
Taylor GS and Barnett JS Evaluation of wearable simulation interface for military training [Article] Hum Factors 2013 55 3 672-690
[40]
van der Vleuten CP and Schuwirth LW Assessing professional competence: from methods to programmes Med Educ 2005 39 3 309-317
Index Terms
- Training transfer validity of virtual reality simulator assessment
Index terms have been assigned to the content through auto-classification.
Recommendations
Exploring Extended Reality as a Simulation Training Tool Through Naturalistic Interactions and Enhanced Immersion
Virtual, Augmented and Mixed Reality. Applications and Case StudiesAbstractAugmented reality (AR), mixed reality (MR), and virtual reality (VR) technologies – collectively extended reality (XR) – have been discussed in the context of training applications since their introduction to the market. While this discussion was ...
Haptics-based virtual reality periodontal training simulator
This paper focuses upon the research and development of a prototype dental simulator for training of periodontal procedures. By the use of virtual reality and haptics technology, the periodontal simulator allows trainees to learn performing diagnosis ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
© The Author(s) 2024.
Publisher
Springer-Verlag
Berlin, Heidelberg
Publication History
Published: 04 November 2024
Accepted: 07 October 2024
Received: 06 March 2024
Author Tags
Qualifiers
- Research-article
Funding Sources
- University Of South-Eastern Norway
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 26 Nov 2024
Other Metrics
Citations
View Options
View options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in