Does the Medium Matter? A Comparison of Augmented Reality Media in Instructing Novices to Perform Complex, Skill-Based Manual Tasks.
Article No.: 249, Pages 1 - 28
Abstract
Past research comparing augmented reality (AR) media such as in-situ projection and head-mounted devices (HMD) has usually considered simple manual activities. It is unknown whether previously reported differences between different AR media also apply to complex, skill-driven tasks. In this paper, we explore the feasibility and challenges in designing AR instructions for expertise-driven, skilled activities. We present findings from a real-world, between-subjects experiment in which novices were instructed to trim and bone sub-primal cuts of pork using two interactive AR prototypes, one utilizing in-situ projection and a second using the Hololens 2. The prototypes and instructions were designed in consultation with experts. We compared novices' task performance and subjective perceptions and gathered experts' feedback. Although both users and experts indicated a subjective preference for in-situ projection, results indicate that when tasks require knowledge, skill and expertise, the choice of the AR medium itself may not be consequential. Rather, in our experiment, the instruction quality influenced comprehension, knowledge retention and task performance. Hence, from an engineering perspective, emphasis ought to be laid on gathering and structuring expert performance and knowledge to create effective instructions, which could be delivered using any AR medium suited to the task and work environment.
Supplemental Material
PDF File
Supplemental file
- Download
- 6.72 MB
References
[1]
2020. Financial needs in the agriculture and agri-food sectors in Germany. Technical Report. European Commission. 80 pages. https://www.fi-compass.eu/sites/default/files/publications/financial_needs_agriculture_agrifood_sectors_Germany.pdf
[2]
2022. Guidelines for meat and meat-based products. Technical Report. German Federal Ministry of Food and Agriculture. 84 pages. https://www.bmel.de/SharedDocs/Downloads/DE/_Ernaehrung/Lebensmittel-Kennzeichnung/LeitsaetzeFleisch.pdf?___blob=publicationFile&v=14
[3]
2022. Labour shortages in the food and farming sector. HC 713. House of Commons Environment, Food and Rural Affairs Committee.
[4]
2022. Vocational training report 2022. Technical Report. German Federal Employment Agency. 129 pages. https://www.bmbf.de/SharedDocs/Downloads/de/2022/berufsbildungsbericht-2022.pdf?___blob=publicationFile&v=1
[5]
James P. Baker and Alan K. Goodboy. 2019. The choice is yours: the effects of autonomy-supportive instruction on students' learning and communication. Communication Education 68, 1 (Jan. 2019), 80--102. https://doi.org/10.1080/03634523.2018.1536793
[6]
James Baumeister, Seung Youb Ssin, Neven A. M. ElSayed, Jillian Dorrian, David P. Webb, James A. Walsh, Timothy M. Simon, Andrew Irlitti, Ross T. Smith, Mark Kohler, and Bruce H. Thomas. 2017. Cognitive Cost of Using Augmented Reality Displays. IEEE Transactions on Visualization and Computer Graphics 23, 11 (Nov. 2017), 2378--2388. https://doi.org/10.1109/TVCG.2017.2735098
[7]
Jonas Blattgerste, Patrick Renner, Benjamin Strenge, and Thies Pfeiffer. 2018. In-Situ Instructions Exceed Side-by-Side Instructions in Augmented Reality Assisted Assembly. In Proceedings of the 11th PErvasive Technologies Related to Assistive Environments Conference. ACM, Corfu Greece, 133--140. https://doi.org/10.1145/3197768.3197778
[8]
Alberto Brandolini. 2017. Introducing Event Storming: An Act of Deliberate Collective Learning. LeanPub.
[9]
Giorgio Brunello and Patricia Wruuck. 2019. Skill Shortages and Skill Mismatch in Europe: A Review of the Literature. SSRN Electronic Journal (2019). https://doi.org/10.2139/ssrn.3390340
[10]
Josef Buchner, Katja Buntins, and Michael Kerres. 2022. The impact of augmented reality on cognitive load and performance: A systematic review. Journal of Computer Assisted Learning 38, 1 (Feb. 2022), 285--303. https://doi.org/10.1111/jcal.12617
[11]
Isidro M. Butaslac, Yuichiro Fujimoto, Taishi Sawabe, Masayuki Kanbara, and Hirokazu Kato. 2022. Systematic Review of Augmented Reality Training Systems. IEEE Transactions on Visualization and Computer Graphics (2022), 1--20. https://doi.org/10.1109/TVCG.2022.3201120
[12]
Kerem A. Bölek, Guido De Jong, and Dylan Henssen. 2021. The effectiveness of the use of augmented reality in anatomy education: a systematic review and meta-analysis. Scientific Reports 11, 1 (July 2021), 15292. https://doi.org/10.1038/s41598-021-94721-4
[13]
Lars Bager Christensen and Morten Pol Engell-N⊘rregård. 2016. Augmented reality in the slaughterhouse - a future operation facility? Cogent Food & Agriculture 2, 1 (June 2016). https://doi.org/10.1080/23311932.2016.1188678
[14]
Richard E Clark. 2012. Learning From Media 2nd Ed.: Arguments, Analysis, and Evidence. IAP.
[15]
European Commission. 2023. Agri-food Industrial ecosystem. https://single-market-economy.ec.europa.eu/sectors/agri-food-industrial-ecosystem_en
[16]
Marie-Paule Daniel and Barbara Tversky. 2012. How to put things together. Cognitive Processing 13, 4 (Nov. 2012), 303--319. https://doi.org/10.1007/s10339-012-0521-5
[17]
Patrick G. Dempsey and Raymond W. McGorry. 2004. Investigation of a Pork Shoulder Deboning Operation. Journal of Occupational and Environmental Hygiene 1, 3 (March 2004), 167--172. https://doi.org/10.1080/15459620490424465
[18]
Hitesh Dhiman, Didarul Alam, Yu Qiao, Matthias Upmann, and Carsten Röcker. 2022. Learn from the Best: Harnessing Expert Skill and Knowledge to Teach Unskilled Workers. In The15th International Conference on PErvasive Technologies Related to Assistive Environments. ACM, Corfu Greece, 93--102. https://doi.org/10.1145/3529190.3529203
[19]
Hitesh Dhiman, Michael Fellmann, and Carsten Röcker. 2022. Towards AI-Enabled Assistant Design Through Grassroots Modeling: Insights from a Practical Use Case in the Industrial Sector. In Perspectives in Business Informatics Research, Ērika Nazaruka, Kurt Sandkuhl, and Ulf Seigerroth (Eds.). Vol. 462. Springer International Publishing, Cham, 96--110. https://doi.org/10.1007/978-3-031-16947-2_7 Series Title: Lecture Notes in Business Information Processing.
[20]
European Foundation for the Improvement of Living and Working Conditions. 2018. New tasks in old jobs: drivers of change and implications for job quality. Publications Office, LU. https://data.europa.eu/doi/10.2806/507642
[21]
David F. Feldon, Soojeong Jeong, and Richard E. Clark. 2021. Fifteen Common but Questionable Principles of Multimedia Learning (3 ed.). Cambridge University Press, 25--40. https://doi.org/10.1017/9781108894333.005
[22]
Benish Fida, Fabrizio Cutolo, Gregorio di Franco, Mauro Ferrari, and Vincenzo Ferrari. 2018. Augmented reality in open surgery. Updates in Surgery 70, 3 (Sept. 2018), 389--400. https://doi.org/10.1007/s13304-018-0567-8
[23]
Markus Funk, Thomas Kosch, and Albrecht Schmidt. 2016. Interactive worker assistance: comparing the effects of in-situ projection, head-mounted displays, tablet, and paper instructions. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM, Heidelberg Germany, 934--939. https://doi.org/10.1145/2971648.2971706
[24]
Julie Gess-Newsome, Joseph A. Taylor, Janet Carlson, April L. Gardner, Christopher D. Wilson, and Molly A. M. Stuhlsatz. 2019. Teacher pedagogical content knowledge, practice, and student achievement. International Journal of Science Education 41, 7 (May 2019), 944--963. https://doi.org/10.1080/09500693.2016.1265158
[25]
Sandra G. Hart and Lowell E. Staveland. 1988. Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research. In Advances in Psychology. Vol. 52. Elsevier, 139--183. https://doi.org/10.1016/S0166-4115(08)62386-9
[26]
Pamela J. Hinds, Michael Patterson, and Jeffrey Pfeffer. 2001. Bothered by abstraction: The effect of expertise on knowledge transfer and subsequent novice performance. Journal of Applied Psychology 86, 6 (2001), 1232--1243. https://doi.org/10.1037/0021-9010.86.6.1232
[27]
Jason Hochreiter, Salam Daher, Gerd Bruder, and Greg Welch. 2018. Cognitive and Touch Performance Effects of Mismatched 3D Physical and Visual Perceptions. In 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). IEEE, Reutlingen, 1--386. https://doi.org/10.1109/VR.2018.8446574
[28]
Robert R. Hoffman, Paul Ward, Paul J. Feltovich, Lia DiBello, Stephen M. Fiore, and Dee H. Andrews. 2013. Accelerated Expertise: Training for High Proficiency in a Complex World (0 ed.). Psychology Press. https://doi.org/10.4324/9780203797327
[29]
Victor Kaptelinin and Bonnie Nardi. 2018. Activity Theory as a Framework for Human-Technology Interaction Research. Mind, Culture, and Activity 25, 1 (Jan. 2018), 3--5. https://doi.org/10.1080/10749039.2017.1393089
[30]
Robin Kay. 2011. Evaluating learning, design, and engagement in web-based learning tools (WBLTs): The WBLT Evaluation Scale. Computers in Human Behavior 27, 5 (Sept. 2011), 1849--1856. https://doi.org/10.1016/j.chb.2011.04.007
[31]
Bui Minh Khuong, Kiyoshi Kiyokawa, Andrew Miller, Joseph J. La Viola, Tomohiro Mashita, and Haruo Takemura. 2014. The effectiveness of an AR-based context-aware assembly support system in object assembly. In 2014 IEEE Virtual Reality (VR). IEEE, Minneapolis, MN, USA, 57--62. https://doi.org/10.1109/VR.2014.6802051
[32]
Christa Krijgsman, Tim Mainhard, Lars Borghouts, Jan van Tartwijk, and Leen Haerens. 2021. Do goal clarification and process feedback positively affect students' need-based experiences? A quasi-experimental study grounded in Self-Determination Theory. Physical Education and Sport Pedagogy 26, 5 (Sept. 2021), 483--503. https://doi.org/10.1080/17408989.2020.1823956
[33]
Sophie Le Bellu, Saadi Lahlou, Valery Nosulenko, and Elena Samoylenko. 2016. Studying activity in manual work: a framework for analysis and training. Le travail humain 79, 1 (2016), 7. https://doi.org/10.3917/th.791.0007
[34]
Bibeg Hang Limbu, Halszka Jarodzka, Roland Klemke, and Marcus Specht. 2018. Using sensors and augmented reality to train apprentices using recorded expert performance: A systematic literature review. Educational Research Review 25 (Nov. 2018), 1--22. https://doi.org/10.1016/j.edurev.2018.07.001
[35]
Renae Low. 2009. Motivation and Multimedia Learning:. In Cognitive Effects of Multimedia Learning, Robert Z. Zheng (Ed.). IGI Global, 154--172. https://doi.org/10.4018/978-1-60566-158-2.ch009
[36]
H. B. Mann and D. R. Whitney. 1947. On a Test of Whether one of Two Random Variables is Stochastically Larger than the Other. The Annals of Mathematical Statistics 18, 1 (March 1947), 50--60. https://doi.org/10.1214/aoms/1177730491
[37]
Michael R. Marner, Andrew Irlitti, and Bruce H. Thomas. 2013. Improving procedural task performance with Augmented Reality annotations. In 2013 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). IEEE, Adelaide, Australia, 39--48. https://doi.org/10.1109/ISMAR.2013.6671762
[38]
Richard E. Mayer. 2021. Evidence-based principles for how to design effective instructional videos. Journal of Applied Research in Memory and Cognition 10, 2 (June 2021), 229--240. https://doi.org/10.1016/j.jarmac.2021.03.007
[39]
Edward McAuley, Terry Duncan, and Vance V. Tammen. 1989. Psychometric Properties of the Intrinsic Motivation Inventory in a Competitive Sport Setting: A Confirmatory Factor Analysis. Research Quarterly for Exercise and Sport 60, 1 (March 1989), 48--58. https://doi.org/10.1080/02701367.1989.10607413
[40]
Norman B. Mendoza, Zi Yan, and Ronnel B. King. 2023. Supporting students' intrinsic motivation for online learning tasks: The effect of need-supportive task instructions on motivation, self-assessment, and task performance. Computers & Education 193 (Feb. 2023), 104663. https://doi.org/10.1016/j.compedu.2022.104663
[41]
Gerhard Messmann and Regina H. Mulder. 2015. Conditions for apprentices' learning activities at work. Journal of Vocational Education & Training 67, 4 (Oct. 2015), 578--596. https://doi.org/10.1080/13636820.2015.1094745
[42]
Athanasios Mouratidis, Willy Lens, and Maarten Vansteenkiste. 2010. How You Provide Corrective Feedback Makes a Difference: The Motivating Role of Communicating in an Autonomy-Supporting Way. Journal of Sport and Exercise Psychology 32, 5 (Oct. 2010), 619--637. https://doi.org/10.1123/jsep.32.5.619
[43]
Lucas-Raphael Müller, Jens Petersen, Amine Yamlahi, Philipp Wise, Tim J. Adler, Alexander Seitel, Karl-Friedrich Kowalewski, Beat Müller, Hannes Kenngott, Felix Nickel, and Lena Maier-Hein. 2022. Robust hand tracking for surgical telestration. International Journal of Computer Assisted Radiology and Surgery 17, 8 (Aug. 2022), 1477--1486. https://doi.org/10.1007/s11548-022-02637-9
[44]
Hong Kok Ng, Slava Kalyuga, and John Sweller. 2013. Reducing transience during animation: a cognitive load perspective. Educational Psychology 33, 7 (Dec. 2013), 755--772. https://doi.org/10.1080/01443410.2013.785050
[45]
Sylvie Ouellet. 2012. Ergonomic analysis of work activity for the purpose of developing training programs: the contribution of ergonomics to vocational didactics. Work 41 (2012), 4572--4576. https://doi.org/10.3233/WOR-2012-0753-4572
[46]
Jocelyn Parong. 2021. Multimedia Learning in Virtual and Mixed Reality (3 ed.). Cambridge University Press, 498--509. https://doi.org/10.1017/9781108894333.051
[47]
Joseph Redmon and Ali Farhadi. 2018. YOLOv3: An Incremental Improvement. arXiv:1804.02767 [cs] (April 2018). http://arxiv.org/abs/1804.02767 arXiv: 1804.02767.
[48]
Dmytro Romanov, Olga Korostynska, Odd Ivar Lekang, and Alex Mason. 2022. Towards human-robot collaboration in meat processing: Challenges and possibilities. Journal of Food Engineering 331 (Oct. 2022), 111117. https://doi.org/10.1016/j.jfoodeng.2022.111117
[49]
Katharina Scheiter and Peter Gerjets. 2007. Learner Control in Hypermedia Environments. Educational Psychology Review 19, 3 (Sept. 2007), 285--307. https://doi.org/10.1007/s10648-007-9046-3
[50]
Martin Schrepp, Andreas Hinderks, and Jörg Thomaschewski. 2014. Applying the User Experience Questionnaire (UEQ) in Different Evaluation Scenarios. In Design, User Experience, and Usability. Theories, Methods, and Tools for Designing the User Experience, David Hutchison, Takeo Kanade, Josef Kittler, Jon M. Kleinberg, Alfred Kobsa, Friedemann Mattern, John C. Mitchell, Moni Naor, Oscar Nierstrasz, C. Pandu Rangan, Bernhard Steffen, Demetri Terzopoulos, Doug Tygar, Gerhard Weikum, and Aaron Marcus (Eds.). Vol. 8517. Springer International Publishing, Cham, 383--392. https://doi.org/10.1007/978-3-319-07668-3_37 Series Title: Lecture Notes in Computer Science.
[51]
Rolf Schwonke, Kirsten Berthold, and Alexander Renkl. 2009. How multiple external representations are used and how they can be made more useful. Applied Cognitive Psychology 23, 9 (Dec. 2009), 1227--1243. https://doi.org/10.1002/acp.1526
[52]
S. S. Shapiro and M. B. Wilk. 1965. An analysis of variance test for normality (complete samples). Biometrika 52, 3-4 (Dec. 1965), 591--611. https://doi.org/10.1093/biomet/52.3-4.591
[53]
James H. Stronge. 2007. Qualities of effective teachers (2nd ed ed.). Association for Supervision and Curriculum Development, Alexandria, Va.
[54]
Keishi Tainaka, Yuichiro Fujimoto, Taishi Sawabe, Masayuki Kanbara, and Hirokazu Kato. 2023. Selection framework of visualization methods in designing AR industrial task-support systems. Computers in Industry 145 (Feb. 2023), 103828. https://doi.org/10.1016/j.compind.2022.103828
[55]
Arthur Tang, Charles Owen, Frank Biocca, and Weimin Mou. 2003. Comparative effectiveness of augmented reality in object assembly. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, Ft. Lauderdale Florida USA, 73--80. https://doi.org/10.1145/642611.642626
[56]
Tamara van Gog, Fred Paas, Nadine Marcus, Paul Ayres, and John Sweller. 2009. The Mirror Neuron System and Observational Learning: Implications for the Effectiveness of Dynamic Visualizations. Educational Psychology Review 21, 1 (March 2009), 21--30. https://doi.org/10.1007/s10648-008-9094-3
[57]
Gabriele Wulf, Markus Raupach, and Felix Pfeiffer. 2005. Self-Controlled Observational Practice Enhances Learning. Research Quarterly for Exercise and Sport 76, 1 (March 2005), 107--111. https://doi.org/10.1080/02701367.2005.10599266
[58]
Jeffrey M. Zacks and Barbara Tversky. 2003. Structuring information interfaces for procedural learning. Journal of Experimental Psychology: Applied 9, 2 (2003), 88--100. https://doi.org/10.1037/1076-898X.9.2.88
[59]
Jeffrey M. Zacks, Barbara Tversky, and Gowri Iyer. 2001. Perceiving, remembering, and communicating structure in events. Journal of Experimental Psychology: General 130, 1 (2001), 29--58. https://doi.org/10.1037/0096-3445.130.1.29
[60]
Fan Zhang, Valentin Bazarevsky, Andrey Vakunov, Andrei Tkachenka, George Sung, Chuo-Ling Chang, and Matthias Grundmann. 2020. MediaPipe Hands: On-device Real-time Hand Tracking. arXiv:2006.10214 [cs] (June 2020). http://arxiv.org/abs/2006.10214 arXiv: 2006.10214.
Index Terms
- Does the Medium Matter? A Comparison of Augmented Reality Media in Instructing Novices to Perform Complex, Skill-Based Manual Tasks.
Recommendations
Multimodal augmented reality: the norm rather than the exception
MVAR '16: Proceedings of the 2016 workshop on Multimodal Virtual and Augmented RealityAugmented reality (AR) is commonly seen as a technology that overlays virtual imagery onto a participant's view of the world. In line with this, most AR research is focused on what we see. In this paper, we challenge this focus on vision and make a case ...
Virtual reality and augmented reality in social learning spaces: a literature review
AbstractIn this survey, we explore Virtual Reality and Augmented Reality within social learning spaces, such as classrooms and museums, while also extending into relevant social interaction concepts found within more reality-based and social immersive ...
The perceptions of CEIT postgraduate students regarding reality concepts: Augmented, virtual, mixed and mirror reality
The purpose of this study is to determine perception of postgraduate Computer Education and Instructional Technologies (CEIT) students regarding the concepts of Augmented Reality (AR), Virtual Reality (VR), Mixed Reality (MR), Augmented Virtuality (AV) ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
Copyright © 2024 Owner/Author.
This work is licensed under a Creative Commons Attribution International 4.0 License.
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 17 June 2024
Accepted: 01 February 2024
Revised: 01 February 2024
Received: 01 October 2023
Published in PACMHCI Volume 8, Issue EICS
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 217Total Downloads
- Downloads (Last 12 months)217
- Downloads (Last 6 weeks)65
Reflects downloads up to 20 Nov 2024
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in