Using Online Videos as the Basis for Developing Design Guidelines: A Case Study of AR-Based Assembly Instructions
Authors: 
Niu Chen, Frances Jihae Sin, Laura Mariah Herman, 
Cuong Nguyen, Ivan Song, Dongwook YoonAuthors Info & Claims
Article No.: 428, Pages 41 - 63
Abstract
Design guidelines serve as an important conceptual tool to guide designers of interactive applications with well-established principles and heuristics. Consulting domain experts is a common way to develop guidelines. However, experts are often not easily accessible, and their time can be expensive. This problem poses challenges in developing comprehensive and practical guidelines. We propose a new guideline development method that uses online public videos as the basis for capturing diverse patterns of design goals and interaction primitives. In a case study focusing on AR-based assembly instructions, we apply our novel Identify-Rationalize pipeline, which distills design patterns from videos featuring AR-based assembly instructions (N=146) into a set of guidelines that cover a wide range of design considerations. The evaluation conducted with 16 AR designers indicated that the pipeline is useful for generating comprehensive guidelines. We conclude by discussing the transferability and practicality of our method.
Supplementary Material
Supplementary Materials 1. Video Data 2. Taxonomy 3. Rating Data 4. AR Guideline Gallery Web App
- Download
- 50.97 MB
Supplementary Materials 1. Video Data 2. Taxonomy 3. Rating Data 4. AR Guideline Gallery Web App
- Download
- 85.93 MB
References
[1]
Saleema Amershi, Dan Weld, Mihaela Vorvoreanu, Adam Fourney, Besmira Nushi, Penny Collisson, Jina Suh, Shamsi Iqbal, Paul N. Bennett, Kori Inkpen, Jaime Teevan, Ruth Kikin-Gil, and Eric Horvitz. 2019. Guidelines for Human-AI Interaction. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI ’19). ACM, New York, NY, USA. Article 3, 13 pages. isbn:978-1-4503-5970-2 https://doi.org/10.1145/3290605.3300233
[2]
Lisa Anthony, YooJin Kim, and Leah Findlater. 2013. Analyzing User-generated Youtube Videos to Understand Touchscreen Use by People with Motor Impairments. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’13). ACM, New York, NY, USA. 1223–1232. isbn:978-1-4503-1899-0 https://doi.org/10.1145/2470654.2466158
[3]
Apple. 2019. Apple Human Interface Guidelines. https://developer.apple.com/design/human-interface-guidelines/
[4]
Reynold Bailey, Ann McNamara, Aaron Costello, Srinivas Sridharan, and Cindy Grimm. 2012. Impact of Subtle Gaze Direction on Short-term Spatial Information Recall. In Proceedings of the Symposium on Eye Tracking Research and Applications (ETRA ’12). ACM, New York, NY, USA. 67–74. isbn:978-1-4503-1221-9 https://doi.org/10.1145/2168556.2168567
[5]
Eric P. S. Baumer, David Mimno, Shion Guha, Emily Quan, and Geri K. Gay. 2017. Comparing grounded theory and topic modeling: Extreme divergence or unlikely convergence? Journal of the Association for Information Science and Technology, 68, 6 (2017), 1397–1410. https://doi.org/10.1002/asi.23786
[6]
Mark Billinghurst, Adrian Clark, and Gun Lee. 2015. A Survey of Augmented Reality. Foundations and Trends in Human-Computer Interaction, 8, 2-3 (2015), 73–272. issn:1551-3955 https://doi.org/10.1561/1100000049
[7]
Mark Blythe and Paul Cairns. 2009. Critical Methods and User Generated Content: The iPhone on YouTube. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’09). ACM, New York, NY, USA. 1467–1476. isbn:978-1-60558-246-7 https://doi.org/10.1145/1518701.1518923
[8]
T.P. Caudell and D.W. Mizell. 1992. Augmented reality: an application of heads-up display technology to manual manufacturing processes. In Proceedings of the Twenty-Fifth Hawaii International Conference on System Sciences. ii, IEEE Computer Society, 659–669 vol.2. https://doi.org/10.1109/HICSS.1992.183317
[9]
Nikolas Chaconas and Tobias Hollerer. 2018. An Evaluation of Bimanual Gestures on the Microsoft HoloLens. In 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). 1–8. https://doi.org/10.1109/VR.2018.8446320
[10]
Dempsey Chang and Keith V. Nesbitt. 2006. Developing Gestalt-based Design Guidelines for Multi-sensory Displays. In Proceedings of the 2005 NICTA-HCSNet Multimodal User Interaction Workshop - Volume 57 (MMUI ’05). Australian Computer Society, Inc., Darlinghurst, Australia, Australia. 9–16. isbn:1-920-68239-2 https://doi.org/10.5555/1151804.1151807
[11]
Viviana Chimienti, Salvatore Iliano, Michele Dassisti, Gino Dini, and Franco Failli. 2010. Guidelines for implementing augmented reality procedures in assisting assembly operations. In International Precision Assembly Seminar. 174–179. https://doi.org/10.1007/978-3-642-11598-1_20
[12]
Marie-Paule Daniel and Barbara Tversky. 2012. How to put things together. Cognitive Processing, 13, 4 (2012), Nov., 303–319. issn:1612-4790 https://doi.org/10.1007/s10339-012-0521-5
[13]
Abhiraj Deshpande and Inki Kim. 2018. The effects of augmented reality on improving spatial problem solving for object assembly. Advanced Engineering Informatics, 38 (2018), 760–775. issn:1474-0346 https://doi.org/10.1016/j.aei.2018.10.004
[14]
Andreas Dünser, Raphaël Grasset, Hartmut Seichter, and Mark Billinghurst. 2007. Applying HCI principles to AR systems design. http://hdl.handle.net/10092/2340
[15]
Stephen Elliott, Alexander Kurz, Peter Beddow, and Jennifer Frey. 2009. Cognitive Load Theory: Instruction-based Research with Applications for Designing Tests.
[16]
P. Fite-Georgel. 2011. Is there a reality in Industrial Augmented Reality? In 2011 10th IEEE International Symposium on Mixed and Augmented Reality. 201–210. https://doi.org/10.1109/ISMAR.2011.6092387
[17]
Nicholas R. J. Frick, Felix Brünker, Björn Ross, and Stefan Stieglitz. 2019. Towards Successful Collaboration: Design Guidelines for AI-based Services enriching Information Systems in Organisations. CoRR, abs/1912.01077 (2019), arxiv:1912.01077. arxiv:1912.01077
[18]
Joseph L Gabbard and D Hix. 2001. Researching usability design and evaluation guidelines for augmented reality (AR) systems. Laboratory for Scientific Visual Analysis, Virginia Tech, USA, https://www.rkriz.net/sv/classes/ESM4714/Student_Proj/class00/gabbard/index.html
[19]
Joseph L. Gabbard, J. Edward Swan, and Deborah Hix. 2006. The Effects of Text Drawing Styles, Background Textures, and Natural Lighting on Text Legibility in Outdoor Augmented Reality. Presence: Teleoperators and Virtual Environments, 15, 1 (2006), 16–32. https://doi.org/10.1162/pres.2006.15.1.16 arxiv:https://doi.org/10.1162/pres.2006.15.1.16.
[20]
Nirit Gavish, Teresa Gutierrez, Sabine Webel, Jorge Rodriguez, and Franco Tecchia. 2011. Design guidelines for the development of virtual reality and augmented reality training systems for maintenance and assembly tasks. In BIO web of conferences. 1, 00029. https://doi.org/10.1051/bioconf/20110100029
[21]
Fabio Giglietto, Luca Rossi, and Davide Bennato. 2012. The open laboratory: Limits and possibilities of using Facebook, Twitter, and YouTube as a research data source. Journal of technology in human services, 30, 3-4 (2012), 145–159. https://doi.org/10.1080/15228835.2012.743797
[22]
Google. 2019. Augmented Reality Design Guidelines. https://designguidelines.withgoogle.com/ar-design/augmented-reality-design-guidelines/
[23]
Elizabeth R Grant and Michael J Spivey. 2003. Eye movements and problem solving: Guiding attention guides thought. Psychological Science, 14, 5 (2003), 462–466. https://doi.org/10.1111/1467-9280.02454
[24]
Martin Groen and Jan Noyes. 2010. Solving problems: How can guidance concerning task-relevancy be provided? Computers in Human Behavior, 26, 6 (2010), 1318 – 1326. issn:0747-5632 https://doi.org/10.1016/j.chb.2010.04.004 Online Interactivity: Role of Technology in Behavior Change
[25]
Gregory M. Hallihan and L.H. Shu. 2013. Considering Confirmation Bias in Design and Design Research. Journal of Integrated Design and Process Science, 17, 4 (2013), 19–35. https://doi.org/10.3233/jid-2013-0019 Publisher: IOS Press
[26]
Mary Hegarty. 1992. Mental animation: Inferring motion from static displays of mechanical systems. Journal of Experimental Psychology: Learning, Memory, and Cognition, 18, 5 (1992), 1084. https://doi.org/10.1037/0278-7393.18.5.1084
[27]
Julie Heiser, Doantam Phan, Maneesh Agrawala, Barbara Tversky, and Pat Hanrahan. 2004. Identification and Validation of Cognitive Design Principles for Automated Generation of Assembly Instructions. In Proceedings of the Working Conference on Advanced Visual Interfaces (AVI ’04). ACM, New York, NY, USA. 311–319. isbn:1-58113-867-9 https://doi.org/10.1145/989863.989917
[28]
S. Henderson and S. Feiner. 2011. Exploring the Benefits of Augmented Reality Documentation for Maintenance and Repair. IEEE Transactions on Visualization and Computer Graphics, 17, 10, 1355–1368. https://doi.org/10.1109/TVCG.2010.245
[29]
CN2 Inc. 2015. Printer Repaired with Augmented Reality. https://youtu.be/btW-MLz74ks
[30]
Eric Jamet, Monica Gavota, and Christophe Quaireau. 2008. Attention guiding in multimedia learning. Learning and Instruction, 18, 2 (2008), 135 – 145. issn:0959-4752 https://doi.org/10.1016/j.learninstruc.2007.01.011
[31]
D. Kalkofen, M. Tatzgern, and D. Schmalstieg. 2009. Explosion Diagrams in Augmented Reality. In 2009 IEEE Virtual Reality Conference. 71–78. https://doi.org/10.1109/VR.2009.4811001
[32]
kingsee青橙视界. 2016. Intelligent Aircraft Maintenance-Kingsee AR Smartglasses Performance Support System (PSS) 航空智能维修. https://youtu.be/zP_5Z1A9Pcw
[33]
Aida Komkaite, Liga Lavrinovica, Maria Vraka, and Mikael B. Skov. 2019. Underneath the Skin: An Analysis of YouTube Videos to Understand Insertable Device Interaction. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI ’19). ACM, New York, NY, USA. Article 214, 12 pages. isbn:978-1-4503-5970-2 https://doi.org/10.1145/3290605.3300444
[34]
S. A. Konz and G. L. Dickey. 1969. Manufacturing Assembly Instructions; A Summary. Ergonomics, 12, 3 (1969), 370–382. https://doi.org/10.1080/00140136908931061 arxiv:https://doi.org/10.1080/00140136908931061.
[35]
Sri Kurniawan and Panayiotis Zaphiris. 2005. Research-derived Web Design Guidelines for Older People. In Proceedings of the 7th International ACM SIGACCESS Conference on Computers and Accessibility (Assets ’05). ACM, New York, NY, USA. 129–135. isbn:1-59593-159-7 https://doi.org/10.1145/1090785.1090810
[36]
F. Lamberti, F. Manuri, A. Sanna, G. Paravati, P. Pezzolla, and P. Montuschi. 2014. Challenges, Opportunities, and Future Trends of Emerging Techniques for Augmented Reality-Based Maintenance. IEEE Transactions on Emerging Topics in Computing, 2, 4 (2014), Dec, 411–421. https://doi.org/10.1109/TETC.2014.2368833
[37]
Joseph J LaViola Jr, Ernst Kruijff, Ryan P McMahan, Doug Bowman, and Ivan P Poupyrev. 2017. 3D user interfaces: theory and practice. Addison-Wesley Professional.
[38]
Karen K. Lee. 2012. Developing and implementing the Active Design Guidelines in New York City. Health & Place, 18, 1 (2012), 5 – 7. issn:1353-8292 https://doi.org/10.1016/j.healthplace.2011.09.009 Active Living Research
[39]
LlamaZoo. 2019. Clarity 3D. https://www.llamazoo.com/clarity-3d/
[40]
Céline Mariage, Jean Vanderdonckt, and Costin Pribeanu. 2005. State of the art of web usability guidelines. The handbook of human factors in web design, 688–700. http://hdl.handle.net/2078/17946
[41]
Bernardo Marques, Samuel Silva, João Alves, António Rocha, Paulo Dias, and Beatriz Sousa Santos. 2022. Remote collaboration in maintenance contexts using augmented reality: Insights from a participatory process. International Journal on Interactive Design and Manufacturing (IJIDeM), 1–20. https://doi.org/10.1007/s12008-021-00798-6
[42]
O. Marques. 2016. Visual Information Retrieval: The State of the Art. IT Professional, 18, 4 (2016), July, 7–9. https://doi.org/10.1109/MITP.2016.70
[43]
Davis L. Masten and Tim M.P. Plowman. 2003. Digital ethnography: The next wave in understanding the consumer experience. Design Management Journal (Former Series), 14, 2 (2003), 75–81. https://doi.org/10.1111/j.1948-7169.2003.tb00044.x arxiv:https://onlinelibrary.wiley.com/doi/pdf/10.1111/j.1948-7169.2003.tb00044.x.
[44]
Richard E Mayer. 2002. Multimedia learning. In Psychology of learning and motivation. 41, Elsevier, 85–139.
[45]
Robert K. Merton. 1968. The Matthew Effect in Science. Science, 159, 3810 (1968), Jan., 56–63. https://doi.org/10.1126/science.159.3810.56 Publisher: American Association for the Advancement of Science
[46]
Microsoft. 2019. Dynamics 365. https://dynamics.microsoft.com/en-ca/mixed-reality/guides/
[47]
Microsoft. 2019. Mixed Reality Design Guidelines. https://docs.microsoft.com/en-us/windows/mixed-reality/design
[48]
Paul Milgram, Haruo Takemura, Akira Utsumi, and Fumio Kishino. 1995. Augmented reality: A class of displays on the reality-virtuality continuum. In Telemanipulator and telepresence technologies. 2351, 282–292. https://doi.org/10.1117/12.197321
[49]
Hans-Christoph Nothdurft. 2002. Attention shifts to salient targets. Vision Research, 42, 10 (2002), 1287 – 1306. issn:0042-6989 https://doi.org/10.1016/S0042-6989(02)00016-0
[50]
Kenneth R Ohnemus. 1997. Web style guides: who, what, where. In Proceedings of the 15th annual international conference on Computer documentation. 189–197. https://doi.org/10.1145/263367.263392
[51]
Susan Palmiter, Jay Elkerton, and Patricia Baggett. 1991. Animated demonstrations vs written instructions for learning procedural tasks: a preliminary investigation. International Journal of Man-Machine Studies, 34, 5 (1991), 687 – 701. issn:0020-7373 https://doi.org/10.1016/0020-7373(91)90019-4
[52]
Cindy M. Robertson, Blair MacIntyre, and Bruce N. Walker. 2008. An Evaluation of Graphical Context when the Graphics Are Outside of the Task Area. In Proceedings of the 7th IEEE/ACM International Symposium on Mixed and Augmented Reality (ISMAR ’08). IEEE Computer Society, Washington, DC, USA. 73–76. isbn:978-1-4244-2840-3 https://doi.org/10.1109/ISMAR.2008.4637328
[53]
Cledja Rolim, Dieter Schmalstieg, Denis Kalkofen, and Veronica Teichrieb. 2015. Design Guidelines for Generating Augmented Reality Instructions. In 2015 IEEE International Symposium on Mixed and Augmented Reality. 120–123. https://repositorio.ufpe.br/handle/123456789/25074
[54]
Marc Ericson C. Santos, Arno in Wolde Lübke, Takafumi Taketomi, Goshiro Yamamoto, Ma. Mercedes T. Rodrigo, Christian Sandor, and Hirokazu Kato. 2016. Augmented reality as multimedia: the case for situated vocabulary learning. Research and Practice in Technology Enhanced Learning, 11, 1 (2016), 20 Jan, 4. issn:1793-7078 https://doi.org/10.1186/s41039-016-0028-2
[55]
Dominique Scapin, Corinne Leulier, Jean Vanderdonckt, Céline Mariage, Christian Bastien, Christelle Farenc, Philippe Palanque, and Rémi Bastide. 2000. A Framework for Organizing Web Usability Guidelines. In 6th Conference on Human Factors and the Web (HF Web 2000), Ph. Kortum and E. Kudzinger (Eds.) (Proceedings. of 6th Conf. on Human Factors and the Web HFWeb’2000). Austin, United States. https://hal.science/hal-01319628
[56]
Hasti Seifi, Farimah Fazlollahi, Michael Oppermann, John Andrew Sastrillo, Jessica Ip, Ashutosh Agrawal, Gunhyuk Park, Katherine J. Kuchenbecker, and Karon E. MacLean. 2019. Haptipedia: Accelerating Haptic Device Discovery to Support Interaction & Engineering Design. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI ’19). Association for Computing Machinery, New York, NY, USA. Article 558, 12 pages. isbn:9781450359702 https://doi.org/10.1145/3290605.3300788
[57]
Roger N Shepard and Jacqueline Metzler. 1971. Mental rotation of three-dimensional objects. Science, 171, 3972 (1971), 701–703. https://doi.org/10.1126/science.171.3972.701
[58]
Maarten Someren, Yvonne Barnard, and Jacobijn Sandberg. 1994. The Think Aloud Method - A Practical Guide to Modelling CognitiveProcesses. https://hdl.handle.net/11245/1.103289
[59]
Anselm Strauss and Juliet Corbin. 1990. Basics of qualitative research. Sage publications. https://doi.org/10.4135/9781452230153
[60]
Timo Strohmann, Laura Höper, and Susanne Robra-Bissantz. 2019. Augmented reality: an application of heads-up display technology to manual manufacturing processes. In Proceedings of the Fifty-Second Hawaii International Conference on System Sciences. https://doi.org/10.1109/HICSS.1992.183317
[61]
J Edward Swan and Joseph L Gabbard. 2005. Survey of user-based experimentation in augmented reality. In Proceedings of 1st International Conference on Virtual Reality. 22, 1–9.
[62]
Arthur Tang, Charles Owen, Frank Biocca, and Weimin Mou. 2002. Experimental evaluation of augmented reality in object assembly task. In Proceedings. International Symposium on Mixed and Augmented Reality. 265–266. https://doi.org/10.1109/ISMAR.2002.1115105
[63]
Arthur Tang, Charles Owen, Frank Biocca, and Weimin Mou. 2003. Comparative Effectiveness of Augmented Reality in Object Assembly. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’03). ACM, New York, NY, USA. 73–80. isbn:1-58113-630-7 https://doi.org/10.1145/642611.642626
[64]
Laura E Thomas and Alejandro Lleras. 2007. Moving eyes and moving thought: On the spatial compatibility between eye movements and cognition. Psychonomic bulletin & review, 14, 4 (2007), 663–668. https://doi.org/10.3758/BF03196818
[65]
Trimble. 2019. Trimble Mixed Reality. https://mixedreality.trimble.com
[66]
Dirk Walther, Ueli Rutishauser, Christof Koch, and Pietro Perona. 2005. Selective visual attention enables learning and recognition of multiple objects in cluttered scenes. Computer Vision and Image Understanding, 100, 1 (2005), 41 – 63. issn:1077-3142 https://doi.org/10.1016/j.cviu.2004.09.004 Special Issue on Attention and Performance in Computer Vision
[67]
Ranxiao Frances Wang and Elizabeth S Spelke. 2002. Human spatial representation: insights from animals. Trends in Cognitive Sciences, 6, 9 (2002), 376 – 382. issn:1364-6613 https://doi.org/10.1016/S1364-6613(02)01961-7
[68]
Michelle Q. Wang Baldonado, Allison Woodruff, and Allan Kuchinsky. 2000. Guidelines for Using Multiple Views in Information Visualization. In Proceedings of the Working Conference on Advanced Visual Interfaces (AVI ’00). ACM, New York, NY, USA. 110–119. isbn:1-58113-252-2 https://doi.org/10.1145/345513.345271
[69]
James R. Williams. 1998. Guidelines for the Use of Multimedia in Instruction. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 42, 20, 1447–1451. https://doi.org/10.1177/154193129804202019
[70]
Yu Xiao and Maria Watson. 2019. Guidance on conducting a systematic literature review. Journal of planning education and research, 39, 1 (2019), 93–112. https://doi.org/10.1177/0739456x17723971
Index Terms
- Using Online Videos as the Basis for Developing Design Guidelines: A Case Study of AR-Based Assembly Instructions
Recommendations
AR Music Visualizers: Application Space and Design Guidelines
CHI EA '22: Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing SystemsMusic visualization has recently become a popular application of augmented-reality (AR) technology. However, despite their various designs and functionalities, their potential benefits and the effectiveness of their designs are not well explored. By ...
Guidelines for Error Message Design
The presented study aims to develop a set of guidelines for error message design. A large amount of research is available in the literature on the topic of warning design. The same is not true for error messages. Although some research exists, the ...
Participatory Evaluation of Human-Data Interaction Design Guidelines
Human-Computer Interaction – INTERACT 2019AbstractThe design of visual analytics tools for facilitating human-data interaction (HDI) plays a key role to help people identifying useful knowledge from large masses of data. Designing data visualization based on guidelines is relevant. However, it is ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
December 2023
482 pages
Copyright © 2023 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 01 November 2023
Published in PACMHCI Volume 7, Issue ISS
Check for updates
Author Tags
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 130Total Downloads
- Downloads (Last 12 months)130
- Downloads (Last 6 weeks)10
Reflects downloads up to 25 Nov 2024
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in