Exploring Real-time Precision Feedback for AR-assisted Manual Adjustment in Mechanical Assembly
Authors: 
Xingyue Tang, Zhuang Chang, Weiping He, Mark Billinghurst, Xiaotian ZhangAuthors Info & Claims
Article No.: 34, Pages 1 - 11
Abstract
Augmented Reality (AR) based manual assembly nowadays enables to guide the process of physical tasks, providing intuitive instructions and detailed information in real-time. However, very limited studies have explored AR manual adjustment tasks with precision requirements. In this paper, we develop an AR-assisted guidance system for manual adjustments with relatively high-precision requirements. We first assessed the accuracy of the special-set OptiTrack system to determine the threshold of precision requirements for our user study. We further evaluated the performance of Number-based and Bar-based precision feedback by comparing orienting assembly errors and task completion time, as well as the usability in the user study. We found that the assembly errors of orientation in the Number-based and Bar-based interfaces were significantly lower than the baseline condition, while there was no significant difference between the Number-based and Bar-based interfaces. Furthermore, the Number-based showed faster task completion time, lower workload, and higher usability than the Bar-based condition.
Supplemental Material
MP4 File
Supplemental video of user views under three interfaces wearing the AR headset.
- Download
- 129.42 MB
References
[1]
[n. d.]. Manual Assembly - an overview | ScienceDirect Topics — sciencedirect.com. https://www.sciencedirect.com/topics/engineering/manual-assembly. [Accessed 10-Jun-2023].
[2]
Salvador S Agati, Rudieri D Bauer, Marcelo da S Hounsell, and Aleksander S Paterno. 2020. Augmented reality for manual assembly in industry 4.0: Gathering guidelines. In 2020 22nd Symposium on Virtual and Augmented Reality (SVR). IEEE, 179–188.
[3]
Tim Ameler, Kai Blohme, Lilith Brandt, Raphael Brüngel, Alice Hensel, Lisa Huber, Francis Kuper, Jessica Swoboda, Maren Warnecke, Michaela Warzecha, 2019. A comparative evaluation of steamvr tracking and the optitrack system for medical device tracking. In 2019 41st annual international conference of the IEEE engineering in medicine and biology society (EMBC). IEEE, 1465–1470.
[4]
Anwesa Barman and Manas Das. 2021. Fundamental understanding and latest developments in magnetic field assisted finishing processes. In Advanced Machining and Finishing. Elsevier, 611–641.
[5]
Eleonora Bottani and Giuseppe Vignali. 2019. Augm ented reality technology in the manufacturing industry: A review of the last decade. Iise Transactions 51, 3 (2019), 284–310.
[6]
John Brooke 1996. SUS-A quick and dirty usability scale. Usability evaluation in industry 189, 194 (1996), 4–7.
[7]
Fabio Bruno, Francesco Caruso, Luigi De Napoli, and Maurizio Muzzupappa. 2006. Visualization of industrial engineering data visualization of industrial engineering data in augmented reality. Journal of visualization 9, 3 (2006), 319–329.
[8]
Andrew Chan, Janelle Aguillon, Doug Hill, and Edmond Lou. 2017. Precision and accuracy of consumer-grade motion tracking system for pedicle screw placement in pediatric spinal fusion surgery. Medical engineering & physics 46 (2017), 33–43.
[9]
Ren-Jung Chang and Jia-Cheng Jau. 2016. Augmented reality in peg-in-hole microassembly operations. International Journal of Automation Technology 10, 3 (2016), 438–446.
[10]
Cyrill Dennler, Laurenz Jaberg, José Spirig, Christoph Agten, Tobias Götschi, Philipp Fürnstahl, and Mazda Farshad. 2020. Augmented reality-based navigation increases precision of pedicle screw insertion. Journal of orthopaedic surgery and research 15 (2020), 1–8.
[11]
Jiahao Ding, Yu Zhu, Mingyu Luo, Minghua Zhu, Xiumin Fan, and Zelin Zhou. 2020. AR Assisted Process Guidance System for Ship Block Fabrication. https://doi.org/10.1007/978-3-030-49698-2_5
[12]
Ashish Doshi, Ross T Smith, Bruce H Thomas, and Con Bouras. 2017. Use of projector based augmented reality to improve manual spot-welding precision and accuracy for automotive manufacturing. The International Journal of Advanced Manufacturing Technology 89 (2017), 1279–1293.
[13]
Sangjun Eom, Seijung Kim, Shervin Rahimpour, and Maria Gorlatova. 2022. AR-assisted surgical guidance system for ventriculostomy. In 2022 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW). IEEE, 402–405.
[14]
John Erkoyuncu and Samir Khan. 2020. Olfactory-based augmented reality support for industrial maintenance. IEEE Access 8 (2020), 30306–30321.
[15]
Shuo Feng, Weiping He, Shaohua Zhang, and Mark Billinghurst. 2022. Seeing is believing: AR-assisted blind area assembly to support hand–eye coordination. The International Journal of Advanced Manufacturing Technology 119, 11-12 (2022), 8149–8158.
[16]
Shuo Feng, Xinjing He, Weiping He, and Mark Billinghurst. 2022. Can you hear it? Stereo sound-assisted guidance in augmented reality assembly. Virtual Reality (2022), 1–11.
[17]
Michele Fiorentino, Antonio E Uva, Michele Gattullo, Saverio Debernardis, and Giuseppe Monno. 2014. Augmented reality on large screen for interactive maintenance instructions. Computers in Industry 65, 2 (2014), 270–278.
[18]
Michele Gattullo, Lucilla Dammacco, Francesca Ruospo, Alessandro Evangelista, Michele Fiorentino, Jan Schmitt, and Antonio E Uva. 2020. Design preferences on industrial augmented reality: a survey with potential technical writers. In 2020 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct). IEEE, 172–177.
[19]
Michele Gattullo, Antonio E Uva, Michele Fiorentino, Giulia Wally Scurati, and Francesco Ferrise. 2017. From paper manual to AR manual: do we still need text?Procedia Manufacturing 11 (2017), 1303–1310.
[20]
Nirit Gavish, Teresa Gutiérrez, Sabine Webel, Jorge Rodríguez, Matteo Peveri, Uli Bockholt, and Franco Tecchia. 2015. Evaluating virtual reality and augmented reality training for industrial maintenance and assembly tasks. Interactive Learning Environments 23, 6 (2015), 778–798.
[21]
Sandra G Hart and Lowell E Staveland. 1988. Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In Advances in psychology. Vol. 52. Elsevier, 139–183.
[22]
Lei Hou, Xiangyu Wang, Leonhard Bernold, and Peter ED Love. 2013. Using animated augmented reality to cognitively guide assembly. Journal of Computing in Civil Engineering 27, 5 (2013), 439–451.
[23]
Tadanobu Inoue, Giovanni De Magistris, Asim Munawar, Tsuyoshi Yokoya, and Ryuki Tachibana. 2017. Deep reinforcement learning for high precision assembly tasks. In 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 819–825.
[24]
Hugh Jack. 2013. Chapter 11 - Manufacturing Design. In Engineering Design, Planning, and Management, Hugh Jack (Ed.). Academic Press, Boston, 419–468. https://doi.org/10.1016/B978-0-12-397158-6.00011-5
[25]
Nor Farzana Syaza Jeffri and Dayang Rohaya Awang Rambli. 2020. Guidelines for the interface design of ar systems for manual assembly. In Proceedings of the 2020 4th international conference on virtual and augmented reality simulations. 70–77.
[26]
Nor Farzana Syaza Jeffri and Dayang Rohaya Awang Rambli. 2021. A review of augmented reality systems and their effects on mental workload and task performance. Heliyon 7, 3 (2021), e06277.
[27]
Seokhee Jeon and Seungmoon Choi. 2009. Haptic augmented reality: Taxonomy and an example of stiffness modulation. Presence 18, 5 (2009), 387–408.
[28]
Sunwook Kim, Maury A Nussbaum, and Joseph L Gabbard. 2019. Influences of augmented reality head-worn display type and user interface design on performance and usability in simulated warehouse order picking. Applied ergonomics 74 (2019), 186–193.
[29]
Wang Li, Junfeng Wang, Sichen Jiao, Meng Wang, and Shiqi Li. 2019. Research on the visual elements of augmented reality assembly processes. Virtual Reality & Intelligent Hardware 1, 6 (2019), 622–634.
[30]
Terje K. Lien. 2014. Manual Assembly. Springer Berlin Heidelberg, Berlin, Heidelberg, 825–828. https://doi.org/10.1007/978-3-642-20617-7_6624
[31]
Jen-Shuo Liu, Barbara Tversky, and Steven Feiner. 2022. Precueing Sequential Rotation Tasks in Augmented Reality. In Proceedings of the 28th ACM Symposium on Virtual Reality Software and Technology. 1–11.
[32]
PA McKeown. 1987. The role of precision engineering in manufacturing of the future. CIRP Annals 36, 2 (1987), 495–501.
[33]
Kristin McLane. [n. d.]. Strategies for Overcoming the Skilled Labor Shortage in Manufacturing — cimx.com. https://www.cimx.com/blog/strategies-for-overcoming-the-skilled-labor-shortage-in-manufacturing. [Accessed 08-Jun-2023].
[34]
Jozef Novak-Marcincin, Jozef Barna, and Jozef Torok. 2014. Precision assembly process with augmented reality technology support. In Key Engineering Materials, Vol. 581. Trans Tech Publ, 106–111.
[35]
Rafael Radkowski. 2015. Investigation of visual features for augmented reality assembly assistance. In Virtual, Augmented and Mixed Reality: 7th International Conference, VAMR 2015, Held as Part of HCI International 2015, Los Angeles, CA, USA, August 2-7, 2015, Proceedings 7. Springer, 488–498.
[36]
Rafael Radkowski, Jordan Herrema, and James Oliver. 2015. Augmented reality-based manual assembly support with visual features for different degrees of difficulty. International Journal of Human-Computer Interaction 31, 5 (2015), 337–349.
[37]
Ionel Simion and Daniel Dobre. [n. d.]. Modeling the Precision Orientation in Fixture Design for Manufacturing. ([n. d.]).
[38]
Sonja Stork, C Stobel, HJ Muller, Mathey Wiesbeck, MF Zah, and A Schubo. 2007. A neuroergonomic approach for the investigation of cognitive processes in interactive assembly environments. In RO-MAN 2007-The 16th IEEE International Symposium on Robot and Human Interactive Communication. IEEE, 750–755.
[39]
Zainab F Syed, Priyanka Aggarwal, Xiaoji Niu, and Naser El-Sheimy. 2008. Civilian vehicle navigation: Required alignment of the inertial sensors for acceptable navigation accuracies. IEEE Transactions on Vehicular Technology 57, 6 (2008), 3402–3412.
[40]
Arthur Tang, Charles Owen, Frank Biocca, and Weimin Mou. 2003. Comparative effectiveness of augmented reality in object assembly. In Proceedings of the SIGCHI conference on Human factors in computing systems. 73–80.
[41]
Yaniel Torres Medina, Sylvie Nadeau, and Kurt Landau. 2019. Assembly guidance systems in aerospace manufacturing. Substance ÉTS (2019).
[42]
Xiangyu Wang, Soh K Ong, and Andrew YC Nee. 2016. A comprehensive survey of augmented reality assembly research. Advances in Manufacturing 4 (2016), 1–22.
[43]
Zhuo Wang, Xiaoliang Bai, Shusheng Zhang, Mark Billinghurst, Weiping He, Peng Wang, Weiqi Lan, Haitao Min, and Yu Chen. 2022. A comprehensive review of augmented reality-based instruction in manual assembly, training and repair. Robotics and Computer-Integrated Manufacturing 78 (2022), 102407. https://doi.org/10.1016/j.rcim.2022.102407
[44]
Zhuo Wang, Xiaoliang Bai, Shusheng Zhang, Weiping He, Yang Wang, Dechuan Han, Sili Wei, Bingzhao Wei, and Chengkun Chen. 2021. M-AR: A Visual Representation of Manual Operation Precision in AR Assembly. International Journal of Human–Computer Interaction 37, 19 (2021), 1799–1814.
[45]
Dian Wu and Fuzhou Du. 2018. A new method of precise orientation adjustment based on matrix similarity for large-scale component. Assembly Automation 38, 2 (2018), 207–215.
Index Terms
- Exploring Real-time Precision Feedback for AR-assisted Manual Adjustment in Mechanical Assembly
Recommendations
Guidelines for the Interface Design of AR Systems for Manual Assembly
ICVARS '20: Proceedings of the 2020 4th International Conference on Virtual and Augmented Reality SimulationsAugmented Reality (AR) allows for the augmentation of relevant in-situ information in real-time, making it an effective assistive tool to support and enhance task performance. Manual assembly is a promising area in which AR can be used to assist users. ...
Evaluating the effectiveness of augmented reality displays for a manual assembly task
The focus of this research was to examine how effectively augmented reality displays, generated with a wearable computer, could be used for aiding an operator performing a manual assembly task. Fifteen subjects were asked to assemble a computer ...
Reflect-AR: Insights into Mirror-Based Augmented Reality Instructions to Support Manual Assembly Tasks
PETRA '23: Proceedings of the 16th International Conference on PErvasive Technologies Related to Assistive EnvironmentsManual assembly tasks can be difficult and tedious without assistance. Here, augmented reality (AR) can help to ease the task load and lower the time and error rates by interactive in-situ instructions. Such approaches are often implemented by video or ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
October 2023
542 pages
ISBN:9798400703287
DOI:10.1145/3611659
Copyright © 2023 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 09 October 2023
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
VRST 2023: 29th ACM Symposium on Virtual Reality Software and Technology
October 9 - 11, 2023
Christchurch, New Zealand
Acceptance Rates
Overall Acceptance Rate 66 of 254 submissions, 26%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 261Total Downloads
- Downloads (Last 12 months)178
- Downloads (Last 6 weeks)10
Reflects downloads up to 18 Nov 2024
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format