Abstract
Collaborative assembly is one of the most common tasks in collaborative robotics, which requires an efficient, simple and user-friendly way of communication with a collaborative robot. This paper presents an augmented reality (AR) interface for controlling a UR5e robot in order to get assembly parts from a robot’s desktop in a collaborative assembly process. This AR interface is based on the concept of a smart mirror that recognizes user’s hand gestures as control signals. A goal of the study is to investigate user perception of a contactless interaction method with a collaborative robot UR5e using the proposed AR interface. The conducted experiments involved 15 participants who are not professional operators of robots and industrial manipulators. During the experiments, the participants demonstrated a quick adaptation and an evident interest to working side by side with the UR5e robot using the AR interface. Our results suggest that the developed AR interface is flexible, intuitive for an operator, scalable in terms of development for different cases of production goals and can be recommended for equipping a workcell for collaborative assembly of various products.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Li, H., et al.: An admittance-controlled amplified force tracking scheme for collaborative lumbar puncture surgical robot system. Int. J. Med. Robot. Comput. Assist. Surg. 18(5) (2022)
Gasparetto, A., Scalera, L.: A brief history of industrial robotics in the 20th century. Adv. Hist. Stud. 8, 24–35 (2019)
Sultanov, R., Sulaiman, S., Li, H., Meshcheryakov, R., Magid, E.: A review on collaborative robots used in industrial and service sectors. In: Proceedings of International Siberian Conference on Control and Communications (SIBCON), pp. 1–7 (2022)
Dobrokvashina, A., Sulaiman, S., Zagirov, A., Chebotareva, E., Kuo-Hsien, H., Magid, E.: Human robot interaction in collaborative manufacturing scenarios: prospective cases. In: Proceedings of International Siberian Conference on Control and Communications (SIBCON), pp. 1–6 (2022)
Rodriguez-Guerra, D., Sorrosal, G., Cabanes, I., Calleja, C.: Human-robot interaction review: challenges and solutions for modern industrial environments. IEEE Access 9, 108557–108578 (2021)
Wang, Y., Wang, J., Feng, J., Liu, J., Liu, X.: Integrated task sequence planning and assignment for human-robot collaborative assembly station. Flex. Serv. Manuf. J. 35(4), 979–1006 (2023)
Petzoldt, C., Harms, M., Freitag, M.: Review of task allocation for human-robot collaboration in assembly. Int. J. Comput. Integr. Manuf. 36(11), 1675–1715 (2023)
Proia, S., Carli, R., Cavone, G., Dotoli, M.: Control techniques for safe, ergonomic, and efficient human-robot collaboration in the digital industry: a survey. IEEE Trans. Autom. Sci. Eng. 19(3), 1798–1819 (2021)
Mateus, J., Claeys, D., Limère, V., Cottyn, J., Aghezzaf, E.: A structured methodology for the design of a human-robot collaborative assembly workplace. Int. J. Adv. Manuf. Technol. 102(5), 2663–2681 (2019)
Angleraud, A., Mehman Sefat, A., Netzev, M., Pieters, R.: Coordinating shared tasks in human-robot collaboration by commands. Front. Robot. AI 8 (2021)
Su, H., Qi, W., Chen, J., Yang, C., Sandoval, J., Laribi, M.A.: Recent advancements in multimodal human-robot interaction. Front. Neurorobot. 17 (2023)
Wang, X., Shen, H., Yu, H., Guo, J., Wei, X.: Hand and arm gesture-based human-robot interaction: a review. In: Proceedings of the 6th International Conference on Algorithms, Computing and Systems, pp. 1–7 (2022)
Neto, P., Simão, M., Mendes, N., Safeea, M.: Gesture-based human-robot interaction for human assistance in manufacturing. Int. J. Adv. Manuf. Technol. 101, 119–135 (2019)
Duan, J., Fang, Y., Zhang, Q., Qin, J.: HRC of intelligent assembly system based on multimodal gesture control. Int. J. Adv. Manuf. Technol. 127(9), 4307–4319 (2023)
Mustafin, M., Chebotareva, E., Li, H., Martínez-García, E.A., Magid, E.: Features of interaction between a human and a gestures-controlled collaborative robot in an assembly task: pilot experiments. In: Proceedings of International Conference on Artificial Life and Robotics, vol. 28, pp. 158–162 (2023)
Mustafin, M., Chebotareva, E., Li, H., Magid, E.: Experimental validation of an interface for a human-robot interaction within a collaborative task. In: Ronzhin, A., Sadigov, A., Meshcheryakov, R. (eds.) ICR 2023. LNCS, vol. 14214, pp. 23–35. Springer, Cham (2023). https://doi.org/10.1007/978-3-031-43111-1_3
Lugaresi, C., Tang, J., Nash, H., McClanahan, C., Uboweja, E., Hays, M.: MediaPipe: a framework for perceiving and processing reality. In: Third Workshop on Computer Vision for AR/VR, IEEE Computer Vision and Pattern Recognition (CVPR), vol. 2019 (2019)
Hand Gesture Recognition Using Mediapipe. https://github.com/Kazuhito00/hand-gesturerecognition-using-mediapipe. Accessed 05 May 2024
Mustafin, M., Chebotareva, E.: Android based educational mobile robot design and pilot evaluations. In: Proceedings of International Conference on Artificial Life and Robotics, (ICAROB), pp. 145–148 (2023)
Schrum, M., Ghuy, M., Hedlund-Botti, E., Natarajan, M., Johnson, M., Gombolay, M.: Concerning trends in likert scale usage in human-robot interaction: towards improving best practices. ACM Trans. Hum.-Robot Interact. 12(3) (2022)
Acknowledgments
This research was funded by the Kazan Federal University Strategic Academic Leadership Program (“PRIORITY-2030”).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Ethics declarations
The authors have no competing interests to declare that are relevant to the content of this article.
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Mustafin, M., Tsoy, T., Bai, Y., Svinin, M., Chebotareva, E. (2024). Augmented Reality Interface for UR5e Robot that Transfers Parts to a Human in Collaborative Assembly. In: Ronzhin, A., Savage, J., Meshcheryakov, R. (eds) Interactive Collaborative Robotics. ICR 2024. Lecture Notes in Computer Science(), vol 14898. Springer, Cham. https://doi.org/10.1007/978-3-031-71360-6_1
Download citation
DOI: https://doi.org/10.1007/978-3-031-71360-6_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-71359-0
Online ISBN: 978-3-031-71360-6
eBook Packages: Computer ScienceComputer Science (R0)