Abstract
Owing to its ability to integrate human flexibility with robotic automation, human-robot collaboration possesses tremendous potential in intelligent manufacturing. A quintessential characteristic of this collaboration is the necessity for robotic arms to cooperate with humans in a dynamically changing environment, wherein humans could be considered as dynamic obstacles. One of the significant challenges in human-robot collaboration is the development of obstacle avoidance strategies for robotic path planning within dynamically changing environments. The inability of traditional two-dimensional path planning methods to handle high-dimensional spaces, therefore, many researchers have turned their attention to deep reinforcement learning, and many deep reinforcement learning methods have been applied to robotic arm path planning. However, most deep reinforcement learning models for robotic arm path planning require a significant amount of training time to achieve convergence. In this study, we introduce an algorithm that synergizes Soft Actor-Critic (SAC) with an improved version of Prioritized Experience Replay (PER)—SAC-iPER. We prioritizes experiences based on task-rewards, employing metrics such as time consumption and collision occurrences, in addition to task completion, to rank experiences. This reward-based ordering significantly boosts the learning process in both speed and quality. The results of this study significantly enhanced the training efficiency of deep reinforcement learning models for robotic arm path planning within human-robot collaboration, paving the way for the development of more efficient human-robot collaborative systems.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Wang, L.: A futuristic perspective on human-centric assembly. J. Manuf. Syst. 62, 199–201 (2022). https://doi.org/10.1016/j.jmsy.2021.11.001
Wang, L., et al.: Symbiotic human-robot collaborative assembly. CIRP Ann. 68(2), 701–726 (2019). https://doi.org/10.1016/j.cirp.2019.05.002
Baratta, A., Cimino, A., Gnoni, M.G., Longo, F.: Human robot collaboration in industry 4.0: a literature review. Procedia Comput. Sci. 217, 1887–1895 (2023). https://doi.org/10.1016/j.procs.2022.12.389
Layne, L.A.: Robot-related fatalities at work in the United States, 1992–2017. Am. J. Ind. Med. 66(6), 454–461 (2023). https://doi.org/10.1002/ajim.23470
Prianto, E., Kim, M., Park, J.-H., Bae, J.-H., Kim, J.-S.: Path planning for multi-arm manipulators using deep reinforcement learning: soft actor-critic with hindsight experience replay. Sensors 20(20), 5911 (2020). https://doi.org/10.3390/s20205911
Kusuma, M., Riyanto., Machbub, C.: Humanoid robot path planning and rerouting using A-star search algorithm. In: 2019 IEEE International Conference on Signals and Systems (ICSigSys), Bandung, Indonesia, pp. 110–115. IEEE (2019). https://doi.org/10.1109/ICSIGSYS.2019.8811093
Xu, Z., Guo, S., Zhang, L.: A path planning method of 6-DOF robot for mirror therapy based on A* algorithm. Technol. Health Care 30(1), 105–116 (2021). https://doi.org/10.3233/THC-202551
Liu, W., Zheng, X., Deng, Z.: Dynamic collision avoidance for cooperative fixed-wing UAV swarm based on normalized artificial potential field optimization. J. Cent. South Univ. 28(10), 3159–3172 (2021). https://doi.org/10.1007/s11771-021-4840-5
Lee, S.-J., Baek, S.-H., Kim, J.-H.: Arm Trajectory Generation Based on RRT* for Humanoid Robot. In: Kim, J.-H., Yang, W., Jo, J., Sincak, P., Myung, H. (eds.) Robot Intelligence Technology and Applications 3. AISC, vol. 345, pp. 373–383. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-16841-8_34
Bhuiyan, T., Kästner, L., Hu, Y., Kutschank, B., Lambrecht, J.: Deep-reinforcement-learning-based path planning for industrial robots using distance sensors as observation. In: 2023 8th International Conference on Control and Robotics Engineering (ICCRE), Niigata, Japan, pp. 204–210. IEEE (2023). https://doi.org/10.1109/ICCRE57112.2023.10155608
Cheng, X., Liu, S.: Dynamic obstacle avoidance algorithm for robot arm based on deep reinforcement learning. In: 2022 IEEE 11th Data Driven Control and Learning Systems Conference (DDCLS), Chengdu, China, pp. 1136–1141. IEEE (2022). https://doi.org/10.1109/DDCLS55054.2022.9858561
Tang, W., Cheng, C., Ai, H., Chen, L.: Dual-arm robot trajectory planning based on deep reinforcement learning under complex environment. Micromachines 13(4), 564 (2022). https://doi.org/10.3390/mi13040564
Liu, Q., Liu, Z., Xiong, B., Xu, W., Liu, Y.: Deep reinforcement learning-based safe interaction for industrial human-robot collaboration using intrinsic reward function. Adv. Eng. Inform. 49, 101360 (2021). https://doi.org/10.1016/j.aei.2021.101360
Prianto, E., Park, J.-H., Bae, J.-H., Kim, J.-S.: Deep reinforcement learning-based path planning for multi-arm manipulators with periodically moving obstacles. Appl. Sci. 11(6), 2587 (2021). https://doi.org/10.3390/app11062587
Ren, J., Huang, X., Huang, R.N.: Efficient deep reinforcement learning for optimal path planning. Electronics 11(21), 3628 (2022). https://doi.org/10.3390/electronics11213628
Joypriyanka, M., Surendran, R.: Priority experience replay DQN for training an agent in virtual reality game for kids with paraplegia. In: 2023 2nd International Conference on Applied Artificial Intelligence and Computing (ICAAIC), Salem, India, pp. 743–749. IEEE (2023). https://doi.org/10.1109/ICAAIC56838.2023.10141031
Acknowledgement
The authors would like to thank the National Natural Science Foundation (52175451 and 52205513).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Sun, D., Wen, J., Wang, J., Yang, X., Hu, Y. (2024). A Path Planning Method Based on Deep Reinforcement Learning with Improved Prioritized Experience Replay for Human-Robot Collaboration. In: Kurosu, M., Hashizume, A. (eds) Human-Computer Interaction. HCII 2024. Lecture Notes in Computer Science, vol 14685. Springer, Cham. https://doi.org/10.1007/978-3-031-60412-6_15
Download citation
DOI: https://doi.org/10.1007/978-3-031-60412-6_15
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-60411-9
Online ISBN: 978-3-031-60412-6
eBook Packages: Computer ScienceComputer Science (R0)