Abstract
Sewer sedimentation tanks accumulate a lot of sludge all year round and require a lot of labor and resources to clean up. Such underwater cleaning has many difficulties in cleaning works. Desilting robots are widely used in sludge cleaning to reduce labor. However, since this method is usually planned based on manual experience, the operation is inefficient, not intelligent, and requires a certain amount of labor. To enable the desilting robot to intelligently plan rational cleaning paths, it uses the complete coverage path planning (CCPP) algorithm. Existing CCPP cannot meet the requirements of various environments. Therefore, in this paper, we propose to use the deep reinforcement learning (DRL) algorithm to learn the sewer sedimentation tank environment and to find the optimal cleaning path. Experiments show that 2000 episodes are trained in a simplified simulation environment using deep Q network (DQN) and double DQN (DDQN), respectively. DQN only got a cumulative reward of 1000 in the 1228th episode, while DDQN got a cumulative reward of 8000 in the 343rd episode and completed the task. Therefore, the slag removal robot using DDQN as the control strategy can adaptively complete the CCPP problem.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Noh, D., Lee, W., Kim, H.R., Cho, I.S., Shim, I.B., Baek, S.: Adaptive coverage path planning policy for a cleaning robot with deep reinforcement learning. In: 2022 IEEE International Conference on Consumer Electronics (ICCE), pp. 1–6. IEEE (2022). https://doi.org/10.1109/ICCE53296.2022.9730307
Mnih, V., et al.: Human-level control through deep reinforcement learning. Nature 518(7540), 529–533 (2015)
Galceran, E., Carreras, M.: A survey on coverage path planning for robotics. Robot. Auton. Syst. 61(12), 1258–1276 (2013)
Kollar, T., Roy, N.: Trajectory optimization using reinforcement learning for map exploration. Int. J. Robot. Res. 27(2), 175–196 (2008)
Lei, X., Zhang, Z., Dong, P.: Dynamic path planning of unknown environment based on deep reinforcement learning. J. Robot. 2018, 1–10 (2018)
Nguyen, Q.P., Low, B.K.H., Jaillet, P.: Inverse reinforcement learning with locally consistent reward functions. Technical report (2015)
Wulfmeier, M., Rao, D., Wang, D.Z., Ondruska, P., Posner, I.: Large-scale cost function learning for path planning using deep inverse reinforcement learning. Int. J. Robot. Res. 36(10), 1073–1087 (2017)
Acknowledgment
This work was partly supported by the Technological Innovation R&D Program (S3264239) funded by the Ministry of SMEs and Startups, and the Technological Innovation R&D Program (S3154675) funded by the Ministry of SMEs and Startups (MSS, Korea).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Zhao, Y., Sun, P., Lim, C. (2023). The Simulation of Adaptive Coverage Path Planning Policy for an Underwater Desilting Robot Using Deep Reinforcement Learning. In: Jo, J., et al. Robot Intelligence Technology and Applications 7. RiTA 2022. Lecture Notes in Networks and Systems, vol 642. Springer, Cham. https://doi.org/10.1007/978-3-031-26889-2_7
Download citation
DOI: https://doi.org/10.1007/978-3-031-26889-2_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-26888-5
Online ISBN: 978-3-031-26889-2
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)