Abstract
This paper proposes a new 3D Lyapunov guidance vector field(3D-LGV) avoidance strategy based on reinforcement learning for the satellite evasion and interception problem. Combining it with the interfered fluid dynamical system (IFDS) enables the satellite to evade and smoothly enter orbit according to the state of the intercepting satellite in real time. 3D-LGV provides an initial flow field approaching an elliptical orbit, while IFDS provides a perturbed flow field based on the intercepting satellite position. The combined potential field of the initial flow field and the disturbed flow field is the planned velocity direction of the satellite. As a decision-making layer, the proximal policy optimization (PPO) dynamically adjusts the perturbed flow field in the IFDS to increase the avoidance success rate in different scenarios. The experimental results show that, compared with the particle swarm optimization with rolling horizon control algorithm, the algorithm proposed in this paper has a shorter decision time and a higher avoidance success rate. At the same time, Monte Carlo simulation shows that the evasion success rate of the proposed algorithm reaches 98%.
Article PDF
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
Code Availability
No software application and custom code is used.
References
Luo, Y.Z., Zhang, J., Tang, G.J.: Survey of Orbital Dynamics and Control of Space Rendezvous. Chin. J. Aeronaut. 27(1), 1–11 (2014)
Le May, S., Gehly, S., Carter, B.A., et al.: Space debris collision probability analysis for proposed global broadband constellations. Acta Astronautica 151, 445–455 (2018)
Wu, Z., Li, J., Zuo, J., et al.: Path planning of UAVs based on collision probability and Kalman filter. IEEE Access 6, 34237–34245 (2018)
Orozco-Rosas, U., Montiel, O., Sepúlveda, R.: Mobile robot path planning using membrane evolutionary artificial potential field. Appl. Soft Comput. 77, 236–251 (2019)
Duhé, J.F., Victor, S., Melchior, P.: Contributions on artificial potential field method for effective obstacle avoidance. Fract. Calc. Appl. Anal. 24(2), 421–446 (2021)
Wen, C., Qiao, D.: Calculating collision probability for long-term satellite encounters through the reachable domain method. Astrodynamics 6(2), 141–159 (2022)
Yan, R., Gong, J., Liu, S., et al.: Gaussian sum reapproximation applied to the probability of collision calculations. Adv. Space Res. 68(9), 3846–3858 (2021)
Khatib, O.: Real-time obstacle avoidance for manipulators and mobile robots. Int. J. Robot. Res. 5(1), 90–98 (1986)
Yao, P., Wang, H., Su, Z.: UAV feasible path planning based on disturbed fluid and trajectory propagation. Chin. J. Aeronaut. 28(4), 1163–1177 (2015)
Wu, J., Wang, H., Li, N., et al.: Formation obstacle avoidance: A fluid-based solution. IEEE Syst. J. 14(1), 1479–1490 (2019)
Harinarayana, T., Krishnan, S.V., Hota, S., et al.: A Lyapunov guidance vector field based continuous curvature path generation for waypoint following of UAVs. In: 2021 International Conference on Unmanned Aircraft Systems (ICUAS), pp. 498–506. IEEE (2021)
Sun, S., Wang, H., Liu, J., et al.: Fast Lyapunov vector field guidance for standoff target tracking based on offline search. IEEE Access 7, 124797–124808 (2019)
Rezende, A.M.C, Gonçalves, V.M, Raffo, G.V, et al.: Robust fixed-wing UAV guidance with circulating artificial vector fields. In: 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 5892–5899. IEEE (2018)
Schulman, J., Wolski, F., Dhariwal, P., et al.: Proximal policy optimization algorithms. arXiv. https://arxiv.org/abs/1707.06347 (2017). Accessed 20 July 2017
Schulman, J., Levine, S., Abbeel, P., et al.: Trust region policy optimization. In: 32nd International Conference on Machine Learning (ICML 2015), pp. 1889–1897. International Machine Learning Society (2015)
Fujimoto, S., Hoof, H., Meger, D.: Addressing function approximation error in actorcritic methods. In: 35th International Conference on Machine Learning (ICML 2018), pp. 2587–2601. International Machine Learning Society (2018)
Lillicrap, T.P., Hunt, J.J., Pritzel, A., et al.: Continuous control with deep reinforcement learning. arXiv. https://arxiv.org/abs/1509.02971 (2015). Accessed 9 Sept 2015
Wang, Y., Wang, H., Wen, J., et al.: Obstacle avoidance of UAV based on neural networks and interfered fluid dynamical system. In: Proceedings of 2020 3rd International Conference on Unmanned Systems (ICUS), pp. 1066–1071. IEEE (2020)
Dogru, O., Velswamy, K., Ibrahim, F., et al.: Reinforcement learning approach to autonomous PID tuning. Comput. Chem. Eng. 161, 107760 (2022)
Ng, A.Y., Harada, D., Russell, S.: Policy invariance under reward transformations: Theory and application to reward shaping. In: 16th International Conference on Machine Learning (ICML 99), pp. 278–287. Machine Learning, Proceedings (1999)
Andrychowicz, M., Wolski, F., Ray, A., et al.: Hindsight experience replay. In: Advances in Neural Information Processing Systems 30 - Proceedings of the 2017 Conference, pp. 5049–5059. Neural information processing systems foundation (2017)
Mnih, V., Badia, A.P., Mirza, M., et al.: Asynchronous methods for deep reinforcement learning. In: 33rd International Conference on Machine Learning (ICML 2016), pp. 2850–2869. International Machine Learning Society (2016)
Schulman, J., Moritz, P., Levine, S., et al.: High-dimensional continuous control using generalized advantage estimation. arXiv. https://arxiv.org/abs/1506.02438 (2015). Accessed 8 June 2015
Haarnoja, T., Zhou, A., Hartikainen, K., et al.: Soft actor-critic algorithms and applications. arXiv. https://arxiv.org/abs/1812.05905 (2018). Accessed 13 Dec 2018
Marini, F., Walczak, B., et al.: Particle swarm optimization (PSO). A tutorial. Chemometr. Intell. Lab. 149, 153–165 (2015)
Kassas, Z.M., Humphreys, T.E.: Receding horizon trajectory optimization in opportunistic navigation environments. IEEE Trans. Aerosp. Electron. Syst. 51(2), 866–877 (2015)
Acknowledgements
The authors would like to express their acknowledgment for the support from the National Natural Science Foundation of China (No. U21B6001) and China Post-doctoral Science Foundation (No. 2022M713006).
Funding
This work is supported by the National Natural Science Foundation of China (No. U21B6001) and China Post-doctoral Science Foundation (No. 2022M713006).
Author information
Authors and Affiliations
Contributions
Model establishment: Yunfei Zhang, Coding: Yunfei Zhang and Yiheng Liu, Data collection and analysis: Menghua Zhang and Jianfa Wu. Giving guidance: Honglun Wang.
Corresponding author
Ethics declarations
Ethics Approval
Not applicable.
Consent to Participate
Not applicable.
Consent for Publication
Not applicable.
Conflict of Interest
The authors declare no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.
About this article
Cite this article
Zhang, Y., Wang, H., Zhang, M. et al. A Novel Method of 3D Lyapunov Guidance Vector Field to Avoid Intercepting Satellite Based on Reinforcement Learning. J Intell Robot Syst 110, 113 (2024). https://doi.org/10.1007/s10846-024-02151-x
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s10846-024-02151-x