Abstract
UAV swarm confrontation game is a type of intelligent game problem. Multi-agent reinforcement learning theory provides an effective solution for this game. However, when using common multi-agent deep reinforcement learning algorithms, such as the multi-agent deep deterministic policy gradient (MADDPG) algorithm, to train the strategy of UAV swarm, there are issues such as slow convergence speed and weak generalization ability on similar tasks. To address these issues, this paper combines the model-agnostic meta-learning (MAML) algorithm in few-shot learning with the original MADDPG algorithm, and proposes an improved MB-MADDPG algorithm, which is applied to the strategy optimization of a UAV swarm confrontation task. Experimental results show that compared with the original algorithm, the improved algorithm can accelerate the convergence while maintaining the training effect, and the success rate of defense after training with both algorithms exceeds 50%.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Zhou, Y., Rao, B., Wang, W.: UAV swarm intelligence: recent advances and future trends. IEEE Access 8, 183856–183878 (2020)
Xia, Z., et al.: Multi-agent reinforcement learning aided intelligent UAV swarm for target tracking. IEEE Trans. Veh. Technol. 71(1), 931–945 (2021)
Lowe, R., Wu, Y.I., Tamar, A., Harb, J., Pieter Abbeel, O., Mordatch, I.: Multi-agent actor-critic for mixed cooperative-competitive environments. In: Advances in Neural Information Processing Systems, vol. 30 (2017)
Finn, C., Abbeel, P., Levine, S.: Model-agnostic meta-learning for fast adaptation of deep networks. In: International Conference on Machine Learning, pp. 1126–1135. PMLR (2017)
Tang, J., Duan, H., Lao, S.: Swarm intelligence algorithms for multiple unmanned aerial vehicles collaboration: a comprehensive review. Artif. Intell. Rev. 56(5), 4295–4327 (2023)
Wu, H., Li, H., Xiao, R., Liu, J.: Modeling and simulation of dynamic ant colony’s labor division for task allocation of UAV swarm. Phys. A 491, 127–141 (2018)
McMahon, D.C.: A neural network trained to select aircraft maneuvers during air combat: a comparison of network and rule based performance. In: 1990 IJCNN International Joint Conference on Neural Networks, pp. 107–112. IEEE (1990)
Guo, J., et al.: Maneuver decision of UAV in air combat based on deterministic policy gradient. In: 2022 IEEE 17th International Conference on Control & Automation (ICCA), pp. 243–248. IEEE (2022)
Gupta, J.K., Egorov, M., Kochenderfer, M.: Cooperative multi-agent control using deep reinforcement learning. In: Sukthankar, G., Rodriguez-Aguilar, J.A. (eds.) AAMAS 2017. LNCS (LNAI), vol. 10642, pp. 66–83. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-71682-4_5
Sukhbaatar, S., Fergus, R., et al.: Learning multiagent communication with backpropagation. In: Advances in Neural Information Processing Systems, vol. 29 (2016)
Foerster, J., Farquhar, G., Afouras, T., Nardelli, N., Whiteson, S.: Counterfactual multi-agent policy gradients. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32 (2018)
Raileanu, R., Denton, E., Szlam, A., Fergus, R.: Modeling others using oneself in multi-agent reinforcement learning. In: International Conference on Machine Learning, pp. 4257–4266. PMLR (2018)
Acknowledgments
This work was supported by the National Natural Science Foundation of China (Grant No.61973244, 72001214, and 61573277) and the open fund of CETC Key Laboratory of Data Link Technology.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Xie, L., Ma, W., Wang, L., Ke, L. (2024). A New Deep Reinforcement Learning Algorithm for UAV Swarm Confrontation Game. In: Tan, Y., Shi, Y. (eds) Data Mining and Big Data. DMBD 2023. Communications in Computer and Information Science, vol 2017. Springer, Singapore. https://doi.org/10.1007/978-981-97-0837-6_14
Download citation
DOI: https://doi.org/10.1007/978-981-97-0837-6_14
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-97-0836-9
Online ISBN: 978-981-97-0837-6
eBook Packages: Computer ScienceComputer Science (R0)