Abstract
Explaining the decision-making policies of deep reinforcement learning is challenging owing to the black-box nature of neural networks. We address this challenge by combining deep learning models and symbolic structures into a neural-logic model that reasons in the form of neural logic programming. The proposed explainable multi-agent reinforcement learning algorithm performs reasoning in a symbolic-represented environment using multi-hop reasoning, a relational path-searching method that uses prior symbolic knowledge. Furthermore, to alleviate the partial observability problem in multi-agent systems, we devised an explainable history module using an attention mechanism to incorporate past experiences while preserving interpretability. Experimental studies demonstrate that the proposed method can effectively learn close-to-optimal policies while generating expressive rules to explain the decisions. Particularly, it can learn more abstract concepts than conventional neural network approaches.
This work was supported by the Fund of State Key Laboratory, China under Grant No. XM2020XT1006.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Evans, R., Grefenstette, E.: Learning explanatory rules from noisy data. J. Artif. Intell. Res. 61, 1–64 (2018)
Hausknecht, M.J., Stone, P.: Deep recurrent Q-learning for partially observable MDPs. In: 2015 AAAI, pp. 29–37 (2015)
Jiang, Z., Luo, S.: Neural logic reinforcement learning. In: Proceedings of the 36th International Conference on Machine Learning, ICML 2019. Proceedings of Machine Learning Research, vol. 97, pp. 3110–3119 (2019)
Leibo, J.Z., Zambaldi, V.F., Lanctot, M., Marecki, J., Graepel, T.: Multi-agent reinforcement learning in sequential social dilemmas. In: Proceedings of the 16th Conference on Autonomous Agents and MultiAgent Systems, AAMAS 2017, pp. 464–473 (2017)
Li, G., Xiao, G., Zhang, J., Liu, J., Shen, Y.: Towards relational multi-agent reinforcement learning via inductive logic programming. In: Artificial Neural Networks and Machine Learning - ICANN 2022–31st International Conference on Artificial Neural Networks, vol. 13530, pp. 99–110 (2022)
Sorokin, I., Seleznev, A., Pavlov, M., Fedorov, A., Ignateva, A.: Deep attention recurrent q-network. CoRR (2015)
Sunehag, P., et al.: Value-decomposition networks for cooperative multi-agent learning based on team reward. In: Proceedings of the 17th International Conference on Autonomous Agents and MultiAgent Systems, AAMAS 2018, pp. 2085–2087 (2018)
Yang, F., Yang, Z., Cohen, W.W.: Differentiable learning of logical rules for knowledge base reasoning. In: Advances in Neural Information Processing Systems, vol. 30, pp. 2319–2328 (2017)
Yang, Y., Song, L.: Learn to explain efficiently via neural logic inductive learning. In: 8th International Conference on Learning Representations, ICLR 2020 (2020)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Ji, B., Li, G., Xiao, G. (2023). Enhancing the Interpretability of Deep Multi-agent Reinforcement Learning via Neural Logic Reasoning. In: Iliadis, L., Papaleonidas, A., Angelov, P., Jayne, C. (eds) Artificial Neural Networks and Machine Learning – ICANN 2023. ICANN 2023. Lecture Notes in Computer Science, vol 14263. Springer, Cham. https://doi.org/10.1007/978-3-031-44204-9_17
Download citation
DOI: https://doi.org/10.1007/978-3-031-44204-9_17
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-44203-2
Online ISBN: 978-3-031-44204-9
eBook Packages: Computer ScienceComputer Science (R0)