Abstract
In recent years, existing studies have sought to enhance the effectiveness of portfolio optimization by modeling asset relations. However, employing conventional graph neural network methodologies for effective aggregation and final representation learning of intricately complex financial information within real-world markets proves challenging. This necessitates the optimization of graph structures to enhance the accuracy of parsing and leveraging financial information. In this paper, we propose an asymmetric graph-based deep reinforcement learning for portfolio optimization. Specifically, leveraging the excellent evaluative capabilities of large language models, we decipher multi-dimensional asymmetric relationships between stocks in multi-dimensional data, constructing asymmetric stock relationship graphs based on news and sectors. We then design a multi-dimensional relationship attention mechanism to jointly represent asymmetric graph information and employ deep reinforcement learning for end-to-end portfolio optimization. Extensive experiments on real datasets from China and the United States have demonstrated the superiority of our method over existing state-of-the-art methods. In the industrial observation conducted at a leading financial technology company, we validated the applicability of our method in real-world market scenarios.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Cheng, D., Yang, F., Wang, X., Zhang, Y., Zhang, L.: Knowledge graph-based event embedding framework for financial quantitative investments. In: SIGIR 2020, pp. 2221–2230. Association for Computing Machinery, New York (2020)
Deng, Y., Bao, F., Kong, Y., Ren, Z., Dai, Q.: Deep direct reinforcement learning for financial signal representation and trading. IEEE Trans. Neural Netw. Learn. Syst. 28(3), 653–664 (2017)
Dimitrios, K., Vasileios, O.: A network analysis of the Greek stock market. Procedia Econ. Finance 33, 340–349 (2015). The Economies of Balkan and Eastern Europe Countries in the Changed World (EBEEC 2015)
Duan, Y., Wang, L., Zhang, Q., Li, J.: FactorVAE: a probabilistic dynamic factor model based on variational autoencoder for predicting cross-sectional stock returns. AAAI 36, 4468–4476 (2022)
Duan, Z., Chen, C., Cheng, D., Liang, Y., Qian, W.: Optimal action space search: an effective deep reinforcement learning method for algorithmic trading. In: Proceedings of the 31st ACM International Conference on Information & Knowledge Management, pp. 406–415 (2022)
Fatouros, G., Soldatos, J., Kouroumali, K., Makridis, G., Kyriazis, D.: Transforming sentiment analysis in the financial domain with chatGPT. Mach. Learn. Appl. 14, 100508 (2023)
Feng, F., Chen, H., He, X., Ding, J., Sun, M., Chua, T.S.: Enhancing stock movement prediction with adversarial training. In: IJCAI (2019)
Han, L., Ding, N., Wang, G., Cheng, D., Liang, Y.: Efficient continuous space policy optimization for high-frequency trading. In: KDD, pp. 4112–4122 (2023)
Jegadeesh, N., Titman, S.: Returns to buying winners and selling losers: implications for stock market efficiency. J. Finance 48, 65–91 (1993)
Li, J., Zhang, Y., Yang, X., Chen, L.: Online portfolio management via deep reinforcement learning with high-frequency data. Inf. Process. Manage. 60(3), 103247 (2023)
Li, W., Bao, R., Harimoto, K., Chen, D., Xu, J., Su, Q.: Modeling the stock relation with graph network for overnight stock movement prediction. In: IJCAI 2020 (2020)
Lin, H., Zhou, D., Liu, W., Bian, J.: Learning multiple stock trading patterns with temporal routing adaptor and optimal transport. In: KDD (2021)
Poterba, J.M., Summers, L.H.: Mean reversion in stock prices: evidence and implications. J. Financ. Econ. 22(1), 27–59 (1988)
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., Klimov, O.: Proximal policy optimization algorithms (2017). ArXiv, abs/1707.06347
Shi, S., Li, J., Li, G., Pan, P., Chen, Q., Sun, Q.: GPM: a graph convolutional network based reinforcement learning framework for portfolio management. Neurocomputing 498, 14–27 (2022)
Soleymani, F., Paquet, E.: Deep graph convolutional reinforcement learning for financial portfolio management-deepPocket. Expert Syst. Appl. 182, 115127 (2021)
Wang, H., Li, S., Wang, T., Zheng, J.: Hierarchical adaptive temporal-relational modeling for stock trend prediction. In: IJCAI (2021)
Wang, J., Zhang, Y., Tang, K., Wu, J., Xiong, Z.: AlphaStock: a buying-winners-and-selling-losers investment strategy using interpretable deep reinforcement attention networks. In: KDD. Association for Computing Machinery (2019)
Wang, Z., Huang, B., Tu, S., Zhang, K., Xu, L.: DeepTrader: a deep reinforcement learning approach for risk-return balanced portfolio management with market conditions embedding. In: AAAI, pp. 643–650 (2021)
Xiang, S., Cheng, D., Shang, C., Zhang, Y., Liang, Y.: Temporal and heterogeneous graph neural network for financial time series prediction. In: CIKM, pp. 3584–3593 (2022)
Xu, K., Zhang, Y., Ye, D., Zhao, P., Tan, M.: Relation-aware transformer for portfolio policy learning. In: Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence. IJCAI (2020)
Xu, W., et al.: HIST: a graph-based framework for stock trend forecasting via mining concept-oriented shared information (2021). ArXiv, abs/2110.13716
Yang, H.W., Zou, Y., Shi, P., Lu, W., Lin, J., Sun, X.: Aligning cross-lingual entities with multi-aspect information, pp. 4431–4441. Association for Computational Linguistics, Hong Kong (2019)
Yang, M., Zheng, X., Liang, Q., Han, B., Zhu, M.: A smart trader for portfolio management based on normalizing flows. In: IJCAI (2022)
Ye, Y., et al.: Reinforcement-learning based portfolio management with augmented asset movement prediction states. In: AAAI, vol. 34, pp. 1112–1119 (2020)
Zeng, Z., Kaur, R., Siddagangappa, S., Rahimi, S., Balch, T., Veloso, M.: Financial time series forecasting using CNN and transformer. In: AAAI (2023)
Zhang, Y., Zhao, P., Wu, Q., Li, B., Huang, J., Tan, M.: Cost-sensitive portfolio selection via deep reinforcement learning. IEEE Trans. Knowl. Data Eng. 34(1), 236–248 (2022)
Zhao, L., Li, W., Bao, R., Harimoto, K., Wu, Y., Sun, X.: Long-term, short-term and sudden event: Trading volume movement prediction with graph-based multi-view modeling. In: IJCAI (2021)
Acknowledgments
This work was supported by the National Key R&D Program of China (Grant no. 2022YFB4501704), the National Natural Science Foundation of China (Grant no. 62102287), and the Shanghai Science and Technology Innovation Action Plan Project (Grant no. 22YS1400600 and 22511100700).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Sun, H., Liu, X., Bian, Y., Zhu, P., Cheng, D., Liang, Y. (2024). Asymmetric Graph-Based Deep Reinforcement Learning for Portfolio Optimization. In: Bifet, A., Krilavičius, T., Miliou, I., Nowaczyk, S. (eds) Machine Learning and Knowledge Discovery in Databases. Applied Data Science Track. ECML PKDD 2024. Lecture Notes in Computer Science(), vol 14949. Springer, Cham. https://doi.org/10.1007/978-3-031-70378-2_11
Download citation
DOI: https://doi.org/10.1007/978-3-031-70378-2_11
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-70377-5
Online ISBN: 978-3-031-70378-2
eBook Packages: Computer ScienceComputer Science (R0)