Nothing Special   »   [go: up one dir, main page]
Skip to main content
Springer Nature Link
Log in

Multi-type data fusion framework based on deep reinforcement learning for algorithmic trading

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

In recent years, research on algorithmic trading based on machine learning has been increasing. One challenge faced is getting an accurate representation of the stock market environment from multi-type data. Most existing algorithmic trading studies analyze the stock market based on a relatively single data source. However, with the complicated stock market environment, different types of data reflect the changes in the stock market from different perspectives, and how to obtain the temporal features of different types of data and integrate them to obtain a deeper representation of the stock market environment are still problems to be solved. To tackle these problems, in this study, we combine deep learning and reinforcement learning (RL) and propose a multi-type data fusion framework with deep reinforcement learning (MSF-DRL) that integrates stock data, technical indicators and candlestick charts, in which technical indicators can reduce the impact of noise in stock data. In the process of learning trading strategies under the MSF-DRL framework, the temporal features of stock data and technical indicators are extracted through a long short-term memory (LSTM) network, and a convolutional neural network (CNN) and bidirectional long short-term memory (BiLSTM) are successively used to extract the features of the candlestick chart. The fused features are used as the input of the RL module, which makes trading decisions on this basis. To verify the effectiveness of the MSF-DRL framework, we conducted comparative experiments on datasets composed of Chinese stocks and some stocks of the S&P 500 stock market index. Compared with the other trading strategies, our trading strategy can obtain more profits and a higher Sharpe ratio.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Notes

  1. https://ta-lib.org/

  2. https://finance.yahoo.com/

References

  1. Fister D, Perc M, Jagrič T (2021) Two robust long short-term memory frameworks for trading stocks. Appl Intell 51(10):7177–7195. https://doi.org/10.1007/s10489-021-02249-x

    Article  Google Scholar 

  2. Soleymani F, Paquet E (2020) Financial portfolio optimization with online deep reinforcement learning and restricted stacked autoencoder—DeepBreath. Expert Syst Appl 156:113456. https://doi.org/10.1016/j.eswa.2020.113456

    Article  Google Scholar 

  3. Thakkar A, Chaudhari K (2021) Fusion in stock market prediction: a decade survey on the necessity, recent developments, and potential future directions. Inf Fusion 65:95–107. https://doi.org/10.1016/j.inffus.2020.08.019

    Article  Google Scholar 

  4. Li L, Zhu F, Sun H, et al. (2021) Multi-source information fusion and deep-learning-based characteristics measurement for exploring the effects of peer engagement on stock price synchronicity. Inf Fusion 69:1–21. https://doi.org/10.1016/j.inffus.2020.11.006

    Article  Google Scholar 

  5. Ozbayoglu AM, Gudelek MU, Sezer OB (2020) Deep learning for financial applications: a survey. Appl Soft Comput 93:106384. https://doi.org/10.1016/j.asoc.2020.106384

    Article  Google Scholar 

  6. Du W, Ding S (2021) A survey on multi-agent deep reinforcement learning: from the perspective of challenges and applications. Artif Intell Rev 54:3215–3238. https://doi.org/10.1007/s10462-020-09938-y

    Article  Google Scholar 

  7. Silver D, Hubert T, Schrittwieser J, et al. (2018) A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play. Science 362:1140–1144. https://doi.org/10.1126/science.aar6404

    Article  MathSciNet  MATH  Google Scholar 

  8. Liu S, Cao J, Wang Y, et al. (2021) Self-play reinforcement learning with comprehensive critic in computer games. Neurocomputing 449:207–213. https://doi.org/10.1016/j.neucom.2021.04.006

    Article  Google Scholar 

  9. Zielinski KMC, Hendges LV, Florindo JB, et al. (2021) Flexible control of discrete event systems using environment simulation and reinforcement learning. Appl Soft Comput 111:107714. https://doi.org/10.1016/j.asoc.2021.107714

    Article  Google Scholar 

  10. Apolinarska AA, Pacher M, Li H, et al. (2021) Robotic assembly of timber joints using reinforcement learning. Autom Constr 125:103569. https://doi.org/10.1016/j.autcon.2021.103569

    Article  Google Scholar 

  11. Chakole JB, Kolhe MS, Mahapurush GD, et al. (2021) A Q-learning agent for automated trading in equity stock markets. Expert Syst Appl 163:113761. https://doi.org/10.1016/j.eswa.2020.113761

    Article  Google Scholar 

  12. Arulkumaran K, Deisenroth MP, Brundage M, Bharath AA (2017) Deep reinforcement learning: a brief survey. IEEE Signal Process Mag 34(6):26–38. https://doi.org/10.1109/MSP.2017.2743240

    Article  Google Scholar 

  13. Sezer OB, Ozbayoglu AM (2018) Algorithmic financial trading with deep convolutional neural networks: Time series to image conversion approach. Appl Soft Comput 70:525–538. https://doi.org/10.1016/j.asoc.2018.04.024

    Article  Google Scholar 

  14. Deng Y, Bao F, Kong Y, Ren Z, Dai Q (2017) Deep direct reinforcement learning for financial signal representation and trading. IEEE Trans Neural Netw Learn Syst 28(3):653–664. https://doi.org/10.1109/TNNLS.2016.2522401

    Article  Google Scholar 

  15. Théate T, Ernst D (2021) An application of deep reinforcement learning to algorithmic trading. Expert Syst Appl 173:114632. https://doi.org/10.1016/j.eswa.2021.114632

    Article  Google Scholar 

  16. Lei K, Zhang B, Li Y, et al. (2020) Time-driven feature-aware jointly deep reinforcement learning for financial signal representation and algorithmic trading. Expert Syst Appl 140:112872. https://doi.org/10.1016/j.eswa.2019.112872

    Article  Google Scholar 

  17. Lee J, Koh H, Choe HJ (2021) Learning to trade in financial time series using high-frequency through wavelet transformation and deep reinforcement learning. Appl Intell 51:6202–6223. https://doi.org/10.1007/s10489-021-02218-4

    Article  Google Scholar 

  18. Ma C, Zhang J, Liu J, et al. (2021) A parallel multi-module deep reinforcement learning algorithm for stock trading. Neurocomputing 449:290–302. https://doi.org/10.1016/j.neucom.2021.04.005

    Article  Google Scholar 

  19. Wu X, Chen H, Chen C, et al. (2020) The autonomous navigation and obstacle avoidance for USVs with ANOA deep reinforcement learning method. Knowl-Based Syst 196:105201. https://doi.org/10.1016/j.knosys.2019.105201

    Article  Google Scholar 

  20. Fan G, Hua Z, Li J (2021) Multi-scale depth information fusion network for image dehazing. Appl Intell 51(10):7262–7280. https://doi.org/10.1007/s10489-021-02236-2

    Article  Google Scholar 

  21. Liu M, Yan X, Wang C, Wang K (2021) Segmentation mask-guided person image generation. Appl Intell 51(2):1161–1176. https://doi.org/10.1007/s10489-020-01907-w

    Article  Google Scholar 

  22. Lippi M, Montemurro MA, Degli Esposti M, Cristadoro G (2019) Natural language statistical features of lstm-generated texts. IEEE Transactions on Neural Networks and Learning Systems 30:3326–3337. https://doi.org/10.1109/TNNLS.2019.2890970

    Article  Google Scholar 

  23. Feng Q, He D, Liu Z, Wang H, Choo KR (2020) SecureNLP: A System for Multi-Party Privacy-Preserving Natural Language Processing. IEEE Transactions on Information Forensics and Security 15:3709–3721. https://doi.org/10.1109/TIFS.2020.2997134

    Article  Google Scholar 

  24. Gridach M (2020) A framework based on (probabilistic) soft logic and neural network for NLP. Appl Soft Comput 93:106232. https://doi.org/10.1016/j.asoc.2020.106232

    Article  Google Scholar 

  25. Xie Y, Liang R, Liang Z, et al. (2019) Speech emotion classification using attention-based LSTM. IEEEACM Trans Audio Speech Lang Process 27:1675–1685. https://doi.org/10.1109/TASLP.2019.2925934

    Article  Google Scholar 

  26. Tu YH, Du J, Lee CH (2019) Speech enhancement based on Teacher–Student deep learning using improved speech presence probability for Noise-Robust speech recognition. IEEEACM Trans Audio Speech Lang Process 27:2080–2091. https://doi.org/10.1109/TASLP.2019.2940662

    Article  Google Scholar 

  27. Li S, Xing X, Fan W, et al. (2021) Spatiotemporal and frequential cascaded attention networks for speech emotion recognition. Neurocomputing 448:238–248. https://doi.org/10.1016/j.neucom.2021.02.094

    Article  Google Scholar 

  28. Fischer T, Krauss C (2018) Deep learning with long short-term memory networks for financial market predictions. Eur J Oper Res 270(2):654–669. https://doi.org/10.1016/j.ejor.2017.11.054

    Article  MathSciNet  MATH  Google Scholar 

  29. Baek Y, Kim HY (2018) Modaugnet: A new forecasting framework for stock market index value with an overfitting prevention LSTM module and a prediction LSTM module. Expert Syst Appl 113:457–480. https://doi.org/10.1016/j.eswa.2018.07.019

    Article  Google Scholar 

  30. Zhou F, Zhou H, Yang Z, Yang L (2019) EMD2FNN: A strategy combining empirical mode decomposition and factorization machine based neural network for stock market trend prediction. Expert Syst Appl 115:136–151. https://doi.org/10.1016/j.eswa.2018.07.065

    Article  Google Scholar 

  31. Lu Y, Chen Y, Zhao D, Li D (2021) MGRL: Graph Neural network based inference in a Markov network with reinforcement learning for visual navigation. Neurocomputing 421:140–150. https://doi.org/10.1016/j.neucom.2020.07.091

    Article  Google Scholar 

  32. Wu M-E, Syu J-H, Lin JC-w, Ho J-M (2021) Portfolio management system in equity market neutral using reinforcement learning. Appl Intell

  33. Wu X, Chen H, Wang J, Troiano L, Loia V, Fujita H (2020) Adaptive stock trading strategies with deep reinforcement learning methods. Inf Sci 538:142–158. https://doi.org/10.1016/j.ins.2020.05.066

    Article  MathSciNet  Google Scholar 

  34. Carta S, Corriga A, Ferreira A, Podda AS, Recupero AS (2021) A multi-layer and multi-ensemble stock trader using deep learning and deep reinforcement learning. Appl Intell 51 (2):889–905. https://doi.org/10.1007/s10489-020-01839-5

    Article  Google Scholar 

  35. Shynkevich Y, McGinnity TM, Coleman SA, et al. (2017) Forecasting price movements using technical indicators: Investigating the impact of varying input window length. Neurocomputing 264:71–88. https://doi.org/10.1016/j.neucom.2016.11.095

    Article  Google Scholar 

  36. Lee SH, Levin D, Finley PD, Heilig CM (2019) Chief complaint classification with recurrent neural networks. J Biomed Inform 93:103158. https://doi.org/10.1016/j.jbi.2019.103158

    Article  Google Scholar 

  37. Mammone N, Ieracitano C, Morabito FC (2020) A deep CNN approach to decode motor preparation of upper limbs from time–frequency maps of EEG signals at source level. Neural Netw 124:357–372. https://doi.org/10.1016/j.neunet.2020.01.027

    Article  Google Scholar 

  38. Liu M, Lu Y, Long S, et al. (2021) An attention-based CNN-biLSTM hybrid neural network enhanced with features of discrete wavelet transformation for fetal acidosis classification. Expert Syst Appl 186:115714. https://doi.org/10.1016/j.eswa.2021.115714

    Article  Google Scholar 

  39. Kim HY, Won CH (2018) Forecasting the volatility of stock price index: a hybrid model integrating LSTM with multiple GARCH-type models. Expert Syst Appl 103:25–37. https://doi.org/10.1016/j.eswa.2018.03.002

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China (Grant Nos. 61972227 and 61902217); the Natural Science Foundation of Shandong Province (Grant Nos. ZR2019MF051, ZR2020MF037, ZR2019BF043 and ZR2020MA036); the NSFC-Zhejiang Joint Fund of the Integration of Informatization and Industrialization (Grant Nos. U1909210); Key Research and Development Project of Shandong Province (Grant Nos. 2019GGX101007 and 2019GSF109112); Science and technology plan for young talents in Colleges and universities of Shandong Province(Grant No. 2020KJN007); the Fostering Project of Dominant Discipline and Talent Team of Shandong Province Higher Education Institutions; and The introduction and education plan of young creative talents in Colleges and universities of Shandong Province.

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, Shandong University of Finance and Economics, Jinan, 250014, China

    Peipei Liu & Yunfeng Zhang

  2. School of Mathmatics, Shandong University, Jinan, Shandong, 250100, China

    Fangxun Bao

  3. Faculty of Engineering and Information Technology, University of Technology Sydney, Sydney, NSW 2007, Australia

    Xunxiang Yao

  4. School of Computer Science and Technology, Shandong University, Jinan, 250101, China

    Caiming Zhang

Authors
  1. Peipei Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yunfeng Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Fangxun Bao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xunxiang Yao
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Caiming Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yunfeng Zhang.

Ethics declarations

Conflict of Interests

The authors declare that they have no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, P., Zhang, Y., Bao, F. et al. Multi-type data fusion framework based on deep reinforcement learning for algorithmic trading. Appl Intell 53, 1683–1706 (2023). https://doi.org/10.1007/s10489-022-03321-w

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-022-03321-w

Keywords

Search

Navigation