research-article

Open access

Reinforcement Learning for Intra-and-Inter-Bank Borrowing and Lending Mean Field Control Game

Authors:

Andrea Angiuli,

Jean-Pierre Fouque,

Mathieu Laurière,

Jimin LinAuthors Info & Claims

ICAIF '22: Proceedings of the Third ACM International Conference on AI in Finance

Pages 369 - 376

https://doi.org/10.1145/3533271.3561743

Published: 26 October 2022 Publication History

All formats PDF

Abstract

We propose a mean field control game (MFCG) model for the intra-and-inter-bank borrowing and lending problem. This framework allows to study the competitive game arising between groups of collaborative banks. The solution is provided in terms of an asymptotic Nash equilibrium between the groups in the infinite horizon. A three-timescale reinforcement learning algorithm is applied to learn the optimal borrowing and lending strategy in a data driven way when the model is unknown. An empirical numerical analysis shows the importance of the three-timescale, the impact of the exploration strategy when the model is unknown, and the convergence of the algorithm.

References

[1]

Clémence Alasseur, Imen Ben Taher, and Anis Matoussi. 2020. An extended mean field game for storage in smart grids. Journal of Optimization Theory and Applications 184, 2(2020), 644–670.

Digital Library

[2]

Andrea Angiuli, Nils Detering, Jean-Pierre Fouque, Mathieu Laurière, and Jimin Lin. 2022. Mean Field Control Games. In preparation (2022).

[3]

Andrea Angiuli, Nils Detering, Jean-Pierre Fouque, and Jimin Lin. 2022. Reinforcement Learning Algorithm for Mixed Mean Field Control Games. arXiv preprint arXiv:2205.02330(2022).

[4]

Andrea Angiuli, Jean-Pierre Fouque, and Mathieu Laurière. 2022. Reinforcement Learning for Mean Field Games, with Applications to Economics. To appear in Machine Learning in Financial Markets: A guide to contemporary practices. arXiv:2106.13755(2022).

[5]

Andrea Angiuli, Jean-Pierre Fouque, and Mathieu Laurière. 2022. Unified reinforcement Q-learning for mean field game and control problems. Mathematics of Control, Signals, and Systems (2022), 1–55.

[6]

Alain Bensoussan, Jens Frehse, Phillip Yam, 2013. Mean field games and mean field type control theory. Vol. 101. Springer.

[7]

Pierre Cardaliaguet and Charles-Albert Lehalle. 2018. Mean field game of controls and an application to trade crowding. Mathematics and Financial Economics 12, 3 (2018), 335–363.

[8]

René Carmona. 2020. Applications of mean field games in financial engineering and economic theory. arXiv preprint arXiv:2012.05237(2020).

[9]

René Carmona and François Delarue. 2018. Probabilistic Theory of Mean Field Games with Applications I-II. Springer.

[10]

René Carmona, Jean-Pierre Fouque, and Li-Hsien Sun. 2015. Mean Field Games and Systemic Risk. Communications in Mathematical Sciences 13, 4 (2015), 911–933.

[11]

Patrick Chan and Ronnie Sircar. 2015. Bertrand and Cournot mean field games. Applied Mathematics & Optimization 71, 3 (2015), 533–569.

Digital Library

[12]

Boualem Djehiche, Alain Tcheukam, and Hamidou Tembine. 2016. Mean-field-type games in engineering. arXiv preprint arXiv:1605.03281(2016).

[13]

Romuald Elie, Julien Perolat, Mathieu Laurière, Matthieu Geist, and Olivier Pietquin. 2020. On the Convergence of Model Free Learning in Mean Field Games. In proc. of AAAI.

[14]

Eyal Even-Dar and Yishay Mansour. 2003. Learning rates for Q-learning. Journal of machine learning Research 5, Dec (2003), 1–25.

Digital Library

[15]

Olivier Guéant, Jean-Michel Lasry, and Pierre-Louis Lions. 2011. Mean field games and applications. In Paris-Princeton lectures on mathematical finance 2010. Springer, 205–266.

[16]

Xin Guo, Anran Hu, Renyuan Xu, and Junzi Zhang. 2019. Learning mean-field games. In Advances in Neural Information Processing Systems. 4966–4976.

[17]

Minyi Huang, Roland P Malhamé, and Peter E Caines. 2006. Large population stochastic dynamic games: closed-loop McKean-Vlasov systems and the Nash certainty equivalence principle. Communications in Information & Systems 6, 3 (2006), 221–252.

[18]

Leslie Pack Kaelbling, Michael L Littman, and Andrew W Moore. 1996. Reinforcement learning: A survey. Journal of artificial intelligence research 4 (1996), 237–285.

Digital Library

[19]

Elias Koutsoupias and Christos Papadimitriou. 1999. Worst-case equilibria. In Annual symposium on theoretical aspects of computer science. Springer, 404–413.

[20]

Volodymyr Kuleshov and Doina Precup. 2014. Algorithms for multi-armed bandit problems. arXiv preprint arXiv:1402.6028(2014).

[21]

Jean-Michel Lasry and Pierre-Louis Lions. 2007. Mean field games. Japanese journal of mathematics 2, 1 (2007), 229–260.

[22]

David Mguni, Joel Jennings, and Enrique Munoz de Cote. 2018. Decentralised learning in systems with many, many strategic agents. In Thirty-Second AAAI Conference on Artificial Intelligence.

[23]

Jayakumar Subramanian and Aditya Mahajan. 2019. Reinforcement Learning in Stationary Mean-field Games. In Proceedings. 18th International Conference on Autonomous Agents and Multiagent Systems.

Digital Library

[24]

Sebastian B Thrun. 1992. Efficient exploration in reinforcement learning. (1992).

[25]

Marco Wiering and Jürgen Schmidhuber. 1998. Efficient model-based exploration. In Proceedings of the Sixth International Conference on Simulation of Adaptive Behavior: From Animals to Animats, Vol. 6. MIT Press Cambridge, MA, 223–228.

[26]

Marco A Wiering. 1999. Explorations in efficient reinforcement learning. Ph.D. Dissertation. University of Amsterdam.

Cited By

Cuchiero CReisinger CRigger S(2023)Optimal bailout strategies resulting from the drift controlled supercooled Stefan problemAnnals of Operations Research10.1007/s10479-023-05293-7336:1-2(1315-1349)Online publication date: 29-Apr-2023
https://doi.org/10.1007/s10479-023-05293-7

Index Terms

Reinforcement Learning for Intra-and-Inter-Bank Borrowing and Lending Mean Field Control Game
1. Mathematics of computing
  1. Mathematical analysis
    1. Mathematical optimization
      1. Continuous optimization
        Stochastic control and optimization
2. Theory of computation
  1. Theory and algorithms for application domains
    1. Algorithmic game theory and mechanism design
      1. Solution concepts in game theory
    2. Machine learning theory
      1. Reinforcement learning

Recommendations

Systemic Risk and Interbank Lending

We propose a simple model of the banking system incorporating a game feature, where the evolution of monetary reserve is modeled as a system of coupled Feller diffusions. The optimization reflects the desire of each bank to borrow from or lend to a ...
Understanding Bank-Run Contagion

We study experimental coordination games to examine through which transmission channels and under which information conditions a panic-based depositor run at one bank may trigger a panic-based depositor run at another bank. We find that withdrawals at ...
Dynamic portfolio rebalancing through reinforcement learning
Abstract
Portfolio managements in financial markets involve risk management strategies and opportunistic responses to individual trading behaviours. Optimal portfolios constructed aim to have a minimal risk with highest accompanying investment returns, ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

ICAIF '22: Proceedings of the Third ACM International Conference on AI in Finance

November 2022

527 pages

ISBN:9781450393768

DOI:10.1145/3533271

Editors:
Daniele Magazzeni
J.P. Morgan AI Research
,
Senthil Kumar
Capital One
,
Rahul Savani
University of Liverpool
,
Renyuan Xu
University of Southern California
,
Carmine Ventre
King's College London
,
Blanka Horvath
University of Oxford
,
Ruimeng Hu
University of California Santa Barbara
,
Tucker Balch
J.P. Morgan AI Research
,
Francesca Toni
Imperial College London

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

ACM: Association for Computing Machinery

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 26 October 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Badges

Best Paper

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

NSF

Conference

ICAIF '22

Sponsor:

ACM

ICAIF '22: 3rd ACM International Conference on AI in Finance

November 2 - 4, 2022

NY, New York, USA

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
479
Total Downloads

Downloads (Last 12 months)237
Downloads (Last 6 weeks)46

Reflects downloads up to 22 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Cuchiero CReisinger CRigger S(2023)Optimal bailout strategies resulting from the drift controlled supercooled Stefan problemAnnals of Operations Research10.1007/s10479-023-05293-7336:1-2(1315-1349)Online publication date: 29-Apr-2023
https://doi.org/10.1007/s10479-023-05293-7

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents