research-article

Deep Hedging: Continuous Reinforcement Learning for Hedging of General Portfolios across Multiple Risk Aversions

Authors:

Phillip Murray,

Mikko PakkanenAuthors Info & Claims

ICAIF '22: Proceedings of the Third ACM International Conference on AI in Finance

Pages 361 - 368

https://doi.org/10.1145/3533271.3561731

Published: 26 October 2022 Publication History

Abstract

We present a method for finding optimal hedging policies for arbitrary initial portfolios and market states. We develop a novel actor-critic algorithm for solving general risk-averse stochastic control problems and use it to learn hedging strategies across multiple risk aversion levels simultaneously. We demonstrate the effectiveness of the approach with a numerical example in a stochastic volatility environment.

References

[1]

Leif BG Andersen, Peter Jäckel, and Christian Kahl. 2010. Simulation of square-root processes. Encyclopedia of Quantitative Finance(2010), 1642–1649.

[2]

Sebastian Becker, Patrick Cheridito, and Arnulf Jentzen. 2020. Pricing and hedging American-style options with deep learning. Journal of Risk and Financial Management 13, 7 (2020), 158.

[3]

Lorenzo Bisi. 2022. Algorithms for risk-averse reinforcement learning. (2022).

[4]

Mark Broadie and Özgür Kaya. 2006. Exact simulation of stochastic volatility and other affine jump diffusion processes. Operations research 54, 2 (2006), 217–231.

[5]

H. Buehler, L. Gonon, J. Teichmann, and B. Wood. 2019. Deep hedging. Quantitative Finance(2019), 1–21. https://doi.org/10.1080/14697688.2019.1571683 arXiv:https://doi.org/10.1080/14697688.2019.1571683

[6]

Hans Buehler, Phillip Murray, Mikko S. Pakkanen, and Ben Wood. March 2022. Deep hedging: Learning to remove the drift. Risk (March 2022).

[7]

Yinlam Chow and Mohammad Ghavamzadeh. 2014. Algorithms for CVaR optimization in MDPs. Advances in neural information processing systems 27 (2014).

[8]

Yinlam Chow, Aviv Tamar, Shie Mannor, and Marco Pavone. 2015. Risk-sensitive and robust decision-making: a cvar optimization approach. Advances in neural information processing systems 28 (2015).

[9]

Will Dabney, Mark Rowland, Marc Bellemare, and Rémi Munos. 2018. Distributional reinforcement learning with quantile regression. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 32.

[10]

Oscar Dowson, David P Morton, and Bernardo K Pagnoncelli. 2020. Multistage stochastic programs with the entropic risk measure. Preprint (2020).

[11]

Jiayi Du, Muyang Jin, Petter N Kolm, Gordon Ritter, Yixuan Wang, and Bofei Zhang. 2020. Deep reinforcement learning for option replication and hedging. The Journal of Financial Data Science 2, 4 (2020), 44–57.

[12]

Hans Föllmer and Alexander Schied. 2010. Convex and coherent risk measures. Encyclopedia of Quantitative Finance(2010), 355–363.

[13]

Javier Garcıa and Fernando Fernández. 2015. A comprehensive survey on safe reinforcement learning. Journal of Machine Learning Research 16, 1 (2015), 1437–1480.

Digital Library

[14]

Michael Gimelfarb, André Barreto, Scott Sanner, and Chi-Guhn Lee. 2021. Risk-Aware Transfer in Reinforcement Learning using Successor Features. Advances in Neural Information Processing Systems 34 (2021), 17298–17310.

[15]

Ben Hambly, Renyuan Xu, and Huining Yang. 2021. Recent advances in reinforcement learning in finance. arXiv preprint arXiv:2112.04553(2021).

[16]

Steven L Heston. 1993. A closed-form solution for options with stochastic volatility with applications to bond and currency options. The review of financial studies 6, 2 (1993), 327–343.

[17]

Blanka Horvath, Josef Teichmann, and Žan Žurič. 2021. Deep hedging under rough volatility. Risks 9, 7 (2021), 138.

[18]

Sergey Ioffe and Christian Szegedy. 2015. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International conference on machine learning. PMLR, 448–456.

[19]

Sebastian Jaimungal. 2022. Reinforcement learning and stochastic optimisation. Finance and Stochastics 26, 1 (2022), 103–129.

[20]

Petter N Kolm and Gordon Ritter. 2019. Dynamic replication and hedging: A reinforcement learning approach. The Journal of Financial Data Science 1, 1 (2019), 159–171.

[21]

Michael Kupper and Walter Schachermayer. 2009. Representation results for law invariant time consistent functions. Mathematics and Financial Economics 2, 3 (2009), 189–210.

[22]

Timothy P Lillicrap, Jonathan J Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, and Daan Wierstra. 2015. Continuous control with deep reinforcement learning. arXiv preprint arXiv:1509.02971(2015).

[23]

Richard S. Sutton and Andrew G. Barto. 2018. Reinforcement Learning: An Introduction(2 ed.). MIT Press, Cambridge.

[24]

Aviv Tamar, Yinlam Chow, Mohammad Ghavamzadeh, and Shie Mannor. 2016. Sequential decision making with coherent risk. IEEE transactions on automatic control 62, 7 (2016), 3323–3338.

[25]

Edoardo Vittori, Michele Trapletti, and Marcello Restelli. 2020. Option hedging with risk averse reinforcement learning. In Proceedings of the First ACM International Conference on AI in Finance. 1–8.

Digital Library

[26]

Magnus Wiese, Ben Wood, Alexandre Pachoud, Ralf Korn, Hans Buehler, Phillip Murray, and Lianjun Bai. 2021. Multi-asset spot and option market simulation. arXiv preprint arXiv:2112.06823(2021).

Cited By

Hirano M(2024)Experimental Analysis of Deep Hedging Using Artificial Market Simulations for Underlying Asset SimulatorsSSRN Electronic Journal10.2139/ssrn.4794316Online publication date: 2024
https://doi.org/10.2139/ssrn.4794316
Iuga IMudakkar SDragolea L(2024)Agricultural commodities market reaction to COVID-19Research in International Business and Finance10.1016/j.ribaf.2024.10228769(102287)Online publication date: Apr-2024
https://doi.org/10.1016/j.ribaf.2024.102287
Pickard RLawryshyn Y(2023)Deep Reinforcement Learning for Dynamic Stock Option Hedging: A ReviewMathematics10.3390/math1124494311:24(4943)Online publication date: 13-Dec-2023
https://doi.org/10.3390/math11244943
Show More Cited By

Index Terms

Deep Hedging: Continuous Reinforcement Learning for Hedging of General Portfolios across Multiple Risk Aversions
1. Theory of computation
  1. Theory and algorithms for application domains
    1. Machine learning theory
      1. Reinforcement learning
        Sequential decision making

Recommendations

Option hedging with risk averse reinforcement learning
ICAIF '20: Proceedings of the First ACM International Conference on AI in Finance

In this paper we show how risk-averse reinforcement learning can be used to hedge options. We apply a state-of-the-art risk-averse algorithm: Trust Region Volatility Optimization (TRVO) to a vanilla option hedging environment, considering realistic ...
CVA Hedging with Reinforcement Learning
ICAIF '23: Proceedings of the Fourth ACM International Conference on AI in Finance

This work considers the problem of a trader who must manage the Credit Valuation Adjustment (CVA) of a derivative, defined as the risk-neutral expectation of losses incurred if the counterparty of the derivative defaults. CVA can be regarded as a hybrid ...
Hedging the exchange rate risk for international portfolios
Abstract
This paper studies exchange rate risk hedging with currency options in international portfolios. We propose a new iterative method to estimate the bandwidth of the kernel density estimator (KDE). Based on KDE, we further estimate the ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

ICAIF '22: Proceedings of the Third ACM International Conference on AI in Finance

November 2022

527 pages

ISBN:9781450393768

DOI:10.1145/3533271

Editors:
Daniele Magazzeni
J.P. Morgan AI Research
,
Senthil Kumar
Capital One
,
Rahul Savani
University of Liverpool
,
Renyuan Xu
University of Southern California
,
Carmine Ventre
King's College London
,
Blanka Horvath
University of Oxford
,
Ruimeng Hu
University of California Santa Barbara
,
Tucker Balch
J.P. Morgan AI Research
,
Francesca Toni
Imperial College London

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

ACM: Association for Computing Machinery

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 26 October 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

ICAIF '22

Sponsor:

ACM

ICAIF '22: 3rd ACM International Conference on AI in Finance

November 2 - 4, 2022

NY, New York, USA

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
219
Total Downloads

Downloads (Last 12 months)80
Downloads (Last 6 weeks)6

Reflects downloads up to 22 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Hirano M(2024)Experimental Analysis of Deep Hedging Using Artificial Market Simulations for Underlying Asset SimulatorsSSRN Electronic Journal10.2139/ssrn.4794316Online publication date: 2024
https://doi.org/10.2139/ssrn.4794316
Iuga IMudakkar SDragolea L(2024)Agricultural commodities market reaction to COVID-19Research in International Business and Finance10.1016/j.ribaf.2024.10228769(102287)Online publication date: Apr-2024
https://doi.org/10.1016/j.ribaf.2024.102287
Pickard RLawryshyn Y(2023)Deep Reinforcement Learning for Dynamic Stock Option Hedging: A ReviewMathematics10.3390/math1124494311:24(4943)Online publication date: 13-Dec-2023
https://doi.org/10.3390/math11244943
Englisch HKrabichler TMüller KSchwarz M(2023)Deep treasury management for banksFrontiers in Artificial Intelligence10.3389/frai.2023.11202976Online publication date: 22-Mar-2023
https://doi.org/10.3389/frai.2023.1120297
Cherrat ERaj SKerenidis IShekhar AWood BDee JChakrabarti SChen RHerman DHu SMinssen PShaydulin RSun YYalovetzky RPistoia M(2023)Quantum Deep HedgingQuantum10.22331/q-2023-11-29-11917(1191)Online publication date: 29-Nov-2023
https://doi.org/10.22331/q-2023-11-29-1191
Stoiljkovic Z(2023)Applying Reinforcement Learning to Option Pricing and HedgingSSRN Electronic Journal10.2139/ssrn.4546371Online publication date: 2023
https://doi.org/10.2139/ssrn.4546371
HIRANO MMinami KImajo K(2023)Adversarial Deep Hedging: Learning to Hedge without Price Process ModelingSSRN Electronic Journal10.2139/ssrn.4520273Online publication date: 2023
https://doi.org/10.2139/ssrn.4520273
Sun SWang RAn B(2023)Reinforcement Learning for Quantitative TradingACM Transactions on Intelligent Systems and Technology10.1145/358256014:3(1-29)Online publication date: 24-Mar-2023
https://dl.acm.org/doi/10.1145/3582560

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents