Nothing Special   »   [go: up one dir, main page]
IDEAS home Printed from https://ideas.repec.org/a/eee/ejores/v274y2019i2p654-670.html
   My bibliography  Save this article

Reverse sensitivity testing: What does it take to break the model?

Author

Listed:
  • Pesenti, Silvana M.
  • Millossovich, Pietro
  • Tsanakas, Andreas
Abstract
Sensitivity analysis is an important component of model building, interpretation and validation. A model comprises a vector of random input factors, an aggregation function mapping input factors to a random output, and a (baseline) probability measure. A risk measure, such as Value-at-Risk and Expected Shortfall, maps the distribution of the output to the real line. As is common in risk management, the value of the risk measure applied to the output is a decision variable. Therefore, it is of interest to associate a critical increase in the risk measure to specific input factors. We propose a global and model-independent framework, termed ‘reverse sensitivity testing’, comprising three steps: (a) an output stress is specified, corresponding to an increase in the risk measure(s); (b) a (stressed) probability measure is derived, minimising the Kullback–Leibler divergence with respect to the baseline probability, under constraints generated by the output stress; (c) changes in the distributions of input factors are evaluated. We argue that a substantial change in the distribution of an input factor corresponds to high sensitivity to that input and introduce a novel sensitivity measure to formalise this insight. Implementation of reverse sensitivity testing in a Monte Carlo setting can be performed on a single set of input/output scenarios, simulated under the baseline model. Thus the approach circumvents the need for additional computationally expensive evaluations of the aggregation function. We illustrate the proposed approach through numerical examples with a simple insurance portfolio and a model of a London Insurance Market portfolio used in industry.

Suggested Citation

  • Pesenti, Silvana M. & Millossovich, Pietro & Tsanakas, Andreas, 2019. "Reverse sensitivity testing: What does it take to break the model?," European Journal of Operational Research, Elsevier, vol. 274(2), pages 654-670.
    • Handle: RePEc:eee:ejores:v:274:y:2019:i:2:p:654-670
    DOI: 10.1016/j.ejor.2018.10.003
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S037722171830852X
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.ejor.2018.10.003?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Aharon Ben-Tal & Dick den Hertog & Anja De Waegenaere & Bertrand Melenberg & Gijs Rennen, 2013. "Robust Solutions of Optimization Problems Affected by Uncertain Probabilities," Management Science, INFORMS, vol. 59(2), pages 341-357, April.
    2. Paul Glasserman & Zongjian Liu, 2010. "Sensitivity Estimates from Characteristic Functions," Operations Research, INFORMS, vol. 58(6), pages 1611-1623, December.
    3. Guangwu Liu & Liu Jeff Hong, 2009. "Kernel estimation of quantile sensitivities," Naval Research Logistics (NRL), John Wiley & Sons, vol. 56(6), pages 511-525, September.
    4. Acerbi, Carlo, 2002. "Spectral measures of risk: A coherent representation of subjective risk aversion," Journal of Banking & Finance, Elsevier, vol. 26(7), pages 1505-1518, July.
    5. Michael C. Fu & L. Jeff Hong & Jian-Qiang Hu, 2009. "Conditional Monte Carlo Estimation of Quantile Sensitivities," Management Science, INFORMS, vol. 55(12), pages 2019-2027, December.
    6. Emanuele Borgonovo, 2017. "Value of Information," International Series in Operations Research & Management Science, in: Sensitivity Analysis, chapter 0, pages 93-100, Springer.
    7. Mara, Thierry A. & Tarantola, Stefano, 2012. "Variance-based sensitivity indices for models with dependent inputs," Reliability Engineering and System Safety, Elsevier, vol. 107(C), pages 115-121.
    8. Asimit, Alexandru V. & Bignozzi, Valeria & Cheung, Ka Chun & Hu, Junlei & Kim, Eun-Seok, 2017. "Robust and Pareto optimality of insurance contracts," European Journal of Operational Research, Elsevier, vol. 262(2), pages 720-732.
    9. Jeremy E. Oakley & Anthony O'Hagan, 2004. "Probabilistic sensitivity analysis of complex models: a Bayesian approach," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 66(3), pages 751-769, August.
    10. Gourieroux, C. & Laurent, J. P. & Scaillet, O., 2000. "Sensitivity analysis of Values at Risk," Journal of Empirical Finance, Elsevier, vol. 7(3-4), pages 225-245, November.
    11. Borgonovo, E., 2007. "A new uncertainty importance measure," Reliability Engineering and System Safety, Elsevier, vol. 92(6), pages 771-784.
    12. Breuer, Thomas & Csiszár, Imre, 2013. "Systematic stress tests with entropic plausibility constraints," Journal of Banking & Finance, Elsevier, vol. 37(5), pages 1552-1559.
    13. Emanuele Borgonovo & William Castaings & Stefano Tarantola, 2011. "Moment Independent Importance Measures: New Results and Analytical Test Cases," Risk Analysis, John Wiley & Sons, vol. 31(3), pages 404-428, March.
    14. Xu, Chonggang & Gertner, George Zdzislaw, 2008. "A general first-order global sensitivity analysis method," Reliability Engineering and System Safety, Elsevier, vol. 93(7), pages 1060-1071.
    15. Philippe Artzner & Freddy Delbaen & Jean‐Marc Eber & David Heath, 1999. "Coherent Measures of Risk," Mathematical Finance, Wiley Blackwell, vol. 9(3), pages 203-228, July.
    16. Yuhong Xu, 2014. "Robust valuation and risk measurement under model uncertainty," Papers 1407.8024, arXiv.org.
    17. Gotoh, Jun-ya & Takano, Yuichi, 2007. "Newsvendor solutions via conditional value-at-risk minimization," European Journal of Operational Research, Elsevier, vol. 179(1), pages 80-96, May.
    18. Emanuele Borgonovo & Gordon B. Hazen & Elmar Plischke, 2016. "A Common Rationale for Global Sensitivity Measures and Their Estimation," Risk Analysis, John Wiley & Sons, vol. 36(10), pages 1871-1895, October.
    19. L. Jeff Hong, 2009. "Estimating Quantile Sensitivities," Operations Research, INFORMS, vol. 57(1), pages 118-130, February.
    20. Ahmed, Shabbir & Cakmak, Ulas & Shapiro, Alexander, 2007. "Coherent risk measures in inventory problems," European Journal of Operational Research, Elsevier, vol. 182(1), pages 226-238, October.
    21. Tapiero, Charles S., 2005. "Value at risk and inventory control," European Journal of Operational Research, Elsevier, vol. 163(3), pages 769-775, June.
    22. Gerber, Hans U., 1974. "On Additive Premium Calculation Principles," ASTIN Bulletin, Cambridge University Press, vol. 7(3), pages 215-222, March.
    23. Harvey M. Wagner, 1995. "Global Sensitivity Analysis," Operations Research, INFORMS, vol. 43(6), pages 948-969, December.
    24. Breuer, Thomas & Jandačka, Martin & Mencía, Javier & Summer, Martin, 2012. "A systematic approach to multi-period stress testing of portfolio credit risk," Journal of Banking & Finance, Elsevier, vol. 36(2), pages 332-340.
    25. Mathieu Cambou & Damir Filipović, 2017. "Model Uncertainty And Scenario Aggregation," Mathematical Finance, Wiley Blackwell, vol. 27(2), pages 534-567, April.
    26. Andreas Tsanakas & Pietro Millossovich, 2016. "Sensitivity Analysis Using Risk Measures," Risk Analysis, John Wiley & Sons, vol. 36(1), pages 30-48, January.
    27. Emanuele Borgonovo & Alessandra Cillo, 2017. "Deciding with Thresholds: Importance Measures and Value of Information," Risk Analysis, John Wiley & Sons, vol. 37(10), pages 1828-1848, October.
    28. Newey, Whitney K & Powell, James L, 1987. "Asymmetric Least Squares Estimation and Testing," Econometrica, Econometric Society, vol. 55(4), pages 819-847, July.
    29. Aven, T. & Nøkland, T.E., 2010. "On the use of uncertainty importance measures in reliability and risk analysis," Reliability Engineering and System Safety, Elsevier, vol. 95(2), pages 127-133.
    30. Plischke, Elmar & Borgonovo, Emanuele & Smith, Curtis L., 2013. "Global sensitivity measures from given data," European Journal of Operational Research, Elsevier, vol. 226(3), pages 536-550.
    31. Rockafellar, R. Tyrrell & Uryasev, Stanislav, 2002. "Conditional value-at-risk for general loss distributions," Journal of Banking & Finance, Elsevier, vol. 26(7), pages 1443-1471, July.
    32. McNeil, Alexander J. & Smith, Andrew D., 2012. "Multivariate stress scenarios and solvency," Insurance: Mathematics and Economics, Elsevier, vol. 50(3), pages 299-308.
    33. Borgonovo, Emanuele & Plischke, Elmar, 2016. "Sensitivity analysis: A review of recent advances," European Journal of Operational Research, Elsevier, vol. 248(3), pages 869-887.
    34. Bellini, Fabio & Klar, Bernhard & Müller, Alfred & Rosazza Gianin, Emanuela, 2014. "Generalized quantiles as risk measures," Insurance: Mathematics and Economics, Elsevier, vol. 54(C), pages 41-48.
    35. Paul Glasserman & Xingbo Xu, 2014. "Robust risk measurement and model risk," Quantitative Finance, Taylor & Francis Journals, vol. 14(1), pages 29-58, January.
    36. Andrea Saltelli, 2002. "Sensitivity Analysis for Importance Assessment," Risk Analysis, John Wiley & Sons, vol. 22(3), pages 579-590, June.
    37. Alison L. Gibbs & Francis Edward Su, 2002. "On Choosing and Bounding Probability Metrics," International Statistical Review, International Statistical Institute, vol. 70(3), pages 419-435, December.
    38. Hans Föllmer & Alexander Schied, 2002. "Convex measures of risk and trading constraints," Finance and Stochastics, Springer, vol. 6(4), pages 429-447.
    39. Xu, Chonggang & Gertner, George Zdzislaw, 2008. "Uncertainty and sensitivity analysis for models with correlated parameters," Reliability Engineering and System Safety, Elsevier, vol. 93(10), pages 1563-1573.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Makam, Vaishno Devi & Millossovich, Pietro & Tsanakas, Andreas, 2021. "Sensitivity analysis with χ2-divergences," Insurance: Mathematics and Economics, Elsevier, vol. 100(C), pages 372-383.
    2. Fissler, Tobias & Pesenti, Silvana M., 2023. "Sensitivity measures based on scoring functions," European Journal of Operational Research, Elsevier, vol. 307(3), pages 1408-1423.
    3. Silvana M. Pesenti, 2021. "Reverse Sensitivity Analysis for Risk Modelling," Papers 2107.01065, arXiv.org, revised May 2022.
    4. Emma Kroell & Silvana M. Pesenti & Sebastian Jaimungal, 2022. "Stressing Dynamic Loss Models," Papers 2211.03221, arXiv.org, revised Oct 2023.
    5. Sebastian Jaimungal & Silvana M. Pesenti & Leandro S'anchez-Betancourt, 2022. "Minimal Kullback-Leibler Divergence for Constrained L\'evy-It\^o Processes," Papers 2206.14844, arXiv.org, revised Aug 2022.
    6. Borgonovo, Emanuele & Rabitti, Giovanni, 2023. "Screening: From tornado diagrams to effective dimensions," European Journal of Operational Research, Elsevier, vol. 304(3), pages 1200-1211.
    7. Peng Liu & Andreas Tsanakas & Yunran Wei, 2024. "Risk sharing with Lambda value at risk under heterogeneous beliefs," Papers 2408.03147, arXiv.org, revised Sep 2024.
    8. Tobias Fissler & Silvana M. Pesenti, 2022. "Sensitivity Measures Based on Scoring Functions," Papers 2203.00460, arXiv.org, revised Jul 2022.
    9. Aigner, Philipp, 2023. "Identifying scenarios for the own risk and solvency assessment of insurance companies," ICIR Working Paper Series 48/23, Goethe University Frankfurt, International Center for Insurance Regulation (ICIR).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Makam, Vaishno Devi & Millossovich, Pietro & Tsanakas, Andreas, 2021. "Sensitivity analysis with χ2-divergences," Insurance: Mathematics and Economics, Elsevier, vol. 100(C), pages 372-383.
    2. Isadora Antoniano‐Villalobos & Emanuele Borgonovo & Sumeda Siriwardena, 2018. "Which Parameters Are Important? Differential Importance Under Uncertainty," Risk Analysis, John Wiley & Sons, vol. 38(11), pages 2459-2477, November.
    3. Andreas Tsanakas & Pietro Millossovich, 2016. "Sensitivity Analysis Using Risk Measures," Risk Analysis, John Wiley & Sons, vol. 36(1), pages 30-48, January.
    4. Tatsuya Sakurahara & Seyed Reihani & Ernie Kee & Zahra Mohaghegh, 2020. "Global importance measure methodology for integrated probabilistic risk assessment," Journal of Risk and Reliability, , vol. 234(2), pages 377-396, April.
    5. Fissler, Tobias & Pesenti, Silvana M., 2023. "Sensitivity measures based on scoring functions," European Journal of Operational Research, Elsevier, vol. 307(3), pages 1408-1423.
    6. Weiwei Li & Dejian Tian, 2023. "Robust optimized certainty equivalents and quantiles for loss positions with distribution uncertainty," Papers 2304.04396, arXiv.org.
    7. Silvana M. Pesenti, 2021. "Reverse Sensitivity Analysis for Risk Modelling," Papers 2107.01065, arXiv.org, revised May 2022.
    8. Mario Brandtner, 2016. "Spektrale Risikomaße: Konzeption, betriebswirtschaftliche Anwendungen und Fallstricke," Management Review Quarterly, Springer, vol. 66(2), pages 75-115, April.
    9. Zhiping Chen & Qianhui Hu & Ruiyue Lin, 2016. "Performance ratio-based coherent risk measure and its application," Quantitative Finance, Taylor & Francis Journals, vol. 16(5), pages 681-693, May.
    10. Tobias Fissler & Silvana M. Pesenti, 2022. "Sensitivity Measures Based on Scoring Functions," Papers 2203.00460, arXiv.org, revised Jul 2022.
    11. Steven Kou & Xianhua Peng, 2016. "On the Measurement of Economic Tail Risk," Operations Research, INFORMS, vol. 64(5), pages 1056-1072, October.
    12. Katja Schilling & Daniel Bauer & Marcus C. Christiansen & Alexander Kling, 2020. "Decomposing Dynamic Risks into Risk Components," Management Science, INFORMS, vol. 66(12), pages 5738-5756, December.
    13. James Ming Chen, 2018. "On Exactitude in Financial Regulation: Value-at-Risk, Expected Shortfall, and Expectiles," Risks, MDPI, vol. 6(2), pages 1-28, June.
    14. Brandtner, Mario & Kürsten, Wolfgang & Rischau, Robert, 2018. "Entropic risk measures and their comparative statics in portfolio selection: Coherence vs. convexity," European Journal of Operational Research, Elsevier, vol. 264(2), pages 707-716.
    15. Ruodu Wang & Ričardas Zitikis, 2021. "An Axiomatic Foundation for the Expected Shortfall," Management Science, INFORMS, vol. 67(3), pages 1413-1429, March.
    16. Borgonovo, Emanuele & Hazen, Gordon B. & Jose, Victor Richmond R. & Plischke, Elmar, 2021. "Probabilistic sensitivity measures as information value," European Journal of Operational Research, Elsevier, vol. 289(2), pages 595-610.
    17. Amarante, Massimiliano & Ghossoub, Mario, 2021. "Aggregation of opinions and risk measures," Journal of Economic Theory, Elsevier, vol. 196(C).
    18. Fernanda Maria Müller & Marcelo Brutti Righi, 2018. "Numerical comparison of multivariate models to forecasting risk measures," Risk Management, Palgrave Macmillan, vol. 20(1), pages 29-50, February.
    19. S. Cucurachi & E. Borgonovo & R. Heijungs, 2016. "A Protocol for the Global Sensitivity Analysis of Impact Assessment Models in Life Cycle Assessment," Risk Analysis, John Wiley & Sons, vol. 36(2), pages 357-377, February.
    20. Silvana M. Pesenti & Pietro Millossovich & Andreas Tsanakas, 2023. "Differential Quantile-Based Sensitivity in Discontinuous Models," Papers 2310.06151, arXiv.org, revised Oct 2024.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:ejores:v:274:y:2019:i:2:p:654-670. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/eor .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.