Nothing Special   »   [go: up one dir, main page]
IDEAS home Printed from https://ideas.repec.org/p/ukc/ukcedp/1908.html
   My bibliography  Save this paper

Macroeconomic simulation comparison with a multivariate extension of the Markov Information Criterion

Author

Listed:
  • Sylvain Barde
Abstract
Comparison of macroeconomic simulation models, particularly agent-based models (ABMs), with more traditional approaches such as VAR and DSGE models has long been identified as an important yet problematic issue in the literature. This is due to the fact that many such simulations have been developed following the great recession with a clear aim to inform policy, yet the methodological tools required for validating these models on empirical data are still in their infancy. The paper aims to address this issue by developing and testing a comparison framework for macroeconomic simulation models based on a multivariate extension of the Markov Information Criterion (MIC) originally developed in Barde (2017). The MIC is designed to measure the informational distance between a set of models and some empirical data by mapping the simulated data to the markov transition matrix of the underlying data generating process, and is proven to perform optimally (i.e. the measurement is unbiased in expectation) for all models reducible to a markov process. As a result, not only can the MIC provide an accurate measure of distance solely on the basis of simulated data, but it can do it for a very wide class of data generating processes. The paper first presents the strategies adopted to address the computational challenges that arise from extending the methodology to multivariate settings and validates the extension on VAR and DGSE models. The paper then carries out a comparison of the benchmark ABM of Caiani et al. (2016) and the DGSE framework of Smets and Wouters (2007), which to our knowledge, is the first direct comparison between a macroeconomic ABM and a DGSE model.

Suggested Citation

  • Sylvain Barde, 2019. "Macroeconomic simulation comparison with a multivariate extension of the Markov Information Criterion," Studies in Economics 1908, School of Economics, University of Kent.
    • Handle: RePEc:ukc:ukcedp:1908
    as

    Download full text from publisher

    File URL: https://www.kent.ac.uk/economics/repec/1908.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Barde, Sylvain, 2016. "Direct comparison of agent-based models of herding in financial markets," Journal of Economic Dynamics and Control, Elsevier, vol. 73(C), pages 329-353.
    2. Lamperti, Francesco & Roventini, Andrea & Sani, Amir, 2018. "Agent-based model calibration using machine learning surrogates," Journal of Economic Dynamics and Control, Elsevier, vol. 90(C), pages 366-389.
    3. Marco Del Negro & Frank Schorfheide, 2006. "How good is what you've got? DSGE-VAR as a toolkit for evaluating DSGE models," Economic Review, Federal Reserve Bank of Atlanta, vol. 91(Q 2), pages 21-37.
    4. Popoyan, Lilit & Napoletano, Mauro & Roventini, Andrea, 2017. "Taming macroeconomic instability: Monetary and macro-prudential policy interactions in an agent-based model," Journal of Economic Behavior & Organization, Elsevier, vol. 134(C), pages 117-140.
    5. Lindé, Jesper & Smets, Frank & Wouters, Rafael, 2016. "Challenges for Central Banks´ Macro Models," Working Paper Series 323, Sveriges Riksbank (Central Bank of Sweden).
    6. Francesco Lamperti & Antoine Mandel & Mauro Napoletano & Alessandro Sapio & Andrea Roventini & Tomas Balint & Igor Khorenzhenko, 2017. "Taming macroeconomic instability," PSE-Ecole d'économie de Paris (Postprint) hal-03399574, HAL.
    7. repec:hal:spmain:info:hdl:2441/3qv4spsglp8tmorvev1h0duo4p is not listed on IDEAS
    8. repec:hal:spmain:info:hdl:2441/7kr9gv74ut9ngo58gia97t83i7 is not listed on IDEAS
    9. Giorgio Fagiolo & Andrea Roventini, 2012. "Macroeconomic Policy in DSGE and Agent-Based Models," Revue de l'OFCE, Presses de Sciences-Po, vol. 0(5), pages 67-116.
    10. Del Negro, Marco & Hasegawa, Raiden B. & Schorfheide, Frank, 2016. "Dynamic prediction pools: An investigation of financial frictions and forecasting performance," Journal of Econometrics, Elsevier, vol. 192(2), pages 391-405.
    11. Giorgio Fagiolo & Mattia Guerini & Francesco Lamperti & Alessio Moneta & Andrea Roventini, 2017. "Validation of Agent-Based Models in Economics and Finance," LEM Papers Series 2017/23, Laboratory of Economics and Management (LEM), Sant'Anna School of Advanced Studies, Pisa, Italy.
    12. Herbert Dawid & Philipp Harting & Sander Hoog & Michael Neugart, 2019. "Macroeconomics with heterogeneous agent models: fostering transparency, reproducibility and replication," Journal of Evolutionary Economics, Springer, vol. 29(1), pages 467-538, March.
    13. Dosi, Giovanni & Fagiolo, Giorgio & Napoletano, Mauro & Roventini, Andrea, 2013. "Income distribution, credit and fiscal policies in an agent-based Keynesian model," Journal of Economic Dynamics and Control, Elsevier, vol. 37(8), pages 1598-1625.
    14. Grazzini, Jakob & Richiardi, Matteo G. & Tsionas, Mike, 2017. "Bayesian estimation of agent-based models," Journal of Economic Dynamics and Control, Elsevier, vol. 77(C), pages 26-47.
    15. Peter R. Hansen & Asger Lunde & James M. Nason, 2011. "The Model Confidence Set," Econometrica, Econometric Society, vol. 79(2), pages 453-497, March.
    16. Dosi, Giovanni & Fagiolo, Giorgio & Roventini, Andrea, 2010. "Schumpeter meeting Keynes: A policy-friendly model of endogenous growth and business cycles," Journal of Economic Dynamics and Control, Elsevier, vol. 34(9), pages 1748-1767, September.
    17. Sylvain Barde, 2017. "A Practical, Accurate, Information Criterion for Nth Order Markov Processes," Computational Economics, Springer;Society for Computational Economics, vol. 50(2), pages 281-324, August.
    18. Kukacka, Jiri & Barunik, Jozef, 2017. "Estimation of financial agent-based models with simulated maximum likelihood," Journal of Economic Dynamics and Control, Elsevier, vol. 85(C), pages 21-45.
    19. Dosi, Giovanni & Fagiolo, Giorgio & Napoletano, Mauro & Roventini, Andrea & Treibich, Tania, 2015. "Fiscal and monetary policies in complex evolving economies," Journal of Economic Dynamics and Control, Elsevier, vol. 52(C), pages 166-189.
    20. Giorgio Fagiolo & Andrea Roventini, 2017. "Macroeconomic Policy in DSGE and Agent-Based Models Redux: New Developments and Challenges Ahead," Journal of Artificial Societies and Social Simulation, Journal of Artificial Societies and Social Simulation, vol. 20(1), pages 1-1.
    21. repec:hal:spmain:info:hdl:2441/13thfd12aa8rmplfudlgvgahff is not listed on IDEAS
    22. Christophe Deissenberg & Sander van Der Hoog & Herbert Dawid, 2008. "EURACE: A Massively Parallel Agent-Based Model of the European Economy," Working Papers halshs-00339756, HAL.
    23. Lux, Thomas, 2018. "Estimation of agent-based models using sequential Monte Carlo methods," Journal of Economic Dynamics and Control, Elsevier, vol. 91(C), pages 391-408.
    24. Caiani, Alessandro & Godin, Antoine & Caverzasi, Eugenio & Gallegati, Mauro & Kinsella, Stephen & Stiglitz, Joseph E., 2016. "Agent based-stock flow consistent macroeconomics: Towards a benchmark model," Journal of Economic Dynamics and Control, Elsevier, vol. 69(C), pages 375-408.
    25. repec:hal:spmain:info:hdl:2441/5fafm6me7k8omq5jbo61urqq27 is not listed on IDEAS
    26. Guerini, Mattia & Moneta, Alessio, 2017. "A method for agent-based models validation," Journal of Economic Dynamics and Control, Elsevier, vol. 82(C), pages 125-141.
    27. Negro, Marco Del & Schorfheide, Frank, 2013. "DSGE Model-Based Forecasting," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 57-140, Elsevier.
    28. Ashraf, Quamrul & Gershman, Boris & Howitt, Peter, 2017. "Banks, market organization, and macroeconomic performance: An agent-based computational analysis," Journal of Economic Behavior & Organization, Elsevier, vol. 135(C), pages 143-180.
    29. Simone Alfarano & Thomas Lux & Friedrich Wagner, 2005. "Estimation of Agent-Based Models: The Case of an Asymmetric Herding Model," Computational Economics, Springer;Society for Computational Economics, vol. 26(1), pages 19-49, August.
    30. Brock, William A. & Hommes, Cars H., 1998. "Heterogeneous beliefs and routes to chaos in a simple asset pricing model," Journal of Economic Dynamics and Control, Elsevier, vol. 22(8-9), pages 1235-1274, August.
    31. Gilli, M. & Winker, P., 2003. "A global optimization heuristic for estimating agent based models," Computational Statistics & Data Analysis, Elsevier, vol. 42(3), pages 299-312, March.
    32. Alessandro Caiani & Alberto Russo & Mauro Gallegati, 2019. "Does inequality hamper innovation and growth? An AB-SFC analysis," Journal of Evolutionary Economics, Springer, vol. 29(1), pages 177-228, March.
    33. Frank Smets & Rafael Wouters, 2007. "Shocks and Frictions in US Business Cycles: A Bayesian DSGE Approach," American Economic Review, American Economic Association, vol. 97(3), pages 586-606, June.
    34. Kimball, Miles S, 1995. "The Quantitative Analytics of the Basic Neomonetarist Model," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 27(4), pages 1241-1277, November.
    35. Joeri Schasfoort & Antoine Godin & Dirk Bezemer & Alessandro Caiani & Stephen Kinsella, 2017. "Monetary Policy Transmission In A Macroeconomic Agent-Based Model," Advances in Complex Systems (ACS), World Scientific Publishing Co. Pte. Ltd., vol. 20(08), pages 1-35, December.
    36. Lindé, J. & Smets, F. & Wouters, R., 2016. "Challenges for Central Banks’ Macro Models," Handbook of Macroeconomics, in: J. B. Taylor & Harald Uhlig (ed.), Handbook of Macroeconomics, edition 1, volume 2, chapter 0, pages 2185-2262, Elsevier.
    37. Dawid, Herbert & Gemkow, Simon & Harting, Philipp & Hoog, Sander van der & Neugart, Michael, 2018. "Agent-based macroeconomic modeling and policy analyses: The Eurace@Unibi Model," Publications of Darmstadt Technical University, Institute for Business Studies (BWL) 110863, Darmstadt Technical University, Department of Business Administration, Economics and Law, Institute for Business Studies (BWL).
    38. Grazzini, Jakob & Richiardi, Matteo, 2015. "Estimation of ergodic agent-based models by simulated minimum distance," Journal of Economic Dynamics and Control, Elsevier, vol. 51(C), pages 148-165.
    39. Francesco Lamperti, 2018. "Empirical validation of simulated models through the GSL-div: an illustrative application," Journal of Economic Interaction and Coordination, Springer;Society for Economic Science with Heterogeneous Interacting Agents, vol. 13(1), pages 143-171, April.
    40. Baptista, Rafa & Farmer, J. Doyne & Hinterschweiger, Marc & Low, Katie & Tang, Daniel & Uluc, Arzu, 2016. "Macroprudential policy in an agent-based model of the UK housing market," Bank of England working papers 619, Bank of England.
    41. Lamperti, Francesco, 2018. "An information theoretic criterion for empirical validation of simulation models," Econometrics and Statistics, Elsevier, vol. 5(C), pages 83-106.
    42. Giorgio Fagiolo & Alessio Moneta & Paul Windrum, 2007. "A Critical Guide to Empirical Validation of Agent-Based Models in Economics: Methodologies, Procedures, and Open Problems," Computational Economics, Springer;Society for Computational Economics, vol. 30(3), pages 195-226, October.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Alperen Bektas & Valentino Piana & René Schumann, 2021. "A meso-level empirical validation approach for agent-based computational economic models drawing on micro-data: a use case with a mobility mode-choice model," SN Business & Economics, Springer, vol. 1(6), pages 1-25, June.
    2. Donovan Platt, 2022. "Bayesian Estimation of Economic Simulation Models Using Neural Networks," Computational Economics, Springer;Society for Computational Economics, vol. 59(2), pages 599-650, February.
    3. Dave, Chetan & Sorge, Marco, 2023. "Fat Tailed DSGE Models: A Survey and New Results," Working Papers 2023-3, University of Alberta, Department of Economics.
    4. Barde, Sylvain, 2024. "Bayesian estimation of large-scale simulation models with Gaussian process regression surrogates," Computational Statistics & Data Analysis, Elsevier, vol. 196(C).
    5. Kukacka, Jiri & Sacht, Stephen, 2023. "Estimation of heuristic switching in behavioral macroeconomic models," Journal of Economic Dynamics and Control, Elsevier, vol. 146(C).
    6. Sylvain Barde, 2022. "Bayesian Estimation of Large-Scale Simulation Models with Gaussian Process Regression Surrogates," Studies in Economics 2203, School of Economics, University of Kent.
    7. Dave, Chetan & Sorge, Marco M., 2021. "Equilibrium indeterminacy and sunspot tales," European Economic Review, Elsevier, vol. 140(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Giorgio Fagiolo & Mattia Guerini & Francesco Lamperti & Alessio Moneta & Andrea Roventini, 2017. "Validation of Agent-Based Models in Economics and Finance," LEM Papers Series 2017/23, Laboratory of Economics and Management (LEM), Sant'Anna School of Advanced Studies, Pisa, Italy.
    2. Lamperti, Francesco & Roventini, Andrea & Sani, Amir, 2018. "Agent-based model calibration using machine learning surrogates," Journal of Economic Dynamics and Control, Elsevier, vol. 90(C), pages 366-389.
    3. Donovan Platt, 2022. "Bayesian Estimation of Economic Simulation Models Using Neural Networks," Computational Economics, Springer;Society for Computational Economics, vol. 59(2), pages 599-650, February.
    4. repec:hal:spmain:info:hdl:2441/20hflp7eqn97boh50no50tv67n is not listed on IDEAS
    5. Platt, Donovan, 2020. "A comparison of economic agent-based model calibration methods," Journal of Economic Dynamics and Control, Elsevier, vol. 113(C).
    6. repec:spo:wpmain:info:hdl:2441/20hflp7eqn97boh50no50tv67n is not listed on IDEAS
    7. Giovanni Dosi & Andrea Roventini, 2019. "More is different ... and complex! the case for agent-based macroeconomics," Journal of Evolutionary Economics, Springer, vol. 29(1), pages 1-37, March.
    8. Vandin, Andrea & Giachini, Daniele & Lamperti, Francesco & Chiaromonte, Francesca, 2022. "Automated and distributed statistical analysis of economic agent-based models," Journal of Economic Dynamics and Control, Elsevier, vol. 143(C).
    9. repec:hal:spmain:info:hdl:2441/13thfd12aa8rmplfudlgvgahff is not listed on IDEAS
    10. Lamperti, Francesco & Roventini, Andrea & Sani, Amir, 2018. "Agent-based model calibration using machine learning surrogates," Journal of Economic Dynamics and Control, Elsevier, vol. 90(C), pages 366-389.
    11. Andrea Vandin & Daniele Giachini & Francesco Lamperti & Francesca Chiaromonte, 2021. "Automated and Distributed Statistical Analysis of Economic Agent-Based Models," Papers 2102.05405, arXiv.org, revised Nov 2023.
    12. Lamperti, Francesco & Bosetti, Valentina & Roventini, Andrea & Tavoni, Massimo & Treibich, Tania, 2021. "Three green financial policies to address climate risks," Journal of Financial Stability, Elsevier, vol. 54(C).
    13. Kukacka, Jiri & Kristoufek, Ladislav, 2021. "Does parameterization affect the complexity of agent-based models?," Journal of Economic Behavior & Organization, Elsevier, vol. 192(C), pages 324-356.
    14. Andrea Vandin & Daniele Giachini & Francesco Lamperti & Francesca Chiaromonte, 2020. "Automated and Distributed Statistical Analysis of Economic Agent-Based Models," LEM Papers Series 2020/31, Laboratory of Economics and Management (LEM), Sant'Anna School of Advanced Studies, Pisa, Italy.
    15. Seri, Raffaello & Martinoli, Mario & Secchi, Davide & Centorrino, Samuele, 2021. "Model calibration and validation via confidence sets," Econometrics and Statistics, Elsevier, vol. 20(C), pages 62-86.
    16. Mario Martinoli & Alessio Moneta & Gianluca Pallante, 2022. "Calibration and Validation of Macroeconomic Simulation Models by Statistical Causal Search," LEM Papers Series 2022/33, Laboratory of Economics and Management (LEM), Sant'Anna School of Advanced Studies, Pisa, Italy.
    17. Donovan Platt, 2019. "A Comparison of Economic Agent-Based Model Calibration Methods," Papers 1902.05938, arXiv.org.
    18. Barde, Sylvain, 2024. "Bayesian estimation of large-scale simulation models with Gaussian process regression surrogates," Computational Statistics & Data Analysis, Elsevier, vol. 196(C).
    19. Francesco Lamperti, 2018. "Empirical validation of simulated models through the GSL-div: an illustrative application," Journal of Economic Interaction and Coordination, Springer;Society for Economic Science with Heterogeneous Interacting Agents, vol. 13(1), pages 143-171, April.
    20. Guerini, Mattia & Moneta, Alessio, 2017. "A method for agent-based models validation," Journal of Economic Dynamics and Control, Elsevier, vol. 82(C), pages 125-141.
    21. Lamperti, Francesco, 2018. "An information theoretic criterion for empirical validation of simulation models," Econometrics and Statistics, Elsevier, vol. 5(C), pages 83-106.
    22. Giorgio Fagiolo & Andrea Roventini, 2017. "Macroeconomic Policy in DSGE and Agent-Based Models Redux: New Developments and Challenges Ahead," Journal of Artificial Societies and Social Simulation, Journal of Artificial Societies and Social Simulation, vol. 20(1), pages 1-1.
    23. repec:hal:spmain:info:hdl:2441/dcditnq6282sbu1u151qe5p7f is not listed on IDEAS
    24. repec:spo:wpmain:info:hdl:2441/dcditnq6282sbu1u151qe5p7f is not listed on IDEAS
    25. Zhang, Jinyu & Zhang, Qiaosen & Li, Yong & Wang, Qianchao, 2023. "Sequential Bayesian inference for agent-based models with application to the Chinese business cycle," Economic Modelling, Elsevier, vol. 126(C).

    More about this item

    Keywords

    Model comparison; Agent-based models; Validation methods;
    All these keywords.

    JEL classification:

    • B41 - Schools of Economic Thought and Methodology - - Economic Methodology - - - Economic Methodology
    • C15 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Statistical Simulation Methods: General
    • C52 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Evaluation, Validation, and Selection
    • C63 - Mathematical and Quantitative Methods - - Mathematical Methods; Programming Models; Mathematical and Simulation Modeling - - - Computational Techniques

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ukc:ukcedp:1908. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Dr Anirban Mitra (email available below). General contact details of provider: https://www.kent.ac.uk/economics/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.