Nothing Special   »   [go: up one dir, main page]
IDEAS home Printed from https://ideas.repec.org/a/fip/fedaer/y2003iq2p17-31nv.88no.2.html
   My bibliography  Save this article

Forecast evaluation with cross-sectional data: The Blue Chip Surveys

Author

Listed:
  • Andrew Bauer
  • Robert A. Eisenbeis
  • Daniel F. Waggoner
  • Tao Zha
Abstract
If economic forecasts are to be used for decision making, then being able to evaluate their accuracy is essential. Assessing accuracy using single variables from a forecast is acceptable as a first pass, but this approach has inherent problems. This article addresses some of these problems by evaluating and comparing the general accuracy of a set of multivariate forecasts over time. ; Using the methodology developed in Eisenbeis, Waggoner, and Zha (2002), the authors compare the economic forecasts in the Blue Chip Economic Indicators Survey. The survey, published monthly since 1977, contains forecasts of many macroeconomic variables over a relatively long time span. The forecasters are a mix of economists from major investment banks, corporations, consulting firms, and academic institutions, many of whom have participated in the survey for several years. The survey thus provides a useful set of forecasts to explore the methodologies and to investigate several aspects of forecast performance over time. ; The methodology assigns each forecast a composite score based on the standard theory of probability and statistics. This single number is easy to interpret and can be used to compare forecasts even if the number of variables being forecast, or their definitions, changes over time. ; The analysis shows that the Blue Chip Consensus Forecast, which is the average of the individual forecasts, performs better than any individual forecaster although several forecasters performed almost as well as the consensus.

Suggested Citation

  • Andrew Bauer & Robert A. Eisenbeis & Daniel F. Waggoner & Tao Zha, 2003. "Forecast evaluation with cross-sectional data: The Blue Chip Surveys," Economic Review, Federal Reserve Bank of Atlanta, vol. 88(Q2), pages 17-31.
    • Handle: RePEc:fip:fedaer:y:2003:i:q2:p:17-31:n:v.88no.2
    as

    Download full text from publisher

    File URL: https://www.frbatlanta.org/-/media/documents/research/publications/economic-review/2001/vol86no1_fisher.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Ottaviani, Marco & Sorensen, Peter Norman, 2006. "The strategy of professional forecasting," Journal of Financial Economics, Elsevier, vol. 81(2), pages 441-466, August.
    2. de Menezes, Lilian M. & W. Bunn, Derek & Taylor, James W., 2000. "Review of guidelines for the use of combined forecasts," European Journal of Operational Research, Elsevier, vol. 120(1), pages 190-204, January.
    3. Robert A. Eisenbeis & Daniel F. Waggoner & Tao Zha, 2002. "Evaluating Wall Street Journal survey forecasters: a multivariate approach," FRB Atlanta Working Paper 2002-8, Federal Reserve Bank of Atlanta.
    4. Zarnowitz, Victor & Lambros, Louis A, 1987. "Consensus and Uncertainty in Economic Prediction," Journal of Political Economy, University of Chicago Press, vol. 95(3), pages 591-621, June.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Michael K Andersson & Ted Aranki & André Reslow, 2017. "Adjusting for information content when comparing forecast performance," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 36(7), pages 784-794, November.
    2. Leon W. Berkelmans, 2008. "Imperfect information and monetary models: multiple shocks and their consequences," Finance and Economics Discussion Series 2008-58, Board of Governors of the Federal Reserve System (U.S.).
    3. Dovern, Jonas & Feldkircher, Martin & Huber, Florian, 2016. "Does joint modelling of the world economy pay off? Evaluating global forecasts from a Bayesian GVAR," Journal of Economic Dynamics and Control, Elsevier, vol. 70(C), pages 86-100.
    4. Baghestani, Hamid, 2006. "An evaluation of the professional forecasts of U.S. long-term interest rates," Review of Financial Economics, Elsevier, vol. 15(2), pages 177-191.
    5. Frank A. G. den Butter & Pieter W. Jansen, 2013. "Beating the random walk: a performance assessment of long-term interest rate forecasts," Applied Financial Economics, Taylor & Francis Journals, vol. 23(9), pages 749-765, May.
    6. Kirdan Lees, 2016. "Assessing forecast performance," Reserve Bank of New Zealand Bulletin, Reserve Bank of New Zealand, vol. 79, pages 1-19., June.
    7. Jon Huntley & Eric Miller, 2009. "An Evaluation of CBO Forecasts: Working Paper 2009-02," Working Papers 41195, Congressional Budget Office.
    8. Dovern, Jonas & Feldkircher, Martin & Huber, Florian, 2015. "Does Joint Modelling of the World Economy Pay Off? Evaluating Multivariate Forecasts from a Bayesian GVAR," VfS Annual Conference 2015 (Muenster): Economic Development - Theory and Policy 112999, Verein für Socialpolitik / German Economic Association.
    9. Hans Christian Müller-Dröge & Tara M. Sinclair & H.O. Stekler, 2014. "Evaluating Forecasts of a Vector of Variables: a German Forecasting Competition," CAMA Working Papers 2014-55, Centre for Applied Macroeconomic Analysis, Crawford School of Public Policy, The Australian National University.
    10. Spencer D. Krane, 2006. "How professional forecasters view shocks to GDP," Working Paper Series WP-06-19, Federal Reserve Bank of Chicago.
    11. Berkelmans, Leon, 2011. "Imperfect information, multiple shocks, and policy's signaling role," Journal of Monetary Economics, Elsevier, vol. 58(4), pages 373-386.
    12. Mihaela SIMIONESCU, 2015. "The Evaluation of Global Accuracy of Romanian Inflation Rate Predictions Using Mahalanobis Distance," Management Dynamics in the Knowledge Economy, College of Management, National University of Political Studies and Public Administration, vol. 3(1), pages 133-149, March.
    13. Carvalho, Fabia A. & Minella, André, 2012. "Survey forecasts in Brazil: A prismatic assessment of epidemiology, performance, and determinants," Journal of International Money and Finance, Elsevier, vol. 31(6), pages 1371-1391.
    14. Stefano Eusepi & Richard Crump & Emanuel Moench & Philippe Andrade, 2014. "Noisy Information and Fundamental Disagreement," 2014 Meeting Papers 797, Society for Economic Dynamics.
    15. Andrew Bauer & Robert A. Eisenbeis & Daniel F. Waggoner & Tao Zha, 2006. "Transparency, expectations and forecasts," Economic Review, Federal Reserve Bank of Atlanta, vol. 91(Q 1), pages 1-25.
    16. Andrade, Philippe & Crump, Richard K. & Eusepi, Stefano & Moench, Emanuel, 2016. "Fundamental disagreement," Journal of Monetary Economics, Elsevier, vol. 83(C), pages 106-128.
    17. Olivier Coibion & Yuriy Gorodnichenko, 2015. "Information Rigidity and the Expectations Formation Process: A Simple Framework and New Facts," American Economic Review, American Economic Association, vol. 105(8), pages 2644-2678, August.
    18. Devereux, Michael B. & Smith, Gregor W. & Yetman, James, 2012. "Consumption and real exchange rates in professional forecasts," Journal of International Economics, Elsevier, vol. 86(1), pages 33-42.
    19. Spencer D. Krane, 2011. "Professional Forecasters' View of Permanent and Transitory Shocks to GDP," American Economic Journal: Macroeconomics, American Economic Association, vol. 3(1), pages 184-211, January.
    20. Kézdi, Gábor & Mátyás, László & Balázsi, László & Divényi, János Károly, 2014. "A közgazdasági adatforradalom és a panelökonometria [The revolution in economic data and panel econometrics]," Közgazdasági Szemle (Economic Review - monthly of the Hungarian Academy of Sciences), Közgazdasági Szemle Alapítvány (Economic Review Foundation), vol. 0(11), pages 1319-1340.
    21. Olga Isengildina‐Massa & Berna Karali & Todd H. Kuethe & Ani L. Katchova, 2021. "Joint Evaluation of the System of USDA's Farm Income Forecasts," Applied Economic Perspectives and Policy, John Wiley & Sons, vol. 43(3), pages 1140-1160, September.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Cukierman, Alex & Lustenberger, Thomas, 2017. "International evidence on professional interest rates forecasts: The impact of forecasting ability," CEPR Discussion Papers 12489, C.E.P.R. Discussion Papers.
    2. Michael P. Clements, 2014. "US Inflation Expectations and Heterogeneous Loss Functions, 1968–2010," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 33(1), pages 1-14, January.
    3. Dick, Christian D. & Schmeling, Maik & Schrimpf, Andreas, 2013. "Macro-expectations, aggregate uncertainty, and expected term premia," European Economic Review, Elsevier, vol. 58(C), pages 58-80.
    4. Broer, Tobias & Kohlhas, Alexandre, 2018. "Forecaster (Mis-)Behavior," CEPR Discussion Papers 12898, C.E.P.R. Discussion Papers.
    5. Clements, Michael P., 2006. "Internal consistency of survey respondentsíforecasts: Evidence based on the Survey of Professional Forecasters," Economic Research Papers 269742, University of Warwick - Department of Economics.
    6. Clements, Michael P., 2021. "Do survey joiners and leavers differ from regular participants? The US SPF GDP growth and inflation forecasts," International Journal of Forecasting, Elsevier, vol. 37(2), pages 634-646.
    7. Clements, Michael P., 2010. "Explanations of the inconsistencies in survey respondents' forecasts," European Economic Review, Elsevier, vol. 54(4), pages 536-549, May.
    8. Pilar Poncela & Eva Senra, 2017. "Measuring uncertainty and assessing its predictive power in the euro area," Empirical Economics, Springer, vol. 53(1), pages 165-182, August.
    9. Yoichi Tsuchiya, 2021. "Crises, market shocks, and herding behavior in stock price forecasts," Empirical Economics, Springer, vol. 61(2), pages 919-945, August.
    10. Schanne, Norbert, 2012. "The formation of experts' expectations on labour markets : do they run with the pack?," IAB-Discussion Paper 201225, Institut für Arbeitsmarkt- und Berufsforschung (IAB), Nürnberg [Institute for Employment Research, Nuremberg, Germany].
    11. Rülke, Jan-Christoph & Silgoner, Maria & Wörz, Julia, 2016. "Herding behavior of business cycle forecasters," International Journal of Forecasting, Elsevier, vol. 32(1), pages 23-33.
    12. Jan Christoph Ruelke & Ralf Fendel & Michael Frenkel, 2011. "Do Professional Forecasters Trust in Taylor-Type Rules? - Evidence from the Wall Street Journal Poll," Post-Print hal-00743770, HAL.
    13. Félix, Luiz & Kräussl, Roman & Stork, Philip, 2018. "Predictable biases in macroeconomic forecasts and their impact across asset classes," CFS Working Paper Series 596, Center for Financial Studies (CFS).
    14. Michael Clements, 2016. "Are Macro-Forecasters Essentially The Same? An Analysis of Disagreement, Accuracy and Efficiency," ICMA Centre Discussion Papers in Finance icma-dp2016-08, Henley Business School, University of Reading.
    15. Berna Karali & Scott H. Irwin & Olga Isengildina‐Massa, 2020. "Supply Fundamentals and Grain Futures Price Movements," American Journal of Agricultural Economics, John Wiley & Sons, vol. 102(2), pages 548-568, March.
    16. Thomas M. Fullerton JR., 2001. "Specification of a Borderplex Econometric Forecasting Model," International Regional Science Review, , vol. 24(2), pages 245-260, April.
    17. Aleksei Smirnov & Egor Starkov, 2019. "Timing of predictions in dynamic cheap talk: experts vs. quacks," ECON - Working Papers 334, Department of Economics - University of Zurich.
    18. Davis, George & Kanago, Bryce, 1998. "High and Uncertain Inflation: Results from a New Data Set," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 30(2), pages 218-230, May.
    19. Crowe, Christopher & Meade, Ellen E., 2008. "Central bank independence and transparency: Evolution and effectiveness," European Journal of Political Economy, Elsevier, vol. 24(4), pages 763-777, December.
    20. Engelberg, Joseph & Manski, Charles F. & Williams, Jared, 2009. "Comparing the Point Predictions and Subjective Probability Distributions of Professional Forecasters," Journal of Business & Economic Statistics, American Statistical Association, vol. 27, pages 30-41.

    More about this item

    Keywords

    Forecasting; Economic indicators;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:fip:fedaer:y:2003:i:q2:p:17-31:n:v.88no.2. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Meredith Rector (email available below). General contact details of provider: https://edirc.repec.org/data/frbatus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.