Nothing Special   »   [go: up one dir, main page]
IDEAS home Printed from https://ideas.repec.org/a/bla/jorssb/v71y2009i2p319-392.html
   My bibliography  Save this article

Approximate Bayesian inference for latent Gaussian models by using integrated nested Laplace approximations

Author

Listed:
  • Håvard Rue
  • Sara Martino
  • Nicolas Chopin
Abstract
Summary. Structured additive regression models are perhaps the most commonly used class of models in statistical applications. It includes, among others, (generalized) linear models, (generalized) additive models, smoothing spline models, state space models, semiparametric regression, spatial and spatiotemporal models, log‐Gaussian Cox processes and geostatistical and geoadditive models. We consider approximate Bayesian inference in a popular subset of structured additive regression models, latent Gaussian models, where the latent field is Gaussian, controlled by a few hyperparameters and with non‐Gaussian response variables. The posterior marginals are not available in closed form owing to the non‐Gaussian response variables. For such models, Markov chain Monte Carlo methods can be implemented, but they are not without problems, in terms of both convergence and computational time. In some practical applications, the extent of these problems is such that Markov chain Monte Carlo sampling is simply not an appropriate tool for routine analysis. We show that, by using an integrated nested Laplace approximation and its simplified version, we can directly compute very accurate approximations to the posterior marginals. The main benefit of these approximations is computational: where Markov chain Monte Carlo algorithms need hours or days to run, our approximations provide more precise estimates in seconds or minutes. Another advantage with our approach is its generality, which makes it possible to perform Bayesian analysis in an automatic, streamlined way, and to compute model comparison criteria and various predictive measures so that models can be compared and the model under study can be challenged.

Suggested Citation

  • Håvard Rue & Sara Martino & Nicolas Chopin, 2009. "Approximate Bayesian inference for latent Gaussian models by using integrated nested Laplace approximations," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 71(2), pages 319-392, April.
    • Handle: RePEc:bla:jorssb:v:71:y:2009:i:2:p:319-392
    DOI: 10.1111/j.1467-9868.2008.00700.x
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/j.1467-9868.2008.00700.x
    Download Restriction: no
    File URL: https://libkey.io/10.1111/j.1467-9868.2008.00700.x?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. J. Durbin & S. J. Koopman, 2000. "Time series analysis of non‐Gaussian observations based on state space models from both classical and Bayesian perspectives," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 62(1), pages 3-56.
    2. Jo Eidsvik & Sara Martino & Håvard Rue, 2009. "Approximate Bayesian Inference in Spatial Generalized Linear Mixed Models," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 36(1), pages 1-22, March.
    3. I. S. Weir & A. N. Pettitt, 2000. "Binary probability maps using a hidden conditional autoregressive Gaussian process with an application to Finnish common toad data," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 49(4), pages 473-484.
    4. Anders Brix & Peter J. Diggle, 2001. "Spatiotemporal prediction for log‐Gaussian Cox processes," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 63(4), pages 823-841.
    5. Thomas Kneib & Ludwig Fahrmeir, 2007. "A Mixed Model Approach for Geoadditive Hazard Regression," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 34(1), pages 207-228, March.
    6. Sudipto Banerjee & Alan E. Gelfand & Andrew O. Finley & Huiyan Sang, 2008. "Gaussian predictive process models for large spatial data sets," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 70(4), pages 825-848, September.
    7. Ole F. Christensen & Gareth O. Roberts & Jeffrey S. Rosenthal, 2005. "Scaling limits for the transient phase of local Metropolis–Hastings algorithms," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 67(2), pages 253-268, April.
    8. Julian Besag & Jeremy York & Annie Mollié, 1991. "Bayesian image restoration, with two applications in spatial statistics," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 43(1), pages 1-20, March.
    9. Fuentes, Montserrat, 2007. "Approximate Likelihood for Large Irregularly Spaced Spatial Data," Journal of the American Statistical Association, American Statistical Association, vol. 102, pages 321-331, March.
    10. Harvey, Andrew C & Shephard, Neil, 1996. "Estimation of an Asymmetric Stochastic Volatility Model for Asset Returns," Journal of Business & Economic Statistics, American Statistical Association, vol. 14(4), pages 429-434, October.
    11. Hannig, J. & Marron, J.S., 2006. "Advanced Distribution Theory for SiZer," Journal of the American Statistical Association, American Statistical Association, vol. 101, pages 484-499, June.
    12. Youngjo Lee & John A. Nelder, 2006. "Double hierarchical generalized linear models (with discussion)," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 55(2), pages 139-185, April.
    13. Sangjoon Kim & Neil Shephard & Siddhartha Chib, 1998. "Stochastic Volatility: Likelihood Inference and Comparison with ARCH Models," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 65(3), pages 361-393.
    14. Rasmus Plenge Waagepetersen, 2007. "An Estimating Function Approach to Inference for Inhomogeneous Neyman–Scott Processes," Biometrics, The International Biometric Society, vol. 63(1), pages 252-258, March.
    15. Borus Jungbacker & Siem Jan Koopman, 2007. "Monte Carlo Estimation for Nonlinear Non-Gaussian State Space Models," Biometrika, Biometrika Trust, vol. 94(4), pages 827-839.
    16. Håvard Rue, 2001. "Fast sampling of Gaussian Markov random fields," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 63(2), pages 325-338.
    17. Tilmann Gneiting & Larissa Stanberry & Eric Grimit & Leonhard Held & Nicholas Johnson, 2008. "Rejoinder on: Assessing probabilistic forecasts of multivariate quantities, with an application to ensemble predictions of surface winds," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 17(2), pages 256-264, August.
    18. Leonhard Knorr-Held & Günter Raßer & Nikolaus Becker, 2002. "Disease Mapping of Stage-Specific Cancer Incidence Data," Biometrics, The International Biometric Society, vol. 58(3), pages 492-501, September.
    19. Michael L. Stein & Zhiyi Chi & Leah J. Welty, 2004. "Approximating likelihoods for large spatial data sets," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 66(2), pages 275-296, May.
    20. Noh, Maengseok & Lee, Youngjo, 2007. "Robust Modeling for Inference From Generalized Linear Model Classes," Journal of the American Statistical Association, American Statistical Association, vol. 102, pages 1059-1072, September.
    21. Noh, Maengseok & Lee, Youngjo, 2007. "REML estimation for binary data in GLMMs," Journal of Multivariate Analysis, Elsevier, vol. 98(5), pages 896-915, May.
    22. A. Azzalini & A. Capitanio, 1999. "Statistical applications of the multivariate skew normal distribution," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 61(3), pages 579-602.
    23. Tilmann Gneiting & Larissa Stanberry & Eric Grimit & Leonhard Held & Nicholas Johnson, 2008. "Assessing probabilistic forecasts of multivariate quantities, with an application to ensemble predictions of surface winds," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 17(2), pages 211-235, August.
    24. Kauermann, Goran, 2005. "Penalized spline smoothing in multivariable survival models with varying coefficients," Computational Statistics & Data Analysis, Elsevier, vol. 49(1), pages 169-186, April.
    25. Bollerslev, Tim, 1986. "Generalized autoregressive conditional heteroskedasticity," Journal of Econometrics, Elsevier, vol. 31(3), pages 307-327, April.
    26. Ruppert,David & Wand,M. P. & Carroll,R. J., 2003. "Semiparametric Regression," Cambridge Books, Cambridge University Press, number 9780521785167, October.
    27. Thomas Kneib & Ludwig Fahrmeir, 2006. "Structured Additive Regression for Categorical Space–Time Data: A Mixed Model Approach," Biometrics, The International Biometric Society, vol. 62(1), pages 109-118, March.
    28. Alexandros Beskos & Omiros Papaspiliopoulos & Gareth O. Roberts & Paul Fearnhead, 2006. "Exact and computationally efficient likelihood‐based estimation for discretely observed diffusion processes (with discussion)," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 68(3), pages 333-382, June.
    29. Ludwig Fahrmeir & Stefan Lang, 2001. "Bayesian inference for generalized additive mixed models based on Markov random field priors," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 50(2), pages 201-220.
    30. Reich, Brian J. & Hodges, James S. & Carlin, Bradley P., 2007. "Spatial Analyses of Periodontal Data Using Conditionally Autoregressive Priors Having Two Classes of Neighbor Relations," Journal of the American Statistical Association, American Statistical Association, vol. 102, pages 44-55, March.
    31. Noel Cressie & Gardar Johannesson, 2008. "Fixed rank kriging for very large spatial data sets," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 70(1), pages 209-226, February.
    32. Gneiting, Tilmann & Raftery, Adrian E., 2007. "Strictly Proper Scoring Rules, Prediction, and Estimation," Journal of the American Statistical Association, American Statistical Association, vol. 102, pages 359-378, March.
    33. Anders Brix & Jesper Moller, 2001. "Space‐time Multi Type Log Gaussian Cox Processes with a View to Modelling Weeds," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 28(3), pages 471-488, September.
    34. McCAUSLAND, William, 2008. "The Hessian Method (Highly Efficient State Smoothing, In a Nutshell)," Cahiers de recherche 2008-03, Universite de Montreal, Departement de sciences economiques.
    35. Song, Peter X.K. & Fan, Yanqin & Kalbfleisch, John D., 2005. "Maximization by Parts in Likelihood Inference," Journal of the American Statistical Association, American Statistical Association, vol. 100, pages 1145-1158, December.
    36. Skaug, Hans J. & Fournier, David A., 2006. "Automatic approximation of the marginal likelihood in non-Gaussian hierarchical models," Computational Statistics & Data Analysis, Elsevier, vol. 51(2), pages 699-709, November.
    37. Chib, Siddhartha & Nardari, Federico & Shephard, Neil, 2002. "Markov chain Monte Carlo methods for stochastic volatility models," Journal of Econometrics, Elsevier, vol. 108(2), pages 281-316, June.
    38. Nicholas G. Polson & Jonathan R. Stroud & Peter Müller, 2008. "Practical filtering with sequential parameter learning," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 70(2), pages 413-428, April.
    39. Leonhard Knorr‐Held & Håvard Rue, 2002. "On Block Updating in Markov Random Field Models for Disease Mapping," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 29(4), pages 597-614, December.
    40. Jiannong Liu & James S. Hodges, 2003. "Posterior bimodality in the balanced one‐way random‐effects model," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 65(1), pages 247-255, February.
    41. Ainsworth, L.M. & Dean, C.B., 2006. "Approximate inference for disease mapping," Computational Statistics & Data Analysis, Elsevier, vol. 50(10), pages 2552-2570, June.
    42. R. A. Rigby & D. M. Stasinopoulos, 2005. "Generalized additive models for location, scale and shape," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 54(3), pages 507-554, June.
    43. Ruppert,David & Wand,M. P. & Carroll,R. J., 2003. "Semiparametric Regression," Cambridge Books, Cambridge University Press, number 9780521780506, October.
    44. Sahu, Sujit K. & Gelfand, Alan E. & Holland, David M., 2007. "High-Resolution SpaceTime Ozone Modeling for Assessing Trends," Journal of the American Statistical Association, American Statistical Association, vol. 102, pages 1221-1234, December.
    45. A. Brezger & L. Fahrmeir & A. Hennerfeind, 2007. "Adaptive Gaussian Markov random fields with applications in human brain mapping," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 56(3), pages 327-345, May.
    46. Leonhard Knorr‐Held, 1999. "Conditional Prior Proposals in Dynamic Models," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 26(1), pages 129-144, March.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Schmidt, Paul & Mühlau, Mark & Schmid, Volker, 2017. "Fitting large-scale structured additive regression models using Krylov subspace methods," Computational Statistics & Data Analysis, Elsevier, vol. 105(C), pages 59-75.
    2. Brezger, Andreas & Lang, Stefan, 2006. "Generalized structured additive regression based on Bayesian P-splines," Computational Statistics & Data Analysis, Elsevier, vol. 50(4), pages 967-991, February.
    3. Håvard Rue & Ingelin Steinsland & Sveinung Erland, 2004. "Approximating hidden Gaussian Markov random fields," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 66(4), pages 877-892, November.
    4. Eidsvik, Jo & Finley, Andrew O. & Banerjee, Sudipto & Rue, Håvard, 2012. "Approximate Bayesian inference for large spatial datasets using predictive process models," Computational Statistics & Data Analysis, Elsevier, vol. 56(6), pages 1362-1380.
    5. Roland Langrock & Théo Michelot & Alexander Sohn & Thomas Kneib, 2015. "Semiparametric stochastic volatility modelling using penalized splines," Computational Statistics, Springer, vol. 30(2), pages 517-537, June.
    6. Lee, Woojoo & Lim, Johan & Lee, Youngjo & del Castillo, Joan, 2011. "The hierarchical-likelihood approach to autoregressive stochastic volatility models," Computational Statistics & Data Analysis, Elsevier, vol. 55(1), pages 248-260, January.
    7. Mahdi Hosseinpouri & Majid Jafari Khaledi, 2019. "An area-specific stick breaking process for spatial data," Statistical Papers, Springer, vol. 60(1), pages 199-221, February.
    8. Congdon, Peter, 2006. "A model for non-parametric spatially varying regression effects," Computational Statistics & Data Analysis, Elsevier, vol. 50(2), pages 422-445, January.
    9. Sudipto Banerjee & Alan E. Gelfand & Andrew O. Finley & Huiyan Sang, 2008. "Gaussian predictive process models for large spatial data sets," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 70(4), pages 825-848, September.
    10. Nadja Klein & Michel Denuit & Stefan Lang & Thomas Kneib, 2013. "Nonlife Ratemaking and Risk Management with Bayesian Additive Models for Location, Scale and Shape," Working Papers 2013-24, Faculty of Economics and Statistics, Universität Innsbruck.
    11. Petropoulos, Fotios & Apiletti, Daniele & Assimakopoulos, Vassilios & Babai, Mohamed Zied & Barrow, Devon K. & Ben Taieb, Souhaib & Bergmeir, Christoph & Bessa, Ricardo J. & Bijak, Jakub & Boylan, Joh, 2022. "Forecasting: theory and practice," International Journal of Forecasting, Elsevier, vol. 38(3), pages 705-871.
      • Fotios Petropoulos & Daniele Apiletti & Vassilios Assimakopoulos & Mohamed Zied Babai & Devon K. Barrow & Souhaib Ben Taieb & Christoph Bergmeir & Ricardo J. Bessa & Jakub Bijak & John E. Boylan & Jet, 2020. "Forecasting: theory and practice," Papers 2012.03854, arXiv.org, revised Jan 2022.
    12. Vinicius Mayrink & Dani Gamerman, 2009. "On computational aspects of Bayesian spatial models: influence of the neighboring structure in the efficiency of MCMC algorithms," Computational Statistics, Springer, vol. 24(4), pages 641-669, December.
    13. Philipp Otto & Osman Dou{g}an & Suleyman Tac{s}p{i}nar & Wolfgang Schmid & Anil K. Bera, 2023. "Spatial and Spatiotemporal Volatility Models: A Review," Papers 2308.13061, arXiv.org.
    14. Hossein Boojari & Majid Khaledi & Firoozeh Rivaz, 2016. "A non-homogeneous skew-Gaussian Bayesian spatial model," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 25(1), pages 55-73, March.
    15. Adrian W. Bowman & Marco Giannitrapani & E. Marian Scott, 2009. "Spatiotemporal smoothing and sulphur dioxide trends over Europe," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 58(5), pages 737-752, December.
    16. Klein, Nadja & Denuit, Michel & Lang, Stefan & Kneib, Thomas, 2013. "Nonlife Ratemaking and Risk Management with Bayesian Additive Models for Location, Scale and Shape," LIDAM Discussion Papers ISBA 2013045, Université catholique de Louvain, Institute of Statistics, Biostatistics and Actuarial Sciences (ISBA).
    17. McCausland, William J. & Miller, Shirley & Pelletier, Denis, 2011. "Simulation smoothing for state-space models: A computational efficiency analysis," Computational Statistics & Data Analysis, Elsevier, vol. 55(1), pages 199-212, January.
    18. Yu, Jun, 2012. "A semiparametric stochastic volatility model," Journal of Econometrics, Elsevier, vol. 167(2), pages 473-482.
    19. Bernhard Baumgartner & Daniel Guhl & Thomas Kneib & Winfried J. Steiner, 2018. "Flexible estimation of time-varying effects for frequently purchased retail goods: a modeling approach based on household panel data," OR Spectrum: Quantitative Approaches in Management, Springer;Gesellschaft für Operations Research e.V., vol. 40(4), pages 837-873, October.
    20. Simon N. Wood & Natalya Pya & Benjamin Säfken, 2016. "Smoothing Parameter and Model Selection for General Smooth Models," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 111(516), pages 1548-1563, October.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:jorssb:v:71:y:2009:i:2:p:319-392. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://edirc.repec.org/data/rssssea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.