research-article

Efficient MCMC for Binomial Logit Models

Authors:

Sylvia Frühwirth-Schnatter,

Rudolf FrühwirthAuthors Info & Claims

ACM Transactions on Modeling and Computer Simulation (TOMACS), Volume 23, Issue 1

Article No.: 3, Pages 1 - 21

https://doi.org/10.1145/2414416.2414419

Published: 01 January 2013 Publication History

Abstract

This article deals with binomial logit models where the parameters are estimated within a Bayesian framework. Such models arise, for instance, when repeated measurements are available for identical covariate patterns. To perform MCMC sampling, we rewrite the binomial logit model as an augmented model which involves some latent variables called random utilities. It is straightforward, but inefficient, to use the individual random utility model representation based on the binary observations reconstructed from each binomial observation. Alternatively, we present in this article a new method to aggregate the random utilities for each binomial observation. Based on this aggregated representation, we have implemented an independence Metropolis-Hastings sampler, an auxiliary mixture sampler, and a novel hybrid auxiliary mixture sampler. A comparative study on five binomial datasets shows that the new aggregation method leads to a superior sampler in terms of efficiency compared to previously published data augmentation samplers.

References

[1]

Albert, J. H. and Chib, S. 1993. Bayesian analysis of binary and polychotomous response data. J. Amer. Statis. Assn. 88, 669--679.

[2]

Crowder, M. J. 1978. Beta-binomial ANOVA for proportions. Appl. Stat. 27, 34--37.

[3]

Cutler, C. D. 1992. kth nearest neighbors and the generalized logistic distribution. In Handbook of the Logistic Distribution, N. Balakrishnan Ed., Marcel Dekker, New York, 512--522.

[4]

Czado, C. and Raftery, A. E. 2006. Choosing the link function and accounting for link uncertainty in generalized linear models using Bayes factors. Statis. Papers 47, 419--442.

[5]

Czado, C. and Song, P. X.-K. 2008. State space mixed models for longitudinal observations with binary and binomial responses. Statis. Papers 49, 691--714.

[6]

Fahrmeir, L. and Tutz, G. 2001. Multivariate Statistical Modelling based on Generalized Linear Models 2nd Ed. Springer Series in Statistics. Springer, Berlin.

[7]

Frühwirth-Schnatter, S. and Frühwirth, R. 2007. Auxiliary mixture sampling with applications to logistic models. Computat. Statis. Data Anal. 51, 3509--3528.

Digital Library

[8]

Frühwirth-Schnatter, S. and Frühwirth, R. 2010. Data augmentation and MCMC for binary and multinomial logit models. In Statistical Modelling and Regression Structures -- Festschrift in Honour of Ludwig Fahrmeir, T. Kneib and G. Tutz Eds., Physica-Verlag, Heidelberg, 111--132.

[9]

Frühwirth-Schnatter, S. and Frühwirth, R. 2012. Bayesian analysis of the multinomial model. Austrian J. Statis. 41, 27--43.

[10]

Frühwirth-Schnatter, S., Frühwirth, R., Held, L., and Rue, H. 2009. Improved auxiliary mixture sampling for hierarchical models of non-Gaussian data. Statis. Comput. 19, 479--492.

Digital Library

[11]

Frühwirth-Schnatter, S. and Wagner, H. 2006. Auxiliary mixture sampling for parameter-driven models of time series of counts with applications to state space modelling. Biometrika 93, 827--841.

[12]

Geyer, C. 1992. Practical Markov chain Monte Carlo. Statis. Sci. 7, 473--511.

[13]

Gramacy, R. B. and Polson, N. G. 2012. Simulation-based regularized logistic regression. Bayesian Anal. 7, 3, 567--590.

[14]

Hilbe, J. M. 2007. Negative Binomial Regression. Cambridge University Press, Cambridge.

[15]

Holmes, C. C. and Held, L. 2006. Bayesian auxiliary variable models for binary and multinomial regression. Bayesian Anal. 1, 145--168.

[16]

Kass, R. E., Carlin, B., Gelman, A., and Neal, R. 1998. Markov chain Monte Carlo in practice: A roundtable discussion. Amer. Statist. 52, 93--100.

[17]

McFadden, D. 1974. Conditional logit analysis of qualitative choice behaviour. In Frontiers of Econometrics, P. Zarembka Ed., Academic, New York, 105--142.

[18]

McNeil, A. J. and Wendin, J. 2007. Bayesian inference for generalized linear mixed models of portfolio credit risk. J. Empir. Finan. 14, 131--149.

[19]

Park, T. and Casella, G. 2008. The Bayesian Lasso. J. Amer. Statis. Assn. 103, 681--686.

[20]

Scott, S. L. 2011. Data augmentation, frequentist estimation, and the Bayesian analysis of multinomial logit models. Statis. Papers 52, 87--109.

[21]

Shephard, N. 1994. Partial non-Gaussian state space. Biometrika 81, 115--131.

[22]

Tüchler, R. 2008. Bayesian variable selection for logistic models using auxiliary mixture sampling. J. Computat. Graphic. Statis. 17, 76--94.

[23]

Wagner, H. and Duller, C. 2012. Bayesian model selection for logistic regression models with random intercept. Computat. Statis. Data Anal. 56, 1256--1274.

Digital Library

[24]

Zelterman, D. and Balakrishnan, N. 1992. Univariate generalized logistic distributions. In Handbook of the Logistic Distribution, N. Balakrishnan Ed., Marcel Dekker, New York, 209--221.

Cited By

Ma XBrynjarsdóttir JLaFramboise T(2024)A double Pólya-Gamma data augmentation scheme for a hierarchical Negative Binomial - Binomial data modelComputational Statistics & Data Analysis10.1016/j.csda.2024.108009199(108009)Online publication date: Nov-2024
https://doi.org/10.1016/j.csda.2024.108009
Zens GFrühwirth-Schnatter SWagner H(2023)Ultimate Pólya Gamma Samplers–Efficient MCMC for Possibly Imbalanced Binary and Categorical DataJournal of the American Statistical Association10.1080/01621459.2023.2259030(1-12)Online publication date: 20-Sep-2023
https://doi.org/10.1080/01621459.2023.2259030
Holmes JSchofield MBarker R(2022)Pólya-Gamma Data Augmentation and Latent Variable Models for Multivariate Binomial DataJournal of the Royal Statistical Society Series C: Applied Statistics10.1111/rssc.1252871:1(194-218)Online publication date: 21-Jan-2022
https://doi.org/10.1111/rssc.12528
Show More Cited By

Index Terms

Efficient MCMC for Binomial Logit Models
1. Mathematics of computing
  1. Probability and statistics
2. Theory of computation
  1. Theory and algorithms for application domains
    1. Machine learning theory
      1. Markov decision processes

Recommendations

Efficient MCMC estimation of inflated beta regression models

This paper introduces a new and computationally efficient Markov chain Monte Carlo (MCMC) estimation algorithm for the Bayesian analysis of zero, one, and zero and one inflated beta regression models. The algorithm is computationally efficient in the ...
Comparison of MCMC Methods for Estimating Stochastic Volatility Models

This article investigates performances of MCMC methods to estimate stochastic volatility models on simulated and real data. There are two efficient MCMC methods to generate latent volatilities from their full conditional distribution. One is the mixture ...
Coordinate sampler: a non-reversible Gibbs-like MCMC sampler
Abstract
We derive a novel non-reversible, continuous-time Markov chain Monte Carlo sampler, called Coordinate Sampler, based on a piecewise deterministic Markov process, which is a variant of the Zigzag sampler of Bierkens et al. (Ann Stat 47(3):1288–1320,...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Transactions on Modeling and Computer Simulation

ACM Transactions on Modeling and Computer Simulation Volume 23, Issue 1

Special Issue on Monte Carlo Methods in Statistics

January 2013

207 pages

ISSN:1049-3301

EISSN:1558-1195

DOI:10.1145/2414416

Issue’s Table of Contents

Copyright © 2013 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 January 2013

Accepted: 01 June 2012

Revised: 01 April 2012

Received: 01 October 2011

Published in TOMACS Volume 23, Issue 1

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
454
Total Downloads

Downloads (Last 12 months)18
Downloads (Last 6 weeks)3

Reflects downloads up to 19 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Ma XBrynjarsdóttir JLaFramboise T(2024)A double Pólya-Gamma data augmentation scheme for a hierarchical Negative Binomial - Binomial data modelComputational Statistics & Data Analysis10.1016/j.csda.2024.108009199(108009)Online publication date: Nov-2024
https://doi.org/10.1016/j.csda.2024.108009
Zens GFrühwirth-Schnatter SWagner H(2023)Ultimate Pólya Gamma Samplers–Efficient MCMC for Possibly Imbalanced Binary and Categorical DataJournal of the American Statistical Association10.1080/01621459.2023.2259030(1-12)Online publication date: 20-Sep-2023
https://doi.org/10.1080/01621459.2023.2259030
Holmes JSchofield MBarker R(2022)Pólya-Gamma Data Augmentation and Latent Variable Models for Multivariate Binomial DataJournal of the Royal Statistical Society Series C: Applied Statistics10.1111/rssc.1252871:1(194-218)Online publication date: 21-Jan-2022
https://doi.org/10.1111/rssc.12528
Reichl J(2020)Estimating marginal likelihoods from the posterior draws through a geometric identityMonte Carlo Methods and Applications10.1515/mcma-2020-206826:3(205-221)Online publication date: 5-Aug-2020
https://doi.org/10.1515/mcma-2020-2068
Dvorzak MWagner H(2015)Sparse Bayesian modelling of underreported count dataStatistical Modelling10.1177/1471082X1558839816:1(24-46)Online publication date: 18-Jun-2015
https://doi.org/10.1177/1471082X15588398
Polson NScott JWindle J(2013)Bayesian Inference for Logistic Models Using Pólya–Gamma Latent VariablesJournal of the American Statistical Association10.1080/01621459.2013.829001108:504(1339-1349)Online publication date: Dec-2013
https://doi.org/10.1080/01621459.2013.829001
Technow FMelchinger A(2013)Genomic prediction of dichotomous traits with Bayesian logistic modelsTheoretical and Applied Genetics10.1007/s00122-013-2041-9126:4(1133-1143)Online publication date: 6-Feb-2013
https://doi.org/10.1007/s00122-013-2041-9

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents