Pseudo-marginal Markov Chain Monte Carlo for Nonnegative Matrix Factorization
Authors: 
Junfu Du, Mingjun ZhongAuthors Info & Claims
Pages 553 - 562
Abstract
A pseudo-marginal Markov chain Monte Carlo (PMCMC) method is proposed for nonnegative matrix factorization (NMF). The sampler jointly simulates the joint posterior distribution for the nonnegative matrices and the matrix dimensions which indicate the number of the nonnegative components in the NMF model. We show that the PMCMC sampler is a generalization of a version of the reversible jump Markov chain Monte Carlo. An illustrative synthetic data was used to demonstrate the ability of the proposed PMCMC sampler in inferring the nonnegative matrices and as well as the matrix dimensions. The proposed sampler was also applied to a nuclear magnetic resonance spectroscopy data to infer the number of nonnegative components.
References
[1]
Andrieu C, Roberts GO (2009) The pseudo-marginal approach for efficient Monte Carlo computations. Ann Stat 37(2):697---725
[2]
Andrieu C, Doucet A, Holenstein R (2010) Particle Markov chain Monte Carlo methods (with discussion). J R Stat Soc B 72(3):269---342
[3]
Boutsidis C, Zouzias A, Mahoney MW, Drineas P (2015) Randomized dimensionality reduction for-means clustering. IEEE Trans Inf Theory 61(2):1045---1062
[4]
Calderhead B, Girolami M (2009) Estimating Bayes factors via thermodynamic integration and population MCMC. Comput Stat Data Anal 53:4028---4045
[5]
Chib S (1995) Marginal likelihood from the Gibbs output. J Am Stat Assoc 90:1313---1321
[6]
Ding C, Li T, Peng W, Park H (2006) Orthogonal nonnegative matrix t-factorizations for clustering. In: Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining, pp 126---135
[7]
Dunn WB, Bailey N, Johnson HE (2005) Measuring the metabolome: current analytical technologies. Analyst 130:606---625
[8]
Friel N, Pettitt AN (2008) Marginal likelihood estimation via power posteriors. J R Stat Soc B 70:589---607
[9]
Green P (1995) Reversible jump Markov chain Monte Carlo computation and Bayesian model determination. Biometrika 82:711---732
[10]
Liu JS (2001) Monte Carlo strategies in scientific computing. Springer, Berlin
[11]
Liu T, Tao D (2015) Classification with noisy labels by importance reweighting. IEEE Trans Pattern Anal Mach Intell 38:447---461
[12]
Liu T, Tao D (2015) On the performance of manhattan nonnegative matrix factorization. IEEE Trans Neural Netw Learn Syst 99:1
[13]
Manganas L, Zhang X, Li Y, Hazel RD, Smith SD, Wagshul ME, Henn F, Benveniste H, Djuric PM, Enikolopov G, Maletic-Savatic M (2007) Magnetic resonance spectroscopy identifies neural progenitor cells in the live human brain. Science 318:980---985
[14]
Rubin D (1987) Comment on the calculation of posterior distributions by data augmentation by Tanner, M.A. and Wong, W. H. J Am Stat Assoc 82:543
[15]
Schachtner R, Po G, Tomé AM, Puntonet CG, Lang EW et al (2014) A new bayesian approach to nonnegative matrix factorization: uniqueness and model order selection. Neurocomputing 138:142---156
[16]
Schmidt MN, Winther O, Hansen LK (2009) Bayesian non-negative matrix factorization. In: International conference on independent component analysis and signal separation. Lecture Notes in Computer Science (LNCS), vol 5441. Springer, Berlin, pp 540---547
[17]
Sun M, Zhang X et al (2015) A stable approach for model order selection in nonnegative matrix factorization. Pattern Recognit Lett 54:97---102
[18]
Tan V, Fevotte C (2012) Automatic relevance determination in nonnegative matrix factorization with the beta-divergence. IEEE Trans Pattern Anal Mach Intell 35:1592---1605
[19]
Xu C, Tao D, Xu C, Rui Y (2014) Large-margin weakly supervised dimensionality reduction. In: Proceedings of the 31st international conference on machine learning (ICML-14), pp 865---873
[20]
Zhong M, Girolami M (2009) Reversible jump MCMC for non-negative matrix factorization. In: Proceedings of the 12th international conference on artificial intelligence and statistics (AISTATS), FL, pp 663---670
[21]
Zhong M, Girolami M, Faulds K, Graham D (2011) Bayesian methods to detect dye-labelled DNA oligonucleotides in multiplexed Raman spectra. J R Stat Soc C (Appl Stat) 60(2):187---206
- Pseudo-marginal Markov Chain Monte Carlo for Nonnegative Matrix Factorization
Recommendations
Pseudo-marginal Hamiltonian Monte Carlo
Bayesian inference in the presence of an intractable likelihood function is computationally challenging. When following a Markov chain Monte Carlo (MCMC) approach to approximate the posterior distribution in this context, one typically either uses MCMC ...
Marginal reversible jump Markov chain Monte Carlo with application to motor unit number estimation
Motor unit number estimation (MUNE) is a method which aims to provide a quantitative indicator of progression of diseases that lead to a loss of motor units, such as motor neurone disease. However the development of a reliable, repeatable and fast real-...
Analyzing Markov chain Monte Carlo output
AbstractMarkov chain Monte Carlo (MCMC) is a sampling‐based method for estimating features of probability distributions. MCMC methods produce a serially correlated, yet representative, sample from the desired distribution. As such it can be difficult to ...
Visual representation of the multivariate correlation structure in a Markov chain. Output analysis that accounts for this multivariate structure is able to more accurately summarize variability in the simulation process. We review the output analysis ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
Copyright © Copyright © 2017 Springer Science+Business Media New York.
Publisher
Kluwer Academic Publishers
United States
Publication History
Published: 01 April 2017
Author Tags
Qualifiers
- Article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 10 Nov 2024
Other Metrics
Citations
View Options
View options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in