On the distribution of posterior probability in Bayesian inference with a large number of observations

Probabilistic generative models work in many applications of image analysis and speech recognition. In general, there is an observation vector \( \vec y \) and a state vector \( \vec x \), and a joint dependency structure among them. The object of interest is, given \( \vec y \), the most likely configuration \( \vec x_{MAP} \) and its posterior distribution. In practice, the exact value of the posterior probability of \( \vec x_{MAP} \) is hard to obtain, especially when there is a large number of observed variables. Here we analyze the distribution of posterior probabilities of \( \vec x_{MAP} \) when there are N = 200–1000 observations. We used a probabilistic model with simple linear dependency structure in which the exact value of the posterior probability of \( \vec x_{MAP} \) is obtainable. Computer experiments show that even identical models generate a variety of posterior distributions, which suggest difficulties in understanding the meaning of posterior probability. Finally, by computing \( P(\vec x * |\vec y) \)’s where \( \vec x \)*’s are neighbors of \( \vec x_{MAP} \), we propose a method of knowing whether the \( \vec x_{MAP} \) is reliable even when the posterior probability \( P(\vec x_{MAP} |\vec y) \) is small.

Authors and Affiliations

1. Department of Computer Science and Systems Engineering, Faculty of Engineering, University of Miyazaki, 1-1 Gakuen Kibanadai-nishi, Miyazaki, 889-2192, Japan

   Akira Date

Corresponding author

Correspondence to Akira Date.

This work was presented in part at the 13th International Symposium on Artificial Life and Robotics, Oita, Japan, January 31–February 2, 2008