Abstract
Belief propagation (BP) is effective for computing marginal probabilities of a high dimensional probability distribution. Loopy belief propagation (LBP) is known not to compute precise marginal probabilities and not to guarantee its convergence. The fixed points of LBP are known to accord with the extrema of Bethe free energy. Hence, the fixed points are analyzed by minimizing the Bethe free energy.
In this paper, we consider the Bethe free energy in Gaussian distributions and analytically clarify the extrema, equivalently, the fixed points of LBP for some particular cases. The analytical results tell us a necessary condition for LBP convergence and the quantities which determine the accuracy of LBP in Gaussian distributions. Based on the analytical results, we perform numerical experiments of LBP and compare the results with analytical solutions.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Heskes, T.: On the uniqueness of loopy belief propagation fixed points. Neural Computation 16(11), 2379–2414 (2004)
Weiss, Y.: Belief propagation and revision in networks with loops. Technical report 1616, MIT AI lab (1997)
Weiss, Y.: Correctness of local probability propagation in graphical models with loops. Neural Computation 12(1), 1–41 (2000)
Weiss, Y., Freeman, W.: Correctness of belief propagation in gaussian graphical models of arbitrary topology. Neural Computation 13(10), 2173–2200 (2001)
Ikeda, S., Tanaka, T., Amari, S.: Stochastic reasoning, free energy, and information geometry. Neural Computation 16(9), 1779–1810 (2004)
Ikeda, S., Tanaka, T., Amari, S.: Information geometry of turbo and low-density parity-check codes. IEEE Trans. Inf. Theory 50(6), 1097–1114 (2004)
Kabashima, Y., Saad, D.: The TAP approach to intensive and extensive connectivity systems. In: Opper, M., Saad, D. (eds.) Advanced Mean Field Methods -Theory and Practice, pp. 65–84. MIT Press, Cambridge (2001)
Yedidia, J., Freeman, W., Weiss, Y.: Bethe free energy, kikuchi approximations, and belief propagation algorithms. Technical Report TR2001-16, Mitsubishi Electric Research Laboratories (2001)
Tanaka, K., Shouno, H., Okada, M.: Accuracy of the bethe approximation for hyperparameter estimation in probabilistic image processing. J.Phys. A, Math. Gen. 37(36), 8675–8696 (2004)
Tanaka, K., Inoue, J., Titterington, D.M.: Loopy belief propagation and probabilistic image processing. In: proceedings of 2003 IEEE International workshop on Neural Networks for Signal Processing, vol. 13, pp. 383–392 (2003)
Nishiyama, Y., Watanabe, S.: Theoretical Analysis of Accuracy of Belief Propagation in Gaussian Models. IEICE Technical Report 107(50), 23–28 (2007)
Nishiyama, Y., Watanabe, S.: Theoretical Analysis of Accuracy of Gaussian Belief Propagation. In: Proceedings of International Conference on Artificial Neural Networks 2007, pp. 29–38 (2007)
Yuille, A.L.: CCCP algorithms to minimize the Bethe and Kikuchi free energies: convergent alternatives to belief propagation. Neural Computation 14(7), 1691–1722 (2002)
Tanaka, K.: Generalized belief propagation formula in probabilistic information processing based on gaussian graphical model. IEICE D-II, Vol. J88-D-II, No. 12, pp. 2368–2379 (in Japanese) (2005)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Nishiyama, Y., Watanabe, S. (2008). On the Minima of Bethe Free Energy in Gaussian Distributions. In: Rutkowski, L., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds) Artificial Intelligence and Soft Computing – ICAISC 2008. ICAISC 2008. Lecture Notes in Computer Science(), vol 5097. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-69731-2_101
Download citation
DOI: https://doi.org/10.1007/978-3-540-69731-2_101
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-69572-1
Online ISBN: 978-3-540-69731-2
eBook Packages: Computer ScienceComputer Science (R0)