research-article

On Bayesian Network Classifiers with Reduced Precision Parameters

Authors:

Sebastian Tschiatschek,

Franz PernkopfAuthors Info & Claims

IEEE Transactions on Pattern Analysis and Machine Intelligence, Volume 37, Issue 4

Pages 774 - 785

https://doi.org/10.1109/TPAMI.2014.2353620

Published: 01 April 2015 Publication History

Abstract

Bayesian network classifier (BNCs) are typically implemented on nowadays desktop computers. However, many real world applications require classifier implementation on embedded or low power systems. Aspects for this purpose have not been studied rigorously. We partly close this gap by analyzing reduced precision implementations of BNCs. In detail, we investigate the quantization of the parameters of BNCs with discrete valued nodes including the implications on the classification rate (CR). We derive worst-case and probabilistic bounds on the CR for different bit-widths. These bounds are evaluated on several benchmark datasets. Furthermore, we compare the classification performance and the robustness of BNCs with generatively and discriminatively optimized parameters, i.e. parameters optimized for high data likelihood and parameters optimized for classification, with respect to parameter quantization. Generatively optimized parameters are more robust for very low bit-widths, i.e. less classifications change because of quantization. However, classification performance is better for discriminatively optimized parameters for all but very low bit-widths. Additionally, we perform analysis for margin-optimized tree augmented network (TAN) structures which outperform generatively optimized TAN structures in terms of CR and robustness.

References

[1]

S. Tschiatschek, P. Reinprecht, M. Mücke, and F. Pernkopf, “Bayesian network classifiers with reduced precision parameters,” in Proc. Eur. Conf. Mach. Learn. Knowl. Discov. Databases, 2012, pp. 74–89.

[2]

S. Tschiatschek, C. E. Cancino Chacón, and F. Pernkopf, “ Bounds for Bayesian network classifiers with reduced precision parameters,” in Proc. Int. Conf. Acoust., Speech, Signal Process., 2013, pp. 3357 –3361.

[3]

N. Friedman, D. Geiger, and M. Goldszmidt, “ Bayesian network classifiers,” Mach. Learn., vol. 29, pp. 131–163, 1997.

[4]

C. Bielza and P. Larrañaga, “Discrete Bayesian network classifiers: A survey,” ACM Comput. Surveys, vol. 47, no. 1, pp. 5:1–5:43, 2014.

[5]

F. Pernkopf, M. Wohlmayr, and S. Tschiatschek, “ Maximum margin Bayesian network classifiers,” IEEE Trans. Pattern Anal. Mach. Intell. , vol. 34, no. 3, pp. 521–531, Mar. 2012.

[6]

D. Husmeier, R. Dybowski, and S. Roberts, Probabilistic Modelling in Bioinformatics and Medical Informatics. New York, NY, USA: Springer, 2004.

[7]

P. Helman, R. Veroff, S. R. Atlas, and C. Willman, “A Bayesian network classification methodology for gene expression data,” Comput. Biol., vol. 11, pp. 581–615, 2004.

[8]

D.-U. Lee, A. Gaffar, R. C. C. Cheung, O. Mencer, W. Luk, and G. Constantinides, “Accuracy-guaranteed bit-width optimization,” IEEE Trans. Comput.-Aided Design Integr. Circuits Syst., vol. 25, no. 10, pp. 1990–2000, Oct. 2006.

[9]

H. Chan and A. Darwiche, “When do numbers really matter?” Artif. Intell. Res., vol. 17, no. 1, pp. 265–287, 2002.

Digital Library

[10]

F. Pernkopf, M. Wohlmayr, and M. Mücke, “Maximum margin structure learning of Bayesian network classifiers,” in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process., 2011, pp. 2076–2079.

[11]

S. Tschiatschek, K. Paul, and F. Pernkopf, “Integer Bayesian network classifiers,” in Proc. Eur. Conf. Mach. Learn. Knowl. Discov. Databases , 2014, pp. 209–224.

[12]

F. Pernkopf and J. Bilmes, “Efficient heuristics for discriminative structure learning of Bayesian network classifiers, ” J. Mach. Learn. Res., vol. 11, pp. 2323– 2360, Aug. 2010.

[13]

R. Peharz and F. Pernkopf, “Exact maximum margin structure learning of Bayesian networks,” in Proc. 29th Int. Conf. Mach. Learn., 2012.

[14]

R. Greiner and W. Zhou, “Structural extension to logistic regression: Discriminative parameter learning of belief net classifiers,” in Proc. Nat. Conf. Artif. Intell., 2002, pp. 167–173.

[15]

T. Roos, H. Wettig, P. Grünwald, P. Myllymäki, and H. Tirri, “On discriminative Bayesian network classifiers and logistic regression,” Mach. Learn., vol. 59, no. 3, pp. 267–296, 2005.

Digital Library

[16]

J. M. Bernardo and A. F. M. Smith, Bayesian Theory, New York, NY, USA: Wiley, 1994.

[17]

D. Heckerman, D. Geiger, and D. Chickering, “ Learning Bayesian networks: The combination of knowledge and statistical data,” Microsoft Res., Redmond, WA, USA, Tech. Rep. MSR-TR-91-05, 1995.

[18]

C. M. Bishop, Pattern Recognition and Machine Learning (Information Science and Statistics) . New York, NY, USA: Springer, 2007.

[19]

J. Pearl, Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference . San Francisco, CA, USA: Morgan Kaufmann, 1988.

[20]

D. Koller and N. Friedman, Probabilistic Graphical Models: Principles and Techniques. Cambridge, MA, USA : MIT Press, 2009.

[21]

I. Kononenko, “Semi-naive Bayesian classifier,” in Proc. Eur. Working Session Learn. Mach. Learn., 1991, pp. 206–219.

[22]

S. Acid, L. M. Campos, and J. G. Castellano, “ Learning Bayesian network classifiers: Searching in a space of partially directed acyclic graphs, ” Mach. Learn., vol. 59, pp. 213–235, 2005.

[23]

F. Pernkopf, R. Peharz, and S. Tschiatschek, “ Introduction to probabilistic graphical models,” in Academic Press Library in Signal Processing, vol. 1, 2014, ch. 18, pp. 989–1064.

[24]

Y. Guo, D. Wilkinson, and D. Schuurmans, “Maximum margin Bayesian networks,” in Proc. Uncertainty Artif. Intell., 2005, pp. 233–242.

[25]

N. Piatkowski, L. Sangkyun, and K. Morik, “The integer approximation of undirected graphical models,” in Proc. Int. Conf. Pattern Recognit. Appl. Methods, 2014, pp. 296–304.

[26]

B. Widrow, I. Kollar, and M. Liu, “Statistical theory of quantization,” IEEE Trans. Instrum. Meas., vol. 45, no. 2, pp. 353–361, Apr. 1996.

[27]

A. Papoulis, Probability, Random Variables, and Stochastic Processes. New York, NY, USA: McGraw-Hill, 1984.

[28]

T. Hastie, R. Tibshirani, and J. Friedman, The Elements of Statistical Learning: Data Mining, Inference, and Prediction. New York, NY, USA : Springer, Aug. 2003.

[29]

Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,” Proc. IEEE, vol. 86, no. 11, pp. 2278–2324, Nov. 1998.

[30]

F. Pernkopf and M. Wohlmayr, “Stochastic margin-based structure learning of Bayesian network classifiers, ” Pattern Recognit., vol. 46, no. 2, pp. 464 –471, 2013.

Digital Library

Cited By

Duan RSun PSatapathy SAgrawal RGarcía Díaz V(2021)Basketball sports neural network model based on nonlinear classificationJournal of Intelligent & Fuzzy Systems: Applications in Engineering and Technology10.3233/JIFS-18943140:4(5917-5926)Online publication date: 1-Jan-2021
https://dl.acm.org/doi/10.3233/JIFS-189431
Pan HYang X(2021)RETRACTED ARTICLE: Intelligent recommendation method integrating knowledge graph and Bayesian networkSoft Computing - A Fusion of Foundations, Methodologies and Applications10.1007/s00500-021-05735-z27:1(483-492)Online publication date: 25-Mar-2021
https://dl.acm.org/doi/10.1007/s00500-021-05735-z
Khasanvis SLi MRahman MBiswas ASalehi-Fashami MAtulasimha JBandyopadhyay SMoritz C(2015)Architecting for Causal Intelligence at NanoscaleComputer10.1109/MC.2015.36748:12(54-64)Online publication date: 1-Dec-2015
https://dl.acm.org/doi/10.1109/MC.2015.367
Show More Cited By

Index Terms

On Bayesian Network Classifiers with Reduced Precision Parameters

Index terms have been assigned to the content through auto-classification.

Recommendations

Bayesian network classifiers with reduced precision parameters
ECMLPKDD'12: Proceedings of the 2012th European Conference on Machine Learning and Knowledge Discovery in Databases - Volume Part I

Bayesian network classifiers (BNCs) are probabilistic classifiers showing good performance in many applications. They consist of a directed acyclic graph and a set of conditional probabilities associated with the nodes of the graph. These conditional ...
Boosted Bayesian network classifiers

The use of Bayesian networks for classification problems has received a significant amount of recent attention. Although computationally efficient, the standard maximum likelihood learning method tends to be suboptimal due to the mismatch between its ...
Bayesian network classifiers versus selective k-NN classifier

In this paper Bayesian network classifiers are compared to the k-nearest neighbor (k-NN) classifier, which is based on a subset of features. This subset is established by means of sequential feature selection methods. Experimental results on classifying ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image IEEE Transactions on Pattern Analysis and Machine Intelligence

IEEE Transactions on Pattern Analysis and Machine Intelligence Volume 37, Issue 4

April 2015

208 pages

ISSN:0162-8828

Issue’s Table of Contents

Copyright © 2014.

Publisher

IEEE Computer Society

United States

Publication History

Published: 01 April 2015

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 26 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Duan RSun PSatapathy SAgrawal RGarcía Díaz V(2021)Basketball sports neural network model based on nonlinear classificationJournal of Intelligent & Fuzzy Systems: Applications in Engineering and Technology10.3233/JIFS-18943140:4(5917-5926)Online publication date: 1-Jan-2021
https://dl.acm.org/doi/10.3233/JIFS-189431
Pan HYang X(2021)RETRACTED ARTICLE: Intelligent recommendation method integrating knowledge graph and Bayesian networkSoft Computing - A Fusion of Foundations, Methodologies and Applications10.1007/s00500-021-05735-z27:1(483-492)Online publication date: 25-Mar-2021
https://dl.acm.org/doi/10.1007/s00500-021-05735-z
Khasanvis SLi MRahman MBiswas ASalehi-Fashami MAtulasimha JBandyopadhyay SMoritz C(2015)Architecting for Causal Intelligence at NanoscaleComputer10.1109/MC.2015.36748:12(54-64)Online publication date: 1-Dec-2015
https://dl.acm.org/doi/10.1109/MC.2015.367
Conte TTrack EDeBenedictis E(2015)Rebooting Computing: New Strategies for Technology ScalingComputer10.1109/MC.2015.36348:12(10-13)Online publication date: 1-Dec-2015
https://dl.acm.org/doi/10.1109/MC.2015.363

View Options

View options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents