Abstract
Inductive Logic Programming (ILP) is a popular approach for learning rules for classification tasks. An important question is how to combine the individual rules to obtain a useful classifier. In some instances, converting each learned rule into a binary feature for a Bayes net learner improves the accuracy compared to the standard decision list approach [3,4,14]. This results in a two-step process, where rules are generated in the first phase, and the classifier is learned in the second phase. We propose an algorithm that interleaves the two steps, by incrementally building a Bayes net during rule learning. Each candidate rule is introduced into the network, and scored by whether it improves the performance of the classifier. We call the algorithm SAYU for Score As You Use. We evaluate two structure learning algorithms Naïve Bayes and Tree Augmented Naïve Bayes. We test SAYU on four different datasets and see a significant improvement in two out of the four applications. Furthermore, the theories that SAYU learns tend to consist of far fewer rules than the theories in the two-step approach.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Alphonse, E., Rouveirol, C.: Lazy propositionalisation for relational learning. In: Horn, W. (ed.) ECAI 2000, Berlin, Allemagne, pp. 256–260. IOS Press, Amsterdam (2000)
Davis, J., Burnside, E., Dutra, I.C., Page, D., Ramakrishnan, R., Santos Costa, V., Shavlik, J.: View learning for statistical relational learning: With an application to mammography. In: IJCAI 2005, Edinburgh, Scotland (2005)
Davis, J., Dutra, I.C., Page, D., Santos Costa, V.: Establishing Entity Equivalence in Multi-Relation Domains. In: International Conference on Intelligence Analysis, Vienna, Va (May 2005)
Davis, J., Santos Costa, V., Ong, I.M., Page, D., Dutra, I.C.: Using Bayesian Classifiers to Combine Rules. In: 3rd MRDM, Seattle, USA (August 2004)
Dutra, I.C., Page, D., Shavlik, J., Santos Costa, V.: An empirical evaluation of bagging in inductive logic programming, September 2002, pp. 48–65 (2002)
Friedman, N., Geiger, D., Goldszmidt, M.: Bayesian networks classifiers. Machine Learning 29, 131–163 (1997)
Goadrich, M., Oliphant, L., Shavlik, J.: Learning Ensembles of First-Order Clauses for Recall-Precision Curves: A Case Study in Biomedical Information Extraction. In: Camacho, R., King, R., Srinivasan, A. (eds.) ILP 2004. LNCS (LNAI), vol. 3194, pp. 98–115. Springer, Heidelberg (2004)
Hoche, S., Wrobel, S.: Relational learning using constrained confidence-rated boosting. In: Rouveirol, C., Sebag, M. (eds.) ILP 2001. LNCS (LNAI), vol. 2157, pp. 51–64. Springer, Heidelberg (2001)
Kok, S., Domingos, P.: Learning the structure of Markov Logic Networks. In: National Conference on Artificial Intelligene, AAAI (2005)
Landwehr, N., Kersting, K., De Raedt, L.: nFOIL: Integrating Naive Bayes and FOIL. In: National Conference on Artificial Intelligene, AAAI (2005)
Mewes, H.W., Frishman, D., Gruber, C., Geier, B., Haase, D., Kaps, A., Lemcke, K., Mannhaupt, G., Pfeiffer, F., Schüller, C., Stocker, S., Weil, B.: Mips: a database for genomes and protein sequences. Nucleic Acids Research 28(1), 37–40 (2000)
Muggleton, S.: Inverse entailment and Progol. New Generation Computing 13, 245–286 (1995)
Neville, J., Jensen, D., Friedland, L., Hay, M.: Learning relational probability trees. In: KDD 2003, pp. 625–630. ACM Press, New York (2003)
Pompe, U., Kononenko, I.: Naive Bayesian classifier within ILP-R. In: Swierstra, S.D. (ed.) PLILP 1995. LNCS, vol. 982, pp. 417–436. Springer, Heidelberg (1995)
Prati, R., Flach, P.: Roccer: an algorithm for rule learning based on roc analysis. In: IJCAI 2005, Edinburgh, Scotland (2005)
Quinlan, J.R.: Boosting first-order learning. In: Arikawa, S., Sharma, A.K. (eds.) ALT 1996. LNCS, vol. 1160, pp. 143–155. Springer, Heidelberg (1996)
Richardson, M., Domingos, P.: Markov logic networks (2004)
Srinivasan, A., King, R.: Feature construction with inductive logic programming: A study of quantitative predictions of biological activity aided by structural attributes. In: Džeroski, S., Lavrač, N. (eds.) ILP 1997. LNCS, vol. 1297, pp. 89–104. Springer, Heidelberg (1997)
Srinivasan, A.: The Aleph Manual (2001)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Davis, J., Burnside, E., de Castro Dutra, I., Page, D., Costa, V.S. (2005). An Integrated Approach to Learning Bayesian Networks of Rules. In: Gama, J., Camacho, R., Brazdil, P.B., Jorge, A.M., Torgo, L. (eds) Machine Learning: ECML 2005. ECML 2005. Lecture Notes in Computer Science(), vol 3720. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11564096_13
Download citation
DOI: https://doi.org/10.1007/11564096_13
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-29243-2
Online ISBN: 978-3-540-31692-3
eBook Packages: Computer ScienceComputer Science (R0)