Abstract
In the paper a new evolutionary algorithm for global induction of linear trees is presented. The learning process consists of searching for both a decision tree structure and hyper-plane weights in all non-terminal nodes. Specialized genetic operators are developed and applied according to the node quality and location. Feature selection aimed at simplification of the splitting hyper-planes is embedded into the algorithm and results in elimination of noisy and redundant features. The proposed approach is verified on both artificial and real-life data and the obtained results are promising.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Blake, C., Keogh, E., Merz, C.: UCI repository of machine learning databases, Irvine, CA: University of California, Dept. of Computer Science (1998), http://www.ics.uci.edu/~mlearn/MLRepository.html
Bot, M., Langdon, W.: Application of genetic programming to induction of linear classification trees. In: Poli, R., Banzhaf, W., Langdon, W.B., Miller, J., Nordin, P., Fogarty, T.C. (eds.) EuroGP 2000. LNCS, vol. 1802, pp. 247–258. Springer, Heidelberg (2000)
Breiman, L., Friedman, J., Olshen, R., Stone, C.: Classification and Regression Trees, Wadsworth Int. Group (1984)
Cantu-Paz, E., Kamath, C.: Inducing oblique decision trees with evolutionary algorithms. IEEE Transactions on Evolutionary Computation 7(1), 54–68 (2003)
Chai, B., Huang, T., Zhuang, X., Zhao, Y., Sklansky, J.: Piecewise-linear classifiers using binary tree structure and genetic algorithm. Pattern Recognition 29(11), 1905–1917 (1996)
Duda, O., Heart, P., Stork, D.: Pattern Classification, 2nd edn. J. Wiley, Chichester (2001)
Esposito, F., Malerba, D., Semeraro, G.: A comparative analysis of methods for pruning decision trees. IEEE Transactions on Pattern Analysis and Machine Intelligence 19(5), 476–491 (1997)
Fayyad, U., Piatetsky-Shapiro, G., Smyth, P., Uthurusamy, R. (eds.): Advances in Knowledge Discovery and Data Mining. AAAI Press, Menlo Park (1996)
Freitas, A.: Data Mining and Knowledge Discovery with Evolutionary Algorithms. Springer, Heidelberg (2002)
Gama, J., Brazdil, P.: Linear tree. Inteligent Data Analysis 3(1), 1–22 (1999)
Koza, J.: Concept formation and decision tree induction using genetic programming paradigm. In: Schwefel, H.-P., Männer, R. (eds.) PPSN 1990. LNCS, vol. 496, pp. 124–128. Springer, Heidelberg (1991)
Krȩtowski, M.: An evolutionary algorithm for oblique decision tree induction. In: Rutkowski, L., Siekmann, J.H., Tadeusiewicz, R., Zadeh, L.A. (eds.) ICAISC 2004. LNCS (LNAI), vol. 3070, pp. 432–437. Springer, Heidelberg (2004)
Krȩtowski, M., Grześ, M.: Global induction of oblique decision trees: an evolutionary approach. In: Proc. of IIPWM 2005, pp. 309–318. Springer, Heidelberg (2005)
Krȩtowski, M., Grześ, M.: Global learning of decision trees by an evolutionary algorithm. In: Information Processing and Security Systems, pp. 401–410. Springer, Heidelberg (2005)
Michalewicz, Z.: Genetic Algorithms + Data Structures = Evolution Programs, 3rd edn. Springer, Heidelberg (1996)
Murthy, S., Kasif, S., Salzberg, S.: A system for induction of oblique decision trees. Journal of Artificial Intelligence Research 2, 1–33 (1994)
Murthy, S.: Automatic construction of decision trees from data: A multi-disciplinary survey. Data Mining and Knowledge Discovery 2, 345–389 (1998)
Nikolaev, N., Slavov, V.: Inductive genetic programming with decision trees. Intelligent Data Analysis 2, 31–44 (1998)
Papagelis, A., Kalles, D.: Breeding decision trees using evolutionary techniques. In: Proc. of ICML 2001, pp. 393–400. Morgan Kaufmann, San Francisco (2001)
Quinlan, J.: C4.5: Programs for Machine Learning. Morgan Kaufmann, San Francisco (1993)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Krętowski, M., Grześ, M. (2006). Evolutionary Learning of Linear Trees with Embedded Feature Selection. In: Rutkowski, L., Tadeusiewicz, R., Zadeh, L.A., Żurada, J.M. (eds) Artificial Intelligence and Soft Computing – ICAISC 2006. ICAISC 2006. Lecture Notes in Computer Science(), vol 4029. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11785231_43
Download citation
DOI: https://doi.org/10.1007/11785231_43
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-35748-3
Online ISBN: 978-3-540-35750-6
eBook Packages: Computer ScienceComputer Science (R0)