Global learning of decision trees by an evolutionary algorithm
In the paper, an evolutionary algorithm for global induction of decision trees is presented. In contrast to greedy, top-down approaches it searches for the whole tree at the moment. Specialised genetic operators are proposed which allow modifying both tests used in the non-terminal nodes and structure of the tree. The proposed approach was validated on both artificial and real-life datasets. Experimental results show that the proposed algorithm is able to find competitive classifiers in terms of accuracy and especially complexity.
KeywordsData mining decision trees evolutionary algorithms global induction
Unable to display preview. Download preview PDF.
- Bennett K., “Global tree optimization: A non-greedy decision tree algorithm”, Computing Science and Statistics 26, pp. 156–160, 1994.Google Scholar
- Blake C, Merz C, “UCI Repository of machine learning databases” [http://www.ics.uci.edu/~mlearn/MLRepository.html] Irvine, CA: University of California, 1998.Google Scholar
- Bobrowski L. “Piecewise-linear classifiers, formal neurons and separability of the learning sets”, Proc. of ICPR'96, IEEE CS Press, pp. 224–228, 1996.Google Scholar
- Bot M., Langdon W., “Application of genetic programming to induction of linear classification trees”, Proc. of EuroGP, LNCS 1802, pp.247–258, 2000.Google Scholar
- Breiman L., Friedman J., Olshen R., Stone C, “Classification and Regression Trees”, Wadsworth International Group, 1984.Google Scholar
- Fayyad U., Irani K., “Multi-interval discretization of continuous-valued attributes for classification learning”, Proc. of IJCAI'93, Morgan Kaufmann, pp. 1022–1027, 1993.Google Scholar
- Fayyad U., Piatetsky-Shapiro G., Smyth P., Uthurusamy R., (eds.) Advances in Knowledge Discovery and Data Mining, AAAI Press, 1996.Google Scholar
- Koza J., “Concept formation and decision tree induction usisng genetic programming paradigm”, Proc. of PPSN 1, LNCS 496, pp. 124–128, 1991.Google Scholar
- Krętowski M., “An evolutionary algorithm for oblique decision tree induction”, Proc. of ICAISC'04, Springer, LNCS 3070, pp.432–437, 2004.Google Scholar
- Michalewicz Z., “Genetic Algorithms + Data Structures = Evolution Programs”, Springer, 1996.Google Scholar
- Murthy S., Salzberg S., “Decision tree induction: How effective is the greedy heuritics?”, Proc. of KDD-95, 1995.Google Scholar
- Quinlan J., “C4.5: Programs for Machine Learning”, Morgan Kauf., 1993.Google Scholar