Global learning of decision trees by an evolutionary algorithm

  • Marek Krętowski
  • Marek Grześ


In the paper, an evolutionary algorithm for global induction of decision trees is presented. In contrast to greedy, top-down approaches it searches for the whole tree at the moment. Specialised genetic operators are proposed which allow modifying both tests used in the non-terminal nodes and structure of the tree. The proposed approach was validated on both artificial and real-life datasets. Experimental results show that the proposed algorithm is able to find competitive classifiers in terms of accuracy and especially complexity.


Data mining decision trees evolutionary algorithms global induction 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

5 References

  1. [1]
    Bennett K., “Global tree optimization: A non-greedy decision tree algorithm”, Computing Science and Statistics 26, pp. 156–160, 1994.Google Scholar
  2. [2]
    Blake C, Merz C, “UCI Repository of machine learning databases” [] Irvine, CA: University of California, 1998.Google Scholar
  3. [3]
    Bobrowski L. “Piecewise-linear classifiers, formal neurons and separability of the learning sets”, Proc. of ICPR'96, IEEE CS Press, pp. 224–228, 1996.Google Scholar
  4. [4]
    Bot M., Langdon W., “Application of genetic programming to induction of linear classification trees”, Proc. of EuroGP, LNCS 1802, pp.247–258, 2000.Google Scholar
  5. [5]
    Breiman L., Friedman J., Olshen R., Stone C, “Classification and Regression Trees”, Wadsworth International Group, 1984.Google Scholar
  6. [6]
    Cantu-Paz E., Kamath C, “Inducing oblique decision trees with evolutionary algorithms”, IEEE Trans. on Evol. Computation 7(1), pp. 54–68, 2003.CrossRefGoogle Scholar
  7. [7]
    Chai B., Huang T., Zhuang X., Zhao Y., Sklansky J., “Piecewise-linear classifiers using binary tree structure and genetic algorithm”, Pattern Recognition 29(11), pp. 1905–1917, 1996.CrossRefGoogle Scholar
  8. [8]
    Fayyad U., Irani K., “Multi-interval discretization of continuous-valued attributes for classification learning”, Proc. of IJCAI'93, Morgan Kaufmann, pp. 1022–1027, 1993.Google Scholar
  9. [9]
    Fayyad U., Piatetsky-Shapiro G., Smyth P., Uthurusamy R., (eds.) Advances in Knowledge Discovery and Data Mining, AAAI Press, 1996.Google Scholar
  10. [10]
    Hayfil L., Rivest R., “Constructing optimal binary decision trees is NP.-complete”, Information Processing Letters 5(1), pp. 15–17, 1976.CrossRefGoogle Scholar
  11. [11]
    Koza J., “Concept formation and decision tree induction usisng genetic programming paradigm”, Proc. of PPSN 1, LNCS 496, pp. 124–128, 1991.Google Scholar
  12. [12]
    Krętowski M., “An evolutionary algorithm for oblique decision tree induction”, Proc. of ICAISC'04, Springer, LNCS 3070, pp.432–437, 2004.Google Scholar
  13. [13]
    Michalewicz Z., “Genetic Algorithms + Data Structures = Evolution Programs”, Springer, 1996.Google Scholar
  14. [14]
    Murthy S., Salzberg S., “Decision tree induction: How effective is the greedy heuritics?”, Proc. of KDD-95, 1995.Google Scholar
  15. [15]
    Murthy S., “Automatic construction of decision trees from data: A multidisciplinary survey”, Data Mining and Know. Disc. 2, pp. 345–389, 1998.MathSciNetCrossRefGoogle Scholar
  16. [16]
    Quinlan J., “C4.5: Programs for Machine Learning”, Morgan Kauf., 1993.Google Scholar
  17. [17]
    Shah S., Sastry P., “New algorithm for learning and pruning oblique decision trees”, IEEE Trans. on SMC — Part C 29(4), pp. 494–505, 1999.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, Inc. 2005

Authors and Affiliations

  • Marek Krętowski
    • 1
  • Marek Grześ
    • 1
  1. 1.Faculty of Computer ScienceBiałystok Technical UniversityBiałystokPoland

Personalised recommendations