Hierarchical Clustering of Multiple Decision Trees

  • Branko Kavšek
  • Nada Lavrač
  • Anuška Ferligoj
Conference paper
Part of the Studies in Classification, Data Analysis, and Knowledge Organization book series (STUDIES CLASS)


Decision tree learning is relatively non-robust: a small change in the training set may significantly change the structure of the induced decision tree. This paper presents a decision tree construction method in which the domain model is constructed by consensus clustering of N decision trees induced in N-fold cross-validation. Experimental results show that consensus decision trees are simpler than C4.5 decision trees, indicating that they may be a more stable approximation of the intended domain model than decision trees, constructed from the entire set of training instances.


Decision Tree Consensus Cluster Concept Hierarchy Decision Tree Learning Decision Tree Induction 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. ADAMS, E.N. (1972). Consensus techniques and the comparison of taxonomic trees. Systematic Zoology, 21, 390–397.CrossRefGoogle Scholar
  2. BREIMAN, L., FRIEDMAN, J., OLSHEN, R., and STONE, C. (1984). Classifica-tion and Regression Trees. Wadsworth International Group, Belmont, CA.Google Scholar
  3. BREIMAN, L. (1996). Bagging predictors. Machine Learning, 24: 123–140.MathSciNetzbMATHGoogle Scholar
  4. DAY, W.H.E. (1983). The role of complexity in comparing classifications. Mathe-matical Biosciences, 66, 97–114.zbMATHCrossRefGoogle Scholar
  5. FAITH, D.P. (1988). Consensus applications in the biological sciences. In: Bock, H.H. (Ed.) Classification and Related Methods of Data Analysis, Amsterdam: North-Holland, 325–332.Google Scholar
  6. FISHER, D.H. (1989). Noise-tolerant conceptual clustering. Proceedings of the Eleventh International Joint Conference on Artificial Intelligence 825–830. San Francisco: Morgan Kaufmann.Google Scholar
  7. GORDON, A.D. (1981). Classification. London: Chapman and Hall.zbMATHGoogle Scholar
  8. HARTIGAN, J.A. (1975). Cluster Algorithms. New York: Wiley.Google Scholar
  9. HUNT, E., MARTIN, J., and STONE, P. (1966). Experiments in Induction. New York: Academic Press.Google Scholar
  10. KONONENKO, I., and BRATKO, I. (1991). Information based evaluation criterion for classifier’s performance. Machine Learning, 6, (1), 67–80.Google Scholar
  11. LANGLEY, P. (1996). Elements of Machine Learning. Morgan Kaufmann.Google Scholar
  12. LECLERC, B. (1988). Consensus applications in the social sciences. In: Bock, H.H. (Ed.) Classification and Related Methods of Data Analysis, Amsterdam: North-Holland, 333–340.Google Scholar
  13. McMORRIS, F.R. and NEUMAN, D. (1983). Consensus functions defined on trees. Mathematical Social Sciences, 4, 131–136.MathSciNetzbMATHCrossRefGoogle Scholar
  14. QUINLAN, J.R. (1986). Induction of decision trees. Machine Learning, 1 (1): 81–106.Google Scholar
  15. QUINLAN, J.R. (1993). C4.5: Programs for Machine Learning. California: Morgan Kaufmann.Google Scholar
  16. REGNIER, S. (1965). Sur quelques aspects mathematiques des problems de classification automatique. I.I. C. Bulletin, 4, 175–191.Google Scholar
  17. SOKAL, R.R. and SNEATH, P.H.A. (1963). Principles of Numerical Taxonomy. San Francisco: Freeman.Google Scholar
  18. WITTEN, I.H. and FRANK, E. (1999). Data Mining: Practical Machine Learning Tools and Techniques with Java Implementations. Morgan Kaufmann, San Francisco.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2002

Authors and Affiliations

  • Branko Kavšek
    • 1
  • Nada Lavrač
    • 1
  • Anuška Ferligoj
    • 2
  1. 1.Institute Jožef StefanLjubljanaSlovenia
  2. 2.University of LjubljanaLjubljanaSlovenia

Personalised recommendations