Design of hierarchical classifiers

  • Richard E. Haskell
  • Ali Noui-Mehidi
Track 2: Artificial Intelligence
Part of the Lecture Notes in Computer Science book series (LNCS, volume 507)


Decision trees provide a powerful method of pattern classification. At each node in a binary tree a decision is made based upon the value of one of many possible attributes or features. The leaves of the tree represent the various classes that can be recognized. Various techniques have been used to select the feature and threshold to use at each node based upon a set of training data. Information theoretic methods are the most popular techniques used for designing each node in the tree. An alternate method uses the Kolmogorov-Smirnov test to design classification trees involving two classes. This paper presents an extension of this method that can produce a single decision tree when there are multiple classes. The relationship between this generalized Kolmogorov-Smirnov method and entropy minimization methods will be described. Experiments comparing classification results using this decision tree with results of using a Bayesian classifier will be presented.


Mahalanobis Distance Terminal Node Tree Node Classifier Design Decision Tree Method 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    G. R. Dattatreya and L. N. Kanal, "Decision Trees in Pattern Recognition," Progress in Pattern Recognition 2, L. N. Kanal and A Rosenfeld (Editors), Elsevier Science Publishers B. V. (North-Holland), 1985.Google Scholar
  2. 2.
    L. Breiman, J. H. Friedman, R. A. Olshen and C. J. Stone, Classification and Regression Trees, Wadsworth & Brooks/Cole, Monterey, CA, 1984.zbMATHGoogle Scholar
  3. 3.
    I. K. Sethi and G. P. R. Sarvarayudu, "Hierarchical Classifier Design Using Mutual Information," IEEE Trans. on Pattern Anal. and Machine Intell., Vol. PAMI-4, pp 441–445, 1982.CrossRefGoogle Scholar
  4. 4.
    J. R. Quinlan, "Learning Efficient Classification Procedures and their Application to Chess End Games," in Machine Learning, An Artificial Intelligence Approach, R. S. Michalski, et al. Eds., Tioga Publishing Co., Palo Alto, CA pp. 463–482, 1983.Google Scholar
  5. 5.
    J. L. Talmon, "A multiclass nonparametric partitioning algorithm,” Pattern Recognition Letters, vol. 4, pp 31–38, 1986.CrossRefGoogle Scholar
  6. 6.
    J. H. Friedman, "A Recursive Partitioning Decision Rule for Nonparametric Classification," IEEE Trans. on Computers, Vol C-26, pp. 404–408, April 1977.Google Scholar
  7. 7.
    E. M. Rounds, "A Combined Nonparametric Approach to Feature Selection and Binary Decision Tree Design," Proc. 1979 IEEE Computer Society Conf. on Pattern Recognition and Image Processign, pp. 38–43, 1979.Google Scholar
  8. 8.
    R. E. Haskell, G. Castelino and B. Mirshab, "Computer Learning Using Binary Tree Classifiers," Proc. 1988 Rochester Forth Conference on Programming Environments, pp. 77–78, June 14–18, 1988.Google Scholar
  9. 9.
    C. E. Shannon, "A Mathematical Theory of Communication," Bell Syst. Tech. J., Vol. 27, pp. 379–423, 1948.MathSciNetzbMATHGoogle Scholar
  10. 10.
    S. Watanabe, "Pattern Recognition as a Quest for Minimum Entropy," Pattern Recognition, Vol. 13, pp. 381–387, 1981.zbMATHMathSciNetCrossRefGoogle Scholar
  11. 11.
    J. R. Quinlan, "Decision Trees as Probabilistic Classifiers," Proc. Fourth Int. Workshop on Machine Learning, U. of Cal, Irvine, pp. 31–37, June 22–25, 1987.Google Scholar
  12. 12.
    B. Mirshab, "A Computer-Based Pattern Learning System With Application to Printed Text Recognition," PhD Dissertation, Oakland University, Rochester, MI, 1989.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1991

Authors and Affiliations

  • Richard E. Haskell
    • 1
  • Ali Noui-Mehidi
    • 1
  1. 1.Department of Computer Science and EngineeringOakland UniversityRochester

Personalised recommendations