Design of hierarchical classifiers
Decision trees provide a powerful method of pattern classification. At each node in a binary tree a decision is made based upon the value of one of many possible attributes or features. The leaves of the tree represent the various classes that can be recognized. Various techniques have been used to select the feature and threshold to use at each node based upon a set of training data. Information theoretic methods are the most popular techniques used for designing each node in the tree. An alternate method uses the Kolmogorov-Smirnov test to design classification trees involving two classes. This paper presents an extension of this method that can produce a single decision tree when there are multiple classes. The relationship between this generalized Kolmogorov-Smirnov method and entropy minimization methods will be described. Experiments comparing classification results using this decision tree with results of using a Bayesian classifier will be presented.
KeywordsMahalanobis Distance Terminal Node Tree Node Classifier Design Decision Tree Method
Unable to display preview. Download preview PDF.
- 1.G. R. Dattatreya and L. N. Kanal, "Decision Trees in Pattern Recognition," Progress in Pattern Recognition 2, L. N. Kanal and A Rosenfeld (Editors), Elsevier Science Publishers B. V. (North-Holland), 1985.Google Scholar
- 4.J. R. Quinlan, "Learning Efficient Classification Procedures and their Application to Chess End Games," in Machine Learning, An Artificial Intelligence Approach, R. S. Michalski, et al. Eds., Tioga Publishing Co., Palo Alto, CA pp. 463–482, 1983.Google Scholar
- 6.J. H. Friedman, "A Recursive Partitioning Decision Rule for Nonparametric Classification," IEEE Trans. on Computers, Vol C-26, pp. 404–408, April 1977.Google Scholar
- 7.E. M. Rounds, "A Combined Nonparametric Approach to Feature Selection and Binary Decision Tree Design," Proc. 1979 IEEE Computer Society Conf. on Pattern Recognition and Image Processign, pp. 38–43, 1979.Google Scholar
- 8.R. E. Haskell, G. Castelino and B. Mirshab, "Computer Learning Using Binary Tree Classifiers," Proc. 1988 Rochester Forth Conference on Programming Environments, pp. 77–78, June 14–18, 1988.Google Scholar
- 11.J. R. Quinlan, "Decision Trees as Probabilistic Classifiers," Proc. Fourth Int. Workshop on Machine Learning, U. of Cal, Irvine, pp. 31–37, June 22–25, 1987.Google Scholar
- 12.B. Mirshab, "A Computer-Based Pattern Learning System With Application to Printed Text Recognition," PhD Dissertation, Oakland University, Rochester, MI, 1989.Google Scholar