Classifier Ensemble Using a Heuristic Learning with Sparsity and Diversity
Classifier ensemble has been intensively studied with the aim of overcoming the limitations of individual classifier components in two prevalent directions, i.e., to diversely generate classifier components, and to sparsely combine multiple classifiers. Currently, most approaches are emphasized only on sparsity or on diversity. In this paper, we investigated classifier ensemble with learning both sparsity and diversity using a heuristic method. We formulated the sparsity and diversity learning problem in a general mathematical framework which is beneficial for learning sparsity and diversity while grouping classifiers. Moreover, we proposed a practical approach based on the genetic algorithm for the optimization process. In order to conveniently evaluate the diversity of component classifiers, we introduced the diversity contribution ability to select proper classifier components and evolve classifier weights. Experimental results on several UCI classification data sets confirm that our approach has a promising sparseness and the generalization performance.
KeywordsClassifier ensemble Sparsity learning Diversity learning Bagging
Unable to display preview. Download preview PDF.
- 8.Schapire, R.: The Strength of Weak Learnability. Mach. Learn. 5(2), 197–227 (1990)Google Scholar
- 13.Yu, Y., Li, Y.F., Zhou, Z.H.: Diversity Regularized Machine. In: IJCAI, pp. 1603–1608 (2011)Google Scholar
- 15.Grove, A., Schuurmans, D.: Boosting in the Limit: Maximizing the Margin of Learned Ensembles. In: AAAI, pp. 692–699 (1998)Google Scholar
- 16.Margineantu, D., Dietterich, T.: Pruning Adaptive Boosting. In: ICML, pp. 211–218 (1997)Google Scholar
- 21.Chen, H., Tino, P., Yao, X.: Predictive Ensemble Pruning by Expectation Propagation. IEEE T-TDE 21(7), 999–1013 (2009)Google Scholar
- 22.Chen, H., Yao, X.: Multiobjective Neural Network Ensembles Based on Regularized Negative Correlation Learning. IEEE T-KDE 22(12), 1738–1751 (2009)Google Scholar