Clustering-Based Ensemble Pruning and Multistage Organization Using Diversity

Abstract

The purpose of ensemble pruning is to reduce the number of predictive models in order to improve efficiency and predictive performance of the ensemble. In clustering-based approach, we are looking for groups of similar models, and then we prune each of them separately in order to increase overall diversity of the ensemble. In this paper we propose two methods for this purpose using classifier clustering on the basis of a criterion based on diversity measure. In the first method we select from each cluster the model with the best predictive performance to form the final ensemble, while the second one employs the multistage organization, where instead of removing the classifiers from the ensemble each classifier group makes the decision independently. The final answer of the proposed framework is the result of the majority voting of the decisions returned by each group. Experimentation results validated through statistical tests confirmed the usefulness of the proposed approaches.