Adaptive Training for Combining Classifier Ensembles

Summary

Classifier ensembles, and multiple classifier systems, have been established in the literature as a means to achieve higher classification accuracy through their combining. This recent interest has been marked by the introduction of a variety of combining methods that improve the overall accuracy. However, it has been noted that for multiple classifier approaches to be useful its members should demonstrate error-independence. Ideally, in terms of ensembles of classifiers, would be a set of classifiers which do not show any coincident errors. That is, each of the classifiers generalized well, and when they did make errors on the test set, these errors were not shared with any other classifier. Generally, training of members of the ensemble to achieve this independence is achieved by training each member independent while manipulating the training data. Although these approaches have been shown to be useful, they might not be sufficient. In this work we present an algorithm to train the members of an ensemble concurrently. The algorithm is applied to an ensemble, based on the weighted average, and the feature based aggregation architecture. An empirical evaluation shows a reduction in the number of training cycles when applying the algorithm on the overall architecture, while maintaining the same or improved performance. The performance of these approaches is also compared to standard approaches proposed in the literature. The results substantiate the use of adaptive training for both the ensemble and the aggregation architecture.