Abstract
Performance of classifier ensembles depends on the precision and on the diversity of the members of the ensemble. In this paper we present an experimental study in which the relationship between the accuracy of the ensemble and both the diversity and the accuracy of base learners is analyzed. We conduct experiments on 8 different ANN ensembles and on 5 multiclass data sets. Experimental results show that a high diversity degree among the base learners does not always imply a high accuracy in the ensemble.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsNotes
- 1.
Note than, by definition, BCE is built using a feature selection process. Nevertheless, in this work BCE, as the other classification models, is built using both, the full feature space (removing the feature selection step) and the feature subsets obtained by applying BF + CFS.
- 2.
Enlarged figures are available at [26].
References
Dietterich, T.G.: Ensemble methods in machine learning. In: Kittler, J., Roli, F. (eds.) MCS 2000. LNCS, vol. 1857, pp. 1–15. Springer, Heidelberg (2000). https://doi.org/10.1007/3-540-45014-9_1
Dietterich, T.G.: Machine-learning research. AI Mag. 18, 97–137 (1997)
Breiman, L.: Bagging predictors. Mach. Learn. 24, 123–140 (1996)
Schapire, R.E.: The strength of weak learnability. Mach. Learn. 5, 197–227 (1990)
Sharkey, A.J.C., Sharkey, N.E.: Combining diverse neural nets. Knowl. Eng. Rev. 12, 231–247 (1997)
Ho, T.K.: The random subspace method for constructing decision forests. IEEE Trans. Pattern Anal. Mach. Intell. 20, 832–844 (1998)
Blum, A.L., Langley, P.: Selection of relevant features and examples in machine learning. Artif. Intell. 97, 245–271 (1997)
Tsymbal, A., Pechenizkiy, M., Cunningham, P.: Diversity in ensemble feature selection (2003)
Anand, R., Mehrotra, K.G., Mohan, C.K., Ranka, S.: An improved algorithm for neural network classification of imbalanced training sets. IEEE Trans. Neural Netw. 4, 962–969 (1993)
Hastie, T., Tibshirani, R.: Classification by pairwise coupling. Ann. Stat. 26, 451–471 (1998)
Dietterich, T.G., Bakiri, G.: Solving multiclass learning problems via error-correcting output codes. J. Artif. Intell. Res. 2, 263–286 (1995)
Murphey, Y.L., Wang, H., Ou, G.: OAHO: an effective algorithm for multi-class learning from imbalanced data. In: Proceedings of International Joint Conference on Neural Networks, pp. 406–411 (2007)
Sesmero, M.P., Alonso-Weber, J.M., Gutierrez, G., Ledezma, A., Sanchis, A.: An ensemble approach of dual base learners for multi-class classification problems. Inf. Fusion. 24, 122–136 (2015)
Dietterich, T.G.: An experimental comparison of three methods for constructing ensembles of decision trees: bagging, boosting, and randomization. Mach. Learn. 40, 139–157 (2000)
Kolen, J.F., Pollack, J.B.: Backpropagation is sensitive to initial conditions. Complex Syst. 4, 269–280 (1990)
Kuncheva, L.I., Whitaker, C.J.: Measures of diversity in classifier ensembles and their relationship with the ensemble accuracy. Mach. Learn. 51, 181–207 (2003)
Tsymbal, A., Pechenizkiy, M., Cunningham, P.: Diversity in search strategies for ensemble feature selection. Inf. Fusion 6, 83–98 (2005)
Zenobi, G., Cunningham, P.: Using diversity in preparing ensembles of classifiers based on different feature subsets to minimize generalization error. In: De Raedt, L., Flach, P. (eds.) ECML 2001. LNCS (LNAI), vol. 2167, pp. 576–587. Springer, Heidelberg (2001). https://doi.org/10.1007/3-540-44795-4_49
Gu, S., Jin, Y.: Generating diverse and accurate classifier ensembles using multi-objective optimization. In: 2014 IEEE Symposium Series on Computational Intelligence in Multi-Criteria Decision-Making, Proceedings, Orlando, pp. 9–15 (2014)
Löfstrüm, T., Johansson, U., Boström, H.: On the use of accuracy and diversity measures for evaluating and selecting ensembles of classifiers. In: Proceedings of the 7th International Conference on Machine Learning and Applications, ICMLA 2008, pp. 127–132 (2008)
Shipp, C.A., Kuncheva, L.I.: Relationships between combination methods and measures of diversity in combining classifiers. Inf. Fusion 3, 135–148 (2002)
Xu, L., Yan, P., Chang, T.: Best first strategy for feature selection. In: 9th International Conference on Pattern Recognition, pp. 706–708 (1988)
Hall, M.A.: Correlation-based feature selection for machine learning (1999). http://www.cs.waikato.ac.nz/~mhall/thesis.pdf
Frank, A., Asuncion, A.: UCI Machine Learning Repository. http://archive.ics.uci.edu/ml/
Sesmero, M.P., Ledezma, A., Alonso-Weber, J.M., Gutierrez, G., Sanchis, A.: Control Learning and Systems Optimization Group. http://www.caos.inf.uc3m.es/datasets/
Sesmero, M.P.: Measures of Diversity and Accuracy. http://www.caos.inf.uc3m.es/diversity-and-accuracy-in-ann-ensembles/
Acknowledgments
This research was supported by the Spanish MINECO under projects TRA2016-78886-C3-1-R and TRA2015-63708-R, and by CAM under project S2013/MIT-3024.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Sesmero, M.P., Alonso-Weber, J.M., Giuliani, A., Armano, G., Sanchis, A. (2018). Measuring Diversity and Accuracy in ANN Ensembles. In: Herrera, F., et al. Advances in Artificial Intelligence. CAEPIA 2018. Lecture Notes in Computer Science(), vol 11160. Springer, Cham. https://doi.org/10.1007/978-3-030-00374-6_11
Download citation
DOI: https://doi.org/10.1007/978-3-030-00374-6_11
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-00373-9
Online ISBN: 978-3-030-00374-6
eBook Packages: Computer ScienceComputer Science (R0)