Abstract
The use of the neural network ensemble approach for solving classification problems is discussed. Methods for forming ensembles of neural networks and methods for combining solutions in ensembles of classifiers are reviewed briefly. The main ideas of comprehensive evolutionary approach for automatic design of neural network ensembles are described. A new variant of a three-stage evolutionary approach to decision making in ensembles of neural networks is proposed for classification problems. The technique and results of a comparative statistical investigation of various methods for producing of ensembles decisions on several well-known test problems are given.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Hansen, L.K., Salamon, P.: Neural network ensembles. IEEE Transactions on Pattern Analysis and Machine Intelligence 12, 993–1001 (1990)
Rastrigin, L.A., Erenstein, R.H.: Method of collective recognition. Energoizdat, Moscow (1981)
Javadi, M., Ebrahimpour, R., Sajedin, A., Faridi, S., Zakernejad, S.: Improving ECG Classification Accuracy Using an Ensemble of Neural Network Modules. PLoS One 6 (2011)
Perrone, M.P., Cooper, L.N.: When networks disagree: ensemble method for neural networks. In: Mammone, R.J. (ed.) Artificial Neural Networks for Speech and Vision, pp. 126–142. Chapman & Hall, New York (1993)
Shimshoni, Y., Intrator, N.: Classification of seismic signals by integrating ensembles of neural networks. IEEE Transactions on Signal Processing 46(5), 1194–1201 (1998)
Wolpert, D.H.: Stacked generalization. Neural Networks 5, 241–259 (1992)
Frank, A., Asuncion, A.: UCI Machine Learning Repository. University of California, School of Information and Computer Science, Irvine (2010), http://archive.ics.uci.edu/ml
Belew, R.K.: Evolving networks: Using genetic algorithm with connectionist learning. Technical report CS90-174, Computer Science and Engineering Department. University of California, San Diego (1991)
Bukhtoyarov, V., Semenkina, O.: Comprehensive evolutionary approach for neural network ensemble automatic design. In: Proceedings of 2010 IEEE World Congress on Computational Intelligence, Barcelona, pp. 1640–1645 (2010)
Woods, K., Kegelmeyer, W.P., Bowyer, K.: Combination of multiple classifiers using local accuracy estimates. IEEE Transactions on Pattern Analysis and Machine Intelligence 19, 405–410 (1997)
Jacobs, R.A., Jordan, M.I., Nowlan, S.J., Hinton, G.E.: Adaptive mixtures of local experts. Neural Computation 3, 79–87 (1991)
Xu, L., Krzyzak, A., Suen, C.Y.: Methods of combining multiple classifiers and their applications to handwriting recognition. Transactions on Systems, Man, and Cybernetics 22, 418–435 (1992)
Polikar, R.: Ensemble based systems in decision making. IEEE Circuits and Systems Magazine 6, 21–45 (2006)
Kuncheva, L.I., Bezdek, J.C., Duin, R.P.W.: Decision templates for multiple classifier fusion: an experimental comparison. Pattern Recognition 34, 299–314 (2001)
Rokach, L., Maimon, O., Arad, O.: Improving supervised learning by sample decomposition. Int. J. Comput. Intell. Appl. 5(1), 37–54 (2005)
Rokach, L., Maimon, O., Lavi, I.: Space decomposition in data mining: A clustering approach. In: Zhong, N., Raś, Z.W., Tsumoto, S., Suzuki, E. (eds.) ISMIS 2003. LNCS (LNAI), vol. 2871, pp. 24–31. Springer, Heidelberg (2003)
Lorena, A.C., Carvalho, A.C., Gama, J.M.: A review on the combination of binary classifiers in multiclass problems. Artificial Intelligence Review 30(1–4), 19–37 (2008)
Rifkin, R., Klautau, A.: In defense of one-vs-all classification. Journal of Machine Learning Research 5, 101–141 (2004)
Sun, Y., Wong, A.C., Kamel, M.S.: Classification of imbalanced data: a review. International Journal of Pattern Recognition and Artificial Intelligence 23(4), 687–719 (2009)
Hong, J.H., Min, J.K., Cho, U.K., Cho, S.B.: Fingerprint classification using one-vs-all support vector machines dynamically ordered with Naive Bayes classifiers. Pattern Recognition 41(2), 662–671 (2008)
Koza, J.R.: The Genetic Programming Paradigm: Genetically Breeding Populations of Computer Programs to Solve Problems. MIT Press, Cambridge (1992)
Galar, M., Fernandez, A., Barrenechea, E., Bustince, H., Herrera, F.: An overview of ensemble methods for binary classifiers in multi-class problems: Experimental study on one-vs-one and one-vs-all schemes. Pattern Recognition 44(8), 1761–1776 (2011)
Sergienko, R.B., Semenkin, E.S., Bukhtoyarov, V.V.: Michigan and Pittsburgh Methods Combining for Fuzzy Classifier Generating with Coevolutionary Algorithm for Strategy Adaptation. In: Proceedings of 2011 IEEE Congress on Evolutionary Computation, New Orleans, LA, USA (2011)
Bailey, R.A.: Design of Comparative Experiments. Cambridge University Press (2008)
Islam, M.M., Yao, X., Murase, K.: A constructive algorithm for training cooperative neural network ensembles. IEEE Trans. Neural Netw. 14, 820–834 (2003)
Breiman, L.: Randomizing outputs to increase prediction accuracy. Mach. Learn. 40, 229–242 (2000)
Garcia, N., Hervas, C.: Ortiz. D.: Cooperative coevolution of artificial neural network ensembles for pattern classification. IEEE Transactions on Evolutionary Computation 9(3), 271–302 (2005)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Bukhtoyarov, V., Semenkin, E. (2013). Evolutionary Three-Stage Approach for Designing of Neural Networks Ensembles for Classification Problems. In: Tan, Y., Shi, Y., Mo, H. (eds) Advances in Swarm Intelligence. ICSI 2013. Lecture Notes in Computer Science, vol 7928. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-38703-6_55
Download citation
DOI: https://doi.org/10.1007/978-3-642-38703-6_55
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-38702-9
Online ISBN: 978-3-642-38703-6
eBook Packages: Computer ScienceComputer Science (R0)