Abstract
We study the formal basis behind Negative Correlation (NC) Learning, an ensemble technique developed in the evolutionary computation literature. We show that by removing an assumption made in the original work, NC can be shown to be a derivative technique of the Ambiguity decomposition by Krogh and Vedelsby. From this formalisation, we calculate parameter bounds, and show significant improvements in empirical tests. We hypothesize that the reason for its success lies in rescaling an estimate of ensemble covariance; then show that during this rescaling, NC varies smoothly between a single neural network and an ensemble system. Finally we unify several other works in the literature, all of which have exploited the Ambiguity decomposition in some way, and term them the Ambiguity Family.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
J. Carney and P. Cunningham. Tuning diversity in bagged neural network ensembles. Technical Report TCD-CS-1999-44, Trinity College Dublin, 1999.
Kalyanmoy Deb. Multi-objective genetic algorithms: Problem difficulties and construction of test problems. Evolutionary Computation, 7(3):205–230, 1999.
Vineet Khare and Xin Yao. Artificial speciation of neural network ensembles. In J.A. Bullinaria, editor, Proc. of the 2002 UK Workshop on Computational Intelligence (UKCI’02), pages 96–103. University of Birmingham, UK, September 2002.
Anders Krogh and Jesper Vedelsby. Neural network ensembles, cross validation, and active learning. NIPS, 7:231–238, 1995.
Yong Liu. Negative Correlation Learning and Evolutionary Neural Network Ensembles. PhD thesis, University College, The University of New South Wales, Australian Defence Force Academy, Canberra, Australia, 1998.
Yong Liu and Xin Yao. Negatively correlated neural networks can produce best ensembles. Australian Journal of Intelligent Information Processing Systems, 4(3/4):176–185, 1997.
Yong Liu and Xin Yao. Ensemble learning via negative correlation. Neural Networks, 12(10):1399–1404, 1999.
David Opitz. Feature selection for ensembles. In Proceedings of 16th National Conference on Artificial Intelligence (AAAI), pages 379–384, 1999.
David W. Opitz and Jude W. Shavlik. Generating accurate and diverse members of a neural-network ensemble. NIPS, 8:535–541, 1996.
Nikunj C. Oza and Kagan Tumer. Input decimation ensembles: Decorrelation through dimensionality reduction. LNCS, 2096:238–247, 2001.
Bruce E. Rosen. Ensemble learning using decorrelated neural networks. Connection Science-Special Issue on Combining Artificial Neural Networks: Ensemble Approaches, 8(3 and 4):373–384, 1996.
N. Ueda and R. Nakano. Generalization error of ensemble estimators. In Proceedings of International Conference on Neural Networks, pages 90–95, 1996.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2003 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Brown, G., Wyatt, J. (2003). Negative Correlation Learning and the Ambiguity Family of Ensemble Methods. In: Windeatt, T., Roli, F. (eds) Multiple Classifier Systems. MCS 2003. Lecture Notes in Computer Science, vol 2709. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-44938-8_27
Download citation
DOI: https://doi.org/10.1007/3-540-44938-8_27
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-40369-2
Online ISBN: 978-3-540-44938-6
eBook Packages: Springer Book Archive