Learning and Evolution by Minimization of Mutual Information
Based on negative correlation learning  and evolutionary learning, evolutionary ensembles with negative correlation learning (EENCL) was proposed for learning and designing of neural network ensembles . The idea of EENCL is to regard the population of neural networks as an ensemble, and the evolutionary process as the design of neural network ensembles. EENCL used a fitness sharing based on the covering set. Such fitness sharing did not make accurate measurement on the similarity in the population. In this paper, a fitness sharing scheme based on mutual information is introduced in EENCL to evolve a diverse and cooperative population. The effectiveness of such evolutionary learning approach was tested on two real-world problems. This paper has also analyzed negative correlation learning in terms of mutual information on a regression task in the different noise conditions.
KeywordsNeural Network Mutual Information Hide Node Individual Network Evolutionary Learning
Unable to display preview. Download preview PDF.
- 3.Y. Liu and X. Yao. Towards designing neural network ensembles by evolution. In Parallel Problem Solving from Nature—PPSN V: Proc. of the Fifth International Conference on Parallel Problem Solving from Nature, volume 1498 of Lecture Notes in Computer Science, pages 623–632. Springer-Verlag, Berlin, 1998.Google Scholar
- 5.J. C. A. van der Lubbe. Information Theory. Prentice-Hall International, Inc., 2nd edition, 1999.Google Scholar
- 7.D. E. Rumelhart, G. E. Hinton, and R. J. Williams. Learning internal representations by error propagation. In D. E. Rumelhart and J. L. McClelland, editors, Parallel Distributed Processing: Explorations in the Microstructures of Cognition, Vol. I, pages 318–362. MIT Press, Cambridge, MA, 1986.Google Scholar
- 9.D. B. Fogel. Evolutionary Computation: Towards a New Philosophy of Machine Intelligence. IEEE Press, New York, NY, 1995.Google Scholar