Ensemble Learning

  • Harri Lappalainen
  • James W. Miskin
Part of the Perspectives in Neural Computing book series (PERSPECT.NEURAL)


This chapter gives a tutorial introduction to ensemble learning, a recently developed Bayesian method. For many problems it is intractable to perform inferences using the true posterior density over the unknown variables. Ensemble Learning allows the true posterior to be approximated by a simpler approximate distribution for which the required inferences are tractable. When we say we are making a model of a system, we are setting up a tool which can be used to make inferences, predictions and decisions. Each model can be seen as a hypothesis, or explanation, which makes assertions about the quantities which are directly observable and those which can only be inferred from their effect on observable quantities.


Cost Function Posterior Distribution Posterior Density Stochastic Approximation Ensemble Learn 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    G. E. Hinton and D. van Camp. Keeping neural networks simple by minimizing the description length of the weights. In Proceedings of the COLT’93, pp. 5–13, Santa Cruz, California, 1993.Google Scholar
  2. 2.
    S. Kullback and R. A. Leible. On information and sufficiency. The Annals of Mathematical Statistics 22: 79–86, 1951.MATHCrossRefGoogle Scholar
  3. 3.
    R M. Neal. Bayesian Learning for Neural Networks. Lecture Notes in Statistics No. 118. Springer-Verlag, 1996.Google Scholar
  4. 4.
    R M. Neal and G E. Hinton. A view of the EM algorithm that justifies incremental, sparse and other variants. In Learning in Graphical Models. M. I. Jordan, editor, 1998.Google Scholar
  5. 5.
    C E. Shannon. A mathematical theory of communication. Bell Systems Technical Journal 27: 379–423 and 623-656, 1948.MathSciNetMATHGoogle Scholar
  6. 6.
    S. M. Stigler. Translation of Laplace’s 1774 memoir on “Probability of causes”. Statistical Science, 1(3): 359–378, 1986.MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer-Verlag London 2000

Authors and Affiliations

  • Harri Lappalainen
  • James W. Miskin

There are no affiliations available

Personalised recommendations