Advertisement

Accelerated learning in Boltzmann Machines using mean field theory

  • H. J. Kappen
  • F. B. Rodríguez
Part II: Cortical Maps and Receptive Fields
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1327)

Abstract

The learning process in Boltzmann Machines is computationally intractible. We present a new approximate learning algorithm for Boltzmann Machines, which is based on mean field theory and the linear response theorem. The computational complexity of the algorithm is cubic in the number of neurons.

In the absence of hidden units, we show how the weights can be directly computed from the fixed point equation of the learning rules. We show that the solutions of this method are close to the optimal and give a significant improvement over the naive mean field approach.

Keywords

Partition Function Firing Rate Linear Response Exact Method Boltzmann Distribution 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    D. Ackley, G. Hinton, and T. Sejnowski. A learning algorithm for Boltzmann Machines. Cognitive Science, 9:147–169, 1985.Google Scholar
  2. 2.
    C. Itzykson and J-M. Drouffe. Statistical field theory. Cambridge monographs on mathematical physics. Cambridge University Press, Cambridge, UK, 1989.Google Scholar
  3. 3.
    C. Peterson and J.R. Anderson. A mean field theory learning algorithm for neural networks. Complex systems, 1:995–1019, 1987.Google Scholar
  4. 4.
    G.E. Hinton. Deterministic Boltzmann learning performs steepest descent in weightspace. Neural Computation, 1:143–150, 1989.Google Scholar
  5. 5.
    H.J. Kappen and F.B. Rodríguez. Efficient learning in Boltzmann Machines using linear response theory. Neural Computation, page Submitted, 1997.Google Scholar
  6. 6.
    G. Parisi. Statistical Field Theory. Frontiers in Physics. Addison-Wesley, 1988.Google Scholar
  7. 7.
    S. Kullback. Information theory and statistics. Wiley, New York, 1959.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1997

Authors and Affiliations

  • H. J. Kappen
    • 1
  • F. B. Rodríguez
    • 2
  1. 1.RWCP SNN Laboratory, Department of BiophysicsUniversity of NijmegenEZ NijmegenThe Netherlands
  2. 2.Instituto de Ingeniería del Conocimiento & Departamento de Ingeniería InformáticaUniversidad Autónoma de MadridMadridSpain

Personalised recommendations