Advertisement

Journal of the Korean Physical Society

, Volume 74, Issue 1, pp 63–72 | Cite as

Learning Process in a Neural Network Model

  • Myoung Won ChoEmail author
Article
  • 5 Downloads

Abstract

The firing and the learning processes are the dynamics in a neural system on fast and slow time scales. In this paper, we study a neural network model that describes the two different dynamics in a unified manner. The model makes it possible to predict the firing probability or the firing correlation in a biological neural system much exactly via a path integral formulation. Also, an ideal learning process in the model system is defined to minimize the path-integral free energy. The learning principle explains how the typical form of plasticity in a biological synapse is determined. Further, the model makes it possible to understand or predict the emergent structures in a biological neural system through the basic principles in (statistical) physics. We investigate how the thermodynamic potentials in the free energy, such as the internal energy, the Gibbs potential, and the entropy, exert effects on the general learning behavior of the model system. In addition, we demonstrate that the derivative of an effective free energy leads to a learning equation for feature map formation.

Keywords

Neural network Learning theory Free energy principle 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. [1]
    D. O. Hebb, The Organization of Behavior: A Neurophysiological Theory (Wiley, New York, 1949).Google Scholar
  2. [2]
    J. C. Principe, N. R. Euliano and W. C. Lefebvre, Neural and Adaptive Systems: Fundamentals through Simulations (John Wiley & Sons, New York, 2000).Google Scholar
  3. [3]
    S. Haykin, Neural Networks and Learning Machines (Pearson Education, New York, 2009).Google Scholar
  4. [4]
    W. Gerstner, R. Kempter, J. L. Van Hemmen and H. Wagner, Nature 383, 76 (1996).ADSCrossRefGoogle Scholar
  5. [5]
    M. W. Cho and M. Y. Choi, Neural Networks 49, 51 (2014).CrossRefGoogle Scholar
  6. [6]
    H. Markram, J. Lubke, M. Frotscher and B. Sakamann, Science 275, 213 (1997).CrossRefGoogle Scholar
  7. [7]
    L. Zhang, H. Tao, C. Holt, W. Harris and M-M. Poo, Nature 95, 37 (1998).ADSCrossRefGoogle Scholar
  8. [8]
    G-Q. Bi and M-M. Poo, J. Neurosci. 18, 10464 (1998).CrossRefGoogle Scholar
  9. [9]
    P. D. Roberts and C. C. Bell, Biol. Cybern. 87, 392 (2002).CrossRefGoogle Scholar
  10. [10]
    L. F. Abbott and S. B. Nelson, Nature Neurosci. 3, 1178 (2000).CrossRefGoogle Scholar
  11. [11]
    M. W. Cho and M. Y. Choi, Neural Networks 31, 46 (2012).CrossRefGoogle Scholar
  12. [12]
    M. W. Cho, J. Korean Phys. Soc. 64, 1213 (2014).ADSCrossRefGoogle Scholar
  13. [13]
    T-W. Lee, Independent Component Analysis (Kluwer Academic Publishers, Boston, 1998).CrossRefzbMATHGoogle Scholar
  14. [14]
    D. H. Ackley, G. Hinton and T. Sejnowski, Cognitive Science 9, 147 (1985).CrossRefGoogle Scholar
  15. [15]
    M. W. Cho and M. Y. Choi, Europhys. Lett. 95, 58005 (2011).ADSCrossRefGoogle Scholar
  16. [16]
    M. W. Cho, J. Korean Phys. Soc. 67, 1679 (2015).ADSCrossRefGoogle Scholar
  17. [17]
    M. W. Cho and M. Y. Choi, Europhy. Lett. 115, 38001 (2016).ADSCrossRefGoogle Scholar
  18. [18]
    M. W. Cho, J. Korean Phys. Soc. 71, 222 (2017).ADSCrossRefGoogle Scholar
  19. [19]
    G. E. Uhlenbeck and L. S. Ornstein, Phys. Rev. 36, 823 (1930).ADSCrossRefGoogle Scholar
  20. [20]
    H. Risken, Fokker-Plank Equation: Method of Solution and Applications (Springer-Verlag, New York, 1989).CrossRefzbMATHGoogle Scholar
  21. [21]
    M. W. Cho, New Physics: Sae Mulli 67, 862 (2017).Google Scholar
  22. [22]
    H. Kleinert, Gauge Fields in Condensed Matter (World Scientific, Singapore, 1989).CrossRefzbMATHGoogle Scholar
  23. [23]
    H. Kleinert and V. Schulte-Fronhlinde, Critical Properties of f4 Theories (World Scientific, Singapore, 2001).CrossRefGoogle Scholar
  24. [24]
    M. W. Cho and M. Y. Choi, Phys. Rev. Lett. 99, 208102 (2007).ADSCrossRefGoogle Scholar
  25. [25]
    M. W. Cho and M. Y. Choi, Europhy. Lett. 101, 48004 (2013).ADSCrossRefGoogle Scholar
  26. [26]
    M. W. Cho, J. Korean Phys. Soc. 67, 1661 (2015).ADSCrossRefGoogle Scholar
  27. [27]
    M. W. Cho, New Physics: Sae Mulli 67, 357 (2017).Google Scholar
  28. [28]
    M. W. Cho, New Physics: Sae Mulli 65, 456 (2015).Google Scholar
  29. [29]
    P. Dayan and L. F. Abbott, Theoretical Neuroscience (MIT Press, London, 2001).zbMATHGoogle Scholar
  30. [30]
    M. W. Cho and M. Y. Choi, J. Korean Phys. Soc. 55, 2532 (2009).ADSCrossRefGoogle Scholar
  31. [31]
    S. LeVay, D. H. Connolly, J. Houde and D. C. V. Essen, J. Neurosci. 5, 486 (1985).CrossRefGoogle Scholar
  32. [32]
    M. W. Cho and S. Kim, Phy. Rev. Lett. 92, 18101 (2004).ADSCrossRefGoogle Scholar
  33. [33]
    M. W. Cho and S. Kim, Phy. Rev. Lett. 94, 68701 (2005).ADSCrossRefGoogle Scholar

Copyright information

© The Korean Physical Society 2019

Authors and Affiliations

  1. 1.Department of Global Medical ScienceSungshin Women’s UniversitySeoulKorea

Personalised recommendations