Skip to main content
SpringerLink
Log in
Book cover

International Conference on Artificial Neural Networks

ICANN 2020: Artificial Neural Networks and Machine Learning – ICANN 2020 pp 180–191Cite as

Nonlinear, Nonequilibrium Landscape Approach to Neural Network Dynamics

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 12397))

Abstract

Distributions maximizing \(S_q\) entropies are not rare in Nature. They have been observed in complex systems in diverse fields, including neuroscience. Nonlinear Fokker-Planck dynamics constitutes one of the main mechanisms that can generate \(S_q\)-maximum entropy distributions. In the present work, we investigate a nonlinear Fokker-Planck equation associated with general, continuous, neural network dynamical models for associative memory. These models admit multiple applications in artificial intelligence, and in the study of mind, because memory is central to many, if not all, the processes investigated by psychology and neuroscience. We explore connections between the nonlinear Fokker-Planck treatment of network dynamics, and the nonequilibrium landscape approach to this dynamics discussed in  [34]. We show that the nonequilibrium landscape approach leads to fundamental relations between the Liapunov function of the network model, the deterministic equations of motion (phase-space flow) of the network, and the form of the diffusion coefficients appearing in the nonlinear Fokker-Planck equations. This, in turn, leads to an H-theorem involving a free energy-like functional related to the \(S_q\) entropy. To illustrate these results, we apply them to the Cohen-Grossberg family of neural network models.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Beggs, J.M., Plenz, D.: Neuronal avalanches in neocortical circuits. J. Neurosci. 23, 11167–11177 (2003). https://doi.org/10.1523/JNEUROSCI.23-35-11167.2003

    Article  Google Scholar 

  2. Brito, S., da Silva, L.R., Tsallis, C.: Role of dimensionality in complex networks. Nature Sci. Rep. 6, 27992.1–27992.8 (2016). https://doi.org/10.1038/srep27992

    Article  Google Scholar 

  3. de Carvalho, L.A.V., Mendes, D.Q., Wedemann, R.S.: Creativity and delusions: the dopaminergic modulation of cortical maps. In: Sloot, P.M.A., Abramson, D., Bogdanov, A.V., Dongarra, J.J., Zomaya, A.Y., Gorbachev, Y.E. (eds.) ICCS 2003. LNCS, vol. 2657, pp. 511–520. Springer, Heidelberg (2003). https://doi.org/10.1007/3-540-44860-8_53

    Chapter  Google Scholar 

  4. Cleeremans, A., Timmermans, B., Pasquali, A.: Consciousness and metarepresentation: a computational sketch. Neural Netw. 20, 1032–1039 (2007). https://doi.org/10.1016/j.neunet.2007.09.011

    Article  Google Scholar 

  5. Cohen, M.A., Grossberg, S.: Absolute stability of global pattern formation and parallel memory storage by competitive neural networks. IEEE Trans. Syst. Man. Cybern. 13, 815–826 (1983). https://doi.org/10.1109/TSMC.1983.6313075

    Article  MathSciNet  MATH  Google Scholar 

  6. Czégel, D., Balogh, S., Pollner, P., Palla, G.: Phase space volume scaling of generalized entropies and anomalous diffusion scaling governed by corresponding non-linear Fokker-Planck equations. Sci. Rep. 8, 1883 (2018). https://doi.org/10.1038/s41598-018-20202-w

    Article  Google Scholar 

  7. da Silva, P.C., da Silva, L.R., Lenzi, E.K., Mendes, R.S., Malacarne, L.C.: Anomalous diffusion and anisotropic nonlinear Fokker-Planck equation. Phys. A 342(1), 16–21 (2004). https://doi.org/10.1016/j.physa.2004.04.054

    Article  Google Scholar 

  8. Fang, X., Kruse, K., Lu, T., Wang, J.: Nonequilibrium physics in biology. Rev. Mod. Phys. 91(4), 045004 (2019). https://doi.org/10.1103/RevModPhys.91.045004

    Article  Google Scholar 

  9. Franck, T.D.: Nonlinear Fokker-Planck Equations: Fundamentals and Applications. Springer, Heidelberg (2005). https://doi.org/10.1007/b137680

    Book  Google Scholar 

  10. Hertz, J.A., Krogh, A., Palmer, R.G., (eds.): Introduction to the Theory of Neural Computation. Lecture Notes, vol. 1. Perseus Books, Cambridge (1991)

    Google Scholar 

  11. Hopfield, J.J.: Neurons with graded responses have collective computational properties like those of two-state neurons. Proc. Natl. Acad. Sci. 81, 3088–3092 (1984). https://doi.org/10.1073/pnas.81.10.3088

    Article  MATH  Google Scholar 

  12. Kandel, E.: Psychiatry, Psychoanalysis, and the New Biology of Mind. American Psychiatric Publishing Inc., Washington (2005)

    Google Scholar 

  13. Lenzi, E.K., Lenzi, M.K., Ribeiro, H.V., Evangelista, L.R.: Extensions and solutions for nonlinear diffusion equations and random walks. Proc. Roy. Soc. A: Math. Phys. Eng. Sci. 475(2231), 20190432 (2019). https://doi.org/10.1098/rspa.2019.0432

    Article  MathSciNet  Google Scholar 

  14. Lotka, A.J.: Elements of Mathematical Biology. Dover, New York (1956)

    MATH  Google Scholar 

  15. de Luca, V.T.F., Wedemann, R.S., Plastino, A.R.: Neuronal asymmetries and Fokker-Planck dynamics. In: Kůrková, V., Manolopoulos, Y., Hammer, B., Iliadis, L., Maglogiannis, I. (eds.) ICANN 2018. LNCS, vol. 11141, pp. 703–713. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01424-7_69

    Chapter  Google Scholar 

  16. Martinez, S., Plastino, A.R., Plastino, A.: Nonlinear Fokker-Planck equations and generalized entropies. Phys. A 259(1–2), 183–192 (1998). https://doi.org/10.1016/S0378-4371(98)00277-5

    Article  Google Scholar 

  17. Papa, A.R.R., da Silva, L.: Earthquakes in the brain. Theory Biosci. 116, 321–327 (1997)

    Google Scholar 

  18. Plastino, A.R., Plastino, A.: Non-extensive statistical mechanics and generalized Fokker-Planck equation. Phys. A 222(1), 347–354 (1995). https://doi.org/10.1016/0378-4371(95)00211-1

    Article  MathSciNet  Google Scholar 

  19. Ribeiro, M.S., Nobre, F.D., Curado, E.M.F.: Classes of N-dimensional nonlinear Fokker-Planck equations associated to Tsallis entropy. Entropy 13(11), 1928–1944 (2011). https://doi.org/10.3390/e13111928

    Article  MathSciNet  MATH  Google Scholar 

  20. Siddiqui, M., Wedemann, R.S., Jensen, H.J.: Avalanches and generalized memory associativity in a network model for conscious and unconscious mental functioning. Phys. A 490, 127–138 (2018). https://doi.org/10.1016/j.physa.2017.08.011

    Article  MathSciNet  Google Scholar 

  21. Souza, A.M.C., Andrade, R.F.S., Nobre, F.D., Curado, E.M.F.: Thermodynamic framework for compact \(q\)-Gaussian distributions. Phys. A 491, 153–166 (2018). https://doi.org/10.1016/j.physa.2017.09.013

    Article  MathSciNet  Google Scholar 

  22. Tagliazucchi, E., Balenzuela, P., Fraiman, D., Chialvo, D.R.: Criticality in large-scale brain fMRI dynamics unveiled by a novel point process analysis. Front. Physiol.|Fractal Physiol. 3, 15 (2012). https://doi.org/10.3389/fphys.2012.00015

    Article  Google Scholar 

  23. Taylor, J.G.: A neural model of the loss of self in schizophrenia. Schizophrenia Bull. 37(6), 1229–1247 (2011). https://doi.org/10.1093/schbul/sbq033

    Article  Google Scholar 

  24. Tirnakli, U., Borges, E.P.: The standard map: From Boltzmann-Gibbs statistics to Tsallis statistics. Nat. Sci. Rep. 6, 23644.1–23644.8 (2016). https://doi.org/10.1038/srep23644

    Article  Google Scholar 

  25. Tsallis, C.: Introduction to Nonextensive Statistical Mechanics, Approaching a Complex World. Springer, New York (2009). https://doi.org/10.1007/978-0-387-85359-8

    Book  MATH  Google Scholar 

  26. Tsallis, C., Stariolo, D.A.: Generalized simulated annealing. Phys. A 233, 395–406 (1996). https://doi.org/10.1016/S0378-4371(96)00271-3

    Article  Google Scholar 

  27. Wang, J., Xu, L., Wang, E.: Potential landscape and flux framework of nonequilibrium networks: robustness, dissipation, and coherence of biochemical oscillations. Proc. Nat. Acad. Sci. 105(34), 12271–12276 (2008). https://doi.org/10.1073/pnas.0800579105, https://www.pnas.org/content/105/34/12271

  28. Wedemann, R.S., Donangelo, R., de Carvalho, L.A.V.: Generalized memory associativity in a network model for the neuroses. Chaos 19(1), 015116 (2009). https://doi.org/10.1063/1.3099608

    Article  MathSciNet  Google Scholar 

  29. Wedemann, R.S., Plastino, A.R.: Asymmetries in synaptic connections and the nonlinear Fokker-Planck formalism. In: Villa, A.E.P., Masulli, P., Pons Rivero, A.J. (eds.) ICANN 2016. LNCS, vol. 9886, pp. 19–27. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-44778-0_3

    Chapter  Google Scholar 

  30. Wedemann, R.S., Plastino, A.R.: q-maximum entropy distributions and memory neural networks. In: Lintas, A., Rovetta, S., Verschure, P.F.M.J., Villa, A.E.P. (eds.) ICANN 2017. LNCS, vol. 10613, pp. 300–308. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-68600-4_35

    Chapter  Google Scholar 

  31. Wedemann, R.S., Plastino, A.R., Tsallis, C.: Curl forces and the nonlinear Fokker-Planck equation. Phys. Rev. E 94(6), 062105 (2016). https://doi.org/10.1103/PhysRevE.94.062105

    Article  MathSciNet  Google Scholar 

  32. Wedemann, R.S., de Carvalho, L.A.V.: Some things psychopathologies can tell us about consciousness. In: Villa, A.E.P., Duch, W., Érdi, P., Masulli, F., Palm, G. (eds.) ICANN 2012. LNCS, vol. 7552, pp. 379–386. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-33269-2_48

    Chapter  Google Scholar 

  33. Wedemann, R.S., de Carvalho, L.A.V., Donangelo, R.: Access to symbolization and associativity mechanisms in a model of conscious and unconscious processes. In: Samsonovich, A.V., Jóhannsdóttir, K.R. (eds.) Biologically Inspired Cognitive Architectures 2011, Frontiers in Artificial Intelligence and Applications, vol. 233, pp. 444–449. IOS Press, Amsterdam, Netherlands (2011). https://doi.org/10.3233/978-1-60750-959-2-444

  34. Yan, H., Zhao, L., Hu, L., Wang, X., Wang, E., Wang, J.: Nonequilibrium landscape theory of neural networks. Proc. Nat. Acad. Sci. 110(45), E4185–E4194 (2013). https://doi.org/10.1073/pnas.1310692110

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Instituto de Matemática e Estatística, Universidade do Estado do Rio de Janeiro, Rua São Francisco Xavier 524, Rio de Janeiro, RJ, 20550-900, Brazil

    Roseli S. Wedemann

  2. CeBio y Departamento de Ciencias Básicas, Universidad Nacional del Noroeste de la Provincia de Buenos Aires, UNNOBA, Conicet, Roque Saenz Peña 456, Junin, Argentina

    Angel R. Plastino

Authors
  1. Roseli S. Wedemann
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Angel R. Plastino
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Roseli S. Wedemann .

Editor information

Editors and Affiliations

  1. Department of Applied Informatics, Comenius University in Bratislava, Bratislava, Slovakia

    Igor Farkaš

  2. Department of Applied Mathematics and Computer Science, Technical University of Denmark, Kgs. Lyngby, Denmark

    Paolo Masulli

  3. Department of Informatics, University of Hamburg, Hamburg, Germany

    Stefan Wermter

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Wedemann, R.S., Plastino, A.R. (2020). Nonlinear, Nonequilibrium Landscape Approach to Neural Network Dynamics. In: Farkaš, I., Masulli, P., Wermter, S. (eds) Artificial Neural Networks and Machine Learning – ICANN 2020. ICANN 2020. Lecture Notes in Computer Science(), vol 12397. Springer, Cham. https://doi.org/10.1007/978-3-030-61616-8_15

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-61616-8_15

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-61615-1

  • Online ISBN: 978-3-030-61616-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation