Advertisement

Global Asymptotic Stability and Stabilization of Long Short-Term Memory Neural Networks with Constant Weights and Biases

  • Shankar A. DekaEmail author
  • Dušan M. Stipanović
  • Boris Murmann
  • Claire J. Tomlin
Technical Note
  • 235 Downloads

Abstract

In this paper, a global asymptotic stability condition for Long Short-Term Memory neural networks is presented. Since this condition is formulated in terms of the networks’ weight matrices and biases that are essentially control variables, the same condition can be viewed as a way to globally asymptotically stabilize these networks. The condition and how to compute numerical values for the weight matrices and biases are illustrated by a number of numerical examples.

Keywords

Neural networks Global asymptotic stability Stabilization 

Mathematics Subject Classification

93D20 68T99 

References

  1. 1.
    Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)CrossRefGoogle Scholar
  2. 2.
    Greff, K., Srivastava, R.K., Koutnk, J., Steunebrink, B.R., Schmidhuber, J.: LSTM: a search space odyssey. IEEE Trans. Neural Netw. Learn. Syst. 28(10), 2222–2232 (2017)MathSciNetCrossRefGoogle Scholar
  3. 3.
    Guckenheimer, J., Holmes, P.: Nonlinear Oscillations, Dynamical Systems, and Bifurcations of Vector Fields. Springer, New York (2002)zbMATHGoogle Scholar
  4. 4.
    Hirsch, M.W., Smale, S., Devaney, R.L.: Differential Equations, Dynamical Systems, and an Introduction to Chaos, 3rd edn. Elsevier Inc., Waltham (2013)zbMATHGoogle Scholar
  5. 5.
    Bertschinger, N., Natschläger, T.: Real-time computation at the edge of chaos in recurrent neural networks. Neural Comput. 16(7), 1413–1436 (2004)CrossRefzbMATHGoogle Scholar
  6. 6.
    Stipanović, D.M., Murmann, B., Causo, M., Lekić, A., Rubies Royo, V., Tomlin, C.J., Beigne, E., Thuries, S., Zarudniev, M., Lesecq, S.: Some local stability properties of an autonomous long short-term memory neural network model. In: 2018 IEEE International Symposium on Circuits and Systems (ISCAS), pp. 1–5. IEEE, Florence (2018). http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=8350958&isnumber=8350884. Accessed 29 Jan 2018
  7. 7.
    LaSalle, J.P.: The Stability and Control of Discrete Processes. Springer, New York (1986)CrossRefzbMATHGoogle Scholar
  8. 8.
    Kalman, R.E., Bertram, J.E.: Control system analysis and design via the second method of lyapunov: I. Continuous-time systems. J. Basic Eng. 82(2), 371–393 (1960)MathSciNetCrossRefGoogle Scholar
  9. 9.
    Graves, A.: Generating sequences with recurrent neural networks. arXiv (2013). ArXiv preprint arXiv:1308.0850
  10. 10.
    Horn, R.A., Johnson, C.R.: Matrix Analysis. Cambridge University Press, Cambridge (1990)zbMATHGoogle Scholar
  11. 11.
    Fleming, W.: Functions of Several Variables. Springer, Berlin (2012)Google Scholar
  12. 12.
    Siljak, D.D.: Decentralized control of complex systems. Academic Press Inc, New York (1991)zbMATHGoogle Scholar
  13. 13.
    Lakshmikantham, V., Trigiante, D.: Theory of Difference Equations: Numerical Methods and Applications, vol. 251. Marcel Dekker, New York (2002)CrossRefzbMATHGoogle Scholar
  14. 14.
    Laurent, T., von Brecht, J.: A recurrent neural network without chaos. arXiv preprint arXiv:1612.06212 (2016)

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Coordinated Science Laboratory, and MechSE DepartmentUniversity of Illinois at Urbana-ChampaignUrbanaUSA
  2. 2.Electrical Engineering DepartmentStanford UniversityStanfordUSA
  3. 3.EECS DepartmentUniversity of CaliforniaBerkeleyUSA

Personalised recommendations