A memristor-based long short term memory circuit

  • Kamilya Smagulova
  • Olga Krestinskaya
  • Alex Pappachen James
Article
First Online:
Received:
Revised:
Accepted:
  • 48 Downloads

Abstract

Long-short term memory (LSTM) is a cognitive architecture that aims to mimic the sequence temporal memory processes in human brain. The state and time-dependent based processing of events is essential to enable contextual processing in several applications such as natural language processing, speech recognition and machine translations. There are many different variants of LSTM and almost all of them are software based. The hardware implementation of LSTM remains as an open problem. In this work, we propose a hardware implementation of LSTM system using memristors. Memristor has proved to mimic behavior of a biological synapse and has promising properties such as smaller size and absence of current leakage among others, making it a suitable element for designing LSTM functions. Sigmoid and hyperbolic tangent functions hardware realization can be performed by using a CMOS-memristor threshold logic circuit. These ideas can be extended for a practical application of implementing sequence learning in real-time sensory processing data.

Keywords

LSTM Memristor Memristor crossbar array 
This is a preview of subscription content, log in to check access

References

  1. 1.
    Benediktsson, J. A., Swain, P. H., & Ersoy, O. K. (1990). Neural network approaches versus statistical methods in classification of multisource remote sensing data. IEEE Transactions on Geoscience and Remote Sensing, 28(4), 540–552.CrossRefGoogle Scholar
  2. 2.
    Specht, D. F. (1990). Probabilistic neural networks. Neural Networks, 3(1), 109–118.CrossRefGoogle Scholar
  3. 3.
    Hopfield, D. A. (1982). Diagonal recurrent neural networks for dynamics control. Proceedings of the National Academy of Sciences of the United States of America, 79, 2554–2561.MathSciNetCrossRefMATHGoogle Scholar
  4. 4.
    Jordan, M. I. (1997). Serial order: A parallel distributed processing approach. Advances in Psychology, 121, 471–495.CrossRefGoogle Scholar
  5. 5.
    Lipton, Z. C., Berkowitz, J., Elkan, C. (2015). A critical review of recurrent neural networks for sequence learning. //arXiv preprint arXiv:1506.00019.
  6. 6.
    Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural Computation, 9(8), 1735–1780.CrossRefGoogle Scholar
  7. 7.
    Chua, L. (1971). Memristor-the missing circuit element. IEEE Transactions on Circuit Theory, 18(5), 507–519.CrossRefGoogle Scholar
  8. 8.
    Strukov, D. B., et al. (2008). The missing memristor found. Nature, 453(7191), 80–83.CrossRefGoogle Scholar
  9. 9.
    Williams, R. S. (2008). How we found the missing memristor. IEEE Spectrum, 45, 12.CrossRefGoogle Scholar
  10. 10.
    Kim, K. H., et al. (2011). A functional hybrid memristor crossbar-array/CMOS system for data storage and neuromorphic applications. Nano Letters, 12(1), 389–395.CrossRefGoogle Scholar
  11. 11.
    Hasan, R., Taha, T. M., & Yakopcic, C. (2017). On-chip training of memristor crossbar based multi-layer neural networks. Microelectronics Journal, 66, 31–40.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.School of EngineeringNazarbayev UniversityAstanaKazakhstan

Personalised recommendations