Skip to main content
SpringerLink
Log in

A memristor-based long short term memory circuit

  • Published:
Analog Integrated Circuits and Signal Processing Aims and scope Submit manuscript

Abstract

Long-short term memory (LSTM) is a cognitive architecture that aims to mimic the sequence temporal memory processes in human brain. The state and time-dependent based processing of events is essential to enable contextual processing in several applications such as natural language processing, speech recognition and machine translations. There are many different variants of LSTM and almost all of them are software based. The hardware implementation of LSTM remains as an open problem. In this work, we propose a hardware implementation of LSTM system using memristors. Memristor has proved to mimic behavior of a biological synapse and has promising properties such as smaller size and absence of current leakage among others, making it a suitable element for designing LSTM functions. Sigmoid and hyperbolic tangent functions hardware realization can be performed by using a CMOS-memristor threshold logic circuit. These ideas can be extended for a practical application of implementing sequence learning in real-time sensory processing data.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Benediktsson, J. A., Swain, P. H., & Ersoy, O. K. (1990). Neural network approaches versus statistical methods in classification of multisource remote sensing data. IEEE Transactions on Geoscience and Remote Sensing, 28(4), 540–552.

    Article  Google Scholar 

  2. Specht, D. F. (1990). Probabilistic neural networks. Neural Networks, 3(1), 109–118.

    Article  Google Scholar 

  3. Hopfield, D. A. (1982). Diagonal recurrent neural networks for dynamics control. Proceedings of the National Academy of Sciences of the United States of America, 79, 2554–2561.

    Article  MathSciNet  MATH  Google Scholar 

  4. Jordan, M. I. (1997). Serial order: A parallel distributed processing approach. Advances in Psychology, 121, 471–495.

    Article  Google Scholar 

  5. Lipton, Z. C., Berkowitz, J., Elkan, C. (2015). A critical review of recurrent neural networks for sequence learning. //arXiv preprint arXiv:1506.00019.

  6. Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural Computation, 9(8), 1735–1780.

    Article  Google Scholar 

  7. Chua, L. (1971). Memristor-the missing circuit element. IEEE Transactions on Circuit Theory, 18(5), 507–519.

    Article  Google Scholar 

  8. Strukov, D. B., et al. (2008). The missing memristor found. Nature, 453(7191), 80–83.

    Article  Google Scholar 

  9. Williams, R. S. (2008). How we found the missing memristor. IEEE Spectrum, 45, 12.

    Article  Google Scholar 

  10. Kim, K. H., et al. (2011). A functional hybrid memristor crossbar-array/CMOS system for data storage and neuromorphic applications. Nano Letters, 12(1), 389–395.

    Article  Google Scholar 

  11. Hasan, R., Taha, T. M., & Yakopcic, C. (2017). On-chip training of memristor crossbar based multi-layer neural networks. Microelectronics Journal, 66, 31–40.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Engineering, Nazarbayev University, 53, Kabanbay Batyr Ave., Astana, Kazakhstan

    Kamilya Smagulova, Olga Krestinskaya & Alex Pappachen James

Authors
  1. Kamilya Smagulova
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Olga Krestinskaya
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Alex Pappachen James
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alex Pappachen James.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Smagulova, K., Krestinskaya, O. & James, A.P. A memristor-based long short term memory circuit. Analog Integr Circ Sig Process 95, 467–472 (2018). https://doi.org/10.1007/s10470-018-1180-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10470-018-1180-y

Keywords

Search

Navigation