A survey on LSTM memristive neural network architectures and applications
- 6 Downloads
The recurrent neural networks (RNN) found to be an effective tool for approximating dynamic systems dealing with time and order dependent data such as video, audio and others. Long short-term memory (LSTM) is a recurrent neural network with a state memory and multilayer cell structure. Hardware acceleration of LSTM using memristor circuit is an emerging topic of study. In this work, we look at history and reasons why LSTM neural network has been developed. We provide a tutorial survey on the existing LSTM methods and highlight the recent developments in memristive LSTM architectures.
Unable to display preview. Download preview PDF.
- 1.K. Adam, K. Smagulova, A.P. James, Memristive LSTM network hardware architecture for time-series predictive modeling problems, in 2018 IEEE Asia Pacific Conference on Circuits and Systems (APCCAS) (IEEE, 2018), pp. 459–462Google Scholar
- 2.K. Adam, K. Smagulova, O. Krestinskaya, A.P. James, Wafer quality inspection using memristive LSTM, ANN, DNN, and HTM, https://arXiv:11809.10438 (2018)
- 3.F. Conti, L. Cavigelli, G. Paulin, I. Susmelj, L. Benini, Chipmunk: A systolically scalable 0.9 mm 2, 3.08 Gop/s/mW at 1.2 mW accelerator for near-sensor recurrent neural network inference, in Custom Integrated Circuits Conference (CICC) (IEEE, 2018), pp. 1–4Google Scholar
- 4.F.A. Gers, J. Schmidhuber, F. Cummins, Learning to forget: Continual prediction with LSTM (IEEE, London, 1999), pp. 850–855Google Scholar
- 5.F.A. Gers, N.N. Schraudolph, J. Schmidhuber, J. Mach. Learn. Res. 3, 115 (2002)Google Scholar
- 6.A. Gomez, Backpropogating an LSTM: a numerical example, Aidan Gomez blog at Medium, 2016Google Scholar
- 9.A. Karpathy, The unreasonable effectiveness of recurrent neural networks, 2015 (2016), http://karpathy.github.io/2015/05/21/rnn-effectiveness
- 11.Z.C. Lipton, J. Berkowitz, C. Elkan, A critical review of recurrent neural networks for sequence learning, https://arXiv:1506.00019 (2015)
- 12.C. Olah, Understanding LSTM networks (2015)Google Scholar
- 13.K. Smagulova, K. Adam, O. Krestinskaya, A.P. James, Design of cmos-memristor circuits for lstm architecture, https://arXiv:1806.02366 (2018)
- 15.Z. Sun, Y. Zhu, Y. Zheng, H. Wu, Z. Cao, P. Xiong, J. Hou, T. Huang, Z. Que, FPGA acceleration of lstm based on data for test flight, in 2018 IEEE International Conference on Smart Cloud (SmartCloud) (IEEE, 2018), pp. 1–6Google Scholar
- 16.I. Sutskever, O. Vinyals, Q.V. Le, Sequence to sequence learning with neural networks, in Advances in Neural Information Processing Systems (2014), pp. 3104–3112Google Scholar