Abstract
This chapter provides a brief overview of learning algorithms and their implementations on hardware. We focus on memristor based systems for leaning, as this is one of the most promising solutions to implement deep neural network on hardware, due to the small on-chip area and low power consumption.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Azghadi MR., Al-Sarawi S, Iannella N, Abbott D (2012) Design and implementation of BCM rule based on spike-timing dependent plasticity. In: The 2012 international joint conference on neural networks (IJCNN), IEEE, pp 1–7
Cheng M, Xia L, Zhu Z, Cai Y, Xie Y, Wang Y, Yang H (2017) Time: a training-in-memory architecture for memristor-based deep neural networks. In: 2017 54th ACM/EDAC/IEEE design automation conference (DAC), pp 1–6, https://doi.org/10.1145/3061639.3062326
Choe Y (2015) Anti-hebbian learning. In: Encyclopedia of computational neuroscience. Springer, pp 191–193
Gerstner W, Kistler WM (2002) Mathematical formulations of hebbian learning. Biol Cybern 87(5–6):404–415
Hasan R, Taha TM (2014) Enabling back propagation training of memristor crossbar neuromorphic processors. In: 2014 International joint conference on neural networks (IJCNN), IEEE, pp 21–28
Hasan R, Taha TM, Yakopcic C (2017) On-chip training of memristor crossbar based multi-layer neural networks. Microelectron J 66:31–40
Hassan AM, Yang C, Liu C, Li HH, Chen Y (2017) Hybrid spiking-based multi-layered self-learning neuromorphic system based on memristor crossbar arrays. In: 2017 Design, automation & test in Europe conference & exhibition (DATE), IEEE, pp 776–781
Hu X, Feng G, Duan S, Liu L (2017) A memristive multilayer cellular neural network with applications to image processing. IEEE Trans Neural Netw Learn Syst
Kataeva I, Merrikh-Bayat F, Zamanidoost E, Strukov D (2015) Efficient training algorithms for neural networks based on memristive crossbar circuits. In: 2015 International joint conference on neural networks (IJCNN), IEEE, pp 1–8
Krestinskaya O, Dolzhikova I, James AP (2018) Hierarchical temporal memory using memristor networks: a survey. IEEE Trans Emerg Top Comput Intell 2(5):380–395. https://doi.org/10.1109/TETCI.2018.2838124
Krestinskaya O, Ibrayev T, James AP (2018) Hierarchical temporal memory features with memristor logic circuits for pattern recognition. IEEE Trans Comput-Aided Des Integr Circuits Syst 37(6):1143–1156
Krestinskaya O, James AP, Chua LO (2018) Neuro-memristive circuits for edge computing: a review. arXiv preprint arXiv:1807.00962
Krestinskaya O, Salama KN, James AP (2018) Analog backpropagation learning circuits for memristive crossbar neural networks. In: 2018 IEEE international symposium on circuits and systems (ISCAS), IEEE, pp 1–5
Krestinskaya O, Salama KN, James AP (2018) Learning in memristive neural network architectures using analog backpropagation circuits. IEEE Trans Circuits Syst I: Regul Papers
Leonard J, Kramer M (1990) Improvement of the backpropagation algorithm for training neural networks. Comput Chem Eng 14(3):337–341
Li B, Wang Y, Wang Y, Chen Y, Yang H (2014) Training itself: mixed-signal training acceleration for memristor-based neural network. In: 2014 19th Asia and South Pacific design automation conference (ASP-DAC), IEEE, pp 361–366
Nair MV, Dudek P (2015) Gradient-descent-based learning in memristive crossbar arrays. In: 2015 International joint conference on neural networks (IJCNN), IEEE, pp 1–7
Rosenthal E, Greshnikov S, Soudry D, Kvatinsky S (2016) A fully analog memristor-based neural network with online gradient training. In: 2016 IEEE international symposium on circuits and systems (ISCAS), pp 1394–1397, https://doi.org/10.1109/ISCAS.2016.7527510
Sheridan PM, Du C, Lu WD (2016) Feature extraction using memristor networks. IEEE Trans Neural Netw Learn Syst 27(11):2327–2336
Smagulova K, Krestinskaya O, James AP (2018) A memristor-based long short term memory circuit. Analog Integr Circuits Signal Process 95(3):467–472
Soudry D, Castro DD, Gal A, Kolodny A, Kvatinsky S (2015) Memristor-based multilayer neural networks with online gradient descent training. IEEE Trans Neural Netw Learn Syst 26(10):2408–2421. https://doi.org/10.1109/TNNLS.2014.2383395
Soudry D, Di Castro D, Gal A, Kolodny A, Kvatinsky S (2013) Hebbian learning rules with memristors. Israel Institute of Technology, Haifa, Israel
Srinivasan N, Ravichandran V, Chan K, Vidhya J, Ramakirishnan S, Krishnan S (2002) Exponentiated backpropagation algorithm for multilayer feedforward neural networks. In: Proceedings of the 9th international conference on neural information processing, 2002. ICONIP’02, vol 1. IEEE, pp 327–331
Wu X, Saxena V, Zhu K (2015) A CMOS spiking neuron for dense memristor-synapse connectivity for brain-inspired computing. In: 2015 International joint conference on neural networks (IJCNN), IEEE, pp 1–6
Zamanidoost E, Bayat FM, Strukov D, Kataeva I (2015) Manhattan rule training for memristive crossbar circuit pattern classifiers. In: 2015 IEEE 9th international symposium on intelligent signal processing (WISP), IEEE, pp 1–6
Zhang Y, Wang X, Friedman EG (2018) Memristor-based circuit design for multilayer neural networks. IEEE Trans Circuits Syst I: Regul Pap 65(2):677–686
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Krestinskaya, O., James, A.P. (2020). Learning Algorithms and Implementation. In: James, A. (eds) Deep Learning Classifiers with Memristive Networks. Modeling and Optimization in Science and Technologies, vol 14. Springer, Cham. https://doi.org/10.1007/978-3-030-14524-8_7
Download citation
DOI: https://doi.org/10.1007/978-3-030-14524-8_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-14522-4
Online ISBN: 978-3-030-14524-8
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)