Abstract
Convolutional Neural Network (CNN) has gotten admirable performance in the domain of image recognition. Nevertheless, the training of CNN on CPU or GPU is energy-intensive and time-consuming. Memristor crossbar is an alternative of the specific chip for CNN application. But it is hard to tune the memristor to certain conductance precisely. This work simulates the performance change of memristor-based CNN when memristor is with stochasticity. The simulation results demonstrate that stochastic memristor-based CNN performs better on CIFAR-10 dataset when memristive stochasticity is low. This is an encouragement for the engineer of memristor crossbar chip and edge computing application.
This work was supported by the National Natural Science Foundation of China under grant 61876209 and the National Key R&D Program of China under Grant 2017YFC1501301.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436–444 (2015)
Jouppi, N.P., et al.: In-datacenter performance analysis of a tensor processing unit. In: 2017 ACM/IEEE 44th Annual International Symposium on Computer Architecture (ISCA), Toronto, pp. 1–12. IEEE (2017)
Chua, L.: Memristor-the missing circuit element. IEEE Trans. Circ. Theory 18(5), 507–519 (1971)
Strukov, D.B., Snider, G.S., Stewart, D.R., Williams, R.S.: The missing memristor found. Nature 453(7191), 80 (2008)
Xia, Q., et al.: Memristor-CMOS hybrid integrated circuits for reconfigurable logic. Nano Lett. 9(10), 3640–3645 (2009)
Yang, J.J., et al.: High switching endurance in \(TaO_x\) memristive devices. Appl. Phys. Lett. 97(23), 232102 (2010)
Soudry, D., Di Castro, D., Gal, A., Kolodny, A., Kvatinsky, S.: Memristor-based multilayer neural networks with online gradient descent training. IEEE Trans. Neural Netw. Learn. Syst. 26(10), 2408–2421 (2015)
Zeng, X., Wen, S., Zeng, Z., Huang, T.: Design of memristor-based image convolution calculation in convolutional neural network. Neural Comput. Appl. 30(2), 503–508 (2018)
Liu, J., Li, Z., Tang, Y., Hu, W., Wu, J.: 3D convolutional neural network based on memristor for video recognition. Pattern Recogn. Lett. (2018). https://doi.org/10.1016/j.patrec.2018.12.005
Yakopcic, C., Alom, M.Z., Taha, T.M.: Extremely parallel memristor crossbar architecture for convolutional neural network implementation. In: 2017 International Joint Conference on Neural Networks (IJCNN), Alaska, pp. 1696–1703. IEEE (2017)
Feinberg, B., Wang, S., Ipek, E.: Making memristive neural network accelerators reliable. In: 2018 IEEE International Symposium on High Performance Computer Architecture (HPCA), Vienna, pp. 52–65. IEEE (2018)
Naous, R., Al-Shedivat, M., Salama, K.N.: Stochasticity modeling in memristors. IEEE Trans. Nanotechnol. 15(1), 15–28 (2016)
Li, C., et al.: Analogue signal and image processing with large memristor crossbars. Nat. Electron. 1(1), 52 (2018)
Krizhevsky, A., Hinton, G.: Learning multiple layers of features from tiny images. Technical report, vol. 1, no. 4, p. 7 (2009)
Glorot, X., Bengio, Y.: Understanding the difficulty of training deep feedforward neural networks. In: Proceedings of the 13th International Conference on Artificial Intelligence and Statistics, pp. 249–256 (2010)
Gao, L., Chen, P.Y., Yu, S.: Programming protocol optimization for analog weight tuning in resistive memories. IEEE Electron Device Lett. 36(11), 1157–1159 (2015)
Merced-Grafals, E.J., Davila, N., Ge, N., Williams, R.S., Strachan, J.P.: Repeatable, accurate, and high speed multi-level programming of memristor 1T1R arrays for power efficient analog computing applications. Nanotechnology 27(36), 365202 (2016)
Zhang, Y., Wang, X., Friedman, E.G.: Memristor-based circuit design for multilayer neural networks. IEEE Transactions on Circuits and Systems I: Regular Papers 65(2), 677–686 (2018)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Wu, K., Wang, X., Li, M. (2019). Better Performance of Memristive Convolutional Neural Network Due to Stochastic Memristors. In: Lu, H., Tang, H., Wang, Z. (eds) Advances in Neural Networks – ISNN 2019. ISNN 2019. Lecture Notes in Computer Science(), vol 11554. Springer, Cham. https://doi.org/10.1007/978-3-030-22796-8_5
Download citation
DOI: https://doi.org/10.1007/978-3-030-22796-8_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-22795-1
Online ISBN: 978-3-030-22796-8
eBook Packages: Computer ScienceComputer Science (R0)