Advertisement

Better Performance of Memristive Convolutional Neural Network Due to Stochastic Memristors

  • Kechuan Wu
  • Xiaoping WangEmail author
  • Mian Li
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11554)

Abstract

Convolutional Neural Network (CNN) has gotten admirable performance in the domain of image recognition. Nevertheless, the training of CNN on CPU or GPU is energy-intensive and time-consuming. Memristor crossbar is an alternative of the specific chip for CNN application. But it is hard to tune the memristor to certain conductance precisely. This work simulates the performance change of memristor-based CNN when memristor is with stochasticity. The simulation results demonstrate that stochastic memristor-based CNN performs better on CIFAR-10 dataset when memristive stochasticity is low. This is an encouragement for the engineer of memristor crossbar chip and edge computing application.

Keywords

Stochastic memristor Convolutional neural network Dataset noise 

References

  1. 1.
    LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436–444 (2015)Google Scholar
  2. 2.
    Jouppi, N.P., et al.: In-datacenter performance analysis of a tensor processing unit. In: 2017 ACM/IEEE 44th Annual International Symposium on Computer Architecture (ISCA), Toronto, pp. 1–12. IEEE (2017)Google Scholar
  3. 3.
    Chua, L.: Memristor-the missing circuit element. IEEE Trans. Circ. Theory 18(5), 507–519 (1971)Google Scholar
  4. 4.
    Strukov, D.B., Snider, G.S., Stewart, D.R., Williams, R.S.: The missing memristor found. Nature 453(7191), 80 (2008)Google Scholar
  5. 5.
    Xia, Q., et al.: Memristor-CMOS hybrid integrated circuits for reconfigurable logic. Nano Lett. 9(10), 3640–3645 (2009)Google Scholar
  6. 6.
    Yang, J.J., et al.: High switching endurance in \(TaO_x\) memristive devices. Appl. Phys. Lett. 97(23), 232102 (2010)Google Scholar
  7. 7.
    Soudry, D., Di Castro, D., Gal, A., Kolodny, A., Kvatinsky, S.: Memristor-based multilayer neural networks with online gradient descent training. IEEE Trans. Neural Netw. Learn. Syst. 26(10), 2408–2421 (2015)MathSciNetGoogle Scholar
  8. 8.
    Zeng, X., Wen, S., Zeng, Z., Huang, T.: Design of memristor-based image convolution calculation in convolutional neural network. Neural Comput. Appl. 30(2), 503–508 (2018)Google Scholar
  9. 9.
    Liu, J., Li, Z., Tang, Y., Hu, W., Wu, J.: 3D convolutional neural network based on memristor for video recognition. Pattern Recogn. Lett. (2018).  https://doi.org/10.1016/j.patrec.2018.12.005Google Scholar
  10. 10.
    Yakopcic, C., Alom, M.Z., Taha, T.M.: Extremely parallel memristor crossbar architecture for convolutional neural network implementation. In: 2017 International Joint Conference on Neural Networks (IJCNN), Alaska, pp. 1696–1703. IEEE (2017)Google Scholar
  11. 11.
    Feinberg, B., Wang, S., Ipek, E.: Making memristive neural network accelerators reliable. In: 2018 IEEE International Symposium on High Performance Computer Architecture (HPCA), Vienna, pp. 52–65. IEEE (2018)Google Scholar
  12. 12.
    Naous, R., Al-Shedivat, M., Salama, K.N.: Stochasticity modeling in memristors. IEEE Trans. Nanotechnol. 15(1), 15–28 (2016)Google Scholar
  13. 13.
    Li, C., et al.: Analogue signal and image processing with large memristor crossbars. Nat. Electron. 1(1), 52 (2018)MathSciNetGoogle Scholar
  14. 14.
    Krizhevsky, A., Hinton, G.: Learning multiple layers of features from tiny images. Technical report, vol. 1, no. 4, p. 7 (2009)Google Scholar
  15. 15.
    Glorot, X., Bengio, Y.: Understanding the difficulty of training deep feedforward neural networks. In: Proceedings of the 13th International Conference on Artificial Intelligence and Statistics, pp. 249–256 (2010)Google Scholar
  16. 16.
    Gao, L., Chen, P.Y., Yu, S.: Programming protocol optimization for analog weight tuning in resistive memories. IEEE Electron Device Lett. 36(11), 1157–1159 (2015)Google Scholar
  17. 17.
    Merced-Grafals, E.J., Davila, N., Ge, N., Williams, R.S., Strachan, J.P.: Repeatable, accurate, and high speed multi-level programming of memristor 1T1R arrays for power efficient analog computing applications. Nanotechnology 27(36), 365202 (2016)Google Scholar
  18. 18.
    Zhang, Y., Wang, X., Friedman, E.G.: Memristor-based circuit design for multilayer neural networks. IEEE Transactions on Circuits and Systems I: Regular Papers 65(2), 677–686 (2018)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.School of Artificial Intelligence and AutomationHuazhong University of Science and TechnologyWuhanChina
  2. 2.Key Laboratory of Image Processing and Intelligent Control of Education Ministry of ChinaWuhanPeople’s Republic of China

Personalised recommendations