Advertisement

Bio-Inspired Deep Spiking Neural Network for Image Classification

  • Jingling Li
  • Weitai Hu
  • Ye Yuan
  • Hong Huo
  • Tao FangEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10635)

Abstract

Spiking neural networks (SNNs) are a kind of data-driven and event-driven hierarchical networks, and they are closer to the biological mechanism than other traditional neural networks. In SNNs, signals are transmitted as spikes between neurons, and spike transmission is easily implemented on hardware platform for large-scale real-time deep network computing. However, the unsupervised learning methods for spike neurons, such as the STDP learning methods, generally are ineffective in training deep spiking neural networks for image classification application. In this paper, the network parameters (weights and bias) obtained from training a convolution neural network (CNN), are converted and utilized in a deep spiking neural network with the similar structure as the CNN, which make the deep SNN be capable of classifying images. Since the CNN is composed of analog neurons, there will be some transfer losses in the process of conversion. After the main sources of transfer losses are analyzed, some reasonable optimization strategies are proposed to reduce the losses while retain a higher accuracy, such as max-pooling, softmax and weight normalization. The deep spiking neural network proposed in this paper is closer to the biological mechanism in the design of neurons and our work is helpful for understanding the spike activity of the brain. The proposed deep SNN is evaluated on CIFAR and MNIST benchmarks and the experimental results have shown that the proposed deep SNN outperforms the state-of-the-art spiking network models.

Keywords

Spiking neural networks Convolution neural networks IF neuron Image classification 

Notes

Acknowledgments

This study was partly supported by the National Natural Science Foundation of China (No. 41571402), the Science Fund for Creative Research Groups of the National Natural Science Foundation of China (No. 61221003), and Shanghai Jiao Tong University Agri-X Fund (No. Agri-X2015004).

References

  1. 1.
    Neftci, E.O., Pedroni, B.U., Joshi, S., et al.: Stochastic synapses enable efficient brain-inspired learning machines. Front. Neurosci. 10, 241 (2016)CrossRefGoogle Scholar
  2. 2.
    MFolowosele, F., Vogelstein, R.J., Etienne-Cummings, R.: Real-time silicon implementation of V1 in hierarchical visual information processing. In: Biomedical Circuits and Systems Conference, pp. 181–184. IEEE Press (2008)Google Scholar
  3. 3.
    LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE Press 86, 2278–2324 (1999). Morgan KaufmannGoogle Scholar
  4. 4.
    Brader, J.M., Senn, W., Fusi, S.: Grid Learning real-world stimuli in a neural network with spike-driven synaptic dynamics. Neural Comput. 19, 2881–2912 (2007)CrossRefzbMATHMathSciNetGoogle Scholar
  5. 5.
    Cao, Y., Chen, Y., Khosla, D.: Spiking deep convolutional neural networks for energy-efficient object recognition. Int. J. Comput. Vis. 113, 54–66 (2015)CrossRefMathSciNetGoogle Scholar
  6. 6.
    Diehl, P.U., Neil, D., Binas, J., et al.: Fast-classifying, high-accuracy spiking deep networks through weight and threshold balancing. In: International Joint Conference on Neural Networks (IJCNN) 2015, IEEE, pp. 1–8. IEEE Press (2015)Google Scholar
  7. 7.
    Hunsberger, E., Eliasmith, C.: Spiking deep networks with LIF neurons. arXiv:1510.08829 (2015)
  8. 8.
    Maass, W., Markram, H.: On the computational power of circuits of spiking neurons. J. Comput. Syst. Sci. 69, 593–616 (2004)CrossRefzbMATHMathSciNetGoogle Scholar
  9. 9.
    Camunas-Mesa, L., Zamarreno-Ramos, C., Linares-Barranco, A., et al.: An event-driven multi-kernel convolution processor module for event-driven vision sensors. IEEE J. Solid-State Circ. 47, 504–517 (2012)CrossRefGoogle Scholar
  10. 10.
    O’Connor, P., Neil, D., Liu, S.C., et al.: Real-time classification and sensor fusion with a spiking deep belief network. Front. Neurosci. 7 (2013)Google Scholar
  11. 11.
    Foster, I., Kesselman, C., Nick, J., Tuecke, S.: The Physiology of the Grid: an Open Grid Services Architecture for Distributed Systems Integration. Technical report, Global Grid Forum (2002)Google Scholar
  12. 12.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. Adv. Neural Inf. Process. Syst. 25, 1097–1105 (2012)Google Scholar
  13. 13.
    Vincent, P., Larochelle, H., Bengio, Y., et al.: Extracting and composing robust features with denoising autoencoders. In: Proceedings of the 25th international conference on Machine learning. ACM, pp. 1096–1103 (2008)Google Scholar
  14. 14.
    Angela, J.Y., Giese, M.A., Poggio, T.A.: Biophysiologically plausible implementations of the maximum operation. Neural Comput. 14, 2857–2881 (2002)CrossRefzbMATHGoogle Scholar
  15. 15.
    Ioffe, S., Szegedy, C.: Batch normalization: Accelerating deep network training by reducing internal covariate shift. In: International Conference on Machine Learning, pp. 448–456 (2015)Google Scholar
  16. 16.
    Krizhevsky, A., Hinton, G.: Learning Multiple Layers of Features from Tiny Images (2009)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Jingling Li
    • 1
    • 2
  • Weitai Hu
    • 1
    • 2
  • Ye Yuan
    • 1
    • 2
  • Hong Huo
    • 1
    • 2
  • Tao Fang
    • 1
    • 2
    Email author
  1. 1.Department of AutomationShanghai Jiao Tong UniversityShanghaiChina
  2. 2.Key Laboratory of System Control and Information Processing, Ministry of EducationShanghaiChina

Personalised recommendations