Bio-Inspired Deep Spiking Neural Network for Image Classification
Spiking neural networks (SNNs) are a kind of data-driven and event-driven hierarchical networks, and they are closer to the biological mechanism than other traditional neural networks. In SNNs, signals are transmitted as spikes between neurons, and spike transmission is easily implemented on hardware platform for large-scale real-time deep network computing. However, the unsupervised learning methods for spike neurons, such as the STDP learning methods, generally are ineffective in training deep spiking neural networks for image classification application. In this paper, the network parameters (weights and bias) obtained from training a convolution neural network (CNN), are converted and utilized in a deep spiking neural network with the similar structure as the CNN, which make the deep SNN be capable of classifying images. Since the CNN is composed of analog neurons, there will be some transfer losses in the process of conversion. After the main sources of transfer losses are analyzed, some reasonable optimization strategies are proposed to reduce the losses while retain a higher accuracy, such as max-pooling, softmax and weight normalization. The deep spiking neural network proposed in this paper is closer to the biological mechanism in the design of neurons and our work is helpful for understanding the spike activity of the brain. The proposed deep SNN is evaluated on CIFAR and MNIST benchmarks and the experimental results have shown that the proposed deep SNN outperforms the state-of-the-art spiking network models.
KeywordsSpiking neural networks Convolution neural networks IF neuron Image classification
This study was partly supported by the National Natural Science Foundation of China (No. 41571402), the Science Fund for Creative Research Groups of the National Natural Science Foundation of China (No. 61221003), and Shanghai Jiao Tong University Agri-X Fund (No. Agri-X2015004).
- 2.MFolowosele, F., Vogelstein, R.J., Etienne-Cummings, R.: Real-time silicon implementation of V1 in hierarchical visual information processing. In: Biomedical Circuits and Systems Conference, pp. 181–184. IEEE Press (2008)Google Scholar
- 3.LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE Press 86, 2278–2324 (1999). Morgan KaufmannGoogle Scholar
- 6.Diehl, P.U., Neil, D., Binas, J., et al.: Fast-classifying, high-accuracy spiking deep networks through weight and threshold balancing. In: International Joint Conference on Neural Networks (IJCNN) 2015, IEEE, pp. 1–8. IEEE Press (2015)Google Scholar
- 7.Hunsberger, E., Eliasmith, C.: Spiking deep networks with LIF neurons. arXiv:1510.08829 (2015)
- 10.O’Connor, P., Neil, D., Liu, S.C., et al.: Real-time classification and sensor fusion with a spiking deep belief network. Front. Neurosci. 7 (2013)Google Scholar
- 11.Foster, I., Kesselman, C., Nick, J., Tuecke, S.: The Physiology of the Grid: an Open Grid Services Architecture for Distributed Systems Integration. Technical report, Global Grid Forum (2002)Google Scholar
- 12.Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. Adv. Neural Inf. Process. Syst. 25, 1097–1105 (2012)Google Scholar
- 13.Vincent, P., Larochelle, H., Bengio, Y., et al.: Extracting and composing robust features with denoising autoencoders. In: Proceedings of the 25th international conference on Machine learning. ACM, pp. 1096–1103 (2008)Google Scholar
- 15.Ioffe, S., Szegedy, C.: Batch normalization: Accelerating deep network training by reducing internal covariate shift. In: International Conference on Machine Learning, pp. 448–456 (2015)Google Scholar
- 16.Krizhevsky, A., Hinton, G.: Learning Multiple Layers of Features from Tiny Images (2009)Google Scholar