Abstract
There has been a great increase in performance of deep neural networks. However, for mobile devices which are not equipped with GPU (Graphics Processing Unit) or powerful CPU (Central Processing Unit), it is still impossible to deal with such a large amount of data in real time. In this paper, preliminary results in spike neural encoding methods reducing the amount of the input and computational load by mimicking the neuronal firing are presented. For this, two neuron models, leaky integrate-and-fire (LIF) model and simplified IF model, are exploited for transforming the input image to the spike image. For the evaluation, MNIST datasets are encoded and tested in deep neural networks for checking the loss of information. The proposed spike encoding modules using neuron models will be able to greatly help reduce required computation by using spike input data in low powered mobile devices.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Krizhevsky, A., Sutskever, I., Hinton, G.: ImageNet classification with deep convolutional neural networks. In: Proceedings of Neural Information Processing Systems (NIPS), pp. 1097–1105 (2012)
Hinton, G., et al.: Deep neural networks for acoustic modeling in speech recognition. IEEE Sig. Process. Mag. 29(6), 82–97 (2012)
Mikolov, T., Karafiat, M., Burget, L., Cernocky, J., Khudanpur, S.: Recurrent neural network based language model. In: Interspeech, vol. 2, p. 3 (2010)
Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. In: Proceedings of Neural Information Processing Systems (NIPS) (2015)
Tavanaei, A., Maida, A.: Bio-inspired spiking convolutional neural network using layer-wise sparse coding and STDP learning. arXiv preprint arXiv:1611.03000 (2016)
Trabelsi, C., et al.: Deep complex networks. arXiv preprint arXiv:1705.09792 (2017)
Courbariaux, M., Hubara, I., Soudry, D., El-Yaniv, R., Bengio, Y.: Binarized neural networks: training deep neural networks with weights and activations constrained to +1 or −1. arXiv preprint arXiv:1602.02830 (2016)
Long, L.: Scalable biologically inspired neural networks with spike time based learning. In: Proceedings Learning and Adaptive Behaviors for Robotic Systems, LAB-RS 2008, pp. 29–34. ECSIS (2008)
Lee, S., Kim, K., Kim, J., Kim, Y., Myung, H.: Spike-inspired deep neural network design using binary weight. In: Proceedings of International Conference on Control, Automation and Systems (ICCAS) (2018)
Kim, K., Kim, J., Lee, S., Kim, Y., Myung, H.: Development of the image to spike conversion algorithm for the deep neural networks. In: Proceedings of International Conference on Control, Automation and Systems (ICCAS) (2018)
Abbott, L.F.: Lapicque’s introduction of the integrate-and-fire model neuron (1907). Brain Res. Bull. 50(5–6), 303–304 (1999)
Xie, X., Qu, H., Yi, Z., Kurths, J.: Efficient training of supervised spiking neural network via accurate synaptic-efficiency adjustment method. IEEE Trans. Neural Netw. Learn. Syst. 28(6), 1411–14247 (2017)
Izhikevich, E.M.: Simple model of spiking neurons. IEEE Trans. Neural Netw. 14(6), 1569–1572 (2013)
Hodgkin, A., Huxley, A.: A quantitative description of membrane current and its application to conduction and excitation in nerve. J. Physiol. 117(4), 500–544 (1952)
Hodgkin, A., Huxley, A., Katz, B.: Measurement of current-voltage relations in the membrane of the giant axon of Loligo. J. Physiol. 116(4), 424–448 (1952)
Hodgkin, A., Huxley, A.: Currents carried by sodium and potassium ions through the membrane of the giant axon of Loligo. J. Physiol. 116(4), 449–472 (1952)
Hodgkin, A., Huxley, A.: The components of membrane conductance in the giant axon of Loligo. J. Physiol. 116(4), 473–496 (1952)
Runge, C.: Über die numerische Auflösung von Differentialgleichungen. Mathematische Annalen 46(2), 167–178 (1895). Springer
LeCun, Y., Cortes, C., Burges, C.: MNIST handwritten digit database. AT&T Labs, (2018). http://yann.lecun.com/exdb/mnist
Krizhevsky, A., Sutskever, I., Hinton, G.: ImageNet classification with deep convolutional neural networks. In: Proceedings of the 25th International Conference on Neural Information Processing Systems (NIPS), pp. 1097–1105 (2012)
Szegedy, C., et al.: Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1–9 (2015)
NVIDIA: DIGITS Deep Learning Framework (2018). https://github.com/NVIDIA/DIGITS
Nemoto, I., Saito, K.: A complex-valued version of Nagumo-Sato model of a single neuron and its behavior. Neural Netw. 15(7), 833–853 (2002)
Iakymchuk, T., Rosado-Muñoz, A., Guerrero-MartÃnez, J., Bataller-Mompeán, M., Francés-VÃllora, J.: Simplified spiking neural network architecture and STDP learning algorithm applied to image classification. EURASIP J. Image Video Process. (2015)
Bologna, L., et al.: A closed-loop neurobotic system for fine touch sensing. J. Neural Eng. 10(4), 046019 (2016)
Thorpe, S., Delorme, A., Van Rullen, R.: Spike-based strategies for rapid processing. Neural Netw. 14(6-7), 715–725 (2001)
Liu, Q., et al.: Benchmarking spike-based visual recognition: a dataset and evaluation. Front. Neurosci. 10, 496 (2016)
Heeger, D.: Poisson model of spike generation. Handout Univ. Standford 5, 1–13 (2000)
Fatahi, M., et al.: evt_MNIST: a spike based version of traditional MNIST. arXiv preprint arXiv:1604.06751 (2016)
Sen, B., Steve, F.: Evaluating rank-order code performance using a biologically-derived retinal model. In: International Joint Conference on IEEE (IJCNN), pp. 2867–2874 (2009)
Serrano-Gotarredona, T., Linares-Barranco, B.: A 128 × 128 1.5% contrast sensitivity 0.9% FPN 3 µs latency 4 mW asynchronous frame-free dynamic vision sensor using transimpedance preamplifiers. J. Solid-State Circ. 48(3), 827–838 (2013)
Acknowledgement
This work was supported by the ICT R&D program of MSIP/IITP (2016-0-00563, Research on Adaptive Machine Learning Technology Development for Intelligent Autonomous Digital Companion). The students are supported by Korea Minister of Ministry of Land, Infrastructure and Transport (MOLIT) as U-City Master and Doctor Course Grant Program.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Kim, Y., Lee, S., Song, W., Myung, H. (2019). Spike Encoding Modules Using Neuron Model in Neural Networks. In: Kim, JH., Myung, H., Lee, SM. (eds) Robot Intelligence Technology and Applications. RiTA 2018. Communications in Computer and Information Science, vol 1015. Springer, Singapore. https://doi.org/10.1007/978-981-13-7780-8_16
Download citation
DOI: https://doi.org/10.1007/978-981-13-7780-8_16
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-13-7779-2
Online ISBN: 978-981-13-7780-8
eBook Packages: Computer ScienceComputer Science (R0)