Abstract
This paper proposes a modification of convolutional neural network (CNN) with biologically inspired structure, retinal structure and ON/OFF rectified linear unit (ON/OFF ReLU). Retinal structure enhances input images by center surround difference of green and red, blue and yellow components and creates positive results and negative results like ON/OFF visual pathway of retina to make totally 12 feature channels. This ON/OFF concept also adopted to each convolutional layer of CNN and we call this ON/OFF ReLU. Conventional ReLU only passes positive results of each convolutional layer so it loses negative information such as how much it was negative and also loses learning chance if results are saturated to zero but proposed model uses both positive and negative information so that additional learning chance also exist through negative results. Experimental results show how much the negative information and retinal structure improves the performance of CNN with public data.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Itti, L., Koch, C.: Computational modelling of visual attention. Nat. Rev. Neurosci. 2(3), 194–203 (2001)
Ban, S.-W., Lee, I., Lee, M.: Dynamic visual selective attention model. Neurocomputing 71(4), 853–856 (2008)
LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)
Hubel, D.H., Wiesel, T.N.: Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex. J. Physiol. 160(1), 106 (1962)
Werblin, F.S., Dowling, J.E.: Organization of the retina of the mudpuppy, Necturus maculosus. II. Intracellular recording. J. Neurophysiol. 32(3), 339–355 (1969)
Gjorgjieva, J., Sompolinsky, H., Meister, M.: Benefits of pathway splitting in sensory coding. J. Neurosci. 34(36), 12127–12144 (2014)
Pitkow, X., Meister, M.: Decorrelation and efficient coding by retinal ganglion cells. Nat. Neurosci. 15(4), 628–635 (2012)
He K., Zhang X., Ren S. and Sun J.: Delving deep into rectifiers: surpassing human-level performance on imagenet classification (2015). arXiv preprint arXiv:1502.01852
Dacey, D., Packer, O.S., Diller, L., Brainard, D., Peterson, B., Lee, B.: Center surround receptive field structure of cone bipolar cells in primate retina. Vision Res. 40(14), 1801–1811 (2000)
Schiller, P.H.: Parallel information processing channels created in the retina. Proc. Natl. Acad. Sci. 107(40), 17087–17094 (2010)
Kim, B., Okuno, H., Yagi, T., Lee, M.: Implementation of visual attention system using artificial retina chip and bottom-up saliency map model. In: Lu, B.-L., Zhang, L., Kwok, J. (eds.) ICONIP 2011, Part III. LNCS, vol. 7064, pp. 416–423. Springer, Heidelberg (2011)
Glorot, X., Bordes, A., Bengio, Y.: Deep sparse rectifier neural networks. In: International Conference on Artificial Intelligence and Statistics (2011)
Hartline, H.K.: The response of single optic nerve fibers of the vertebrate eye to illumination of the retina. Am. J. Physiol. 121, 400–415 (1938)
Acknowledgement
This work was supported by the Industrial Strategic Technology Development Program (10044009) funded by the Ministry of Trade, Industry and Energy (MOTIE, Korea) and was supported by Regional Specialized Industry R&D program funded by the Ministry of Trade, Industry and Energy (R0002982).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Kim, J., Kim, S., Lee, M. (2015). Convolutional Neural Network with Biologically Inspired ON/OFF ReLU. In: Arik, S., Huang, T., Lai, W., Liu, Q. (eds) Neural Information Processing. ICONIP 2015. Lecture Notes in Computer Science(), vol 9492. Springer, Cham. https://doi.org/10.1007/978-3-319-26561-2_38
Download citation
DOI: https://doi.org/10.1007/978-3-319-26561-2_38
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-26560-5
Online ISBN: 978-3-319-26561-2
eBook Packages: Computer ScienceComputer Science (R0)