Adaptive Orthogonal Characteristics of Bio-inspired Neural Networks

  • Naohiro IshiiEmail author
  • Toshinori Deguchi
  • Masashi Kawaguchi
  • Hiroshi Sasaki
  • Tokuro Matsuo
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11871)


Neural networks researches are developed for the machine learning and the recent deep learning. Adaptive mechanisms of neural networks are prominent in the visual pathway. In this paper, adaptive orthogonal properties are studied, First, this paper proposes a model of the bio-inspired asymmetric neural networks. The prominent features are the nonlinear characteristics as the squaring and rectification functions, which are observed in the retinal and visual cortex networks. In this paper, the proposed asymmetric network with Gabor filters and the conventional energy model are analyzed from the orthogonality characteristics. Second, it is shown that the biological asymmetric network is effective for generating the orthogonality function using the network correlation computations. Finally, the asymmetric networks with nonlinear characteristics are able to generate independent subspaces, which will be useful for the creation of features spaces and efficient computations in the learning.


Asymmetric neural network Gabor filter Correlation and orthogonality analysis Energy model Linear and nonlinear pathways 


  1. 1.
    Reichard, W.: Autocorrelation, a Principle for the Evaluation of Sensory Information by the Central Nervous System. Sensory Communication. MIT Press, Cambridge (2012)Google Scholar
  2. 2.
    Barlow, H.B.: Possible Principles Underlying the Transformations of Sensory Messages. In: Rosenblith, W.A. (ed.) Sensory Communication. MIT Press, Cambridge (2012)Google Scholar
  3. 3.
    Adelson, E.H., Bergen, J.R.: Spatiotemporal energy models for the perception of motion. J. Opt. Soc. Am. A 2(2), 284–298 (1985)CrossRefGoogle Scholar
  4. 4.
    Pan, H., Jiang, H.: Learning convolutional neural networks using hybrid orthogonal projection and estimation. In: Proceedings Machine Learning Research, ACML 2017, vol. 77, pp. 1–16 (2017)Google Scholar
  5. 5.
    Widrow, B., Greenblatt, A., Kim, Y., Park, D.: The No-Prop algorithm: a new learning algorithm for multilayer neural networks. Neural Netw. 37, 182–188 (2013)CrossRefGoogle Scholar
  6. 6.
    Huang, l., Liu, X., Lang, B., Yu, A.W., Wang, Y., Li, B.: Orthogonal weight normalization: solution to optimization over multiple dependent stiefel manifolds in deep neural networks. In: 32nd AAAI Conference on AI, AAAI-18, pp. 3271–3278 (2018)Google Scholar
  7. 7.
    Shi, W., Gong, Y., Cheng, D., Tao, X., Zheng, N.: Entropy orthogonality based deep discriminative feature learning for object recognition. Pattern Recogn. 81, 71–80 (2018)CrossRefGoogle Scholar
  8. 8.
    Simonceli, E.P., Heeger, D.J.: A model of neuronal responses in visual area MT. Vision. Res. 38, 743–761 (1996)CrossRefGoogle Scholar
  9. 9.
    Heeger D.J.: Models of motion perception, University of Pennsylvania, Department of Computer and Information Science. Technical rep No. MS-CIS-87-91, September 1987Google Scholar
  10. 10.
    Grzywacz, N.M., Yuille, A.L.: A model for the estimate of local image velocity by cells in the visual cortex. Proc. R. Soc. Lond. vol. B 239, 129–161 (1990)CrossRefGoogle Scholar
  11. 11.
    Marmarelis, P.Z., Marmarelis, V.Z.: Analysis of Physiological Systems – The White Noise Approach. Plenum Press, New York (1978)CrossRefGoogle Scholar
  12. 12.
    Sakuranaga, M., Naka, K.-I.: Signal transmission in the Catfish Retina III Transmission to Type-C Cell. J. Neurophysiol. 53(2), 411–428 (1985)CrossRefGoogle Scholar
  13. 13.
    Naka, K.-I., Sakai, H.M., Ishii, N.: Generation of transformation of second order nonlinearity in catfish retina. Ann. Biomed. Eng. 16, 53–64 (1988)CrossRefGoogle Scholar
  14. 14.
    Ishii, N., Deguchi, T., Kawaguchi, M., Sasaki, H.: Motion Detection in Asymmetric Neural Networks. In: Cheng, L., Liu, Q., Ronzhin, A. (eds.) ISNN 2016. LNCS, vol. 9719, pp. 409–417. Springer, Cham (2016). Scholar
  15. 15.
    Ishii, N., Deguchi, T., Kawaguchi, M., Sasaki, H.: Distinctive Features of asymmetric neural networks with gabor filters. In: de Cos Juez, F., et al. (eds.) Intelligent Hybrid Artificial Systems, HAIS 2018. LNCS, vol. 10870, pp. 185–196. Springer, Cham (2018). Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Naohiro Ishii
    • 1
    Email author
  • Toshinori Deguchi
    • 2
  • Masashi Kawaguchi
    • 3
  • Hiroshi Sasaki
    • 4
  • Tokuro Matsuo
    • 1
  1. 1.Advanced Institute of Industrial TechnologyTokyoJapan
  2. 2.Gifu CollegeNational Institute of TechnologyGifuJapan
  3. 3.Suzuka CollegeNational Institute of TechnologyMieJapan
  4. 4.Fukui University of TechnologyFukuiJapan

Personalised recommendations