Science China Technological Sciences

, Volume 62, Issue 4, pp 628–634 | Cite as

Emergence of higher-level neuron properties using a hierarchical statistical distribution model

  • Ning Xian
  • YiMin Deng
  • HaiBin DuanEmail author


Essential to visual tasks such as object recognition is the formation of effective representations that generalize from specific instances of visual input. Neurons in primary visual cortex are typically hypothesized to efficiently encode image structures such as edge and textures from natural scenes. Here this paper proposed a novel hierarchical statistical distribution model to generalize higher-level neuron properties and encode distributed regularities that characterize local image regions. Two layers of our hierarchical model are presented to extract spiking activities of excitatory neurons decorrelated by inhibitory neurons and to construct the statistical patterns of input data, respectively. Trained on whitened natural images, parameters including neural connecting weights and distribution coding weights are estimated by their corresponding learning rules. To prove the feasibility and effectiveness of our model, several experiments on natural images are conducted. Adapting our model to natural scenes yields a distributed representation for higher-order statistical regularities. Comparison results provide insight into higher-level neurons which encode more abstract and invariant properties.


statistical distribution spiking activity object classification localization 


  1. 1.
    van Hateren J H. Theoretical predictions of spatiotemporal receptive fields of fly LMCs, and experimental validation. J Comp Physiol A, 1992, 171: 157–170CrossRefGoogle Scholar
  2. 2.
    Freeman J, Ziemba C M, Heeger D J, et al. A functional and perceptual signature of the second visual area in primates. Nat Neurosci, 2013, 16: 974–981CrossRefGoogle Scholar
  3. 3.
    Field G D, Gauthier J L, Sher A, et al. Functional connectivity in the retina at the resolution of photoreceptors. Nature, 2010, 467: 673–677CrossRefGoogle Scholar
  4. 4.
    Dan Y, Atick J J, Reid R C. Efficient coding of natural scenes in the lateral geniculate nucleus: Experimental test of a computational theory. J Neurosci, 1996, 16: 3351–3362CrossRefGoogle Scholar
  5. 5.
    Olshausen B A, Field D J. Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature, 1996, 381: 607–609CrossRefGoogle Scholar
  6. 6.
    Duan H B, Li P. Bio-inspired Computation in Unmanned Aerial Vehicles. Berlin, Heidelberg: Springer, 2014CrossRefGoogle Scholar
  7. 7.
    Duan H, Deng Y, Wang X, et al. Biological eagle-eye: Based visual imaging guidance simulation platform for unmanned flying vehicles. IEEE Aerosp Electron Syst Mag, 2013, 28: 36–45CrossRefGoogle Scholar
  8. 8.
    Itti L, Koch C. Computational modelling of visual attention. Nat Rev Neurosci, 2001, 2: 194–203CrossRefGoogle Scholar
  9. 9.
    Deng Y M, Duan H B. Avian contrast sensitivity inspired contour detector for unmanned aerial vehicle landing. Sci China Tech Sci, 2017, 60: 1958–1965CrossRefGoogle Scholar
  10. 10.
    Duan H, Deng Y, Wang X, et al. Small and dim target detection via lateral inhibition filtering and artificial bee colony based selective visual attention. PLoS ONE, 2013, 8: e72035CrossRefGoogle Scholar
  11. 11.
    Krüger N, Janssen P, Kalkan S, et al. Deep hierarchies in the primate visual cortex: What can we learn for computer vision? IEEE Trans Pattern Anal Mach Intell, 2013, 35: 1847–1871CrossRefGoogle Scholar
  12. 12.
    Pajares G, Guijarro M, Herrera P J, et al. Combining classifiers through fuzzy cognitive maps in natural images. IET Comput Vis, 2009, 3: 112–123CrossRefGoogle Scholar
  13. 13.
    Wang K, Gu X F, Yu T, et al. Classification of hyperspectral remote sensing images using frequency spectrum similarity. Sci China Tech Sci, 2013, 56: 980–988CrossRefGoogle Scholar
  14. 14.
    Sohn K, Zhou G, Lee C, et al. Learning and selecting features jointly with point-wise gated boltzmann machines. In: International Conference on Machine Learning. Atlanta, 2013. 217–225Google Scholar
  15. 15.
    Li H, Duan H B. Verification of monocular and binocular pose estimation algorithms in vision-based UAVs autonomous aerial refueling system. Sci China Tech Sci, 2016, 59: 1730–1738CrossRefGoogle Scholar
  16. 16.
    Balakrishnan N, Hariharakrishnan K, Schonfeld D. A new image representation algorithm inspired by image submodality models, redundancy reduction, and learning in biological vision. IEEE Trans Pattern Anal Machine Intell, 2005, 27: 1367–1378CrossRefGoogle Scholar
  17. 17.
    Spratling M W. Image segmentation using a sparse coding model of cortical area V1. IEEE Trans Image Process, 2013, 22: 1631–1643MathSciNetCrossRefzbMATHGoogle Scholar
  18. 18.
    Lee T S. Image representation using 2D Gabor wavelets. IEEE Trans Pattern Anal Machine Intell, 1996, 18: 959–971CrossRefGoogle Scholar
  19. 19.
    Derpanis K G, Gryn J M. Three-dimensional nth derivative of Gaussian separable steerable filters. In: IEEE International Conference on Image Processing. Genoa: IEEE, 2005. 553–556Google Scholar
  20. 20.
    Zylberberg J, Murphy J T, DeWeese M R. A sparse coding model with synaptically local plasticity and spiking neurons can account for the diverse shapes of V1 simple cell receptive fields. PLoS Comput Biol, 2011, 7: e1002250MathSciNetCrossRefGoogle Scholar
  21. 21.
    Doi E, Gauthier J L, Field G D, et al. Efficient coding of spatial information in the primate retina. J Neurosci, 2012, 32: 16256–16264CrossRefGoogle Scholar
  22. 22.
    King P D, Zylberberg J, DeWeese M R. Inhibitory interneurons decorrelate excitatory cells to drive sparse code formation in a spiking model of V1. J Neurosci, 2013, 33: 5475–5485CrossRefGoogle Scholar
  23. 23.
    Gu J, Han H, Li X, et al. Hierarchical spatial pyramid max pooling based on SIFT features and sparse coding for image classification. IET Comput Vision, 2013, 7: 144–150CrossRefGoogle Scholar
  24. 24.
    Riesenhuber M, Poggio T. Hierarchical models of object recognition in cortex. Nat Neurosci, 1999, 2: 1019–1025CrossRefGoogle Scholar
  25. 25.
    Serre T, Wolf L, Bileschi S, et al. Robust object recognition with cortex-like mechanisms. IEEE Trans Pattern Anal Mach Intell, 2007, 29: 411–426CrossRefGoogle Scholar
  26. 26.
    Deng Y, Duan H. Hybrid C2 features and spectral residual approach to object recognition. Optik-Int J Light Electron Opt, 2013, 124: 3590–3595CrossRefGoogle Scholar
  27. 27.
    Farabet C, Couprie C, Najman L, et al. Learning hierarchical features for scene labeling. IEEE Trans Pattern Anal Mach Intell, 2013, 35: 1915–1929CrossRefGoogle Scholar
  28. 28.
    Jarrett K, Kavukcuoglu K, Ranzato M A, et al. What is the best multistage architecture for object recognition? In: IEEE International Conference on Computer Vision. Kyoto: IEEE, 2009. 2146–2153Google Scholar
  29. 29.
    Karklin Y, Lewicki M S. A hierarchical Bayesian model for learning nonlinear statistical regularities in nonstationary natural signals. Neural Comput, 2005, 17: 397–423CrossRefzbMATHGoogle Scholar
  30. 30.
    Karklin Y, Lewicki M S. Emergence of complex cell properties by learning to generalize in natural scenes. Nature, 2009, 457: 83–86CrossRefGoogle Scholar
  31. 31.
    Faivre O, Juusola M. Visual coding in locust photoreceptors. PLoS ONE, 2008, 3: e2173CrossRefGoogle Scholar
  32. 32.
    Mika S, Ratsch G, Weston J, et al. Fisher discriminant analysis with kernels. In: IEEE Conference on Neural Networks for Signal Processing. Madison: IEEE, 1999. 41–48Google Scholar
  33. 33.
    Li F F, Fergus R, Perona P. Learning generative visual models from few training examples: An incremental Bayesian approach tested on 101 object categories. Comput Vis Image Underst, 2007, 106: 59–70CrossRefGoogle Scholar
  34. 34.
    Hou X D, Zhang L Q. Saliency detection: A spectral residual approach. In: IEEE Conference on Computer Vision and Pattern Recognition. Minneapolis: IEEE, 2007. 1–8Google Scholar

Copyright information

© Science China Press and Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  1. 1.School of Automation Science and Electrical EngineeringBeihang UniversityBeijingChina

Personalised recommendations