Neural Computing and Applications

, Volume 31, Issue 3, pp 887–893 | Cite as

Sparse coding network model based on fast independent component analysis

  • Guanzheng Wang
  • Rubin WangEmail author
Original Article


Neurobiological studies have shown that neurons in the primary visual cortex (V1) may employ sparse presentations to represent stimuli. We describe a network model for sparse coding which includes input layer, base functional layer and output layer. We simulated standard sparse coding and sparse coding based on fast independent component analysis (ICA). The duration of training bases, the convergence speed of objective function and the sparsity of coefficient matrix were compared, respectively. The results show that sparse coding based on fast ICA is more effective than standard sparse coding.


Sparse coding Independent component analysis Bases Natural images Sparsity 


Compliance with ethical standards

Conflict of interest

The authors do not have any type of conflict of interest.


  1. 1.
    Treichler DG (1967) Are you missing the boat in training aids. Film Audio Vis Commun 1:14–16Google Scholar
  2. 2.
    Olshausen BA, Field DJ (1997) Sparse coding with an over complete basis set: a strategy employed by V1. Vis Res 37:3313–3325CrossRefGoogle Scholar
  3. 3.
    Field DJ (1994) What is the goal of sensory coding? Neural Comput 6:559–601CrossRefGoogle Scholar
  4. 4.
    Olshausen BA, Field DJ (1996) Emergence of simple cell receptive properties by learning a sparse code for natural images. Nature 381:607–609CrossRefGoogle Scholar
  5. 5.
    Simoncelli EP (2003) Vision and the statistics of the visual environment. Curr Opin Neurobiol 1(13):144–149CrossRefGoogle Scholar
  6. 6.
    Delgutte B, Hammond B, Cariani P (1998) Psychophysical and physiological advances in hearing. Whurr Publishers, LondonGoogle Scholar
  7. 7.
    Ruderman DL, Bialek W (1994) Statistics of natural images: scaling in the woods. Phys Rev Lett 73(6):814–817CrossRefGoogle Scholar
  8. 8.
    Kandel ER, Schwartz JH, Jessel TM (2000) Principles of neural science, vol 4. McGraw-Hill Medical, New yorkGoogle Scholar
  9. 9.
    Hyvarinen A (1999) Survey on independent component analysis. Neural Comput Surv 2(4):94–128Google Scholar
  10. 10.
    Olshausen BA, Field DJ (2004) Sparse coding of sensory inputs. Curr Opin Neurobiol 14:481–487CrossRefGoogle Scholar
  11. 11.
    Lewick M (2002) Efficient coding of natural sounds. Nat Neurosci 5:356–363CrossRefGoogle Scholar
  12. 12.
    Vinje W, Gallant J (2002) Natural stimulation of the non-classical receptive field increases information transmission efficiency in V1. J Neurosci 22:2904–2915CrossRefGoogle Scholar
  13. 13.
    Hubel DH, Wiesel TN (1977) Functional architecture of macaque monkey visual cortex. Proc R Soc Lond B 198:1–59CrossRefGoogle Scholar
  14. 14.
    Hyvarinen A, Hoyer PO (2002) A two-layer sparse coding model learn simple and complex cell receptive fields and topography from natural images. Vis Res 41(18):2413–2423CrossRefGoogle Scholar
  15. 15.
    Simoncelli EP, Olshausen BA (2001) Natural image statistics and neural representation. Annu Rev Neurosci 24:1193–1216CrossRefGoogle Scholar
  16. 16.
    Hoyer PO, Hyvarinen A (2000) Independent component analysis applied to feature extraction from colour and stereo images. Netw Comput Neural Syst 11(3):191–210CrossRefzbMATHGoogle Scholar
  17. 17.
    Haken H (2007) Towards a unifying model of neural net activity in the visual cortex. Cogn Neurodyn 1(1):15–25CrossRefGoogle Scholar
  18. 18.
    Hyvarinen A, Hoyer PO, Mika OI (2001) Topographic independent component analysis. Neural Comput 13(7):1527–1558CrossRefzbMATHGoogle Scholar
  19. 19.
    Li S, Wu S (2007) Robustness of neural codes and its implication on natural image processing. Cogn Neurodyn 1(3):261–272CrossRefGoogle Scholar
  20. 20.
    Hateren JH, Ruderman DL (1998) Independent component analysis of natural image sequences yields spatiotemporal filters similar to simple cells in primary visual cortex. Proc R Soc Ser B 265:2315–2320CrossRefGoogle Scholar
  21. 21.
    Gong HY, Zhang YY, Liang PJ, Zhang PM (2010) Neural coding properties based on spike timing and pattern correlation of retinal ganglion cells. Cogn Neurodyn 4(4):337–346CrossRefGoogle Scholar
  22. 22.
    Saglam Murat, Hayashida Yuki, Murayama Nobuki (2009) A retinal circuit model accounting for wide-field amacrine cells. Cogn Neurodyn 3(1):25–32CrossRefGoogle Scholar
  23. 23.
    Pillow JW, Shlens J, Paninski L, Sher A, Litke AM (2008) Spatio-temporal correlations and visual signaling in a complete neuronal population. Nature 454:995–999CrossRefGoogle Scholar
  24. 24.
    Li CG, Li YK (2011) Fast and robust image segmentation by small-world neural oscillator networks. Cogn Neurodyn 5(2):209–220CrossRefGoogle Scholar
  25. 25.
    Vialatte FB, Dauwels J, Maurice M, Yamaguchi Y, Cichocki A (2009) On the synchrony of steady state visual evoked potentials and oscillatory burst events. Cogn Neurodyn 3(3):251–261CrossRefGoogle Scholar
  26. 26.
    Huberman AD, Feller MB, Chapman B (2008) Mechanisms underlying development of visual maps and receptive fields. Annu Rev Neurosci 31:479–509CrossRefGoogle Scholar
  27. 27.
    Han JW, Zhao SJ, Hu XT, Guo L, Liu TM (2014) Encoding brain network response to free viewing of videos. Cogn Neurodyn 8(5):389–397CrossRefGoogle Scholar
  28. 28.
    Wang RB, Tsuda I, Zhang ZK (2015) A new work mechanism on neuronal activity. Int J Neural Syst 25(03):1450037CrossRefGoogle Scholar
  29. 29.
    Wang RB (2015) Can the activities of the large scale cortical network be expressed by neural energy? Cogn Neurodyn 1:1–5CrossRefGoogle Scholar
  30. 30.
    Wang ZY, Wang RB, Fang RY (2015) Energy coding in neural network with inhibitory neurons. Cogn Neurodyn 9(2):129–144MathSciNetCrossRefGoogle Scholar
  31. 31.
    Wang ZY, Wang RB (2014) Energy distribution property and energy coding of a structural neural network. Front Comput Neurosci 8(8):14Google Scholar
  32. 32.
    Wang RB, Zhang ZK (2011) Phase synchronization motion and neural coding in dynamic transmission of neural information. IEEE Trans Neural Netw 22(7):1097–1106CrossRefGoogle Scholar
  33. 33.
    Wang RB, Zhang ZK (2007) Energy coding in biological neural network. Cogn Neurodyn 1(3):203–212CrossRefGoogle Scholar

Copyright information

© The Natural Computing Applications Forum 2017

Authors and Affiliations

  1. 1.Institute for Cognitive NeurodynamicsEast China University of Science and TechnologyShanghaiChina

Personalised recommendations