# Sparse coding network model based on fast independent component analysis

Original Article

First Online:

- 100 Downloads

## Abstract

Neurobiological studies have shown that neurons in the primary visual cortex (V1) may employ sparse presentations to represent stimuli. We describe a network model for sparse coding which includes input layer, base functional layer and output layer. We simulated standard sparse coding and sparse coding based on fast independent component analysis (ICA). The duration of training bases, the convergence speed of objective function and the sparsity of coefficient matrix were compared, respectively. The results show that sparse coding based on fast ICA is more effective than standard sparse coding.

## Keywords

Sparse coding Independent component analysis Bases Natural images Sparsity## Notes

### Compliance with ethical standards

### Conflict of interest

The authors do not have any type of conflict of interest.

## References

- 1.Treichler DG (1967) Are you missing the boat in training aids. Film Audio Vis Commun 1:14–16Google Scholar
- 2.Olshausen BA, Field DJ (1997) Sparse coding with an over complete basis set: a strategy employed by V1. Vis Res 37:3313–3325CrossRefGoogle Scholar
- 3.Field DJ (1994) What is the goal of sensory coding? Neural Comput 6:559–601CrossRefGoogle Scholar
- 4.Olshausen BA, Field DJ (1996) Emergence of simple cell receptive properties by learning a sparse code for natural images. Nature 381:607–609CrossRefGoogle Scholar
- 5.Simoncelli EP (2003) Vision and the statistics of the visual environment. Curr Opin Neurobiol 1(13):144–149CrossRefGoogle Scholar
- 6.Delgutte B, Hammond B, Cariani P (1998) Psychophysical and physiological advances in hearing. Whurr Publishers, LondonGoogle Scholar
- 7.Ruderman DL, Bialek W (1994) Statistics of natural images: scaling in the woods. Phys Rev Lett 73(6):814–817CrossRefGoogle Scholar
- 8.Kandel ER, Schwartz JH, Jessel TM (2000) Principles of neural science, vol 4. McGraw-Hill Medical, New yorkGoogle Scholar
- 9.Hyvarinen A (1999) Survey on independent component analysis. Neural Comput Surv 2(4):94–128Google Scholar
- 10.Olshausen BA, Field DJ (2004) Sparse coding of sensory inputs. Curr Opin Neurobiol 14:481–487CrossRefGoogle Scholar
- 11.Lewick M (2002) Efficient coding of natural sounds. Nat Neurosci 5:356–363CrossRefGoogle Scholar
- 12.Vinje W, Gallant J (2002) Natural stimulation of the non-classical receptive field increases information transmission efficiency in V1. J Neurosci 22:2904–2915CrossRefGoogle Scholar
- 13.Hubel DH, Wiesel TN (1977) Functional architecture of macaque monkey visual cortex. Proc R Soc Lond B 198:1–59CrossRefGoogle Scholar
- 14.Hyvarinen A, Hoyer PO (2002) A two-layer sparse coding model learn simple and complex cell receptive fields and topography from natural images. Vis Res 41(18):2413–2423CrossRefGoogle Scholar
- 15.Simoncelli EP, Olshausen BA (2001) Natural image statistics and neural representation. Annu Rev Neurosci 24:1193–1216CrossRefGoogle Scholar
- 16.Hoyer PO, Hyvarinen A (2000) Independent component analysis applied to feature extraction from colour and stereo images. Netw Comput Neural Syst 11(3):191–210CrossRefzbMATHGoogle Scholar
- 17.Haken H (2007) Towards a unifying model of neural net activity in the visual cortex. Cogn Neurodyn 1(1):15–25CrossRefGoogle Scholar
- 18.Hyvarinen A, Hoyer PO, Mika OI (2001) Topographic independent component analysis. Neural Comput 13(7):1527–1558CrossRefzbMATHGoogle Scholar
- 19.Li S, Wu S (2007) Robustness of neural codes and its implication on natural image processing. Cogn Neurodyn 1(3):261–272CrossRefGoogle Scholar
- 20.Hateren JH, Ruderman DL (1998) Independent component analysis of natural image sequences yields spatiotemporal filters similar to simple cells in primary visual cortex. Proc R Soc Ser B 265:2315–2320CrossRefGoogle Scholar
- 21.Gong HY, Zhang YY, Liang PJ, Zhang PM (2010) Neural coding properties based on spike timing and pattern correlation of retinal ganglion cells. Cogn Neurodyn 4(4):337–346CrossRefGoogle Scholar
- 22.Saglam Murat, Hayashida Yuki, Murayama Nobuki (2009) A retinal circuit model accounting for wide-field amacrine cells. Cogn Neurodyn 3(1):25–32CrossRefGoogle Scholar
- 23.Pillow JW, Shlens J, Paninski L, Sher A, Litke AM (2008) Spatio-temporal correlations and visual signaling in a complete neuronal population. Nature 454:995–999CrossRefGoogle Scholar
- 24.Li CG, Li YK (2011) Fast and robust image segmentation by small-world neural oscillator networks. Cogn Neurodyn 5(2):209–220CrossRefGoogle Scholar
- 25.Vialatte FB, Dauwels J, Maurice M, Yamaguchi Y, Cichocki A (2009) On the synchrony of steady state visual evoked potentials and oscillatory burst events. Cogn Neurodyn 3(3):251–261CrossRefGoogle Scholar
- 26.Huberman AD, Feller MB, Chapman B (2008) Mechanisms underlying development of visual maps and receptive fields. Annu Rev Neurosci 31:479–509CrossRefGoogle Scholar
- 27.Han JW, Zhao SJ, Hu XT, Guo L, Liu TM (2014) Encoding brain network response to free viewing of videos. Cogn Neurodyn 8(5):389–397CrossRefGoogle Scholar
- 28.Wang RB, Tsuda I, Zhang ZK (2015) A new work mechanism on neuronal activity. Int J Neural Syst 25(03):1450037CrossRefGoogle Scholar
- 29.Wang RB (2015) Can the activities of the large scale cortical network be expressed by neural energy? Cogn Neurodyn 1:1–5CrossRefGoogle Scholar
- 30.Wang ZY, Wang RB, Fang RY (2015) Energy coding in neural network with inhibitory neurons. Cogn Neurodyn 9(2):129–144MathSciNetCrossRefGoogle Scholar
- 31.Wang ZY, Wang RB (2014) Energy distribution property and energy coding of a structural neural network. Front Comput Neurosci 8(8):14Google Scholar
- 32.Wang RB, Zhang ZK (2011) Phase synchronization motion and neural coding in dynamic transmission of neural information. IEEE Trans Neural Netw 22(7):1097–1106CrossRefGoogle Scholar
- 33.Wang RB, Zhang ZK (2007) Energy coding in biological neural network. Cogn Neurodyn 1(3):203–212CrossRefGoogle Scholar

## Copyright information

© The Natural Computing Applications Forum 2017