Neural Processing Letters

, Volume 30, Issue 2, pp 133–144 | Cite as

Linear Multilayer ICA Using Adaptive PCA

  • Yoshitatsu Matsuda
  • Kazunori Yamaguchi


Linear multilayer independent component analysis (LMICA) is an approximate algorithm for ICA. In LMICA, approximate independent components are efficiently estimated by optimizing only highly dependent pairs of signals when all the sources are super-Gaussian. In this paper, the nonlinear functions in LMICA are generalized, and a new method using adaptive PCA is proposed for the selection of pairs of highly dependent signals. In this method, at first, all the signals are sorted along the first principal axis of their higher-order correlation matrix. Then, the sorted signals are divided into two groups so that relatively highly correlated signals are collected in each group. Lastly, each of them is sorted recursively. This process is repeated until each group consists of only one or two signals. Because a well-known adaptive PCA algorithm named PAST is utilized for calculating the first principal axis, this method is quite simple and efficient. Some numerical experiments verify the effectiveness of LMICA with this improvement.


Independent component analysis Adaptive PCA Massive data analysis Image processing 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Amari S (1998) Natural gradient works efficiently. Neural Comput 10: 251–276CrossRefGoogle Scholar
  2. 2.
    Amari S, Cichocki A (1996) A new learning algorithm for blind signal separation. In: Touretzky D, Mozer M, Hasselmo M (eds) Advances in neural information processing systems 8. MIT Press, Cambridge, MA, pp 757–763Google Scholar
  3. 3.
    Bell AJ, Sejnowski TJ (1997) The independent components of natural scenes are edge filters. Vision Res 37(23): 3327–3338CrossRefGoogle Scholar
  4. 4.
    Burnham KP, Anderson DR (2002) Model selection and multimodel inference: a practical-theoretic approach, 2nd edn. Springer, Berlin, NYGoogle Scholar
  5. 5.
    Cardoso J-F (1999) High-order contrasts for independent component analysis. Neural Comput 11(1): 157–192CrossRefMathSciNetGoogle Scholar
  6. 6.
    Cichocki A, Amari S (2002) Adaptive blind signal and image processing: learning algorithms and applications. Wiley, LondonCrossRefGoogle Scholar
  7. 7.
    Cichocki A, Kasprzak W, Amari S (1995) Multi-layer neural networks with a local adaptive learning rule for blind separation of source signals. In: Proceedings of NOLTA95, vol 1. Las Vegas, USA, pp 61–66Google Scholar
  8. 8.
    Cichocki A, Zdunek R (2007) Multilayer nonnegative matrix factorization using projected gradient approaches. Int J Neural Syst 17(6): 431–446CrossRefGoogle Scholar
  9. 9.
    Hyvärinen A (1999) Fast and robust fixed-point algorithms for independent component analysis. IEEE Trans Neural Netw 10(3): 626–634CrossRefGoogle Scholar
  10. 10.
    Hyvärinen A, Karhunen J, Oja E (2001) Independent component analysis. Wiley, LondonCrossRefGoogle Scholar
  11. 11.
    Matsuda Y, Yamaguchi K (2005) An efficient MDS-based topographic mapping algorithm. Neurocomputing 64: 285–299CrossRefGoogle Scholar
  12. 12.
    Matsuda Y, Yamaguchi K (2007) Extraction of approximate independent components from large natural scenes. In: ICONIP 2007, vol 4984. Springer, Kitakyushu, pp 635–642Google Scholar
  13. 13.
    Matsuda Y, Yamaguchi K (2007) Linear multilayer ICA generating hierarchical edge detectors. Neural Comput 19: 218–230MATHCrossRefGoogle Scholar
  14. 14.
    Matsuda Y, Yamaguchi K (2008) A connection-limited neural network by infomax and infomin. In: Proceedings of IJCNN2008, Hong Kong, IEEE, pp 2531–2537Google Scholar
  15. 15.
    Oja E (1982) A simplified neuron model as a principal component analyzer. J Math Biol 15: 267–273MATHCrossRefMathSciNetGoogle Scholar
  16. 16.
    Oja E (1992) Principal components, minor components, and linear neural networks. Neural Netw 5: 927–935CrossRefGoogle Scholar
  17. 17.
    Olshausen BA, Field DJ (1996) Emergence of simple-cell receptive field properties by learning a sparce code for natural images. Nature 381: 607–609CrossRefGoogle Scholar
  18. 18.
    Ouyang S, Bao Z, Liao G-S (2000) Robust recursive least squares learning algorithm for principal component analysis. IEEE Trans Neural Netw 11(1): 215–221CrossRefGoogle Scholar
  19. 19.
    Sanger TD (1989) Optimal unsupervised learning in a single-layer linear feedforward neural network. Neural Netw 2: 459–473CrossRefGoogle Scholar
  20. 20.
    Särelä J, Vigário R (2003) Overlearning in marginal distribution-based ica: analysis and solutions. J Mach Learn Res 4: 1447–1469CrossRefGoogle Scholar
  21. 21.
    van Hateren JH, van der Schaaf A (1998) Independent component filters of natural images compared with simple cells in primary visual cortex. Proc Royal Soc Lond B 265: 359–366CrossRefGoogle Scholar
  22. 22.
    Yang B (1995) Projection approximation subspace tracking. IEEE Trans Signal Process 43(1): 95–107MATHCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC. 2009

Authors and Affiliations

  1. 1.Department of Integrated Information TechnologyAoyama Gakuin UniversitySagamihara-Shi, KanagawaJapan
  2. 2.Department of General Systems Studies, Graduate School of Arts and SciencesThe University of TokyoTokyoJapan

Personalised recommendations