Skip to main content
SpringerLink
Log in

Linear Multilayer ICA Using Adaptive PCA

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

Linear multilayer independent component analysis (LMICA) is an approximate algorithm for ICA. In LMICA, approximate independent components are efficiently estimated by optimizing only highly dependent pairs of signals when all the sources are super-Gaussian. In this paper, the nonlinear functions in LMICA are generalized, and a new method using adaptive PCA is proposed for the selection of pairs of highly dependent signals. In this method, at first, all the signals are sorted along the first principal axis of their higher-order correlation matrix. Then, the sorted signals are divided into two groups so that relatively highly correlated signals are collected in each group. Lastly, each of them is sorted recursively. This process is repeated until each group consists of only one or two signals. Because a well-known adaptive PCA algorithm named PAST is utilized for calculating the first principal axis, this method is quite simple and efficient. Some numerical experiments verify the effectiveness of LMICA with this improvement.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Amari S (1998) Natural gradient works efficiently. Neural Comput 10: 251–276

    Article  Google Scholar 

  2. Amari S, Cichocki A (1996) A new learning algorithm for blind signal separation. In: Touretzky D, Mozer M, Hasselmo M (eds) Advances in neural information processing systems 8. MIT Press, Cambridge, MA, pp 757–763

    Google Scholar 

  3. Bell AJ, Sejnowski TJ (1997) The independent components of natural scenes are edge filters. Vision Res 37(23): 3327–3338

    Article  Google Scholar 

  4. Burnham KP, Anderson DR (2002) Model selection and multimodel inference: a practical-theoretic approach, 2nd edn. Springer, Berlin, NY

    Google Scholar 

  5. Cardoso J-F (1999) High-order contrasts for independent component analysis. Neural Comput 11(1): 157–192

    Article  MathSciNet  Google Scholar 

  6. Cichocki A, Amari S (2002) Adaptive blind signal and image processing: learning algorithms and applications. Wiley, London

    Book  Google Scholar 

  7. Cichocki A, Kasprzak W, Amari S (1995) Multi-layer neural networks with a local adaptive learning rule for blind separation of source signals. In: Proceedings of NOLTA95, vol 1. Las Vegas, USA, pp 61–66

  8. Cichocki A, Zdunek R (2007) Multilayer nonnegative matrix factorization using projected gradient approaches. Int J Neural Syst 17(6): 431–446

    Article  Google Scholar 

  9. Hyvärinen A (1999) Fast and robust fixed-point algorithms for independent component analysis. IEEE Trans Neural Netw 10(3): 626–634

    Article  Google Scholar 

  10. Hyvärinen A, Karhunen J, Oja E (2001) Independent component analysis. Wiley, London

    Book  Google Scholar 

  11. Matsuda Y, Yamaguchi K (2005) An efficient MDS-based topographic mapping algorithm. Neurocomputing 64: 285–299

    Article  Google Scholar 

  12. Matsuda Y, Yamaguchi K (2007) Extraction of approximate independent components from large natural scenes. In: ICONIP 2007, vol 4984. Springer, Kitakyushu, pp 635–642

  13. Matsuda Y, Yamaguchi K (2007) Linear multilayer ICA generating hierarchical edge detectors. Neural Comput 19: 218–230

    Article  MATH  Google Scholar 

  14. Matsuda Y, Yamaguchi K (2008) A connection-limited neural network by infomax and infomin. In: Proceedings of IJCNN2008, Hong Kong, IEEE, pp 2531–2537

  15. Oja E (1982) A simplified neuron model as a principal component analyzer. J Math Biol 15: 267–273

    Article  MATH  MathSciNet  Google Scholar 

  16. Oja E (1992) Principal components, minor components, and linear neural networks. Neural Netw 5: 927–935

    Article  Google Scholar 

  17. Olshausen BA, Field DJ (1996) Emergence of simple-cell receptive field properties by learning a sparce code for natural images. Nature 381: 607–609

    Article  Google Scholar 

  18. Ouyang S, Bao Z, Liao G-S (2000) Robust recursive least squares learning algorithm for principal component analysis. IEEE Trans Neural Netw 11(1): 215–221

    Article  Google Scholar 

  19. Sanger TD (1989) Optimal unsupervised learning in a single-layer linear feedforward neural network. Neural Netw 2: 459–473

    Article  Google Scholar 

  20. Särelä J, Vigário R (2003) Overlearning in marginal distribution-based ica: analysis and solutions. J Mach Learn Res 4: 1447–1469

    Article  Google Scholar 

  21. van Hateren JH, van der Schaaf A (1998) Independent component filters of natural images compared with simple cells in primary visual cortex. Proc Royal Soc Lond B 265: 359–366

    Article  Google Scholar 

  22. Yang B (1995) Projection approximation subspace tracking. IEEE Trans Signal Process 43(1): 95–107

    Article  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Integrated Information Technology, Aoyama Gakuin University, 5-10-1 Fuchinobe, Sagamihara-Shi, Kanagawa, 229-8558, Japan

    Yoshitatsu Matsuda

  2. Department of General Systems Studies, Graduate School of Arts and Sciences, The University of Tokyo, 3-8-1, Komaba, Meguro-Ku, Tokyo, 153-8902, Japan

    Kazunori Yamaguchi

Authors
  1. Yoshitatsu Matsuda
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kazunori Yamaguchi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yoshitatsu Matsuda.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Matsuda, Y., Yamaguchi, K. Linear Multilayer ICA Using Adaptive PCA. Neural Process Lett 30, 133–144 (2009). https://doi.org/10.1007/s11063-009-9114-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-009-9114-4

Keywords

Search

Navigation