Fahlman-Type Activation Functions Applied to Nonlinear PCA Networks Provide a Generalised Independent Component Analysis
It has been shown experimentally that Oja’s nonlinear principal component analysis (PCA) algorithm is capable of performing an independent component analysis (ICA) on a specific data set . However, the dynamic stability requirements of the nonlinear PCA algorithm restrict its use to data which has sub-gaussian probability densities . The restriction is particularly severe as this precludes the application of the algorithm from performing ICA on naturally occurring data such as speech, music and certain visual images. We have shown that the nonlinear PCA algorithm can be considered as minimising an information theoretic contrast function and develop a more direct link between ICA and the algorithm function . To remove the sub-gaussian restriction and enable a generalised ICA which will span the full range of possible data kurtosis, we propose the use of Fahlman type activation functions  in the nonlinear PCA algorithm. We show that variants of these functions satisfy all the dynamic and asymptotic stability requirements of the algorithm and successfully remove the sub-gaussian restriction. We also report on simulations which demonstrate the blind separating ability of the nonlinear PCA algorithm with the Fahlman type functions on mixtures of super-Gaussian data (natural speech).
KeywordsIndependent Component Analysis Independent Component Analysis Natural Speech Robust Principal Component Analysis Blind Separation
Unable to display preview. Download preview PDF.
- S.E. Fahlman. Faster-learning variations on back-propagation: an empirical study. In Proc. Connectionist Models Summer School, pages 38–51. Morgan-Kaufmann, 1988.Google Scholar
- M. Girolami and C. Fyfe. Higher order cumulant maximisation using nonlinear Hebbian and anti-Hebbian learning for adaptive blind separation of source signals. In Proc. IWSIP-96, IEEE/IEE International Workshop on Signal and Image Processing, pages 141–144. Elsevier, 1996.Google Scholar
- M. Girolami and C. Fyfe. Kurtosis extrema and identification of independent components: A network approach. In Proceedings ICASSP-97, 1997. To appear.Google Scholar
- M. Girolami and C. Fyfe. Stochastic ICA contrast maximisation using Oja’s nonlinear PCA algorithm. International Journal of Neural Systems, 1997. In press.Google Scholar
- J. Karhunen. Neural approaches to independent component analysis and source separation. In Proc. ESANN96, 1996.Google Scholar
- E. Oja. The nonlinear PCA learning rule and signal separation-mathematical analysis. Technical Report A26, Helsinki University of Technology, 1995.Google Scholar