Fahlman-Type Activation Functions Applied to Nonlinear PCA Networks Provide a Generalised Independent Component Analysis

Abstract

It has been shown experimentally that Oja’s nonlinear principal component analysis (PCA) algorithm is capable of performing an independent component analysis (ICA) on a specific data set [7]. However, the dynamic stability requirements of the nonlinear PCA algorithm restrict its use to data which has sub-gaussian probability densities [6]. The restriction is particularly severe as this precludes the application of the algorithm from performing ICA on naturally occurring data such as speech, music and certain visual images. We have shown that the nonlinear PCA algorithm can be considered as minimising an information theoretic contrast function and develop a more direct link between ICA and the algorithm function [6]. To remove the sub-gaussian restriction and enable a generalised ICA which will span the full range of possible data kurtosis, we propose the use of Fahlman type activation functions [2] in the nonlinear PCA algorithm. We show that variants of these functions satisfy all the dynamic and asymptotic stability requirements of the algorithm and successfully remove the sub-gaussian restriction. We also report on simulations which demonstrate the blind separating ability of the nonlinear PCA algorithm with the Fahlman type functions on mixtures of super-Gaussian data (natural speech).