Independent Component Analysis for Non-Normal Factor Analysis

Summary

Independent component analysis (ICA) was developed in the signal processing and neural computation communities. Its original purpose was to solve what is called the blind source separation problem: when linear mixtures of some original source signals are observed, the goal is to recover the source signals, using minimum assumptions on the mixing matrix (i.e. blindly). This leads to a linear model that is very similar to the one used in factor analysis. What makes ICA fundamentally different from conventional factor analysis is that the source signals are assumed to be non-Gaussian,in addition to the basic assumption of their independence. In fact, this implies that the model can be uniquely estimated from the data, using supplementary information that is not contained in the covariance matrix. Interestingly, a very close connection can be found with projection pursuit: The basic form of the ICA model can be estimated by finding the projections that are maximally non-Gaussian, which is the goal of projection pursuit as well. On the other hand, the dimension of the observed data vector is often first reduced by principal component analysis, in which case ICA can be viewed as a method of determining the factor rotation using the non-Gaussianity of the factors.