Multivariate techniques for preprocessing noisy data for source separation using ICA

Abstract

Independent component analysis (ICA) is a popular technique for separating sources from observed linear mixtures of the sources. If the measured signals are corrupted by noise, then they are generally preprocessed before applying ICA. We make use of a recently developed technique known as the iterative principal component analysis (IPCA) to preprocess the noisy signals and estimate the signal subspace prior to application of ICA. This preprocessing technique is consistent with the assumptions made in ICA, is invariant to any scaling of the data, and accounts for heteroscedastic errors. Through a simulated example, we show that if the measured signals are contaminated with a high-level of additive noise and outliers, the use of the proposed preprocessing technique results in more precise extraction of the independent sources as compared to the use of PCA as a preprocessing method. Application of the technique to an experimental chemometric data set shows that pure species spectra are more accurately extracted from mixture spectra using ICA, if the data is preprocessed using IPCA.

Appendix: The IPCA algorithm

The algorithm for estimating the source subspace using IPCA is given below. We use [U,S,V] = svd(X,m) to denote the truncated singular value decomposition of a m × N data matrix X, where U is the m × m matrix of left singular vectors corresponding to the non-zero singular values, S is a m × m diagonal matrix containing the non-zero singular values ordered from the largest to the smallest along the diagonal, and V is a m × N matrix of right singular vectors.

• Step 1. Set iteration counter k = 1 and λ ⁰ to be zero.

• Step 2. Set estimates of the non-zero elements of $\Upsigma_{\varepsilon }^{k}$ to be a small fraction (say 0.0001) of the corresponding elements of S _x .

• Step 3. Obtain the transformed matrix X _s  = L ⁻¹ X where LL ^T = $\Upsigma_{\varepsilon }^{k}$ .

• Step 4. Let [U,S,V] = svd(X _s ,m). Obtain estimate, $M^{k} = U_{n + 1 \ldots \ldots m}^{T} L,$ where $U_{n + 1 \ldots \ldots m}$ is the sub-matrix of U corresponding last m–n columns.

• Step 5. Let λ ^k be the sum of the last m–n singular values. Stop if relative change in λ is less than the specified tolerance; else continue.

• Step 6. Obtain the solution for the non-zero elements of the error covariance matrix by solving the optimization problem given by (14). Denote the solution as $\Upsigma_{\varepsilon }^{k + 1}$.

• Step 7. Increment iteration counter k and return to step 3.

It is to be noted that in order to apply above algorithm, an initial guess of the number of sources has to be specified. In order to determine the actual number of sources, we can iteratively apply above algorithm for different guesses of the number of sources as described in the text and check whether the converged singular values satisfy the required theoretical condition.