IVA using complex multivariate GGD: application to fMRI analysis

Abstract

Examples of complex-valued random phenomena in science and engineering are abound, and joint blind source separation (JBSS) provides an effective way to analyze multiset data. Thus there is a need for flexible JBSS algorithms for efficient data-driven feature extraction in the complex domain. Independent vector analysis (IVA) is a prominent recent extension of independent component analysis to multivariate sources, i.e., to perform JBSS, but its effectiveness is determined by how well the source models used match the true latent distributions and the optimization algorithm employed. The complex multivariate generalized Gaussian distribution (CMGGD) is a simple, yet effective parameterized family of distributions that account for full second- and higher-order statistics including noncircularity, a property that has been often omitted for convenience. In this paper, we marry IVA and CMGGD to derive, IVA-CMGGD, with a number of numerical optimization implementations including steepest descent, the quasi-Newton method Broyden–Fletcher–Goldfarb–Shanno (BFGS), and its limited-memory sibling limited-memory BFGS all in the complex-domain. We demonstrate the performance of our algorithm on simulated data as well as a 14-subject real-world complex-valued functional magnetic resonance imaging dataset against a number of competing algorithms.

Appendices

Appendix

Derivation of gradient

Since (15) is a real-valued function of complex variables, it suffices to compute the gradient with respect to ${\mathbf {w}}_n^{[k]*}$ using Wirtinger calculus. First, by applying the chain rule we can write:

$$\begin{aligned} \frac{\partial J({\mathbf {W}})}{\partial {\mathbf {w}}_n^{[k]*}} = \frac{\partial J({\mathbf {W}})}{\partial {\mathbf {y}}_n} \frac{\partial {\mathbf {y}}_n}{\partial w_n^{[k]*}} + \frac{\partial J({\mathbf {W}})}{\partial {\mathbf {y}}_n^*} \frac{\partial {\mathbf {y}}_n^*}{\partial w_n^{[k]*}}. \end{aligned}$$

(28)

Due to (12), the first term in (28) is equal to 0 leaving only the second term. Next, we subdivide the IVA-CMGGD cost function in (15) into two terms:

$$\begin{aligned} J({\mathbf {W}}) = J_1({\mathbf {W}}) + J_2({\mathbf {W}}), \end{aligned}$$

(29)

where

$$\begin{aligned} J_1({\mathbf {W}}) = \frac{1}{2}\sum _{n=1}^N{{\mathbb {E}}}\left\{ \left( \frac{\eta _n}{2}\underline{{\mathbf {y}}}_n^H\underline{{\mathbf {C}}}_n^{-1}\underline{{\mathbf {y}}}_n\right) ^{\beta _n} \right\} , \end{aligned}$$

(30)

and

$$\begin{aligned} J_2({\mathbf {W}})=- \sum _{k=1}^K\left( \log \begin{vmatrix}{\mathbf {W}}^{[k]}\end{vmatrix}^2\right) . \end{aligned}$$

(31)

Differentiating $J_1({\mathbf {W}})$ yields:

$$\begin{aligned} \frac{\partial J_1({\mathbf {W}})}{\partial {\mathbf {w}}_n^{[k]*}}&= \frac{1}{2}{{\mathbb {E}}}\left\{ \beta _n\left( \frac{\eta _n}{2}\underline{{\mathbf {y}}}_n^H\underline{{\mathbf {C}}}_n^{-1}\underline{{\mathbf {y}}}_n\right) ^{\beta _n-1}\left( \frac{\eta _n}{2}{\mathbf {e}}_k^\top \underline{{\mathbf {C}}}_n^{-1}\underline{{\mathbf {y}}}_n\right) {\mathbf {x}}^{[k]*}\right\} \nonumber \\&= {{\mathbb {E}}} \left\{ \frac{\beta _n\eta _n^{\beta _n}}{2^{\beta _n+1}} \frac{\left( {\mathbf {e}}_k^\top \underline{{\mathbf {C}}}^{-1}_n\underline{{\mathbf {y}}}_n\right) {\mathbf {x}}^{[k]*}}{\left( \underline{{\mathbf {y}}}_n^H\underline{{\mathbf {C}}}_n^{-1}\underline{{\mathbf {y}}}_n\right) ^{(1-\beta _n)}} \right\} , \end{aligned}$$

(32)

where ${\mathbf {e}}_k$ is defined as in (16) and $\frac{\partial {\mathbf {y}}_n^{*}}{\partial {\mathbf {w}}_n^{[k]*}}={\mathbf {x}}^{[k]*}$.

In order to differentiate $J_2({\mathbf {W}})$, we utilize a decoupling procedure (Anderson et al. 2012a), originally established in Li and Zhang (2007). The purpose is to factorize each summand in (31) into the product of two terms: one dependent on ${\mathbf {w}}_n^{[k]}$ and the other independent of it. By defining $\tilde{{\mathbf {W}}}_n^{[k]}$ to be the $(N-1)\times N$ matrix containing rows of ${\mathbf {W}}^{[k]}$ other than the nth, and by defining

$$\begin{aligned} {\bar{\omega }}_n^{[k]}=\sqrt{\left| \det \left( \tilde{{\mathbf {W}}}_n^{[k]}\tilde{{\mathbf {W}}}_n^{[k]H}\right) \right| }, \end{aligned}$$

(33)

the decoupling procedure admits the following representation for $J_2({\mathbf {W}})$:

$$\begin{aligned} J_2({\mathbf {W}})=-\sum _{k=1}^K\log \left( \left| {\mathbf {w}}_n^{[k]H}{\mathbf {h}}_n^{[k]*}\right| ^2{\bar{\omega }}_n^{[k]2}\right) , \end{aligned}$$

(34)

where ${\mathbf {h}}_n^{[k]}$ is a unit-length vector orthogonal to each of $\left\{ {\mathbf {w}}_n^{[m]}\right\} _{m\ne k}$. Then, the gradient of $J_2({\mathbf {W}})$ can be computed as:

$$\begin{aligned} \frac{\partial J_2({\mathbf {W}})}{\partial {\mathbf {w}}_n^{[k]*}}&= \frac{{\mathbf {h}}_n^{[k]*}{\mathbf {h}}_n^{[k]\top }{\mathbf {w}}_n^{[k]}}{{\mathbf {w}}_n^{[k]H}{\mathbf {h}}_n^{[k]*}{\mathbf {h}}_n^{[k]\top }{\mathbf {w}}_n^{[k]}} \nonumber \\&= \frac{{\mathbf {h}}_n^{[k]*}}{{\mathbf {h}}_n^{[k]H}{\mathbf {w}}_n^{[k]*}}. \end{aligned}$$

(35)

Summing (32) and (35) yields the result in (16).