Three-Way Generalized Structured Component Analysis

Abstract

Generalized structured component analysis (GSCA) is a component-based approach to structural equation modeling, where components of observed variables are used as proxies for latent variables. GSCA has thus far focused on analyzing two-way (e.g., subjects by variables) data. In this paper, GSCA is extended to deal with three-way data that contain three different types of entities (e.g., subjects, variables, and occasions) simultaneously. The proposed method, called three-way GSCA, permits each latent variable to be loaded on two types of entities, such as variables and occasions, in the measurement model. This enables to investigate how these entities are associated with the latent variable. The method aims to minimize a single least squares criterion to estimate parameters. An alternating least squares algorithm is developed to minimize this criterion. We conduct a simulation study to evaluate the performance of three-way GSCA. We also apply three-way GSCA to real data to demonstrate its empirical usefulness.

Appendix

The ALS algorithm repeats the following three steps until convergence.

Step 1. Update weights (w _p ’s) for fixed ${\mathbf{c}}_{p}^{\text{J} }$, ${\mathbf{c}}_{p}^{\text{K} }$, and B. This is equivalent to minimizing

$$ \begin{aligned} \phi = & \sum\limits_{p = 1}^{P} {\text{SS} \left( {{\mathbf{X}}_{p} - {\varvec{\upgamma}}_{p} ({\mathbf{c}}_{p}^{\text{K} } \, \otimes \,{\mathbf{c}}_{p}^{\text{J} } )^{\prime}} \right) + \text{SS} \left( {{\varvec{\upgamma}}_{p} {\mathbf{e}}_{p}^{{\prime }} + {\varvec{\Gamma}}^{( - p)} - {\varvec{\upgamma}}_{p} {\mathbf{b}}_{p}^{{\prime }} - {\varvec{\Gamma}}^{( - p)} {\mathbf{B}}^{( - p)} } \right)} \\ & = \sum\limits_{p = 1}^{P} {\text{SS} \left( {{\mathbf{X}}_{p} - {\varvec{\upgamma}}_{p} ({\mathbf{c}}_{p}^{\text{K} } \, \otimes \,{\mathbf{c}}_{p}^{\text{J} } )^{\prime}} \right) + \text{SS} \left( {{\varvec{\upgamma}}_{p} {\mathbf{t}}_{p} - {\varvec{\Delta}}_{p} } \right)} \\ & = \sum\limits_{p = 1}^{P} {\text{SS} \left( {{\mathbf{X}}_{p} - {\varvec{\upgamma}}_{p} {\mathbf{q}}_{p}^{{\prime }} } \right) + \text{SS} \left( {{\varvec{\upgamma}}_{p} {\mathbf{t}}_{p} - {\varvec{\Delta}}_{p} } \right)} , \\ \end{aligned} $$

(A.1)

subject to ${\varvec{\upgamma}}_{p}^{{\prime }} {\varvec{\upgamma}}_{p} = 1$, where ${\mathbf{q}}_{p} = ({\mathbf{c}}_{p}^{\text{K} } \, \otimes \,{\mathbf{c}}_{p}^{\text{J} } )$, ${\mathbf{t}}_{p} = {\mathbf{e}}_{p}^{{\prime }} - {\mathbf{b}}_{p}^{{\prime }}$, and ${\varvec{\Delta}}_{p} = {\varvec{\Gamma}}^{( - p)} {\mathbf{B}}^{( - p)} - {\varvec{\Gamma}}^{( - p)}$. In (A.1), $\varvec\Gamma ^{{\left( { - p} \right)}}$ and ${\mathbf{B}}^{{\left( { - p} \right)}}$ indicate Γ and B, whose columns are all zero vectors except the pth column, respectively, and ${\mathbf{e}}_{p}^{{\prime }}$ indicates a 1 by P vector, whose elements are all zero except the pth element being unity. Based on (1), (A.1) can be re-expressed as

$$ \begin{aligned} \phi = & \sum\limits_{p = 1}^{P} {\text{SS} \left( {{\mathbf{X}}_{p} - {\mathbf{X}}_{p} {\mathbf{w}}_{p} {\mathbf{q}}_{p}^{{\prime }} } \right) + \text{SS} \left( {{\mathbf{X}}_{p} {\mathbf{w}}_{p} {\mathbf{t}}_{p} - {\varvec{\Delta}}_{p} } \right)} \\ = & \sum\limits_{p = 1}^{P} {\left( {\text{tr} \left( {{\mathbf{X^{\prime}}}_{p} {\mathbf{X}}_{p} } \right) - 2{\mathbf{w}}_{p}^{{\prime }} {\mathbf{X}}_{p}^{{\prime }} {\mathbf{X}}_{p} {\mathbf{q}}_{p} + {\mathbf{w}}_{p}^{{\prime }} {\mathbf{X}}_{p}^{{\prime }} {\mathbf{X}}_{p} {\mathbf{w}}_{p} {\mathbf{q}}_{p}^{{\prime }} {\mathbf{q}}_{p} } \right)} \\ & \quad + {\mathbf{w}}_{p}^{{\prime }} {\mathbf{X}}_{p}^{{\prime }} {\mathbf{X}}_{p} {\mathbf{w}}_{p} {\mathbf{t}}_{p} {\mathbf{t}}_{p}^{{\prime }} - 2{\mathbf{w}}_{p}^{{\prime }} {\mathbf{X}}_{p}^{{\prime }} \Delta_{p} {\mathbf{t}}_{p}^{{\prime }} + \text{tr} \left( {{\varvec{\Delta}}_{p}^{{\prime }} {\varvec{\Delta}}_{p} } \right). \\ \end{aligned} $$

(A.2)

Solving $\frac{\partial \phi }{{\partial} \mathbf{w}_{p} }$ = 0, w _p is updated by

$$ {\hat{\mathbf{w}}}_{p} = \left( {{\mathbf{q}}_{p} {\mathbf{q}}_{p}^{{\prime }} {\mathbf{X}}_{p}^{{\prime }} {\mathbf{X}}_{p} + {\mathbf{t}}_{p} {\mathbf{t}}_{p}^{{\prime }} {\mathbf{X}}_{p}^{{\prime }} {\mathbf{X}}_{p} } \right)^{ - 1} \left( {{\mathbf{X}}_{p}^{{\prime }} {\mathbf{X}}_{p} {\mathbf{q}}_{p} + {\mathbf{X}}_{p}^{{\prime }} {\varvec{\Delta}}_{p} {\mathbf{t}}_{p}^{{\prime }} } \right). $$

(A.3)

Subsequently, ${\varvec{\upgamma}}_{p}$ is updated by $\varvec\gamma_{p} = {\mathbf{X}}_{p} \widehat{{\mathbf{w}}}_{p}$ and normalized to satisfy the constraint ${\varvec{\upgamma}}_{p}^{{\prime }} {\varvec{\upgamma}}_{p} = 1$.

Step 2. Update ${\mathbf{c}}_{p}^{\text{J} }$ and ${\mathbf{c}}_{p}^{\text{K} }$ for fixed w _p and B. This is equivalent to applying parallel factor analysis (PARAFAC) (Harshman 1970), subject to ${\mathbf{c}}_{p}^{{\text{J} {\prime }}} {\mathbf{c}}_{p}^{\text{J} } = 1$, and ${\mathbf{c}}_{p}^{{\text{K} {\prime }}} {\mathbf{c}}_{p}^{\text{K} } = 1$. We can simply use the ALS algorithm for PARAFAC to update ${\mathbf{c}}_{p}^{\text{J} }$ and ${\mathbf{c}}_{p}^{\text{K} }$ (Acar and Yener 2009; Harshman 1970; Olivieri et al. 2015).

Step 3. Update B for fixed w _p , ${\mathbf{c}}_{p}^{\text{J} }$ and ${\mathbf{c}}_{p}^{\text{K} }$. This is equivalent to minimizing

$$ \begin{aligned} \phi_{B} = & {\text{SS}}\left( {{\varvec{\Gamma}} - {\varvec{\Gamma}} {\mathbf{B}}} \right) \\ = & {\text{SS}}\left( {{\text{vec}}\left( {\varvec{\Gamma}} \right) - \left( {{\mathbf{I}}_{p} \otimes {\varvec{\Gamma}}} \right){\text{vec}}\left( {\mathbf{B}} \right)} \right) \\ = & {\text{SS}}\left( {{\text{vec}}\left( {\varvec{\Gamma}} \right) - {\mathbf{\varvec\Psi u}}} \right) \\ \end{aligned} $$

(A.4)

where vec(S) is a super vector formed by stacking all columns of S in order, u denotes free parameters to be estimated in vec(B), and Ψ is a matrix consisting of the columns of ${\mathbf{I}}_{p} \, \otimes \,{\varvec{\Gamma}}$ corresponding to the free parameters of vec(B) The estimate of u is obtained by

$$ {\hat{\mathbf{u}}} = \left( {{\varvec{\Psi}}^{{\prime }} {\varvec{\Psi}}} \right)^{ - 1} {\varvec{\Psi}}^{{\prime }} \,{\text{vec}}\left( {\varvec{\Gamma}} \right). $$

(A.5)

Then, $\widehat{\mathbf{{B}}}$ is reconstructed from $\widehat{\mathbf{{u}}}$.