Some new aspects of taxicab correspondence analysis

Abstract

Correspondence analysis (CA) and nonsymmetric correspondence analysis are based on generalized singular value decomposition, and, in general, they are not equivalent. Taxicab correspondence analysis (TCA) is a $\hbox {L}_{1}$ variant of CA, and it is based on the generalized taxicab singular value decomposition (GTSVD). Our aim is to study the taxicab variant of nonsymmetric correspondence analysis. We find that for diagonal metric matrices GTSVDs of a given data set are equivalent; from which we deduce the equivalence of TCA and taxicab nonsymmetric correspondence analysis. We also attempt to show that TCA stays as close as possible to the original correspondence matrix without calculating a dissimilarity (or similarity) measure between rows or columns. Further, we discuss some new geometric and distance aspects of TCA.

Appendix

1.1 Singular value decomposition

Let $\mathbf{X}$ be a data set of dimension $I\times J$, where $I$ observations are described by the $J$ variables. The ordinary SVD can be described as successive maximization of the $L_{2}$-norm of the linear combination of the columns of $\mathbf{X}$ subject to a quadratic constraint; that is, it is based on the following optimization problem

$$\begin{aligned} max\left| \left| \mathbf{Xu}\right| \right| _{2}\quad \text { subject to }\,\left| \left| \mathbf{u}\right| \right| _{2}=1\mathbf{;} \end{aligned}$$

(36)

or equivalently, it can also be described as maximization of the the $L_{2}$-norm of the linear combination of the rows of $\mathbf{X}$

$$\begin{aligned} max\left| \left| \mathbf{X}^{\prime }\mathbf{v}\right| \right| _{2}\quad \text {subject to }\,\left| \left| \mathbf{v}\right| \right| _{2}=1\mathbf{.} \end{aligned}$$

(37)

Equation (36) is the dual of (37), and they can be reexpressed as matrix norms

$$\begin{aligned} \lambda _{1}&= \max _{\mathbf{u\in \mathbb {R} }^{J}}\frac{\left| \left| \mathbf{Xu}\right| \right| _{2}}{ \left| \left| \mathbf{u}\right| \right| _{2}}, \nonumber \\&= \max _{\mathbf{v\in \mathbb {R} }^{I}}\frac{\left| \left| \mathbf{X}^{\prime }\mathbf{v}\right| \right| _{2}}{\left| \left| \mathbf{v}\right| \right| _{2}},\nonumber \\&= \max _{\mathbf{u\in \mathbb {R} }^{J},\mathbf{v\in \mathbb {R} }^{I}}\frac{\mathbf{v}^{\prime }\mathbf{Xu}}{\left| \left| \mathbf{u} \right| \right| _{2}\left| \left| \mathbf{v}\right| \right| _{2}}. \end{aligned}$$

(38)

The solution to (38), $\lambda _{1},\$is the square root of the greatest eigenvalue of the matrix $\mathbf{X}^{\prime }\mathbf{X}$ or $\mathbf{XX}^{\prime }.$ The first principal axes, $\mathbf{u}_{1}$ and $\mathbf{v}_{1},$ can be defined as

$$\begin{aligned} \mathbf{u}_{1}=\arg \max _{\mathbf{u}}\left| \left| \mathbf{Xu}\right| \right| _{2}\,\text { such that }\,\left| \left| \mathbf{u} _{1}\right| \right| _{2}=1, \end{aligned}$$

(39)

where $\mathbf{u}_{1}$ is the eigenvector of the matrix $\mathbf{X}^{\prime }\mathbf{X }$ associated with the greatest eigenvalue $\lambda _{1}^{2};$ and

$$\begin{aligned} \mathbf{v}_{1}=\arg \max _{\mathbf{v}}\left| \left| \mathbf{X}^{\prime }\mathbf{v} \right| \right| _{2}\,\text { such that }\,\left| \left| \mathbf{v} _{1}\right| \right| _{2}=1. \end{aligned}$$

(40)

Let $\mathbf{a}_{1}$ and $\mathbf{b}_{1}$ be defined as

$$\begin{aligned} \mathbf{a}_{1}=\mathbf{Xu}_{1}\, \text { and }\, \mathbf{b}_{1}=\mathbf{X}^{\prime }\mathbf{v}_{1}; \end{aligned}$$

(41)

then

$$\begin{aligned} \left| \left| \mathbf{a}_{1}\right| \right| _{2}=\mathbf{v} _{1}^{\prime }\mathbf{a}_{1}=\left| \left| \mathbf{b}_{1}\right| \right| _{2}=\mathbf{u}_{1}^{\prime }\mathbf{b}_{1}=\lambda _{1}. \end{aligned}$$

(42)

Equations (41) and (42) are named transition formulas, because $\mathbf{v}_{1}$ and $\mathbf{a}_{1},$ and, $\mathbf{u}_{1}$ and $\mathbf{b}_{1},$ are related by

$$\begin{aligned} \mathbf{u}_{1}=\mathbf{b}_{1}/\lambda _{1} \quad \text { and} \quad \mathbf{v}_{1}=\mathbf{a} _{1}/\lambda _{1}. \end{aligned}$$

(43)

To obtain $\mathbf{a}_{2}$ and $\mathbf{b}_{2},$ and axes $\mathbf{u}_{2}$ and $\mathbf{v }_{2}$, we repeat the above procedure on the residual dataset

$$\begin{aligned} \mathbf{X}_{2}\mathbf{=X}_{1}\mathbf{-a}_{1}\mathbf{b}_{1}^{^{\prime }}/\lambda _{1}, \end{aligned}$$

(44)

where $\mathbf{X}_{1}\mathbf{=X.}$ We note that $rank(\mathbf{X}_{2}\mathbf{)=}rank(\mathbf{X_{1})-}1,$ because by (41) and (42)

$$\begin{aligned} \mathbf{X}_{2}\mathbf{u}_{1}=\mathbf{0}\, \text { and}\, \mathbf{X}_{2}^{\prime }\mathbf{v}_{1}= \mathbf{0}. \end{aligned}$$

(45)

Classical SVD can be described as the sequential repetition of the above procedure for $k=rank(\mathbf{X)}$ times till the residual matrix becomes $\mathbf{0}$; thus, using $\alpha =1,\ldots ,k$ as indices, the matrix $\mathbf{X}$ can be written as

$$\begin{aligned} \mathbf{X}=\sum _{\alpha =1}^{k}\mathbf{a}_{\alpha }\mathbf{b}_{\alpha }^{\prime }/\lambda _{\alpha }, \end{aligned}$$

(46)

which, by (43), can be rewritten in a much more familiar form

$$\begin{aligned} \mathbf{X}=\sum _{\alpha =1}^{k}\lambda _{\alpha }\mathbf{v}_{\alpha }\mathbf{u} _{\alpha }^{\prime }. \end{aligned}$$

(47)

Further, we have

$$\begin{aligned} \lambda _{\alpha }=\left| \left| \mathbf{a}_{\alpha }\right| \right| _{2}=\left| \left| \mathbf{b}_{\alpha }\right| \right| _{2}\quad \text {and }\, \lambda _{\alpha }\text {'}s \, \text {are decreasing for} \quad \alpha =1,\ldots ,k; \end{aligned}$$

(48)

and

$$\begin{aligned} Tr(\mathbf{X}^{\prime }\mathbf{X})&= Tr(\mathbf{XX}^{\prime })=\sum _{\alpha =1}^{k}\lambda _{\alpha }^{2}, \\&= \sum _{\alpha =1}^{k}\left| \left| \mathbf{a}_{\alpha }\right| \right| _{2}^{2}=\sum _{\alpha =1}^{k}\left| \left| \mathbf{b} _{\alpha }\right| \right| _{2}^{2}. \nonumber \end{aligned}$$

(49)

1.2 Taxicab singular value decomposition

TSVD consists of maximizing the $L_{1}$ norm of the linear combination of the columns of X subject to the $L_{\infty }$ norm constraint; more precisely, it is based on the following optimization problem

$$\begin{aligned} max\left| \left| \mathbf{Xu}\right| \right| _{1}\quad \text { subject to }\,\left| \left| \mathbf{u}\right| \right| _{\infty }=1 \mathbf{;} \end{aligned}$$

(50)

or equivalently, it can also be described as maximization of the $L_{1}$ norm of the linear combination of the rows of the matrix $\mathbf{X}$

$$\begin{aligned} max\left| \left| \mathbf{X}^{\prime }\mathbf{v}\right| \right| _{1}\quad \text {subject to }\, \left| \left| \mathbf{v}\right| \right| _{\infty }=1\mathbf{.} \end{aligned}$$

(51)

Equation (50) is the dual of (51), and they can be reexpressed as matrix norms

$$\begin{aligned} \lambda _{1}&= \max _{\mathbf{u\in \mathbb {R} }^{J}}\frac{\left| \left| \mathbf{Xu}\right| \right| _{1}}{ \left| \left| \mathbf{u}\right| \right| _{\infty }}, \nonumber \\&= \max _{\mathbf{v\in \mathbb {R} }^{I}}\frac{\left| \left| \mathbf{X}^{\prime }\mathbf{v}\right| \right| _{1}}{\left| \left| \mathbf{v}\right| \right| _{\infty }}, \nonumber \\&= \max _{\mathbf{u\in \mathbb {R} }^{J},\mathbf{v\in \mathbb {R} }^{I}}\frac{\mathbf{v}^{\prime }\mathbf{Xu}}{\left| \left| \mathbf{u} \right| \right| _{\infty }\left| \left| \mathbf{v}\right| \right| _{\infty }}, \end{aligned}$$

(52)

which is a well known and much discussed matrix norm related to Grothendieck problem, see for instance, Alon and Naor (2006). The solution to (52), $\lambda _{1},\$is a combinatorial optimization problem given by

$$\begin{aligned} \max \mathbf{||Xu||}_{1}\quad \text {subject to }\,\mathbf{u}\in \left\{ -1,+1\right\} ^{J}. \end{aligned}$$

(53)

Equation (53) characterizes the robustness of the method, in the sense that the weights affected to the columns (similarly to the rows by duality) are uniformly distributed on $\{-1,+1\}$. The vectors, $\mathbf{u}_{1}$ and $\mathbf{v} _{1},$ are defined as

$$\begin{aligned} \mathbf{u}_{1}=\arg \max _{\mathbf{u}}\left| \left| \mathbf{Xu}\right| \right| _{1}\, \text { such that }\,\left| \left| \mathbf{u} _{1}\right| \right| _{\infty }=1, \end{aligned}$$

(54)

and

$$\begin{aligned} \mathbf{v}_{1}=\arg \max _{\mathbf{v}}\left| \left| \mathbf{X}^{\prime }\mathbf{v} \right| \right| _{1}\, \text { such that }\,\left| \left| \mathbf{v} _{1}\right| \right| _{\infty }=1. \end{aligned}$$

(55)

Let $\mathbf{a}_{1}$ and $\mathbf{b}_{1}$ be

$$\begin{aligned} \mathbf{a}_{1}=\mathbf{Xu}_{1}\, \text { and }\, \mathbf{b}_{1}=\mathbf{X}^{\prime }\mathbf{v}_{1} \text {;} \end{aligned}$$

(56)

then

$$\begin{aligned} \left| \left| \mathbf{a}_{1}\right| \right| _{1}=\mathbf{v} _{1}^{\prime }\mathbf{a}_{1}=\left| \left| \mathbf{b}_{1}\right| \right| _{1}=\mathbf{u}_{1}^{\prime }\mathbf{b}_{1}=\lambda _{1}. \end{aligned}$$

(57)

Equations (56) and (57) are named transition formulas, because $\mathbf{v}_{1}$ and $\mathbf{a}_{1},$ and, $\mathbf{u}_{1}$ and $\mathbf{b}_{1},$ are related by

$$\begin{aligned} \mathbf{u}_{1}=sgn(\mathbf{b}_{1}) \quad \text {and } \quad \mathbf{v}_{1}=sgn(\mathbf{a}_{1}), \end{aligned}$$

(58)

where $sgn(\mathbf{g}_{1})=(sgn(g_{1}(1)),\ldots ,sgn(g_{1}(J))^{\prime },$ and $sgn(g_{1}(j))=1$ if $g_{1}(j)>0,$ $sgn(g_{1}(j))=-1$ otherwise. Note that (58) is completely different from (43).

To obtain $\mathbf{a}_{2}$, $\mathbf{b}_{2},$ and axes $\mathbf{u}_{2}$ and $\mathbf{v} _{2}$, we repeat the above procedure on the residual dataset

$$\begin{aligned} \mathbf{X}_{2}&= \mathbf{X}_{1}\mathbf{-X}_{1}\mathbf{u}_{1}\mathbf{v}_{1}^{\prime } \mathbf{X}_{1}/\lambda _{1}, \nonumber \\&= \mathbf{X}_{1}\mathbf{-a}_{1}\mathbf{b}_{1}^{\prime }/\lambda _{1} \end{aligned}$$

(59)

where $\mathbf{X}_{1}\mathbf{=X.}$ We note that $rank(\mathbf{X}_{2}\mathbf{)=}rank(\mathbf{X}_{1}\mathbf{)-}1,$ because by (56), (57) and (58)

$$\begin{aligned} \mathbf{X}_{2}\mathbf{u}_{1}=\mathbf{0}\, \text { and }\, \mathbf{X}_{2}^{\prime }\mathbf{v}_{1}= \mathbf{0;} \end{aligned}$$

(60)

which implies that

$$\begin{aligned} \mathbf{u}_{1}^{\prime }\mathbf{b}_{\alpha }=0 \, \text {and}\,\mathbf{v}_{1}^{\prime } \mathbf{a}_{\alpha }=0\,\text { for }\, \alpha =2,\ldots ,k. \end{aligned}$$

(61)

TSVD is described as the sequential repetition of the above procedure for $k=rank(\mathbf{X)}$ times till the residual matrix becomes $\mathbf{0}$; thus the matrix $\mathbf{X}$ can be written as

$$\begin{aligned} \mathbf{X}=\sum _{\alpha =1}^{k}\mathbf{a}_{\alpha }\mathbf{b}_{\alpha }^{\prime }/\lambda _{\alpha }. \end{aligned}$$

(62)

It is important to note that (62) has the same form as (4) and (46), but it can not be rewritten as (47).

Further, similar to (57), we have

$$\begin{aligned} \lambda _{\alpha }=\left| \left| \mathbf{a}_{\alpha }\right| \right| _{1}=\left| \left| \mathbf{b}_{\alpha }\right| \right| _{1} \quad \text {for} \quad \alpha =1,\ldots ,k. \end{aligned}$$

(63)

But the dispersion measures $\lambda _{\alpha }$’s in (63) will not satisfy (49), because the Pythagorean theorem is not satisfied in $L_{1}.$

In TSVD, the optimization problem (50), (51) or (52) can be accomplished by two algorithms. The first one is based on complete enumeration (53); this can be applied, with the present state of desktop computing power, say, if $min(I,J)\simeq 25.$ The second one is based on iterating the transition formulas (56), (57) and (58), similar to Wold (1966) NIPALS (nonlinear iterative partial alternating least squares) algorithm, also named criss-cross regression by Gabriel and Zamir (1979). It is easy to show that this is also an ascent algorithm. The criss-cross algorithm is nonlinear and can be summarized in the following way, where $\mathbf{b}$ is a starting value:

Step 1: $\mathbf{u=}sgn\mathbf{(b)}$, $\mathbf{a=Xu}$ and $\lambda (\mathbf{u)} =\left| \left| \mathbf{Xu}\right| \right| _{1};$

Step 2: $\mathbf{v=}sgn\mathbf{(a),}$ $\mathbf{b=X}^{\prime }\mathbf{v}$ and $\lambda ( \mathbf{v)}=\left| \left| \mathbf{X}^{\prime }\mathbf{v}\right| \right| _{1};$

Step 3: If $\lambda (\mathbf{v)-}\lambda (\mathbf{u)>}0\mathbf{,}$ go to Step 1; otherwise, stop.

This is an ascent algorithm; that is, it increases the value of the objective function $\lambda$ at each iteration. The convergence of the algorithm is superlinear (very fast, at most two iterations); however it could converge to a local maximum; so we restart the algorithm $I$ times using each row of $\mathbf{X}$ as a starting value. The iterative algorithm is statistically consistent in the sense that as the sample size increases there will be some observations in the direction of the principal axes, so the algorithm will find the optimal solution.