New Approaches to Principal Component Analysis for Trees

Abstract

Object Oriented Data Analysis is a new area in statistics that studies populations of general data objects. In this article we consider populations of tree-structured objects as our focus of interest. We develop improved analysis tools for data lying in a binary tree space analogous to classical Principal Component Analysis methods in Euclidean space. Our extensions of PCA are analogs of one dimensional subspaces that best fit the data. Previous work was based on the notion of tree-lines.

In this paper, a generalization of the previous tree-line notion is proposed: k-tree-lines. Previously proposed tree-lines are k-tree-lines where k=1. New sub-cases of k-tree-lines studied in this work are the 2-tree-lines and tree-curves, which explain much more variation per principal component than tree-lines. The optimal principal component tree-lines were computable in linear time. Because 2-tree-lines and tree-curves are more complex, they are computationally more expensive, but yield improved data analysis results.

We provide a comparative study of all these methods on a motivating data set consisting of brain vessel structures of 98 subjects.

Appendix

1.1 6.1 Proof of Lemma 1

The approach taken here is to count all possible 2-tree-lines on a given data set. A polynomial bound on this number will suffice to conclude that we have a problem that can be solved in polynomial time, as the process of calculating the total distance of a given 2-tree-line to the points in the data set is a linear-time process.

For a given data set, the number of possible k-tree-lines depends on the size of its support tree only, and not on the number of data trees in it. In this section, it will be assumed that the support tree is a full tree, i.e. all levels of the support tree include all the nodes on those levels. Another simplification is that we will assume the starting tree for the 2-tree-lines considered is the root node. This approach will give an upper bound on the 2-tree-line count, since arranging the same number of nodes in a full tree and starting from the root node would give the highest number of possible 2-tree-lines. These two assumptions will enable us to disregard the structure of an arbitrary starting tree and the support tree in finding an upper bound that depends on the node count only.

Let:

We know that:

$$f(n) = f_1(n)+f_2(n), \quad \forall n \geq0.$$

We will write a recursive formula for f(n). If we consider the most trivial case where our tree is only the root node, we obtain the initial condition for the recursion:

To get the recursive formula, assume that we know the values of f ₁(n) and f ₂(n), and we are looking for f ₁(n+1) and f ₂(n+1). First let us count the 2-tree-lines that end at (n+1)st level with a single node. This single node can be either one of the two children of a 2-tree-line ending at level n with a single node, or it can be one of the four children of a 2-tree-line ending at level n with two nodes. Therefore:

$$f_1(n+1)=2f_1(n)+4f_2(n).$$

For f ₂(n+1), first consider f ₁(n). These lines end with a single node at nth level, and have two children, where both of them need to be added. Since the order of the addition matters, each such line gives us two options for extension. For f ₂(n), we need to choose two nodes out of the four children of nth level nodes. However, not all of the 2-combinations of these are available. Now let us name the nodes on nth level as a and b, b being the last added node. Let us name their children as a ₁,a ₂ and b ₁,b ₂, respectively. Now the possible choices for addition are (a ₁,a ₂), (a ₂,a ₁), (b ₁,a ₁), (b ₁,a ₂), (b ₂,a ₁), (b ₂,a ₂). Summing all the choices up, we get

$$f_2(n+1)=2f_1(n)+6f_2(n).$$

These two formulas are valid for all n greater than 1. Now let us write these two in matrix form:

$$\left[\begin{array}{c} f_1(n+1) \\ f_2(n+1)\end{array}\right] = \left[\begin{array}{c@{\quad}c}2 & 4 \\2 & 6\end{array}\right] \left[\begin{array}{c} f_1(n) \\ f_2(n)\end{array}\right].$$

Using this formula, we can write

$$\left[\begin{array}{c} f_1(n+1) \\ f_2(n+1)\end{array}\right] = {\left[\begin{array}{c@{\quad}c}2 & 4 \\2 & 6\end{array}\right]}^{n}\left[\begin{array}{c} f_1(1) \\ f_2(1)\end{array}\right].$$

To further simplify this, we can re-write the coefficient matrix using spectral decomposition:

$$\left[\begin{array}{c@{\quad}c} 2 & 4 \\ 2 & 6\end{array}\right] = {\left[\begin{array}{c@{\quad}c}\sqrt{3}-1 & 1 \\1& \frac{1-\sqrt{3}}{2}\end{array}\right]} {\left[\begin{array}{c@{\quad}c} \lambda_1 & 0 \\ 0 & \lambda_2\end{array}\right] } {\left[\begin{array}{c@{\quad}c}\sqrt{3}-1 & 1 \\1& \frac{1-\sqrt{3}}{2}\end{array}\right]}^{-1}$$

where $\lambda_{1} = 4+2\sqrt{3}$ and $\lambda_{2} = 4-2\sqrt{3}$, the eigenvalues of coefficient matrix. Now we can get the nth multiple of this easily:

$${\left[\begin{array}{c@{\quad}c} 2 & 4 \\ 2 & 6\end{array}\right]}^{n} = {\left[\begin{array}{c@{\quad}c}\sqrt{3}-1 & 1 \\1& \frac{1-\sqrt{3}}{2}\end{array}\right]} {\left[\begin{array}{c@{\quad}c} \lambda_1 & 0 \\ 0 & \lambda_2\end{array}\right] }^{n} {\left[\begin{array}{c@{\quad}c}\sqrt{3}-1 & 1 \\1& \frac{1-\sqrt{3}}{2}\end{array}\right]}^{-1}.$$

Insert this into f(n+1) formula along with the initial conditions, and do the necessary simplifications:

Summing these up, we get the desired quantity:

$$f(n+1) = f_1(n+1)+f_2(n+1) = \frac{(\lambda_1)^n + (\lambda_2)^n}{2}.$$

We know that a full support tree with n levels has ∼2ⁿ nodes. If we call the total number of nodes in the support tree m, we have n=log₂(m). So for a problem with full support tree size m, the total number of 2-tree-lines is:

$$\frac{\lambda_1^{(\log_2m) - 1} + \lambda_2^{(\log_2m) - 1}}{2}.$$

So the order of the problem of finding all 2-tree-lines is:

$$O\biggl(\frac{1}{2\lambda_1}m^{\log_2\lambda_1}\biggr) = O\bigl (m^{2.9}\bigr).$$

1.2 6.2 Proof of Theorem 1

Lemma 1 already establishes the order for the total count of 2-tree-lines. To prove Theorem 1, we also need the maximum length of these 2-tree-lines.

The full support tree with m nodes has a depth log₂(m+1). And it is easy to see that the 2-tree-line with maximum number of nodes in it that can be defined on this support tree has 1+2∗(log₂(m+1)) nodes. We obtain this number by using the observation that a 2-tree-line starting from the root can contain at most two nodes from each level, except the root level. Therefore, the maximum number of nodes contained in each 2-tree-line has an order of O(logm). Combine this with Lemma 1, and we see that the order of all nodes contained in the list of all 2-tree-lines is O(m ^2.9logm). The final step is to show that the brute-force method needs to account every node on the 2-tree-line list only once to form the list. This step is rather trivial, so it will not be elaborated here.

1.3 6.3 Proof of Proposition 1

The projection of a data point onto an object is, naturally, a point on that object. In our case, this implies the fact that the projection of a data point t _i onto a 2-tree-line K, P _K (t _i ), is a tree that is contained in Pa(K). Therefore we can write:

(1)

Let K ^∗ be any maximal 2-tree-line that can be extended from K. Naturally, K ^∗⊇K, and

$$ \sum_{v \in \mathit{MP}(K)}{w(v)} \geq\sum _{v \in \mathit{Pa}(K^*)}{w(v)}.$$

(2)

Now, using (1) and (2), we can show

Which proves that any maximal 2-tree-line extending from K will have a worse objective function value than $\sum_{t_{i} \in T}{|t_{i}|} -\sum_{v \in \mathit{MP}(K)}{w(v)}$, and therefore ∑ _v∈MP(K) w(v) provides a lower bound.