Clustering in Hilbert’s Projective Geometry: The Case Studies of the Probability Simplex and the Elliptope of Correlation Matrices

Abstract

Clustering categorical distributions in the probability simplex is a fundamental task met in many applications dealing with normalized histograms. Traditionally, differential-geometric structures of the probability simplex have been used either by (i) setting the Riemannian metric tensor to the Fisher information matrix of the categorical distributions, or (ii) defining the dualistic information-geometric structure induced by a smooth dissimilarity measure, the Kullback–Leibler divergence. In this work, we introduce for this clustering task a novel computationally-friendly framework for modeling the probability simplex termed Hilbert simplex geometry. In the Hilbert simplex geometry, the distance function is described by a polytope. We discuss the pros and cons of those different statistical modelings, and benchmark experimentally these geometries for center-based k-means and k-center clusterings. Furthermore, since a canonical Hilbert metric distance can be defined on any bounded convex subset of the Euclidean space, we also consider Hilbert’s projective geometry of the elliptope of correlation matrices and study its clustering performances.

Appendices

8 Isometry of Hilbert Simplex Geometry to a Normed Vector Space

Consider the Hilbert simplex metric space $(\varDelta ^d,\rho _\mathrm {HG})$ where $\varDelta ^d$ denotes the d-dimensional open probability simplex and $\rho _\mathrm {HG}$ the Hilbert cross-ratio metric. Let us recall the isometry ([25], 1991) of the open standard simplex to a normed vector space $(V^d,\Vert \cdot \Vert _\mathrm {NH})$. Let $V^d=\{v\in \mathbb {R}^{d+1} \ :\ \sum _i v^i=0\}$ denote the d-dimensional vector space sitting in $\mathbb {R}^{d+1}$. Map a point $p=(\lambda ^0,\ldots ,\lambda ^{d})\in \varDelta ^d$ to a point $v(x)=(v^0,\ldots , v^{d})\in V^d$ as follows:

$$\begin{aligned} v^i = \frac{1}{d+1} \left( d\log \lambda ^i -\sum _{j\ne {i}}\log \lambda ^j \right) = \log \lambda ^i - \frac{1}{d+1}\sum _j \log \lambda ^j. \end{aligned}$$

We define the corresponding norm $\Vert \cdot \Vert _\mathrm {NH}$ in $V^d$ by considering the shape of its unit ball $B_V=\{v\in V^d \ :\ |v^i-v^j|\le 1, \forall i\not =j\}$. The unit ball $B_V$ is a symmetric convex set containing the origin in its interior, and thus yields a polytope norm $\Vert \cdot \Vert _\mathrm {NH}$ (Hilbert norm) with $2\left( {\begin{array}{c}d+1\\ 2\end{array}}\right) =d(d+1)$ facets. Reciprocally, let us notice that a norm induces a unit ball centered at the origin that is convex and symmetric around the origin.

The distance in the normed vector space between $v\in V^d$ and $v'\in V^d$ is defined by:

$$\begin{aligned} \rho _V(v,v')= \Vert v-v'\Vert _\mathrm {NH}= \inf \left\{ \tau \ :\ v'\in \tau (B_V\oplus \{v\}) \right\} , \end{aligned}$$

where $A\oplus B=\{a+b \ :\ a\in A,b\in B\}$ is the Minkowski sum.

The reverse map from the normed space $V^d$ to the probability simplex $\varDelta ^d$ is given by:

$$\begin{aligned} \lambda ^i = \frac{\exp ({v^i})}{\sum _j \exp (v^j)}. \end{aligned}$$

Thus we have $(\varDelta ^d,\rho _\mathrm {HG})\cong (V^d,\Vert \cdot \Vert _\mathrm {NH})$. In 1D, $(V^1,\Vert \cdot \Vert _\mathrm {NH})$ is isometric to the Euclidean line.

Note that computing the distance in the normed vector space requires naively $O(d^2)$ time.

Unfortunately, the norm $\Vert \cdot \Vert _\mathrm {NH}$ does not satisfy the parallelogram law.³ Notice that a norm satisfying the parallelogram law can be associated with an inner product via the polarization identity. Thus the isometry of the Hilbert geometry to a normed vector space is not equipped with an inner product. However, all norms in a finite dimensional space are equivalent. This implies that in finite dimension, $(\varDelta ^d,\rho _\mathrm {HG})$ is quasi-isometric to the Euclidean space $\mathbb {R}^d$. An example of Hilbert geometry in infinite dimension is reported in [25]. Hilbert spaces are not CAT spaces except when $\mathscr {C}$ is an ellipsoid [76].

9 Hilbert Geometry with Finslerian/Riemannian Structures

In a Riemannian geometry, each tangent plane $T_pM$ of a d-dimensional manifold M is equivalent to $\mathbb {R}^d$: $T_pM\simeq \mathbb {R}^d$. The inner product at each tangent plane $T_pM$ can be visualized by an ellipsoid shape, a convex symmetric object centered at point p. In a Finslerian geometry, a norm $\Vert \cdot \Vert _p$ is defined in each tangent plane $T_pM$, and this norm is visualized as a symmetric convex object with non-empty interior. Finslerian geometry thus generalizes Riemannian geometry by taking into account generic symmetric convex objects instead of ellipsoids for inducing norms at each tangent plane. Any Hilbert geometry induced by a compact convex domain $\mathscr {C}$ can be expressed by an equivalent Finslerian geometry by defining the norm in $T_p$ at p as follows [76]:

$$\begin{aligned} \Vert v\Vert _p = F_\mathscr {C}(p,v) =\frac{\Vert v\Vert }{2} \left( \frac{1}{pp^+} + \frac{1}{pp^-} \right) , \end{aligned}$$

where $F_\mathscr {C}$ is the Finsler metric, $\Vert \cdot \Vert$ is an arbitrary norm on $\mathbb {R}^d$, and $p^+$ and $p^-$ are the intersection points of the line passing through p with direction v:

$$ p^+=p+t^+v,\quad p^-=p+t^-v. $$

A geodesic $\gamma$ in a Finslerian geometry satisfies:

$$\begin{aligned} d_\mathscr {C}(\gamma (t_1),\gamma (t_2)) = \int _{t_1}^{t_2} F_\mathscr {C}(\gamma (t),\dot{\gamma }(t)) \mathrm {d}t. \end{aligned}$$

In $T_pM$, a ball of center c and radius r is defined by:

$$\begin{aligned} B(c,r)=\{ v \ : \ F_\mathscr {C}(c,v) \le r \}. \end{aligned}$$

Thus any Hilbert geometry induces an equivalent Finslerian geometry, and since Finslerian geometries include Riemannian geometries, one may wonder which Hilbert geometries induce Riemannian structures? The only Riemannian geometries induced by Hilbert geometries are the hyperbolic Cayley–Klein geometries [27, 29, 30] with the domain $\mathscr {C}$ being an ellipsoid. The Finslerian modeling of information geometry has been studied in [71, 72].

There is not a canonical way of defining measures in a Hilbert geometry since Hilbert geometries are Finslerian but not necessary Riemannian geometries [76]. The Busemann measure is defined according to the Lebesgue measure $\lambda$ of $\mathbb {R}^d$: Let $B_p$ denote the unit ball wrt. to the Finsler norm at point $p\in \mathscr {C}$, and $B_e$ the Euclidean unit ball. Then the Busemann measure for a Borel set $\mathscr {B}$ is defined by [76]:

$$ \mu _\mathscr {C}(\mathscr {B}) = \int _\mathscr {B}\frac{\lambda (B_e)}{\lambda (B_p)} \mathrm {d}\lambda (p). $$

The existence and uniqueness of center points of a probability measure in Finsler geometry have been investigated in [77].

10 Bounding Hilbert Norm with Other Norms

Let us show that $\Vert v\Vert _\mathrm {NH}\le \beta _{d,c} \Vert v\Vert _c$, where $\Vert \cdot \Vert _c$ is any norm. Let $v=\sum _{i=0}^{d} e_i x_i$, where $\{e_i\}$ is a basis of $\mathbb {R}^{d+1}$. We have:

$$ \Vert v\Vert _c \le \sum _{i=0}^{d} |x_i| \Vert e_i\Vert _c \le \Vert x\Vert _2 \underbrace{\sqrt{\sum _{i=0}^{d} \Vert e_i\Vert ^2_c}}_{\beta _d}, $$

where the first inequality comes from the triangle inequality, and the second inequality is from the Cauchy–Schwarz inequality. Thus we have:

$$ \Vert v\Vert _\mathrm {NH}\le \beta _d \Vert x\Vert _2, $$

with $\beta _d=\sqrt{d+1}$ since $\Vert e_i\Vert _\mathrm {NH}\le 1$.

Let $\alpha _{d,c}=\min _{\{v \ :\ \Vert v\Vert _c = 1\}} \Vert v\Vert _\mathrm {NH}$. Consider $u=\frac{v}{\Vert v\Vert _c}$. Then $\Vert u\Vert _c=1$ so that $\Vert v\Vert _\mathrm {NH}\ge \alpha _{d,c} \Vert v\Vert _c$. To find $\alpha _d$, we consider the unit $\ell _2$ ball in $V^d$, and find the smallest $\lambda >0$ so that $\lambda B_V$ fully contains the Euclidean ball (Fig. 14).

Fig. 14

Polytope balls $B_V$ and the Euclidean unit ball $B_E$. From the figure the smallest polytope ball has radius $\approx 1.5$

Therefore, we have overall:

$$ \alpha _d \Vert x\Vert _2 \le \Vert v\Vert _\mathrm {NH}\le \sqrt{d+1} \Vert x\Vert _2 $$

In general, note that we may consider two arbitrary norms $\Vert \cdot \Vert _l$ and $\Vert \cdot \Vert _u$ so that:

$$ \alpha _{d,l} \Vert x\Vert _l \le \Vert v\Vert _\mathrm {NH}\le \beta _{d,u} \Vert x\Vert _u. $$

11 Funk Directed Metrics and Funk Balls

The Funk metric [78] wrt a convex domain $\mathscr {C}$ is defined by

$$ F_\mathscr {C}(x,y) = \log \left( \frac{\Vert x-a\Vert }{\Vert y-a\Vert } \right) , $$

where a is the intersection of the domain boundary and the affine ray R(x, y) starting from x and passing through y. Correspondingly, the reverse Funk metric is

$$ F_\mathscr {C}(y,x) = \log \left( \frac{\Vert y-b\Vert }{\Vert x-b\Vert } \right) , $$

where b is the intersection of R(y, x) with the boundary. The Funk metric is not a metric distance.

The Hilbert metric is simply the arithmetic symmetrization:

$$ H_\mathscr {C}(x,y)=\frac{F_\mathscr {C}(x,y)+F_\mathscr {C}(y,x)}{2}. $$

It is interesting to explore clustering based on the Funk geometry, which we leave as a future work.