Learning a Gaussian Process Model on the Riemannian Manifold of Non-decreasing Distribution Functions

Abstract

In this work, we consider the problem of learning regression models from a finite set of functional objects. In particular, we introduce a novel framework to learn a Gaussian process model on the space of Strictly Non-decreasing Distribution Functions (SNDF). Gaussian processes (GPs) are commonly known to provide powerful tools for non-parametric regression and uncertainty estimation on vector spaces. On top of that, we define a Riemannian structure of the SNDF space and we learn a GP model indexed by SNDF. Such formulation enables to define an appropriate covariance function, extending the Matérn family of covariance functions. We also show how the full Gaussian process methodology, namely covariance parameter estimation and prediction, can be put into action on the SNDF space. The proposed method is tested using multiple simulations and validated on real-world data.

The authors thank the ANITI program (Artificial Natural Intelligence Toulouse Institute) and the ANR Project RISCOPE (Risk-based system for coastal flooding early warning). JM Loubes acknowledges the funding by DEEL-IRT and C. Samir acknowledges the funding by CNRS Prime.

A Proofs

Proof (Proof of Proposition 1)

Proof

Let $F_1,\dots ,F_n$ in ${{\mathcal {F}}}$. For $i=1,\ldots ,n$, let $g_i = \log _{1}( \phi _i )$. Consider the matrix $\tilde{C}=(<g_i,g_j>)_{\{i,j\}}$. This matrix is a Grammian matrix in ${\mathbb {R}}^{n \times n}$ hence there exists a non negative diagonal matrix D and an orthogonal matrix P such that

$$\begin{aligned} \tilde{C}=PDP^{'} = PD^{1/2} D^{1/2}P^{'}. \end{aligned}$$

Let $e_1,\dots ,e_n$ be the canonical basis of ${\mathbb {R}}^n$. Then $e_i^t \tilde{C} e_j= u_i^t u_j$ where $u_i^t=e_i^t P D^{1/2}$. Note that the $u_i$’s are vectors in ${\mathbb {R}}^n$ that depend on the $f_1,\dots ,f_n$. We get that

$$\begin{aligned} <g_i,g_j>\,= u_i^tu_j, \end{aligned}$$

and for any $F_1,\dots ,F_n$ in ${{\mathcal {F}}}$ there are $u_1,\dots ,u_n$ in ${\mathbb {R}}^n$ such that

$$\begin{aligned} \Vert \log _{1}( \phi _i) - \log _{1}( \phi _j)\Vert = \Vert u_i-u_j\Vert . \end{aligned}$$

So any covariance matrix that can be written as $[ K(\Vert \log _{1}( \phi _i) - \log _{1}( \phi _j) \Vert ) ]_{i,j}$ can be seen as a covariance matrix $[ K(\Vert u_i-u_j\Vert ) ]_{i,j}$ on ${\mathbb {R}}^n$ and inherits its properties. The invertibility and non-negativity of this covariance matrix entail the invertibility and non-negativity of the first one, which proves the result.

Proof (Proof of Theorem 1)

Proof

Let $\theta _1 , \theta _2 \in \varTheta$, with $\theta _1 \ne \theta _2$. Then, there exists $t^* \in [0, \pi /4]$ so that $K_{\theta _1}(0) - K_{\theta _1}(2 t^*) \ne K_{ \theta _2 }(0) - K_{ \theta _2 }( 2t^*)$.

For $i \in {\mathbb {N}}$, let $c_i:[0,1] \rightarrow {\mathbb {R}}$ be defined by $c_i(t) = t^* \cos (2 \pi i t)$. Then, $c_i \in T_1({\mathcal {H}})$. Let $\tilde{e}_i = \exp _{1}( c_i )$. Then, for $t \in [0,1]$

$$\begin{aligned} \tilde{e}_i(t) = \cos ( t^* ) + \frac{\sin ( t^* )}{t^*} t^* \cos (2 \pi i t) \ge \cos ( t^* ) - \sin (t^*) \ge 0. \end{aligned}$$

It follows that $\tilde{e}_i \in {\mathcal {H}}$ and we can let $\tilde{F}_i(t) = \int _{0}^t \tilde{e}_i(s)^2 ds$. Letting $\bar{e}_i = \exp _{1}( -c_i )$, we obtain similarly that $\bar{e}_i \in {\mathcal {H}}$ and we let $\bar{F}_i(t) = \int _{0}^t \bar{e}_i(s)^2 ds$.

Consider the 2n elements $(F_1,...,F_{2n})$ composed by the pairs $( \tilde{F}_i,\bar{F_i})$ for $i=1,\dots ,n$. Consider a Gaussian process Z on ${{\mathcal {F}}}$ with mean function zero and covariance function $K_{ \theta _1 }$. Then, the Gaussian vector $W = (Z(F_i))_{i=1,...,2n}$ has covariance matrix C given by

$$\begin{aligned} C_{i,j} = {\left\{ \begin{array}{ll} K_{ \theta _1 }(0) &{} \text{ if } i=j \\ K_{ \theta _1 }(2 t^*) &{} \text{ if } i \text{ odd } \text{ and } j=i+1 \\ K_{ \theta _1 }(2 t^*) &{} \text{ if } i \text{ even } \text{ and } j=i-1 \\ K_{ \theta _1 }( \sqrt{2} t^*) &{} \text{ else. } \end{array}\right. } \end{aligned}$$

Hence, we have $C = D + M$ where M is the matrix with all components equal to $K_{ \theta _1 }( \sqrt{2} t^*)$ and where D is block diagonal, composed of n blocks of size $2 \times 2$, with each block $B_{2,2}$ equal to

$$\begin{aligned} \begin{pmatrix} K_{ \theta _1 }(0) - K_{ \theta _1 }( \sqrt{2} t^*) &{} K_{ \theta _1 }(2 t^*) - K_{ \theta _1 }( \sqrt{2} t^*) \\ K_{ \theta _1 }(2 t^*) - K_{ \theta _1 }( \sqrt{2} t^*) &{} K_{ \theta _1 }(0) - K_{ \theta _1 }( \sqrt{2} t^*) \end{pmatrix}. \end{aligned}$$

Hence, in distribution, $W = M + E$, with M and E independent, $M=(z,....,z)$ where $z \sim {\mathcal {N}}(0,K_{ \theta _1 }( \sqrt{2} t^*))$ and where the n pairs $(E_{2k+1},E_{2k+2})$, $k=0,...,n-1$ are independent, with distribution ${\mathcal {N}}(0,B_{2,2})$. Hence, with $\bar{W}_1 = (1/n) \sum _{k=0}^{n-1} W_{2k+1}$, $\bar{W}_2 = (1/n) \sum _{k=0}^{n-1} W_{2k+2}$ and $\bar{E} = (1/n) \sum _{k=0}^{n-1} (E_{2k+1},E_{2k+2})^t$, we have

$$\begin{aligned} \hat{B}&:= \frac{1}{n} \sum _{i=0}^{n-1} \begin{pmatrix} W_{2i+1} - \bar{W}_1 \\ W_{2i+2} - \bar{W}_2 \end{pmatrix} \begin{pmatrix} W_{2i+1} - \bar{W}_1 \\ W_{2i+2} - \bar{W}_2 \end{pmatrix}^t \\&= \frac{1}{n} \sum _{i=0}^{n-1} \begin{pmatrix} E_{2i+1} \\ E_{2i+2} \end{pmatrix} \begin{pmatrix} E_{2i+1} \\ E_{2i+2} \end{pmatrix}^t - \bar{E} \bar{E}^t \\&\rightarrow _{n \rightarrow \infty }^{p} B_{2,2}. \end{aligned}$$

Hence, there exists a subsequence $n' \rightarrow \infty$ so that, almost surely $\hat{B} \rightarrow B_{2,2}$ as $n' \rightarrow \infty$. Hence, almost surely $\hat{B}_{1,1} - \hat{B}_{1,2} \rightarrow K_{ \theta _1 }(0) - K_{ \theta _1 }(2 t^*)$ as $n' \rightarrow \infty$. Hence, the event $\{ \hat{B}_{2,2} \rightarrow _{n' \rightarrow \infty } K_{ \theta _1 }(0) - K_{ \theta _1 }(2 t^*) \}$ has probability one under ${\mathbb {P}}_{\theta _1}$. With the same arguments, we can show that the event $\{ \hat{B}_{2,2} \rightarrow _{n'' \rightarrow \infty } K_{ \theta _2 }(0) - K_{ \theta _2 }(2 t^*)$} has probability one under ${\mathbb {P}}_{\theta _2}$, where $n''$ is a subsequence extracted from $n'$. Since these two events have zero intersection, it follows that ${\mathbb {P}}_{\theta _1}$ and ${\mathbb {P}}_{\theta _2}$ are orthogonal. Hence, $\theta$ is microergodic.