A Fast Algorithm to Estimate the Square Root of Probability Density Function

Abstract

A fast maximum likelihood estimator based on a linear combination of Gaussian kernels is introduced to represent the square root of probability density function. It is shown that, if the kernel centres and kernel width are known, then the underlying problem can be formulated as a Riemannian optimization one. The first order Riemannian geometry of the sphere manifold and vector transport are explored, and then the well-known Riemannian conjugate gradient algorithm is used to estimate the model parameters. For completeness the k-means clustering algorithm and a grid search are applied to determine the centers and kernel width respectively. Illustrative examples are employed to demonstrate that the proposed approach is effective in constructing the estimate of the square root of probability density function.

Appendix A

To integrate $q_{i,j}=\int K_{\sigma }\big (\varvec{x},\varvec{c}_i \big ) K_{\sigma }\big (\varvec{x},\varvec{c}_j \big ) d\varvec{x}$, we let $\varvec{x}=[x_1,...x_m]^\mathrm{T}$, we have

$$\begin{aligned}&q_{i,j}=\frac{1}{ (2 \pi \sigma ^2)^{m} } \int ... \int \exp \left( -\frac{\Vert \varvec{x}- \varvec{c}_i \Vert ^2 }{2\sigma ^2} -\frac{\Vert \varvec{x}- \varvec{c}_j \Vert ^2 }{2\sigma ^2} \right) \nonumber \\&\ \ \ dx_1...dx_m\nonumber \\&=\frac{1}{ (2 \pi \sigma ^2)^{m} } \prod _{l=1}^{m} \int \exp \Big ( -\frac{(x_l-c_{i,l})^2}{2\sigma ^2} - \frac{(x_l- c_{j,l})^2}{2\sigma ^2} \Big )dx_l \end{aligned}$$

(23)

in which

$$\begin{aligned}&\int \exp \Big ( -\frac{(x_l-c_{i,l})^2}{2\sigma ^2} - \frac{(x_l-c_{j,l})^2}{2\sigma ^2} \Big )dx_l \nonumber \\&=\int \exp \Big ( -\frac{ x_l^2- (c_{i,l} +c_{j,l} )x_l +(c_{i,l}^2 +c_{j,l}^2)/2 }{ \sigma ^2 } \Big )dx_l\nonumber \\&=\exp \Big (-\frac{ \frac{ c_{j,l}^2 + c_{i,l}^2 }{2 }-\big (\frac{ c_{i,l} +c_{j,l} }{2} \big )^2 }{ \sigma ^2 } \Big ) \nonumber \\&\ \ \ \times \int \exp \Big ( -\frac{ \big [x_l-(c_{i,l} +c_{j,l} ) \big ]^2}{ \sigma ^2 } \Big )dx_l \end{aligned}$$

(24)

By making use of $\int \frac{1}{\sqrt{2\pi \sigma ^2}} \exp \Big (-\frac{(x_l-c)^2}{2\sigma ^2} \Big )dx_l =1$, i.e. Gaussian density integrates to one, we have

$$\begin{aligned}&\ \ \int \exp \Big ( -\frac{(x_l-c_{i,l})^2}{2\sigma ^2}- \frac{(x_l-c_{j,l})^2}{2\sigma ^2} \Big )dx_l\nonumber \\&=\sqrt{ \pi \sigma ^2 }\exp \Big (-\frac{ \frac{ c_{j,l}^2 + c_{i,l}^2 }{2 }-\big (\frac{ c_{i,l} +c_{j,l}}{2} \big )^2 }{ \sigma ^2 } \Big )\nonumber \\&=\sqrt{ \pi \sigma ^2 }\exp \Big (-\frac{ (c_{j,l} - c_{i,l})^2 }{4 \sigma ^2 } \Big ) \end{aligned}$$

(25)

Hence

$$\begin{aligned} q_{i,j}= \frac{ 1}{ (4 \pi \sigma ^2)^{m} } \exp \left( -\frac{\Vert \varvec{c}_i- \varvec{c}_j \Vert ^2 }{4\sigma ^2} \right) =K_{\sqrt{2}\sigma }\big (\varvec{c}_i,\varvec{c}_j\big ) \end{aligned}$$

(26)