Soft Image Segmentation: On the Clustering of Irregular, Weighted, Multivariate Marked Networks

Abstract

The contribution exposes and illustrates a general, flexible formalism, together with an associated iterative procedure, aimed at determining soft memberships of marked nodes in a weighted network. Gathering together spatial entities which are both spatially close and similar regarding their features is an issue relevant in image segmentation, spatial clustering, and data analysis in general. Unoriented weighted networks are specified by an “exchange matrix”, determining the probability to select a pair of neighbors. We present a family of membership-dependent free energies, whose local minimization specifies soft clusterings. The free energy additively combines a mutual information, as well as various energy terms, concave or convex in the memberships: within-group inertia, generalized cuts (extending weighted Ncut and modularity), and membership discontinuities (generalizing Dirichlet forms). The framework is closely related to discrete Markov models, random walks, label propagation and spatial autocorrelation (Moran’s I), and can express the Mumford-Shah approach. Four small datasets illustrate the theory.

Appendix

1.1 Computing the Exchange Matrix E(A, f)

Defining an exchange matrix E both weight-compatible (that is obeying $E\mathbf{1}=f$, where the regional weights f are given) and reflecting the spatial structure contained in the binary adjacency matrix $A=(a_{ij})$ is a crucial, necessary step in the “ZED formalism” under consideration. Two constructions, not trivial, nor that difficult either, have been investigated in this paper, namely the diffusive specification and the Metropolis-Hastings specification.

The Diffusive Exchange Matrix. Consider a time-continuous Markov chain W on the n pixels, whose infinitesimal generator or rate matrix is proportional to the adjacency matrix A, and conveniently normalized so that f constitutes the stationary distribution of W. The resulting exchange matrix $E=\varPi W$ turns out to be symmetric and p.s.d., and given by

(20)

where $\varPi = \text{ diag }(f)$, and

$$\begin{aligned} \varPsi = \varPi ^{-1/2}\frac{LA}{\mathrm {trace}{(LA)}} \varPi ^{-1/2} \qquad \qquad (LA)_{ij} = \delta _{ij} \, a_{i\bullet } - a_{ij} \end{aligned}$$

LA is the Laplacian of matrix A, and matrix exponentiation (20) can be carried out by the spectral decomposition of $\varPsi$. Specification (20) describes a diffusive process at time $t>0$, with limits $\lim _{t\rightarrow 0} E(A, f, t)=\varPi$ (“frozen network”, consisting of n isolated nodes: spatial autarchy), and $\lim _{t\rightarrow \infty } E(A, f, t)=ff'$ (“complete network”, with independent selection of the node pairs: complete mobility). Identity $\text{ trace }(E(t)) = 1-t + 0(t^2)$ shows t to measure, for $t\ll 1$, the proportion of distinct regional pairs in the joint distribution E.

The Metropolis-Hastings Exchange Matrix. The natural random walk with Markov transition matrix $a_{ij}/a_{i\bullet }$ correctly describes the spatial structure of the network, but its stationary distribution is $g_i=a_{i\bullet }/a_{\bullet \bullet }$ instead of $f_i$. Applying the Metropolis-Hastings algorithm defines a recalibrated random walk with stationary distribution f, ending up in a weight-compatible exchange matrix of the form:

$$\begin{aligned} E =\varPi -LB\qquad \text{ where }\quad B=(b_{ij}),\, \, b_{ij}=\min (\kappa _i,\kappa _j)\cdot \frac{a_{ij}}{a_{\bullet \bullet }}\, \, \text{ and } \, \, \kappa _i=\frac{f_i}{g_i} \end{aligned}$$

(21)

and $(LB)_{ij} = \delta _{ij} \, b_{i\bullet } - b_{ij}$ is the Laplacian of B. Expression (21) does not require spectral decomposition, and its computation is much faster than (20) for increasing n (Fig. 15). However, E in (21) is not p.s.d in general, thus threatening the concavity of $\mathcal{C}^\kappa [Z]$ (Sect. 5.1).

Fig. 15.

Deterministic profiling : CPU time for computing the exchange matrices E(f, A, t) (20) and $E_{M.-H.}(f, A)$ (21), as a function of the number of pixels n in a regular setting and performed with Python 2.7.12 on a CPU Intel Core i7 two Core with a frequency $3.1\,\text{ GHz }$ (Mac OS X 10.10.5).

1.2 Testing Spatial Autocorrelation

Under the null hypothesis $H_0$ of stationarity and absence of spatial autocorrelation, univariate features are independent, and follow a distribution with common mean and variance inversely proportional to the size of the region, namely $E(X_{ik})=\mu _k$ and $\text{ Cov }(X_{ik},X_{jk})=\delta _{ij}\sigma ^2_k/f_i$ [40]. Under normal approximation, the expected value of the multivariate Moran’s I (2) reads

$$\begin{aligned} E_0(I) = \frac{\text{ tr }(W)-1}{n-1} \qquad \text{ where } \quad w_{ij} = \frac{e_{ij}}{f_i} \end{aligned}$$

and its the variance reads

$$\begin{aligned} \text{ Var }_0(I) = \frac{2}{n^2-1} \left[ \text {trace}(W^2) - 1 - \frac{(\text {trace}(W) - 1)^2}{n-1}\right] \end{aligned}$$

Spatial autocorrelation is thus significant at level $\alpha$ if $z= |I - E_0(I)|/ \sqrt{\text{ Var }_0}(I)\, \ge \, u_{1-\frac{\alpha }{2}}$, where $u_{1-\frac{\alpha }{2}}$ is the quantile of the standard normal distribution.

Alternatively, a permutation test can be performed (e.g. [41]), by generating a series of values $\hat{I}$ of the transformed Moran index, where $\hat{I}$ obtains as (2) with replaced by . The plain specification, which consists in replacing the profile $x_{ik}$ of region i by the profile $\hat{x}_{ik}=x_{\pi (i)k}$ of another region $\pi (i)$ (where $\pi$ denotes a permutation), that is in defining $\hat{D}_{ij}=D_{\pi (i),\pi (j)}$, is somehow flawed in the weighted case, in view of the heteroscedasticity of the distribution of $X_{ik}$. Instead, the quantities $\sqrt{f_i}(x_{ik}-\bar{x}_k)$ (with $\bar{x}_k=\sum _i f_i x_{ik}$) for $i=1,\ldots , n$ are expected to follow the same distribution under $H_0$, thus insuring the validity of the weight-corrected specification, with (see Fig. 2)

$$\begin{aligned} \hat{x}_{ik}=\bar{x}_k+\sqrt{\frac{f_{\pi (i)}}{f_i}}(x_{\pi (i)k}-\bar{x}_k) \qquad \text{ and }\qquad \hat{D}_{ij}=\Vert \hat{x}_i-\hat{x}_j\Vert ^2. \end{aligned}$$

(22)