Highly Corrupted Image Inpainting Through Hypoelliptic Diffusion

Abstract

We present a new biomimetic image inpainting algorithm, the Averaging and Hypoelliptic Evolution (AHE) algorithm, inspired by the one presented in Boscain et al. (SIAM J. Imaging Sci. 7(2):669–695, 2014) and based upon a semi-discrete variation of the Citti–Petitot–Sarti model of the primary visual cortex V1. The AHE algorithm is based on a suitable combination of sub-Riemannian hypoelliptic diffusion and ad hoc local averaging techniques. In particular, we focus on highly corrupted images (i.e., where more than the 80% of the image is missing), for which we obtain high-quality reconstructions.

Appendix A: Sub-Riemannian Geometry

In this Appendix, we recall some standard definitions of sub-Riemannian geometry and hypoelliptic operators. Classical texts are [2, 3, 22, 30].

Definition 1

A (n, m)-sub-Riemannian manifold is given by a triple $(M,{\blacktriangle },{\mathbf {g}})$, where

• M is a connected smooth manifold of dimension n;

• ${\blacktriangle }$ is a smooth distribution of constant rank $m< n$ satisfying the Hörmander condition. That is, ${\blacktriangle }$ is a smooth map that associates to $q\in M$ an m-dimensional subspace ${\blacktriangle }(q)$ of $T_qM$, such that $\forall ~q\in M$ we have

  $$\begin{aligned} T_qM={{{\mathrm{span}}}}\lbrace [X_1,[\ldots [X_{k-1},X_k]]](q)~|~X_i\in \mathrm {Vec}_H(M)\rbrace . \end{aligned}$$

  Here, $\mathrm {Vec}_H(M)$ denotes the set of horizontal smooth vector fields on M, i.e.,

  $$\begin{aligned} \mathrm {Vec}_H(M)=\lbrace X\in \mathrm {Vec}(M)\ |\ X(q)\in {\blacktriangle }(q)~\ \forall ~q\in M\rbrace . \end{aligned}$$

• ${\mathbf {g}}_q$ is a Riemannian metric on ${\blacktriangle }(q)$, smooth as function of q.

A Lipschitz continuous curve $q(\cdot ):[0,T]\rightarrow M$ is said to be horizontal if $\dot{q}(t)\in {\blacktriangle }(q(t))$ for almost every $t\in [0,T]$. Given an horizontal curve $q(\cdot ):[0,T]\rightarrow M$, the length of $q(\cdot )$ is

$$\begin{aligned} \ell (q(\cdot ))=\int _0^T \sqrt{ {\mathbf {g}}_{q(t)} (\dot{q}(t),\dot{q}(t))}~\hbox {d}t. \end{aligned}$$

The distance induced by the sub-Riemannian structure on M is the function

$$\begin{aligned} d(q_0,q_1)= & {} \inf \lbrace \ell (q(\cdot ))\mid q(0)=q_0,q(T)\\= & {} q_1, q(\cdot )\ \mathrm {horizontal}\rbrace . \end{aligned}$$

The connectedness assumption for M and the Hörmander condition guarantee the finiteness and the continuity of $d(\cdot ,\cdot )$ with respect to the topology of M (Chow’s Theorem, see for instance [2]). The function $d(\cdot ,\cdot )$ is called the Carnot-Carathéodory distance and gives to M the structure of metric space.

Locally, the pair $({\blacktriangle },{\mathbf {g}})$ can be specified by assigning a set of m smooth vector fields spanning ${\blacktriangle }$, that are moreover orthonormal for ${\mathbf {g}}$, i.e.,

$$\begin{aligned} {\blacktriangle }(q)={{{\mathrm{span}}}}\lbrace X_1(q),\ldots ,X_m(q)\rbrace ,~~~{\mathbf {g}}_q(X_i(q),X_j(q))=\delta _{ij}. \end{aligned}$$

(14)

Such a set $\lbrace X_1,\ldots ,X_m\rbrace$ is called a local orthonormal frame for the sub-Riemannian structure. When $({\blacktriangle },{\mathbf {g}})$ can be defined by m globally defined vector fields as in (14) we say that the sub-Riemannian manifold is trivializable.

Given a trivializable (n, m)-sub-Riemannian manifold, the problem of finding a curve realizing the distance between two fixed points $q_0,q_1\in M$ is naturally formulated as the following optimal control problem

$$\begin{aligned} \left\{ \begin{aligned} \phantom {o}&\dot{q}(t)=\sum _{i=1}^m u_i(t) X_i(q(t)), \ \ u_i(\cdot ) \in L^\infty ([0,T],\mathbb {R}), \\ \phantom {o}&\int \limits _0^T \sqrt{\sum \nolimits _{i=1}^m u_i^2(t)}~\hbox {d}t \, \rightarrow \, \min ,\\ \phantom {o}&q(0)=q_0, \ \ \ q(T)=q_1.\\ \end{aligned} \right. \end{aligned}$$

1.1 A.1 Diffusion in a Sub-Riemannian Manifold

Given a sub-Riemannian manifold $(M,{\blacktriangle },{\mathbf {g}})$ and a smooth volume $\omega$ on M, the sub-Riemannian heat equation is the diffusion equation:

$$\begin{aligned} \partial _t \psi =\varDelta _H \psi , \end{aligned}$$

(15)

where $\varDelta _H$ is the sub-Riemannian (or horizontal) Laplacian, defined by

$$\begin{aligned} \varDelta _H \varphi ={{\mathrm{div}}}_\omega {{\mathrm{grad}}}_H \varphi , \qquad \varphi \in C^2(M). \end{aligned}$$

Here, ${{\mathrm{div}}}_\omega$ is the divergence with respect to the volume $\omega$ and ${{\mathrm{grad}}}_H\varphi$ is the horizontal gradient of $\varphi$. That is, it is the unique vector field satisfying, for every $q\in M$,

$$\begin{aligned} {\mathbf {g}_q}({{\mathrm{grad}}}_H \varphi (q),v)= d_q\varphi (v) \text{ for } \text{ every } v\in {\blacktriangle }(q). \end{aligned}$$

If $\lbrace X_1,\ldots ,X_m\rbrace$ is a local orthonormal frame, it follows that ${{\mathrm{grad}}}_H \varphi = \sum _{i=1}^m(X_i\varphi )X_i$, and thus that

$$\begin{aligned} \varDelta _H\varphi = \sum _{i=1}^m \left( X_i^2\varphi + ({{\mathrm{div}}}_\omega X_i)X_i\varphi \right) . \end{aligned}$$

Thanks to the Hörmander condition assumed in the definition of the sub-Riemannian manifold, the celebrated Hörmander Theorem [24], implies the following.

Theorem 1

The operators $\varDelta _H$ (operating on functions $\varphi :M\rightarrow \mathbb {R}$) and $\varDelta _H-\partial _t$ (operating on functions $\psi :M\times \mathbb {R}\rightarrow \mathbb {R}$) are hypoelliptic.

We recall that a second-order differential operator L is said to be hypoelliptic if for every distribution $\varphi$ defined on an open set $\Omega$ of a manifold N, the condition $L\varphi \in {C}^\infty (\Omega )$ implies that $\varphi \in {C}^\infty (\Omega )$. In particular, the hypoellipticity of $\varDelta _H-\partial _t$ implies that any solution to the heat equation (15) on $M\times ]t_0,t_1[$ is smooth.

Remark 2

The sub-Riemannian structure studied in this paper is the one on $PT\mathbb {R}^2$ for which the distribution is given by the vector fields

$$\begin{aligned} X_1(q) = \cos \theta \frac{\partial }{\partial x} + \sin \theta \frac{\partial }{\partial y}, \ \ \ X_2(q) = \frac{\partial }{\partial \theta }. \end{aligned}$$

The metric $\mathbf {g}$ is then chosen such that $\lbrace X_1,X_2\rbrace$ are orthogonal, and ${\mathbf {g}}(X_1,X_1)=1$, ${\mathbf {g}}(X_2,X_2)=1/\beta$, for some given $\beta >0$. By taking as volume on $PT\mathbb {R}^2$ the Lebesgue measure, i.e., $\omega = dx\,dy\,d\theta$, since $X_1$ and $X_2$ are divergence free, one immediately gets

$$\begin{aligned} \varDelta _H= (X_1)^2 + \beta (X_2)^2. \end{aligned}$$