Analysis of Inpainting via Clustered Sparsity and Microlocal Analysis

Abstract

Recently, compressed sensing techniques in combination with both wavelet and directional representation systems have been very effectively applied to the problem of image inpainting. However, a mathematical analysis of these techniques which reveals the underlying geometrical content is missing. In this paper, we provide the first comprehensive analysis in the continuum domain utilizing the novel concept of clustered sparsity, which besides leading to asymptotic error bounds also makes the superior behavior of directional representation systems over wavelets precise. First, we propose an abstract model for problems of data recovery and derive error bounds for two different recovery schemes, namely ℓ ₁ minimization and thresholding. Second, we set up a particular microlocal model for an image governed by edges inspired by seismic data as well as a particular mask to model the missing data, namely a linear singularity masked by a horizontal strip. Applying the abstract estimate in the case of wavelets and of shearlets we prove that—provided the size of the missing part is asymptotic to the size of the analyzing functions—asymptotically precise inpainting can be obtained for this model. Finally, we show that shearlets can fill strictly larger gaps than wavelets in this model.

Appendix: Decay of Shearlet Coefficients Related to Line Singularity

We present the idea of a continuous shearlet system in order to prove various auxiliary results. For ι∈{h,w}, a>0, s∈R, and t∈R ², define

It is easy to show that $\sigma^{\iota}_{a,s,t} = a^{-3/2}\sigma ^{\iota ,a,s}(S_{s}^{\iota}A_{a^{-1}}^{\iota}(\cdot-t))$ for some smooth function σ ^ι,a,s. For s=±a, we similarly define the continuous version of the “seam” elements σ _a,±a,t . The discrete shearlet system $\{\sigma^{\iota}_{j,\ell,k}\}$ is then obtained by sampling $\sigma_{a,s,t}^{\iota}$ on the discrete set of points

To prove that the choice of Λ _j offers clustered sparsity for the shearlet frame, we need some auxiliary results. The following lemma gives the decay estimate of the shearlet elements.

Note that if we define $\langle|t|_{a,s;\iota}\rangle:= \langle |S_{s}^{\iota}A_{a^{-1}}^{\iota}t|\rangle$, then

$$\bigl|\sigma^\iota_{a,s,t}(x)\bigr|\le c_N a^{-3/2} \bigl\langle|x-t|_{a,s;\iota }\bigr\rangle^{-N}. $$

The following lemma is needed later for estimating the decay coefficients of the shearlet aligned with the singularity.

Lemma 16

Let the line segment with respect to (a,s,t;v) be $\mathit{Seg}(a,s,t;v) :=\{S_{s}^{v} A_{a^{-1}}^{v}(x-t_{1},-t_{2}): |x|\le\rho\}$. Then

• 1. Given the line

  $$\mathit{Line}(a,s,t;v):=\bigl\{S_{s}^v A_{a^{-1}}^v(x-t_1,-t_2): x\in\mathbf{R} \bigr\}, $$

  the closest point P _L to the origin on this line satisfies

  $$d_1^2:=\|P_L\|_2^2 = \frac{a^{-4}}{1+s^2}t_2^2. $$

• 2. Set $x_{0} =\frac{a^{-1}s}{1+s^{2}}t_{2}+t_{1}$. If P _S is the closest point on the segment Seg(a,s,t;v) to the origin, then

  

Proof

Let $L(x):=S_{s}^{v} A_{a^{-1}}^{v}(x-t_{1},-t_{2})$. Then

Solving $\frac{d}{dx}\|L(x)\|_{2} = 2(x-t_{1})a^{-2}(1+s^{2})-2a^{-3}st_{2}=0$, we have $x_{0} =\frac{a^{-1}s}{1+s^{2}}t_{2}+t_{1}$. It follows that

Note that P _L ∈Seg(a,s,t;v) if and only if x∈[−ρ,ρ], in which case d ₂=0. Otherwise,

which completes the proof. □

We need another auxiliary lemma. Note that

Lemma 17

Define $R_{N}(x_{0},y_{0}):=\int_{y_{0}}^{\infty}\langle|(x_{0},\alpha)|\rangle ^{-N}d\alpha$ (which may be thought of as a ray integral). Then for y ₀≥0,

$$R_N(x_0,y_0)\le\pi\bigl\langle|x_0| \bigr\rangle^{-1}\bigl\langle\bigl|(x_0,y_0)\bigr|\bigr \rangle^{2-N}. $$

Proof

Choose β∈(0,1). Then

$$\int_{0}^\infty|f(\alpha)|d\alpha\le\Bigl(\sup_{t\in(0,\infty )}|f(\alpha )|^\beta\Bigr)\int_0^\infty|f(\alpha)|^{1-\beta}d\alpha. $$

If we set (1−β)N=2 and f(t)=〈|(x ₀,y ₀+α)|〉^−N, then we obtain

$$R_N(x_0,y_0)\le\Bigl(\sup_{v\in R(x_0,y_0)}\langle|v|\rangle^{2-N}\Bigr) \int _0^\infty \bigl\langle\bigl|(x_0,y_0+\alpha)\bigr|\bigr\rangle^{-2}d\alpha. $$

Since

fixing M=2 and recalling the classic identity $\pi= \int_{-\infty }^{\infty}(1+\alpha^{2})^{-1}d\alpha$ yield the bound

$$\int_0^\infty\bigl\langle\bigl|(x_0,y_0+ \alpha)\bigr|\bigr\rangle^{-2}d\alpha\le\pi \bigl\langle|x_0| \bigr\rangle^{-1}. $$

Furthermore, since y ₀≥0,

$$\sup_{v\in R(x_0,y_0)}\langle|v|\rangle^{2-N}=\bigl\langle \bigl|(x_0,y_0)\bigr|\bigr\rangle^{2-N}. $$

This completes the proof. □

Now we can estimate the decay of the shearlet coefficients aligned with the line singularity as follows.

Lemma 18

Retaining the notation as above, we have

Proof

We have

(22)

where we use an affine transformation of variables to turn the anisotropic norm |(x,0)| _a,s,t;v into the Euclidean norm |w|. Application of the same transformation to [−ρ,ρ]×{0} yields Seg(a,s,t;v). The integral in (22) is along a curve traversing Seg(a,s,t;v) at speed $\nu_{1}=a^{-1}\sqrt{1+s^{2}}$. If we let Ray(a,s,t;v) denote the ray starting from P _S and initially traversing Seg(a,s,t;v), then

 □

Next, we estimate the decay of the shearlet coefficients associated with those shearlets not aligned with the line singularity.

Lemma 19

Let t=(t ₁,t ₂). We consider the following three cases:

• (i) t ₁≠0 and t ₂≠0. Then we have

  

  when 1≤|s|<a ⁻¹

  

  and for s=±a ⁻¹

  

• (ii) If exactly one of t ₁ or t ₂ is 0, then we have

  

• (iii) t ₁=t ₂=0. Then we have

  

Proof

First, it is easy to show that

$$\frac{\partial^L}{\partial\xi_1^L}\frac{\partial^M}{\partial\xi _2^M}|\hat{\sigma}_{a,s,0}^v |\le c_{L,M} a^{3/2} a^{L} a^{2M}. $$

By definition of the line singularity , we have

For t ₁≠0 and t ₂≠0, when we repeatedly apply integration by parts, we have

where

$$h_{L,M}(\xi_2) = \int D^{L,M}\bigl(\hat{w}( \xi_1)\hat{\sigma }_{a,s,0}^v(\xi_1, \xi_2)\bigr)d\xi_1, $$

and for some function f which is sufficiently differentiable we define the multi index,

$$D^{L,M}f(\eta_1,\eta_2) = \biggl( \frac{\partial}{\partial\eta_1} \biggr)^L \biggl(\frac{\partial}{\partial\eta_2} \biggr)^M f(\eta_1,\eta_2). $$

The next step is to estimate the term |h _L,M (ξ ₂)|.

Let Ξ _a,s (ξ ₂) be the support of the function

$$\xi_1\mapsto D^{L,M} \bigl(\hat{w}(\xi_1)\hat{ \sigma}^v_{a,s,0}(\xi_1,\xi_2)\bigr). $$

Note that for fixed a,s, the function $\xi_{1}\mapsto\hat{w}(\xi _{1})\times \hat{\sigma}_{a,s,0}^{v}(\xi_{1},\xi_{2})$ is supported inside $[c a^{-1}|s|,\frac{1}{2}a^{-1}s)$ for a constant $c < \frac{1}{2}$. h _L,M can then be written as

$$h_{L,M}(\xi_2) = \int_{\varXi_{a,s}(\xi_2)}D^{L,M} \bigl(\hat{w}(\xi_1)\hat {\sigma}^v_{a,s,0}( \xi_1,\xi_2)\bigr)d\xi_1. $$

We then rewrite the integrand as

Thus |h _L,M (ξ ₂)| is bounded by

where

$$N^{L-\ell,M}(a,s) = \bigl\|D^{L-\ell,M}\hat{\sigma}_{a,s,0}^v( \xi_1,\xi_2)\bigr\|_{L^\infty(\varXi_{a,s}(\xi_2))} $$

Consequently, we have

Therefore,

Using the same approach, it is not difficult to show that for |s|<a ⁻¹,

and for s=±a ⁻¹

The proofs for other cases are similar with simple modifications of the above procedure. □