Journal of Mathematical Imaging and Vision

, Volume 48, Issue 2, pp 205–234 | Cite as

Analysis of Inpainting via Clustered Sparsity and Microlocal Analysis

  • Emily J. King
  • Gitta Kutyniok
  • Xiaosheng Zhuang


Recently, compressed sensing techniques in combination with both wavelet and directional representation systems have been very effectively applied to the problem of image inpainting. However, a mathematical analysis of these techniques which reveals the underlying geometrical content is missing. In this paper, we provide the first comprehensive analysis in the continuum domain utilizing the novel concept of clustered sparsity, which besides leading to asymptotic error bounds also makes the superior behavior of directional representation systems over wavelets precise. First, we propose an abstract model for problems of data recovery and derive error bounds for two different recovery schemes, namely 1 minimization and thresholding. Second, we set up a particular microlocal model for an image governed by edges inspired by seismic data as well as a particular mask to model the missing data, namely a linear singularity masked by a horizontal strip. Applying the abstract estimate in the case of wavelets and of shearlets we prove that—provided the size of the missing part is asymptotic to the size of the analyzing functions—asymptotically precise inpainting can be obtained for this model. Finally, we show that shearlets can fill strictly larger gaps than wavelets in this model.


1 minimization Cluster coherence Inpainting Parseval frames Sparse representation Data recovery Shearlets Meyer wavelets 


  1. 1.
    Aharon, M., Elad, M., Bruckstein, A.: K-SVD: an algorithm for designing overcomplete dictionaries for sparse representation. IEEE Trans. Signal Process. 54, 4311–4322 (2006) CrossRefGoogle Scholar
  2. 2.
    Ballester, C., Bertalmio, M., Caselles, V., Sapiro, G., Verdera, J.: Filling-in by joint interpolation of vector fields and gray levels. IEEE Trans. Image Process. 10(8), 1200–1211 (2001) CrossRefMATHMathSciNetGoogle Scholar
  3. 3.
    Bertalmio, M., Bertozzi, A., Sapiro, G.: Navier-Stokes, fluid dynamics, and image and video inpainting. In: Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition 2001 (CVPR 2001), pp. I355–I362. IEEE Press, New York (2001) Google Scholar
  4. 4.
    Bertalmío, M., Sapiro, G., Caselles, V., Ballester, C.: Image inpainting. In: Proceedings of SIGGRAPH 2000, New Orleans, pp. 417–424 (2000) Google Scholar
  5. 5.
    Cai, J.F., Cha, R.H., Shen, Z.: Simultaneous cartoon and texture inpainting. Inverse Probl. Imaging 4(3), 379–395 (2010) CrossRefMATHMathSciNetGoogle Scholar
  6. 6.
    Cai, J.F., Dong, B., Osher, S., Shen, Z.: Image restoration: Total variation, wavelet frames, and beyond. J. Am. Math. Soc. 25, 1033–1089 (2012) CrossRefMATHMathSciNetGoogle Scholar
  7. 7.
    Candès, E.J., Donoho, D.L.: New tight frames of curvelets and optimal representations of objects with piecewise C 2 singularities. Commun. Pure Appl. Math. 57(2), 219–266 (2004) CrossRefMATHGoogle Scholar
  8. 8.
    Candès, E.J., Donoho, D.L.: Continuous curvelet transform. I. Resolution of the wavefront set. Appl. Comput. Harmon. Anal. 19(2), 162–197 (2005) CrossRefMATHMathSciNetGoogle Scholar
  9. 9.
    Chan, T.F., Kang, S.H.: Error analysis for image inpainting. J. Math. Imaging Vis. 26(1–2), 85–103 (2006) CrossRefMathSciNetGoogle Scholar
  10. 10.
    Chan, T.F., Kang, S.H., Shen, J.: Euler’s elastica and curvature based inpainting. SIAM J. Appl. Math. 63(2), 564–592 (2002) MATHMathSciNetGoogle Scholar
  11. 11.
    Chan, T.F., Shen, J.: Mathematical models for local nontexture inpaintings. SIAM J. Appl. Math. 62(3), 1019–1043 (2002) (electronic) CrossRefMATHMathSciNetGoogle Scholar
  12. 12.
    Christensen, O.: An Introduction to Frames and Riesz Bases. Applied and Numerical Harmonic Analysis. Birkhäuser, Boston (2003) CrossRefMATHGoogle Scholar
  13. 13.
    Chui, C.K., Shi, X.L.: Inequalities of Littlewood-Paley type for frames and wavelets. SIAM J. Math. Anal. 24(1), 263–277 (1993) CrossRefMATHMathSciNetGoogle Scholar
  14. 14.
    Daubechies, I.: Ten Lectures on Wavelets. CBMS-NSF Regional Conference Series in Applied Mathematics, vol. 61. SIAM, Philadelphia (1992) CrossRefMATHGoogle Scholar
  15. 15.
    Dong, B., Ji, H., Li, J., Shen, Z., Xu, Y.: Wavelet frame based blind image inpainting. Appl. Comput. Harmon. Anal. 32(2), 268–279 (2012) CrossRefMATHMathSciNetGoogle Scholar
  16. 16.
    Donoho, D.L., Elad, M.: Optimally sparse representation in general (nonorthogonal) dictionaries via l 1 minimization. Proc. Natl. Acad. Sci. USA 100(5), 2197–2202 (2003) (electronic) CrossRefMATHMathSciNetGoogle Scholar
  17. 17.
    Donoho, D.L., Huo, X.: Uncertainty principles and ideal atomic decomposition. IEEE Trans. Inf. Theory 47(7), 2845–2862 (2001) CrossRefMATHMathSciNetGoogle Scholar
  18. 18.
    Donoho, D.L., Kutyniok, G.: Microlocal analysis of the geometric separation problem. Commun. Pure Appl. Math. 66, 1–47 (2013) CrossRefMATHMathSciNetGoogle Scholar
  19. 19.
    Drori, I.: Fast 1 minimization by iterative thresholding for multidimensional NMR spectroscopy. EURASIP J. Adv. Signal Process. 2007, 020248 (2007) CrossRefMathSciNetGoogle Scholar
  20. 20.
    Elad, M., Bruckstein, A.M.: A generalized uncertainty principle and sparse representation in pairs of bases. IEEE Trans. Inf. Theory 48(9), 2558–2567 (2002) CrossRefMATHMathSciNetGoogle Scholar
  21. 21.
    Elad, M., Starck, J.L., Querre, P., Donoho, D.L.: Simultaneous cartoon and texture image inpainting using morphological component analysis (MCA). Appl. Comput. Harmon. Anal. 19(3), 340–358 (2005) CrossRefMATHMathSciNetGoogle Scholar
  22. 22.
    Engan, K., Aase, S., Hakon Husoy, J.: Method of optimal directions for frame design. In: IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP ’99), vol. 5, pp. 2443–2446 (1999) Google Scholar
  23. 23.
    Foucart, S.: Recovering jointly sparse vectors via hard thresholding pursuit. In: Proceedings of SampTA 2011, Singapore (2011) Google Scholar
  24. 24.
    Grohs, P.: Continuous shearlet frames and resolution of the wavefront set. Monatshefte Math. 164(4), 393–426 (2011) CrossRefMATHMathSciNetGoogle Scholar
  25. 25.
    Grohs, P., Kutyniok, G.: Parabolic molecules. Preprint (2012) Google Scholar
  26. 26.
    Guo, K., Labate, D.: Analysis and detection of surface discontinuities using the 3D continuous shearlet transform. Appl. Comput. Harmon. Anal. 30(2), 231–242 (2011) CrossRefMATHMathSciNetGoogle Scholar
  27. 27.
    Guo, K., Labate, D.: The construction of smooth parseval frames of shearlets. Math. Model. Nat. Phenom. 8(1), 32–55 (2013) CrossRefMathSciNetGoogle Scholar
  28. 28.
    Hennenfent, G., Fenelon, L., Herrmann, F.J.: Nonequispaced curvelet transform for seismic data reconstruction: a sparsity-promoting approach. Geophysics 75(6), WB203–WB210 (2010) CrossRefGoogle Scholar
  29. 29.
    Hennenfent, G., Herrmann, F.J.: Application of stable signal recovery to seismic interpolation. In: SEG International Exposition and 76th Annual Meeting. SEG, Tulsa (2006) Google Scholar
  30. 30.
    Herrmann, F.J., Hennenfent, G.: Non-parametric seismic data recovery with curvelet frames. Geophys. J. Int. 173, 233–248 (2008) CrossRefGoogle Scholar
  31. 31.
    Hörmander, L.: The Analysis of Linear Partial Differential Operators. I. Classics in Mathematics. Springer, Berlin (2003). Distribution theory and Fourier analysis. Reprint of the second (1990) edition [Springer, Berlin; MR1065993 (91m:35001a)] Google Scholar
  32. 32.
    Jing, Z.: On the stability of wavelet and Gabor frames (Riesz bases). J. Fourier Anal. Appl. 5(1), 105–125 (1999) CrossRefMATHMathSciNetGoogle Scholar
  33. 33.
    King, E.J.: Wavelet and frame theory: frame bound gaps, generalized shearlets, Grassmannian fusion frames, and p-adic wavelets. Ph.D. thesis, University of Maryland, College Park (2009) Google Scholar
  34. 34.
    King, E.J., Kutyniok, G., Zhuang, X.: Analysis of data separation and recovery problems using clustered sparsity. In: SPIE Proceedings: Wavelets and Sparsity XIV, vol. 8138 (2011) Google Scholar
  35. 35.
    Kowalski, M., Torrésani, B.: Sparsity and persistence: mixed norms provide simple signal models with dependent coefficients. Signal Image Video Process. 3, 251–264 (2009) CrossRefMATHGoogle Scholar
  36. 36.
    Kutyniok, G.: Geometric separation by single pass alternating thresholding. Appl. Comput. Harmon. Anal. (to appear) Google Scholar
  37. 37.
    Kutyniok, G., Labate, D.: Resolution of the wavefront set using continuous shearlets. Trans. Am. Math. Soc. 361(5), 2719–2754 (2009) CrossRefMATHMathSciNetGoogle Scholar
  38. 38.
    Kutyniok, G., Labate, D. (eds.): Shearlets: Multiscale Analysis for Multivariate Data. Applied and Numerical Harmonic Analysis. Birkhäuser, Boston (2012) Google Scholar
  39. 39.
    Kutyniok, G., Lemvig, J., Lim, W.: Compactly supported shearlets. In: Approximation Theory XIII, San Antonio, TX, 2010. Springer, Berlin (2010) Google Scholar
  40. 40.
    Kutyniok, G., Lemvig, J., Lim, W.: Shearlets and optimally sparse approximations. In: Shearlets: Multiscale Analysis for Multivariate Data. Springer, Berlin (2012) CrossRefGoogle Scholar
  41. 41.
    Kutyniok, G., Lemvig, J., Lim, W.Q.: Optimally sparse approximations of 3d functions by compactly supported shearlet frames. SIAM J. Appl. Math. 44, 2962–3017 (2012) MATHMathSciNetGoogle Scholar
  42. 42.
    Kutyniok, G., Lim, W.: Compactly supported shearlets are optimally sparse. J. Approx. Theory 163, 1564–1589 (2011) CrossRefMATHMathSciNetGoogle Scholar
  43. 43.
    Meyer, Y.: Principe d’incertitude, bases hilbertiennes et algèbres d’opérateurs. Astérisque 145–146(4), 209–223 (1987). Séminaire Bourbaki, Vol. 1985/86 Google Scholar
  44. 44.
    Meyer, Y.: Oscillating Patterns in Image Processing and Nonlinear Evolution Equations. University Lecture Series, vol. 22. Am. Math. Soc., Providence (2001). The fifteenth Dean Jacqueline B. Lewis memorial lectures MATHGoogle Scholar
  45. 45.
    Nam, S., Davies, M., Elad, M., Gribonval, R.: Cosparse analysis modeling—uniqueness and algorithms. In: International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2011). IEEE Press, New York (2011) Google Scholar
  46. 46.
    Nam, S., Davies, M.E., Elad, M., Gribonval, R.: The cosparse analysis model and algorithms. Appl. Comput. Harmon. Anal. 34(1), 30–56 (2013) CrossRefMATHMathSciNetGoogle Scholar
  47. 47.
    Olshausen, B.A., Field, D.J.: Sparse coding with an overcomplete basis set: a strategy employed by V1? Vis. Res. 37(23), 3311–3325 (1997) CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  • Emily J. King
    • 1
  • Gitta Kutyniok
    • 1
  • Xiaosheng Zhuang
    • 2
  1. 1.Department of MathematicsTechnische Universität BerlinBerlinGermany
  2. 2.Department of MathematicsCity University of Hong KongHong KongChina

Personalised recommendations