Denoising by Inpainting

  • Robin Dirk Adam
  • Pascal PeterEmail author
  • Joachim Weickert
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10302)


The filling-in effect of diffusion processes has been successfully used in many image analysis applications. Examples include image reconstructions in inpainting-based compression or dense optic flow computations. As an interesting side effect of diffusion-based inpainting, the interpolated data are smooth, even if the known image data are noisy: Inpainting averages information from noisy sources. Since this effect has not been investigated for denoising purposes so far, we propose a general framework for denoising by inpainting. It averages multiple inpainting results from different selections of known data. We evaluate two concrete implementations of this framework: The first one specifies known data on a shifted regular grid, while the second one employs probabilistic densification to optimise the known pixel locations w.r.t. the inpainting quality. For homogeneous diffusion inpainting, we demonstrate that our regular grid method approximates the quality of its corresponding diffusion filter. The densification algorithm with homogeneous diffusion inpainting, however, shows edge-preserving behaviour. It resembles space-variant diffusion and offers better reconstructions than homogeneous diffusion filters.


Diffusion Denoising Inpainting Densification 


  1. 1.
    Belhachmi, Z., Bucur, D., Burgeth, B., Weickert, J.: How to choose interpolation data in images. SIAM J. Appl. Math. 70(1), 333–352 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  2. 2.
    Bertalmío, M., Sapiro, G., Caselles, V., Ballester, C.: Image inpainting. In: Proceedings of the SIGGRApPH 2000, New Orleans, LI, pp. 417–424, July 2000Google Scholar
  3. 3.
    Bruhn, A., Weickert, J.: A confidence measure for variational optic flow methods. In: Klette, R., Kozera, R., Noakes, L., Weickert, J. (eds.) Geometric Properties from Incomplete Data, Computational Imaging and Vision, vol. 31, pp. 283–297. Springer, Dordrecht (2006)CrossRefGoogle Scholar
  4. 4.
    Buades, A., Coll, B., Morel, J.M.: A review of image denoising algorithms, with a new one. Multiscale Model. Simul. 4(2), 490–530 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    Charbonnier, P., Blanc-Féraud, L., Aubert, G., Barlaud, M.: Deterministic edge-preserving regularization in computed imaging. IEEE Trans. Image Proc. 6(2), 298–311 (1997)CrossRefGoogle Scholar
  6. 6.
    Chen, Y., Yu, W., Pock, T.: On learning optimized reaction diffusion processes for effective image restoration. In: Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, pp. 5261–5269, June 2015Google Scholar
  7. 7.
    Chen, Y., Ranftl, R., Pock, T.: A bi-level view of inpainting-based image compression. In: Proceedings of the 19th Computer Vision Winter Workshop, Křtiny, Czech Republic, pp. 19–26, February 2014Google Scholar
  8. 8.
    Craven, P., Wahba, G.: Smoothing noisy data with spline functions. Numer. Math. 31(4), 377–403 (1978)MathSciNetCrossRefzbMATHGoogle Scholar
  9. 9.
    Dabov, K., Foi, A., Katkovnik, V., Egiazarian, K.: Image denoising by sparse 3-D transform-domain collaborative filtering. IEEE Trans. Image Proc. 16(8), 2080–2095 (2007)MathSciNetCrossRefGoogle Scholar
  10. 10.
    Efros, A.A., Leung, T.K.: Texture synthesis by non-parametric sampling. In: Proceedings of the Seventh IEEE International Conference on Computer Vision, Corfu, Greece, vol. 2, pp. 1033–1038, September 1999Google Scholar
  11. 11.
    Fritsch, D.S.: A medial description of greyscale image structure by gradient-limited diffusion. In: Proceedings of SPIE Visualization in Biomedical Computing 1992, vol. 1808, pp. 105–117. SPIE Press, Bellingham (1992)Google Scholar
  12. 12.
    Galić, I., Weickert, J., Welk, M., Bruhn, A., Belyaev, A., Seidel, H.P.: Image compression with anisotropic diffusion. J. Math. Imaging Vis. 31(2–3), 255–269 (2008)MathSciNetzbMATHGoogle Scholar
  13. 13.
    Hoeltgen, L., Setzer, S., Weickert, J.: An optimal control approach to find sparse data for laplace interpolation. In: Heyden, A., Kahl, F., Olsson, C., Oskarsson, M., Tai, X.-C. (eds.) EMMCVPR 2013. LNCS, vol. 8081, pp. 151–164. Springer, Heidelberg (2013). doi: 10.1007/978-3-642-40395-8_12 CrossRefGoogle Scholar
  14. 14.
    Horn, B., Schunck, B.: Determining optical flow. Artif. Intell. 17, 185–203 (1981)CrossRefGoogle Scholar
  15. 15.
    Iijima, T.: Basic theory on normalization of pattern (in case of typical one-dimensional pattern). Bull. Electrotech. Lab. 26, 368–388 (1962). (in Japanese)Google Scholar
  16. 16.
    Mainberger, M., Hoffmann, S., Weickert, J., Tang, C.H., Johannsen, D., Neumann, F., Doerr, B.: Optimising spatial and tonal data for homogeneous diffusion inpainting. In: Bruckstein, A.M., ter Haar Romeny, B.M., Bronstein, A.M., Bronstein, M.M. (eds.) SSVM 2011. LNCS, vol. 6667, pp. 26–37. Springer, Heidelberg (2012). doi: 10.1007/978-3-642-24785-9_3 CrossRefGoogle Scholar
  17. 17.
    Masnou, S., Morel, J.M.: Level lines based disocclusion. In: Proceedings of the 1998 IEEE International Conference on Image Processing, Chicago, IL, vol. 3, pp. 259–263, October 1998Google Scholar
  18. 18.
    Nagel, H.H., Enkelmann, W.: An investigation of smoothness constraints for the estimation of displacement vector fields from image sequences. IEEE Trans. Pattern Anal. Mach. Intell. 8, 565–593 (1986)CrossRefGoogle Scholar
  19. 19.
    Perona, P., Malik, J.: Scale space and edge detection using anisotropic diffusion. IEEE Trans. Pattern Anal. Mach. Intell. 12, 629–639 (1990)CrossRefGoogle Scholar
  20. 20.
    Peter, P., Weickert, J., Munk, A., Krivobokova, T., Li, H.: Justifying tensor-driven diffusion from structure-adaptive statistics of natural images. In: Tai, X.-C., Bae, E., Chan, T.F., Lysaker, M. (eds.) EMMCVPR 2015. LNCS, vol. 8932, pp. 263–277. Springer, Cham (2015). doi: 10.1007/978-3-319-14612-6_20 Google Scholar
  21. 21.
    Roth, S., Black, M.J.: Fields of experts. Int. J. Comput. Vis. 82(2), 205–229 (2009)CrossRefGoogle Scholar
  22. 22.
    Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Physica D 60(1), 259–268 (1992)MathSciNetCrossRefzbMATHGoogle Scholar
  23. 23.
    Scherzer, O., Weickert, J.: Relations between regularization and diffusion filtering. J. Math. Imaging Vis. 12(1), 43–63 (2000)MathSciNetCrossRefzbMATHGoogle Scholar
  24. 24.
    Schmaltz, C., Peter, P., Mainberger, M., Ebel, F., Weickert, J., Bruhn, A.: Understanding, optimising, and extending data compression with anisotropic diffusion. Int. J. Comput. Vis. 108(3), 222–240 (2014)MathSciNetCrossRefGoogle Scholar
  25. 25.
    Schönlieb, C.B.: Partial Differential Equation Methods for Image Inpainting. Cambridge University Press, Cambridge (2015)CrossRefzbMATHGoogle Scholar
  26. 26.
    Tikhonov, A.N., Arsenin, V.Y.: Solutions of Ill-Posed Problems. Wiley, Washington (1977)zbMATHGoogle Scholar
  27. 27.
    Weickert, J.: Anisotropic Diffusion in Image Processing. Teubner, Stuttgart (1998)zbMATHGoogle Scholar
  28. 28.
    Weickert, J., Steidl, G., Mrázek, P., Welk, M., Brox, T.: Diffusion filters and wavelets: what can they learn from each other? In: Paragios, N., Chen, Y., Faugeras, O. (eds.) Handbook of Mathematical Models in Computer Vision, pp. 3–16. Springer, New York (2006)Google Scholar
  29. 29.
    Weyrich, N., Warhola, G.T.: Wavelet shrinkage and generalized cross validation for image denoising. IEEE Trans. Image Proc. 7(1), 82–90 (1998)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Robin Dirk Adam
    • 1
  • Pascal Peter
    • 1
    Email author
  • Joachim Weickert
    • 1
  1. 1.Mathematical Image Analysis Group, Faculty of Mathematics and Computer Science, Campus E1.7Saarland UniversitySaarbrückenGermany

Personalised recommendations