Multimedia Tools and Applications

, Volume 56, Issue 3, pp 469–483 | Cite as

Simultaneous inpainting for image structure and texture using anisotropic heat transfer model

  • Chuan QinEmail author
  • Shuozhong Wang
  • Xinpeng Zhang


We propose a PDE-based image inpainting method using anisotropic heat transfer model, which can simultaneously propagate the structure and texture information. In structure inpainting, the propagating direction and intensity are related to image contents, and the strength of propagation along gradient direction is made inversely proportional to the magnitude of gradient. In texture inpainting, the added texture term reflects periodicity along the texture and its perpendicular direction. For numerical implementation, the step size of finite difference is adaptively chosen according to the curvature, leading to fewer iteration steps and satisfactory inpainting quality. Compared with other high order PDE methods and layered methods, the proposed approach is more concise and doesn’t need image decomposition. Experiments are carried out to show effectiveness of the method.


Image inpainting Structure Texture PDE Heat transfer Anisotropic Finite difference 



This work was supported by the Natural Science Foundation of China (60872116, 60832010, and 60773079), the Shanghai Rising-Star Program (10QH14011), the Key Scientific Research Project of Shanghai Education Committee (10ZZ59), the Shanghai Specialized Research Foundation for Excellent Young Teacher in University (slg09005), and the OECE Innovation Foundation of USST (GDCX-Y-103).


  1. 1.
    Ballester C, Bertalmío M, Caselles V, Garrido L, Marques A, Ranchin F (2007) An inpainting-based deinterlacing method. IEEE Trans Image Process 16(10):2476–2491MathSciNetCrossRefGoogle Scholar
  2. 2.
    Bertalmio M, Bertozzi AL, Sapiro G (2001) Navier-stokes, fluid dynamics, and image and video inpainting, in Proc. IEEE Int. Conf. Comput. Vision and Pattern Recognit., Dec., Kauai, HI I:355–362Google Scholar
  3. 3.
    Bertalmio M, Sapiro G, Caselles V, Ballester C (2000) Image inpainting, in Proc. ACM SIGGRAPH Conf. Comput. Graphics 417–424Google Scholar
  4. 4.
    Bertalmio M, Vese L, Sapiro G, Osher S (2003) Simultaneous structure and texture image inpainting. IEEE Trans Image Process 12(8):882–889CrossRefGoogle Scholar
  5. 5.
    Chan TF, Shen J (2001) Nontexture inpainting by curvature-driven diffusions. J Vis Commun Image Represent 12(4):436–449CrossRefGoogle Scholar
  6. 6.
    Chan TF, Shen J (2001) Mathematical models for local non-texture inpaintings. SIAM J Appl Math 62(3):1019–1043MathSciNetGoogle Scholar
  7. 7.
    Criminisi A, Pérez P, Toyama K (2004) Region filling and object removal by exemplar-based image inpainting. IEEE Trans Image Process 13(9):1200–1212CrossRefGoogle Scholar
  8. 8.
    Efros AA, Leung TK (1999) Texture synthesis by non-parametric sampling. in Proc. IEEE Int. Conf. Comput. Vision 2:1033–1038CrossRefGoogle Scholar
  9. 9.
    Hsu HJ, Wang JF, Liao SC (2007) A hybrid algorithm with artifact detection mechanism for region filling after object removal from a digital photograph. IEEE Trans Image Process 16(6):1611–1622MathSciNetCrossRefGoogle Scholar
  10. 10.
    Liu D, Sun X, Wu F, Li S, Zhang YQ (2007) Image compression with edge-based inpainting. IEEE Trans Circuits Syst Video Technol 17(10):1273–1287CrossRefGoogle Scholar
  11. 11.
    Mairal J, Elad M, Sapiro G (2008) Sparse representation for color image restoration. IEEE Trans Image Process 17(1):53–69MathSciNetCrossRefGoogle Scholar
  12. 12.
    Perona P, Malik J (1990) Scale-space and edge detection using anisotropic diffusion. IEEE Trans Pattern Anal Mach Intell 12(7):629–639CrossRefGoogle Scholar
  13. 13.
    Rane SD, Sapiro G, Bertalmio M (2003) Structure and texture filling-in of missing image blocks in wireless transmission and compression applications. IEEE Trans Image Process 12(3):296–303MathSciNetCrossRefGoogle Scholar
  14. 14.
    Robles-Kelly A, Hancock ER (2004) Vector field smoothing via heat flow. in Proc. Int. Conf. Pattern Recognit 2:94–97Google Scholar
  15. 15.
    Shih TK (2004) Adaptive digital image inpainting. in Proc. Int. Conf. Advanced Inform. Networking and Applications 1:71–76Google Scholar
  16. 16.
    Tschumperlé D, Deriche R (2005) Vector-valued image regularization with PDEs: a common framework for different applications. IEEE Trans. Pattern Anal. Mach. Intell 27(4):506–517CrossRefGoogle Scholar
  17. 17.
    Witkin A, Kass M (1991) Reaction-diffusion textures. ACM SIGGRAPH Comput. Graphics 25(4):299–308CrossRefGoogle Scholar
  18. 18.
    Xu Z, Sun J (2010) Image inpainting by patch propagation using patch sparsity. IEEE Trans. Image Process 19(5):1153–1165MathSciNetCrossRefGoogle Scholar
  19. 19.
    Zhu ZJ, Li ZG, Rahardja S, Franti P (2009) Recovering real-world scene: high-quality image inpainting using multi-exposed references. Electron Lett 45(25):1310–1312CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2010

Authors and Affiliations

  1. 1.School of Optical-Electrical and Computer EngineeringUniversity of Shanghai for Science and TechnologyShanghaiChina
  2. 2.School of Communication and Information EngineeringShanghai UniversityShanghaiChina

Personalised recommendations