Scale-Invariant Image Inpainting Using Gradient-Based Image Composition

  • Mrinmoy GhoraiEmail author
  • Soumitra Samanta
  • Bhabatosh Chanda
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10481)


In this paper, we propose a novel scale-invariant image inpainting algorithm that combines several inpainted images obtained from multiple pyramids of different coarsest scales. To achieve this, first we build the pyramids and then we run an image inpainting algorithm individually on each of the pyramids to obtain different inpainted images. Finally, we combine those inpainted images by a gradient based approach to obtain the final inpainted image. The motivation of this approach is to solve the problem of appearing artifacts in traditional single pyramid-based approach since the results depend on the starting scale of the pyramid. Here we assume that most of the inpainted images produced by the pyramids are quite good. However, some of them may have artifacts and these artifacts are eliminated by gradient based image composition. We test the proposed algorithm on a large number of natural images and compare the results with some of the existing methods to demonstrate the efficacy and superiority of the proposed method.


Inpainting Scale-invariant Multiple pyramid Image composition 


  1. 1.
    Barnes, C., Shechtman, E., Finkelstein, A., Goldman, D.B.: Patch-Match: a randomized correspondence algorithm for structural image editing. ACM Trans. Graph. 28(3), 1–11 (2009)CrossRefGoogle Scholar
  2. 2.
    Criminisi, A., Pérez, P., Toyama, K.: Region filling and object removal by exemplar-based inpainting. IEEE Trans. Image Process. 13(9), 1200–1212 (2004)CrossRefGoogle Scholar
  3. 3.
    Darabi, S., Shechtman, E., Barnes, C., Goldman, D.B., Sen, P.: Image melding: combining inconsistent images using patch-based synthesis. ACM Trans. Graph. (TOG) 31(4), 82:1–82:10 (2012)CrossRefGoogle Scholar
  4. 4.
    Efros, A., Leung, T.: Texture synthesis by non-parametric sampling. In: Proceedings of the IEEE International Conference on Computer Vision, vol. 2, pp. 1033–1038 (1999)Google Scholar
  5. 5.
    Fadili, M.J., Starck, J.L., Murtagh, F.: Inpainting and zooming using sparse representations. Comput. J. 52(1), 64–79 (2009)CrossRefGoogle Scholar
  6. 6.
    Fang, C.-W., Lien, J.-J.J.: Rapid image completion system using multiresolution patch-based directional and nondirectional approaches. IEEE Trans. Image Process. 18(12), 2769–2779 (2009)CrossRefzbMATHMathSciNetGoogle Scholar
  7. 7.
    Freedman, G., Fattal, R.: Image and video upscaling from local self-examples. ACM Trans. Graph. (TOG) 30(2), 12 (2011)CrossRefGoogle Scholar
  8. 8.
    Ghorai, M., Chanda, B.: An image inpainting algorithm using higher order singular value decomposition. In: 2014 22nd International Conference on Pattern Recognition (ICPR), pp. 2867–2872. IEEE (2014)Google Scholar
  9. 9.
    Ghorai, M., Chanda, B.: An image inpainting method using pLSA-based search space estimation. Mach. Vis. Appl. 26(1), 69–87 (2015)CrossRefGoogle Scholar
  10. 10.
    Komodakis, N., Tziritas, G.: Image completion using efficient belief propagation via priority scheduling and dynamic pruning. IEEE Trans. Image Process. 16(11), 2649–2661 (2007)CrossRefMathSciNetGoogle Scholar
  11. 11.
    Le Meur, O., Gautier, J., Guillemot, C.: Examplar-based inpainting based on local geometry. In: 2011 18th IEEE International Conference on Image Processing, pp. 3401–3404. IEEE (2011)Google Scholar
  12. 12.
    Li, Z., He, H., Tai, H.-M., Yin, Z., Chen, F.: Color-direction patch-sparsity-based image inpainting using multidirection features. IEEE Trans. Image Process. 24(3), 1138–1152 (2015)CrossRefMathSciNetGoogle Scholar
  13. 13.
    Liu, Y., Caselles, V.: Exemplar-based image inpainting using multiscale graph cuts. IEEE Trans. Image Process. 22(5), 1699–1711 (2013)CrossRefzbMATHMathSciNetGoogle Scholar
  14. 14.
    Mertens, T., Kautz, J., Van Reeth, F.: Exposure fusion: a simple and practical alternative to high dynamic range photography. In: Computer Graphics Forum, vol. 28, pp. 161–171. Wiley Online Library (2009)Google Scholar
  15. 15.
    Qin, Y., Wang, F.: A curvature constraint exemplar-based image inpainting. In: International Conference on Image Processing and Signal Processing, pp. 263–267 (2010)Google Scholar
  16. 16.
    Shen, B., Hu, W., Zhang, Y., Zhang, Y.: Image inpainting via sparse representation. In: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 697–700 (2009)Google Scholar
  17. 17.
    Sun, J., Yuan, L., Jia, J., Shum, H.: Image completion with structure propagation. In: Proceedings of the SIGGRAPH2005, pp. 861–868 (2005)Google Scholar
  18. 18.
    Wohlberg, B.: Inpainting by joint optimization of linear combinations of exemplars. IEEE Signal Process. Lett. 18(1), 75–78 (2011)CrossRefGoogle Scholar
  19. 19.
    Wu, J., Ruan, Q.: Object removal by cross isophotes exemplar-based inpainting. In: Proceedings of the 18th International Conference on Pattern Recognition, vol. 3, pp. 810–813 (2006)Google Scholar
  20. 20.
    Xu, Z., Sun, J.: Image inpainting by patch propagation using patch sparsity. IEEE Trans. Image Process. 19(5), 1153–1165 (2010)CrossRefzbMATHMathSciNetGoogle Scholar
  21. 21.
    Yang, M.-C., Wang, C.-H., Hu, T.-Y., Wang, Y.-C.F.: Learning context-aware sparse representation for single image super-resolution. In: 2011 18th IEEE International Conference on Image Processing, pp. 1349–1352. IEEE (2011)Google Scholar
  22. 22.
    Zhang, K., Gao, X., Tao, D., Li, X.: Single image super-resolution with multiscale similarity learning. IEEE Trans. Neural Netw. Learn. Syst. 24(10), 1648–1659 (2013)CrossRefGoogle Scholar
  23. 23.
    Zhang, W., Cham, W.-K.: Gradient-directed multiexposure composition. IEEE Trans. Image Process. 21(4), 2318–2323 (2012)CrossRefzbMATHMathSciNetGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Mrinmoy Ghorai
    • 1
    Email author
  • Soumitra Samanta
    • 1
  • Bhabatosh Chanda
    • 1
  1. 1.Indian Statistical InstituteKolkataIndia

Personalised recommendations