Advertisement

Statistics of Patch Offsets for Image Completion

  • Kaiming He
  • Jian Sun
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7573)

Abstract

Image completion involves filling missing parts in images. In this paper we address this problem through the statistics of patch offsets. We observe that if we match similar patches in the image and obtain their offsets (relative positions), the statistics of these offsets are sparsely distributed. We further observe that a few dominant offsets provide reliable information for completing the image. With these offsets we fill the missing region by combining a stack of shifted images via optimization. A variety of experiments show that our method yields generally better results and is faster than existing state-of-the-art methods.

Keywords

Salient Object Coherence Measure Image Editing Image Inpainting Similar Patch 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
Download to read the full conference paper text

References

  1. 1.
    Bertalmio, M., Sapiro, G., Caselles, V., Ballester, C.: Image inpainting. In: SIGGRAPH, pp. 417–424 (2000)Google Scholar
  2. 2.
    Ballester, C., Bertalmio, M., Caselles, V., Sapiro, G., Verdera, J.: Filling-in by joint interpolation of vector fields and gray levels. In: TIP, pp. 1200–1211 (2001)Google Scholar
  3. 3.
    Levin, A., Zomet, A., Weiss, Y.: Learning how to inpaint from global image statistics. In: ICCV, pp. 305–312 (2003)Google Scholar
  4. 4.
    Bertalmio, M., Vese, L., Sapiro, G., Osher, S.: Simultaneous structure and texture image inpainting. In: CVPR (2003)Google Scholar
  5. 5.
    Roth, S., Black, M.J.: Fields of experts: a framework for learning image priors. In: CVPR, pp. 860–867 (2005)Google Scholar
  6. 6.
    Efros, A.A., Leung, T.K.: Texture synthesis by non-parametric sampling. In: ICCV, pp. 1033–1038 (1999)Google Scholar
  7. 7.
    Criminisi, A., Perez, P., Toyama, K.: Object removal by exemplar-based inpainting. In: CVPR (2003)Google Scholar
  8. 8.
    Drori, I., Cohen-Or, D., Yeshurun, H.: Fragment-based image completion. In: SIGGRAPH, pp. 303–312 (2003)Google Scholar
  9. 9.
    Wexler, Y., Shechtman, E., Irani, M.: Space-time video completion. In: CVPR (2004)Google Scholar
  10. 10.
    Sun, J., Yuan, L., Jia, J., Shum, H.-Y.: Image completion with structure propagation. In: SIGGRAPH, pp. 861–868 (2005)Google Scholar
  11. 11.
    Simakov, D., Caspi, Y., Shechtman, E., Irani, M.: Summarizing visual data using bidirectional similarity. In: CVPR, pp. 1–8 (2008)Google Scholar
  12. 12.
    Barnes, C., Shechtman, E., Finkelstein, A., Goldman, D.B.: PatchMatch: A randomized correspondence algorithm for structural image editing. In: SIGGRAPH, pp. 1–8 (2009)Google Scholar
  13. 13.
  14. 14.
    Komodakis, N., Tziritas, G.: Image completion using efficient belief propagation via priority scheduling and dynamic pruning. In: TIP, pp. 2649–2661 (2007)Google Scholar
  15. 15.
    Pritch, Y., Kav-Venaki, E., Peleg, S.: Shift-map image editing. In: ICCV (2009)Google Scholar
  16. 16.
    Agarwala, A., Dontcheva, M., Agrawala, M., Drucker, S., Colburn, A., Curless, B., Salesin, D., Cohen, M.: Interactive digital photomontage. In: SIGGRAPH (2004)Google Scholar
  17. 17.
    Kwatra, V., Schödl, A., Essa, I., Turk, G., Bobick, A.: Graphcut textures: image and video synthesis using graph cuts. In: ACM SIGGRAPH 2003 Papers (2003)Google Scholar
  18. 18.
    Boykov, Y., Veksler, O., Zabih, R.: Fast approximate energy minimization via graph cuts. In: TPAMI, pp. 1222–1239 (2001)Google Scholar
  19. 19.
    Liu, T., Sun, J., Zheng, N.-N., Tang, X., Shum, H.-Y.: Learning to detect a salient object. In: CVPR (2007)Google Scholar
  20. 20.
    Zontak, M., Irani, M.: Internal statistics of a single natural image. In: CVPR (2011)Google Scholar
  21. 21.
  22. 22.
    Darabi, S., Shechtman, E., Barnes, C., Goldman, D.B., Sen, P.: Image Melding: Combining Inconsistent Images using Patch-based Synthesis. In: SIGGRAPH (2012)Google Scholar
  23. 23.
    He, K., Sun, J.: Computing nearest-neighbor fields via propagation-assisted kd-trees. In: CVPR (2012)Google Scholar
  24. 24.
    Multi-label Graph Cuts, http://vision.csd.uwo.ca/code/
  25. 25.
    Pérez, P., Gangnet, M., Blake, A.: Poisson image editing. In: SIGGRAPH (2003)Google Scholar
  26. 26.
    Farbman, Z., Fattal, R., Lischinski, D.: Convolution pyramids. In: SIGGRAPH Asia, 175:1–175:8 (2011)Google Scholar
  27. 27.
    Mansfield, A., Prasad, M., Rother, C., Sharp, T., Kohli, P., Van Gool, L.: Transforming image completion. In: BMVC (2011)Google Scholar
  28. 28.
    Komodakis, N., Tziritas, G.: Image completion using global optimization. In: CVPR, pp. 442–452 (2006)Google Scholar
  29. 29.
    Katkovnik, V., Foi, A., Egiazarian, K., Astola, J.: From local kernel to nonlocal multiple-model image denoising. In: IJCV, pp. 1–32 (2010)Google Scholar
  30. 30.
    Fergus, R., Singh, B., Hertzmann, A., Roweis, S.T., Freeman, W.T.: Removing camera shake from a single photograph. In: SIGGRAPH, pp. 787–794 (2006)Google Scholar
  31. 31.
    Glasner, D., Bagon, S., Irani, M.: Super-resolution from a single image. In: ICCV (2009)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Kaiming He
    • 1
  • Jian Sun
    • 1
  1. 1.Microsoft Research AsiaChina

Personalised recommendations

Cite paper