Depth Map Inpainting under a Second-Order Smoothness Prior

  • Daniel Herrera C.
  • Juho Kannala
  • L’ubor Ladický
  • Janne Heikkilä
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7944)


Many 3D reconstruction methods produce incomplete depth maps. Depth map inpainting can generate visually plausible structures for the missing areas. We present an inpainting method that encourages flat surfaces without favouring fronto-parallel planes. Moreover, it uses a color image to guide the inpainting and align color and depth edges. We implement the algorithm efficiently through graph-cuts. We compare the performance of our method with another inpainting approach used for large datasets and we show the results using several datasets. The depths inpainted with our method are visually plausible and of higher quality.


depth map inpainting second order prior graph cut 


  1. 1.
    Rother, C., Kolmogorov, V., Lempitsky, V., Szummer, M.: Optimizing binary mrfs via extended roof duality. In: CVPR (2007)Google Scholar
  2. 2.
    Seitz, S., Curless, B., Diebel, J., Scharstein, D., Szeliski, R.: A comparison and evaluation of multi-view stereo reconstruction algorithms. In: CVPR, pp. 519–528. IEEE (2006)Google Scholar
  3. 3.
    Woodford, O., Torr, P., Reid, I., Fitzgibbon, A.: Global stereo reconstruction under second-order smoothness priors. PAMI 31(12), 2115–2128 (2009)CrossRefGoogle Scholar
  4. 4.
    Herrera C.,D., Kannala, J., Heikkilä, J.: Generating dense depth maps using a patch cloud and local planar surface models. In: 3DTV-CON. IEEE (2011)Google Scholar
  5. 5.
    Levin, A., Lischinski, D., Weiss, Y.: Colorization using optimization. In: SIGGRAPH (2004)Google Scholar
  6. 6.
    Silberman, N., Hoiem, D., Kohli, P., Fergus, R.: Indoor segmentation and support inference from RGBD images. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part V. LNCS, vol. 7576, pp. 746–760. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  7. 7.
    Yang, Q., Yang, R., Davis, J., Nistér, D.: Spatial-depth super resolution for range images. In: CVPR. IEEE (2007)Google Scholar
  8. 8.
    Gandhi, V., Cech, J., Horaud, R.: High-resolution depth maps based on ToF-stereo fusion. In: ICRA, pp. 4742–4749 (2012)Google Scholar
  9. 9.
    Fitzgibbon, A., Wexler, Y., Zisserman, A., et al.: Image-based rendering using image-based priors. In: ICCV, vol. 2, pp. 1176–1183 (2003)Google Scholar
  10. 10.
    Gargallo, P., Sturm, P.: Bayesian 3d modeling from images using multiple depth maps. In: CVPR, vol. 2, pp. 885–891. IEEE (2005)Google Scholar
  11. 11.
    Kolmogorov, V., Zabin, R.: What energy functions can be minimized via graph cuts? PAMI 26(2), 147–159 (2004)CrossRefGoogle Scholar
  12. 12.
    Boykov, Y., Veksler, O., Zabih, R.: Fast approximate energy minimization via graph cuts. PAMI 23, 1222–1239 (2001)CrossRefGoogle Scholar
  13. 13.
    Billionnet, A., Jaumard, B.: A decomposition method for minimizing quadratic pseudoboolean functions. Operations Research Letters 8, 161–163 (1989)Google Scholar
  14. 14.
    Scharstein, D., Pal, C.: Learning conditional random fields for stereo. In: CVPR (2007)Google Scholar
  15. 15.
    Strecha, C., von Hansen, W., Van Gool, L., Fua, P., Thoennessen, U.: On benchmarking camera calibration and multi-view stereo for high resolution imagery. In: CVPR (2008)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Daniel Herrera C.
    • 1
  • Juho Kannala
    • 1
  • L’ubor Ladický
    • 2
  • Janne Heikkilä
    • 1
  1. 1.Center for Machine Vision ResearchUniversity of OuluFinland
  2. 2.Visual Geometry GroupUniversity of OxfordUK

Personalised recommendations