Virtual View Synthesis Based on DIBR and Image Inpainting

  • Yuhan Gao
  • Hui Chen
  • Weisong Gao
  • Tobi Vaudrey
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8333)

Abstract

In 3DTV research, virtual view synthesis is a key component to the technology. Depth-image-based-rendering (DIBR) is an important method to realize virtual view synthesis. However, DIBR always results in hole problems where the depth and colour values are not known. Hole-filling methods often cause other problems, such as edge-ghosting and cracks. This paper proposes an algorithm that uses the depth and colour images to address the holes. It exploits the assumption of a virtual view between two laterally aligned reference cameras. The hole-filling method is performed on the blended depth image by morphological operations, and inpainting of the holes is obtained with the position information provided by the filtered depth maps. A new interpolation method to eliminate edge-ghosting is also presented, which additionally uses a post-processing technique to improve image quality. The main novelty of this paper is the unique image blending, which is more efficient than pre-processing depth maps. It is also the first method that is using morphological closing in the depth map de-noising process. The method proposed in this paper can effectively remove holes and edge-ghosting. Experimental quantitative and qualitative results show the proposed algorithm improves quality dramatically on traditional methods.

Keywords

Virtual View Synthesis Depth-Image-Based-Rendering (DIBR) Image Inpainting Interpolation 

References

  1. 1.
    Vazquez, C., Tam, W.J., Speranza, F.: Sterescopic Imaging: Filling Disoccluded Areas in Depth Image-Based Rendering. In: Proceedings of SPIE, Orlando, FL, USA, vol. 6392 (2006)Google Scholar
  2. 2.
    Domaski, M., Gotfryd, M., Wegner, K.: View Synthesis For Multiview Video Transmission. In: International Conference on Computer Vision and Pattern Recognition, Florida, USA, pp. 433–439 (2009)Google Scholar
  3. 3.
    Narayanan, P., Kumar, P., Reddy, K.: Depth+Texture Representation For Image Based Rendering. In: Proceedings of Fourth Indian Conference on Computer Vision, Graphics and Image Processing, Kolkata, Indian, pp. 113–118 (2004)Google Scholar
  4. 4.
    Fehn, C.: Depth-image-based rendering(DIBR), Compression, and Transmission For a New Approach on 3DTV. In: Proceedings of the SPIE, San Jose, CA, USA, vol. 5291, pp. 93–104 (2004)Google Scholar
  5. 5.
    McMillan, L.: An Image-Based Approach to Three-Dimensional Computer Graphics. Technical Report. University of North Carolina at Chapel Hill, Chapel Hill, NC, USA (1997)Google Scholar
  6. 6.
    Mori, Y., Fukushima, N., Yendo, T., Fujii, T., Tanimoto, M.: View Generation With 3D Warping Using Depth Information for FTV. Signal Processing: Image Communication 24(1), 65–72 (2009)Google Scholar
  7. 7.
    Li, M., Chen, H., Li, R., Chang, X.: An Improved Virtual View Rendering Method Based on Depth Image. In: International Conference on Computer Communication, Jinan, China, pp. 381–384 (2011)Google Scholar
  8. 8.
    Zhang, L., Tam, J., Wang, D.: Stereoscopic Image Generation Based on Depth Images For 3DTV. IEEE Transactions on Broadcasting 51(2), 191–199 (2005)CrossRefGoogle Scholar
  9. 9.
    Criminisi, A., Perez, P., Toyama, K.: Region Filling and Object Removal by Exemplar-Based Image Inpainting. IEEE Transactions on Image Processing 13(9), 1200–1212 (2004)CrossRefGoogle Scholar
  10. 10.
    Oh, K., Yea, S., Vetro, A., Ho, Y.: Virtual View Synthesis Method and Self Evaluation Metrics for Free Viewpoint Television and 3D Video. International Journal of Imaging Systems and Technology 20(4), 378–390 (2010)CrossRefGoogle Scholar
  11. 11.
    Jung, J., Ho, Y.: Virtual View Synthesis Using Temporal Hole Filling with Bilateral Coefficients. In: IEEE International Conference on Research, Innovation and Vision for the Future, pp. 1–4 (2012)Google Scholar
  12. 12.
    Herk, M.V.: A Fast Algorithm for Local Minimum and Maximum Filters on Rectangular and Octagonal Kernels. Patt. Recog. Letters 13, 517–521 (1992)CrossRefGoogle Scholar
  13. 13.
    Canny, J.: A Computational Approach To Edge Detection. IEEE Transactions on Pattern Analysis and Machine Intelligence 8(6), 679–698 (1986)CrossRefGoogle Scholar
  14. 14.
    Yu, Y., Acton, S.: Speckle Reducing Anisotropic Diffusion. IEEE Transactions on Image Processing 11, 1260–1270 (2002)CrossRefMathSciNetGoogle Scholar
  15. 15.
    Microsoft Research, Image-Based Realities-3D Video Download, http://research.microsoft.com/en-us/um/people/sbkang/3dvideodownload/.
  16. 16.
    Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing 13(4), 600–612 (2004)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  • Yuhan Gao
    • 1
  • Hui Chen
    • 1
  • Weisong Gao
    • 2
  • Tobi Vaudrey
    • 3
  1. 1.School of Information Science and EngineeringShandong UniversityJinanChina
  2. 2.Multimedia R & D CenterHisenseChina
  3. 3.Department of Computer ScienceThe University of AucklandAucklandNew Zealand

Personalised recommendations