Virtual View Synthesis Based on DIBR and Image Inpainting
In 3DTV research, virtual view synthesis is a key component to the technology. Depth-image-based-rendering (DIBR) is an important method to realize virtual view synthesis. However, DIBR always results in hole problems where the depth and colour values are not known. Hole-filling methods often cause other problems, such as edge-ghosting and cracks. This paper proposes an algorithm that uses the depth and colour images to address the holes. It exploits the assumption of a virtual view between two laterally aligned reference cameras. The hole-filling method is performed on the blended depth image by morphological operations, and inpainting of the holes is obtained with the position information provided by the filtered depth maps. A new interpolation method to eliminate edge-ghosting is also presented, which additionally uses a post-processing technique to improve image quality. The main novelty of this paper is the unique image blending, which is more efficient than pre-processing depth maps. It is also the first method that is using morphological closing in the depth map de-noising process. The method proposed in this paper can effectively remove holes and edge-ghosting. Experimental quantitative and qualitative results show the proposed algorithm improves quality dramatically on traditional methods.
KeywordsVirtual View Synthesis Depth-Image-Based-Rendering (DIBR) Image Inpainting Interpolation
- 1.Vazquez, C., Tam, W.J., Speranza, F.: Sterescopic Imaging: Filling Disoccluded Areas in Depth Image-Based Rendering. In: Proceedings of SPIE, Orlando, FL, USA, vol. 6392 (2006)Google Scholar
- 2.Domaski, M., Gotfryd, M., Wegner, K.: View Synthesis For Multiview Video Transmission. In: International Conference on Computer Vision and Pattern Recognition, Florida, USA, pp. 433–439 (2009)Google Scholar
- 3.Narayanan, P., Kumar, P., Reddy, K.: Depth+Texture Representation For Image Based Rendering. In: Proceedings of Fourth Indian Conference on Computer Vision, Graphics and Image Processing, Kolkata, Indian, pp. 113–118 (2004)Google Scholar
- 4.Fehn, C.: Depth-image-based rendering(DIBR), Compression, and Transmission For a New Approach on 3DTV. In: Proceedings of the SPIE, San Jose, CA, USA, vol. 5291, pp. 93–104 (2004)Google Scholar
- 5.McMillan, L.: An Image-Based Approach to Three-Dimensional Computer Graphics. Technical Report. University of North Carolina at Chapel Hill, Chapel Hill, NC, USA (1997)Google Scholar
- 6.Mori, Y., Fukushima, N., Yendo, T., Fujii, T., Tanimoto, M.: View Generation With 3D Warping Using Depth Information for FTV. Signal Processing: Image Communication 24(1), 65–72 (2009)Google Scholar
- 7.Li, M., Chen, H., Li, R., Chang, X.: An Improved Virtual View Rendering Method Based on Depth Image. In: International Conference on Computer Communication, Jinan, China, pp. 381–384 (2011)Google Scholar
- 11.Jung, J., Ho, Y.: Virtual View Synthesis Using Temporal Hole Filling with Bilateral Coefficients. In: IEEE International Conference on Research, Innovation and Vision for the Future, pp. 1–4 (2012)Google Scholar
- 15.Microsoft Research, Image-Based Realities-3D Video Download, http://research.microsoft.com/en-us/um/people/sbkang/3dvideodownload/.