Super-Resolved Free-Viewpoint Image Synthesis Using Semi-global Depth Estimation and Depth-Reliability-Based Regularization

  • Keita Takahashi
  • Takeshi Naemura
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7087)


A method for synthesizing high-quality free-viewpoint images from a set of multi-view images is presented. First, an accurate depth map is estimated from a given target viewpoint using modified semi-global stereo matching. Then, a high-resolution image from that viewpoint is obtained through super-resolution reconstruction. The depth estimation results from the first step are used for the second step. First, the depth values are used to associate pixels between the input images and the latent high-resolution image. Second, the pixel-wise reliabilities of the depth information are used for regularization to adaptively control the strength of the super-resolution reconstruction. Experimental results using real images showed the effectiveness of our method.


free-viewpoint image semi-global stereo super-resolution depth reliability regularization 


  1. 1.
    Kubota, A., et al.: Multiview Imaging and 3DTV. IEEE Signal Processing Magazine 24(6), 10–111 (2007)CrossRefGoogle Scholar
  2. 2.
    Taguchi, Y., Koike, T., Takahashi, K., Naemura, T.: TransCAIP: A Live 3D TV System Using a Camera Array and an Integral Photography Display with Interactive Control of Viewing Parameters. IEEE Trans. Visualization and Computer Graphics 15(5), 841–852 (2009)CrossRefGoogle Scholar
  3. 3.
    Park, S.-C., et al.: Super-resolution Image Reconstruction: A Technical Overview. IEEE Signal Processing Magazine 20(3), 21–36 (2003)CrossRefGoogle Scholar
  4. 4.
    Matusik, W., Buehler, C., Raskar, R., Gortler, S.-J., McMillan, L.: Image-Based Visual Hulls. In: Proc. ACM SIGGRAPH, pp. 369–374 (2000)Google Scholar
  5. 5.
    Yang, R., Welch, G., Bishop, G.: Real-Time Consensus-Based Scene Reconstruction Using Commodity Graphics Hardware. In: Proceedings of Pacific Graphics, pp. 225–235 (2002)Google Scholar
  6. 6.
    Hirschmueller, H.: Accurate and Efficient Stereo Processing by Semi-Global Matching and Mutual Information. In: IEEE CVPR, pp. 807–814 (2005)Google Scholar
  7. 7.
    Hirschmuller, H.: Stereo Processing by Semiglobal Matching and Mutual Information. In: IEEE TPAMI, vol. 30(2), pp. 328–341 (2008)Google Scholar
  8. 8.
    Tung, T., Nobuhara, S., Matsuyama, T.: Simultaneous Super-Resolution and 3D Video Using Graph-Cuts. In: IEEE CVPR, pp. 1–8 (2008)Google Scholar
  9. 9.
    Goldluecke, B., Cremers, D.: Superresolution Texture Maps for Multiview Reconstruction. In: IEEE ICCV, pp. 1677–1684 (2009)Google Scholar
  10. 10.
    Mudenagudi, U., Gupta, A., Goel, L., Kushal, A., Kalra, P., Banerjee, S.: Super Resolution of Images of 3D Scenecs. In: Yagi, Y., Kang, S.B., Kweon, I.S., Zha, H. (eds.) ACCV 2007, Part II. LNCS, vol. 4844, pp. 85–95. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  11. 11.
    Takahashi, K., Ishii, M., Naemura, T.: Super-Resolution Plane Sweeping for Free-Viewpoint Image Synthesis. In: IEEE ICIP, pp. 2013–2016 (2011)Google Scholar
  12. 12.

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Keita Takahashi
    • 1
  • Takeshi Naemura
    • 2
  1. 1.The University of Electro-CommunicationsChofu-shiJapan
  2. 2.The University of TokyoBunkyo-kuJapan

Personalised recommendations