Appearance-Based Virtual-View Generation for Fly Through in a Real Dynamic Scene

  • Shigeyuki Baba
  • Hideo Saito
  • Sundar Vedula
  • Kong Man Cheung
  • Takeo Kanade
Conference paper
Part of the Eurographics book series (EUROGRAPH)


We present appearance-based Virtual view generation which allows viewers to fly through a real dynamic scene. The scene is captured by synchronized multiple cameras. Arbitrary views are generated by interpolating two original camera-view images near the given viewpoint. The quality of the generated synthetic view is determined by the precision, consistency and density of correspondences between the two images. All or most of previous work that uses interpolation extracts the correspondences from these two images. However, not only is it difficult to do so reliably (the task requires a good stereo algorithm), but also the two images alone sometimes do not have enough information, due to problems such as occlusion. Instead, we take advantage of the fact that we have many views, from which we can extract much more reliable and comprehensive 3D geometry of the scene as a 3D model. The dense and precise correspondences between the two images, to be used for interpolation, are derived from this constructed 3D model. Our method of 3D modeling from multiple images uses the Multiple Baseline Stereo method and Shape from Silhouette method.


Optical Center Virtual Image Virtual View Virtual Camera Warped Image 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Chen, S., Williams, L.:View Interpolation for Image Synthesis. Proc. of SIGGRAPH’ 93. (1982) 279-288Google Scholar
  2. 2.
    Debevec, P., Taylor, C., Malik, J.: Modeling and Rendering Architecture from Photographs: A Hybrid Geometry and Image-Based Approach. Proc. of SIGGRAPH’ 96. (1996)Google Scholar
  3. 3.
    Avidan S., Shashua, A.: Novel View Synthesis by Cascading Trilinear Tensors. IEEE TVCG.Vol.4. No.4 (1998) 293–306Google Scholar
  4. 4.
    Potmesil, M.: Generating Octree Models of 3D Objects from Their Silhouettes in a Sequence of Images. Computer Vision, Graphics and Image Processing. 40 (1987) 277–283CrossRefGoogle Scholar
  5. 5.
    Gortier, S.J., Grzeszczuk, R., Szeliski, R., Cohen, M.F. The Lumigraph. Proc. of SIGGRAPH’96. (1996)Google Scholar
  6. 6.
    Chein, C.H., Aggarawal, J.K.: Identification of 3D Objects from Multiple Silhouettes using Quadtrees / Octrees. Computer Vision, Graphics and Image Processing. 36 (1986) 100–113CrossRefGoogle Scholar
  7. 7.
    Kanade, T., Rander, P.W., Narayanan, P.J.: Virtualized Reality: Constructing Virtual Worlds from Real Scenes. IEEE Multimedia. Vol.4. No.1 (1997)Google Scholar
  8. 8.
    Kanade, T., Saito, H., Vedula, S.: The 3D Room: Digitizing Time-Varying 3D Events by Synchronized Multiple Video Streams. CMU-RI-TR-98-34 (1998)Google Scholar
  9. 9.
    Katayama, A., Tanaka, K., Oshino, T., Tamura, H.:A Viewpoint Dependent Stereoscopic Display Using Interpolation of Multi-Viewpoint Images. SPIE Proc. Vol.2409. Stereoscopic Displays and Virtual Reality Systems II (1995) 11-20.Google Scholar
  10. 10.
    Levoy, M., Hanrahan, P.:Light Field Rendering. Proc. of SIGGRAPH’96 (1996)Google Scholar
  11. 11.
    Okutomi, M., Kanade, T.: A Multiple-Baseline Stereo.IEEE Trans, on PAMI. Vol. 15. No.4 (1993) 353–363CrossRefGoogle Scholar
  12. 12.
    Narayanan, P.J., Rander, P.W., Kanade, T.: Constructing Virtual Worlds Using Dense Stereo. Proc. ICCV’98 (1998)Google Scholar
  13. 13.
    Seitz, S.M., Dyler, C.R: View Morphing. Proc. of SIGGRAPH’96 (1996) 21-30Google Scholar
  14. 14.
    Tsai, R.: A Versatile Camera Calibration Technique for High-Accuracy 3D Machine Vision Metrology Using Off-The-Shelf TV Cameras and Lenses. IEEE J. of Robotics and Automation RA-3. 4. (1987) 323–344CrossRefGoogle Scholar
  15. 15.
    Vedula, S., Rander, P.W., Saito, H., Kanade, T.:Modeling, Combining and Rendering Dynamic Real-World Events from Image Sequences. Proc. 4th Conf. Virtual Systems and Multimedia. Vol.1 (1998) 326–332Google Scholar
  16. 16.
    Wheeler, M.D., Sato,Y., Ikeuchi, K.: Consensus Surfaces for Modeling 3D Objects from Multiple Range Images. DARPA Image Understanding Workshop (1997)Google Scholar
  17. 17.
    Faugeras, O., Laveau, S., Robert, L., Csurka, G., Zeller, C.: 3-D Reconstruction of Urban Scenes from Sequences of Images. INRIA Technical Report. No.2572 (1995Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2000

Authors and Affiliations

  • Shigeyuki Baba
    • 1
    • 2
  • Hideo Saito
    • 1
    • 3
  • Sundar Vedula
    • 1
  • Kong Man Cheung
    • 1
  • Takeo Kanade
    • 1
  1. 1.Carnegie Mellon UniversityPittsburghUSA
  2. 2.Sony CorporationTokyoJapan
  3. 3.Department of Information and Computer ScienceKeio UniversityYokohamaJapan

Personalised recommendations