Multi viewpoint stereo from uncalibrated video sequences

  • Reinhard Koch
  • Marc Pollefeys
  • Luc Van Gool
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1406)


This contribution describes an automatic 3D surface modeling system that extracts dense metric 3D surfaces from an uncalibrated video sequence. A static 3D scene is observed from multiple viewpoints by freely moving a video camera around the object. No restrictions on camera movement and internal camera parameters like zoom are imposed, as the camera pose and intrinsic parameters are calibrated from the sequence.

Dense surface reconstructions are obtained by first treating consecutive images of the sequence as stereoscopic pairs and computing dense disparity maps for all image pairs. All viewpoints are then linked by controlled correspondence linking for each image pixel. The correspondence linking algorithm allows for accurate depth estimation as well as image texture fusion from all viewpoints simultaneously. By keeping track of surface visibility and measurement uncertainty it can cope with occlusions and measurement outliers. The correspondence linking is applied to increase the robustness and geometrical resolution of surface depth as well as to remove highlights and specular reflections, and to create super-resolution texture maps for increased realism.

The major impact of this work is the ability to automatically generate geometrically correct and visually pleasing 3D surface models from image sequences alone, which allows the economic model generation for a wide range of applications. The resulting textured 3D surface model are highly realistic VRML representations of the scene.


Fill Rate Depth Estimate Epipolar Line Depth Error Texture Enhancement 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    P. Beardsley, P. Torr and A. Zisserman: 3D Model Acquisition from Extended Image Sequences. In: B. Buxton, R. Cipolla(Eds.) Computer Vision — ECCV 96, Cambridge, UK., vol.2, pp.683–695. Lecture Notes in Computer Science, Vol. 1064. Springer Verlag, 1996.Google Scholar
  2. 2.
    Y. Chen and G. Medioni: Object Modeling by Registration of Multiple Range Images. Proceedings 1991 IEEE International Conference on Robotics and Automation, pp. 2724–2729, Sacramento (CA), 1991.Google Scholar
  3. 3.
    I. J. Cox, S. L. Hingorani, and S. B. Rao: A Maximum Likelihood Stereo Algorithm. Computer Vision and Image Understanding, Vol. 63, No. 3, May 1996.Google Scholar
  4. 4.
    L.Falkenhagen: Hierarchical Block-Based Disparity Estimation Considering Neighborhood Constraints. International Workshop on SNHC and 3D Imaging, September 5–9, 1997, Rhodes, Greece.Google Scholar
  5. 5.
    O.Faugeras: Three-Dimensional Computer Vision — a geometric viewpoint. MIT-Press, 1993.Google Scholar
  6. 6.
    O. Faugeras: What can be seen in three dimensions with an uncalibrated stereo rig. Proc. ECCV'92, pp.563–578.Google Scholar
  7. 7.
    O. Faugeras, Q.-T. Luong and S. Maybank: Camera self-calibration — Theory and experiments. Proc. ECCV'92, pp.321–334.Google Scholar
  8. 8.
    G.L.Gimel'farb: Symmetrical approach to the problem of automating stereoscopic measurements in photogrammetry. Cybernetics, 1979, 15(2), 235–247; Consultants Bureau, N.Y.zbMATHGoogle Scholar
  9. 9.
    G.L.Gimel'farb and R.M.Haralick: Terrain reconstruction from multiple views. Proc. 7th Int. Conf. on Computer Analysis of Images and Patterns (CAIP'97), Kiel, Germany, Sept. 1997. (G.Sommer, K.Daniilidis, and J.Pauli, Eds.). Lecture Notes in Computer Science 1296, Springer: Berlin e.a., 1997, pp.694–701.Google Scholar
  10. 10.
    C.G. Harris and J.M. Pike: 3D Positional Integration from Image Sequences. 3rd Alvey Vision Conf, pp. 233–236, 1987.Google Scholar
  11. 11.
    R. Hartley: Estimation of relative camera positions for uncalibrated cameras. Proc. ECCV'92, pp.579–87.Google Scholar
  12. 12.
    A. Heyden and K. åström: Euclidean Reconstruction from Image Sequences with Varying and Unknown Focal Length and Principal Point. Proc. CVPR'97.Google Scholar
  13. 13.
    M. Irani and S. Peleg: Super resolution from image sequences. Tenth International Conference on Pattern Recognition (Atlantic City, NJ, June 16–21, 1990), IEEE Catalog No. 90CH2898-5, 1990, subconference C, 115–220.Google Scholar
  14. 14.
    R. Koch: 3-D Modeling of Human Heads from Stereoscopic Image Sequences. Proc. DAGM'96, Informatik Aktuell, Springer Heidelberg, Germany, Sept. 1996.Google Scholar
  15. 15.
    R. Koch: Surface Segmentation and Modeling of 3-D Polygonal Objects from Stereoscopic Image Pairs. Proc. ICPR'96, Vienna 1996.Google Scholar
  16. 16.
    R. Koch: Automatische Oberflächenmodellierung starrer dreidimensionaler Objekte aus stereoskopischen Bildfolgen. Ph.D. Thesis, Fortschr.-Ber. 10/499. Düsseldorf: VDI Verlag 1997.Google Scholar
  17. 17.
    A. Kochan: Eine Methodenbank zur Evaluierung von Stereo-Vision-Verfahren. Ph.D. Thesis, TU Berlin, June 1991.Google Scholar
  18. 18.
    L. Matthies, T. Kanade, and R. Szeliski: Kaiman filter-based algorithms for estimating depth from image sequences. International Journal of Computer Vision, vol. 3, 1989, 209–236.CrossRefGoogle Scholar
  19. 19.
    M. Okutomi and T. Kanade: A multiple-baseline stereo. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 15, no. 4, 1993, 353–363.CrossRefGoogle Scholar
  20. 20.
    E. Ofek, E. Shilat, A. Rappoport, M. Werman: Highlight and Reflection-Independent Multiresolution Textures from Image Sequences. IEEE Computer Graphics and Applications vol. 17 (2), March–April 1997.Google Scholar
  21. 21.
    M. Pollefeys, L. Van Gool and M. Proesmans: Euclidean 3D Reconstruction from Image Sequences with Variable Focal Lengths. Proc. ECCV'96, vol.1, pp. 31–42.Google Scholar
  22. 22.
    M. Pollefeys and L. Van Gool: Self-calibration from the absolute conic on the plane at infinity. Proc. CAIP'97.Google Scholar
  23. 23.
    M. Pollefeys, R. Koch and L. Van Gool: Self-Calibration and Metric Reconstruction in spite of Varying and Unknown Internal Camera Parameters. Proc. ICCV'98, Bombay, India, 1998.Google Scholar
  24. 24.
    P.H.S. Torr: Motion Segmentation and Outlier Detection. PhD thesis, Dept. Eng. Science, University of Oxford, UK, 1995.Google Scholar
  25. 25.
    B. Triggs: The Absolute Quadric. Proc. CVPR'97.Google Scholar
  26. 26.
    Z. Zhang and O. Faugeras: 3D Dynamic Scene Analysis. Springer Verlag, 1992.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1998

Authors and Affiliations

  • Reinhard Koch
    • 1
  • Marc Pollefeys
    • 1
  • Luc Van Gool
    • 1
  1. 1.Katholieke Universiteit LeuvenLeuvenBelgium

Personalised recommendations