Multi viewpoint stereo from uncalibrated video sequences
This contribution describes an automatic 3D surface modeling system that extracts dense metric 3D surfaces from an uncalibrated video sequence. A static 3D scene is observed from multiple viewpoints by freely moving a video camera around the object. No restrictions on camera movement and internal camera parameters like zoom are imposed, as the camera pose and intrinsic parameters are calibrated from the sequence.
Dense surface reconstructions are obtained by first treating consecutive images of the sequence as stereoscopic pairs and computing dense disparity maps for all image pairs. All viewpoints are then linked by controlled correspondence linking for each image pixel. The correspondence linking algorithm allows for accurate depth estimation as well as image texture fusion from all viewpoints simultaneously. By keeping track of surface visibility and measurement uncertainty it can cope with occlusions and measurement outliers. The correspondence linking is applied to increase the robustness and geometrical resolution of surface depth as well as to remove highlights and specular reflections, and to create super-resolution texture maps for increased realism.
The major impact of this work is the ability to automatically generate geometrically correct and visually pleasing 3D surface models from image sequences alone, which allows the economic model generation for a wide range of applications. The resulting textured 3D surface model are highly realistic VRML representations of the scene.
KeywordsFill Rate Depth Estimate Epipolar Line Depth Error Texture Enhancement
Unable to display preview. Download preview PDF.
- 1.P. Beardsley, P. Torr and A. Zisserman: 3D Model Acquisition from Extended Image Sequences. In: B. Buxton, R. Cipolla(Eds.) Computer Vision — ECCV 96, Cambridge, UK., vol.2, pp.683–695. Lecture Notes in Computer Science, Vol. 1064. Springer Verlag, 1996.Google Scholar
- 2.Y. Chen and G. Medioni: Object Modeling by Registration of Multiple Range Images. Proceedings 1991 IEEE International Conference on Robotics and Automation, pp. 2724–2729, Sacramento (CA), 1991.Google Scholar
- 3.I. J. Cox, S. L. Hingorani, and S. B. Rao: A Maximum Likelihood Stereo Algorithm. Computer Vision and Image Understanding, Vol. 63, No. 3, May 1996.Google Scholar
- 4.L.Falkenhagen: Hierarchical Block-Based Disparity Estimation Considering Neighborhood Constraints. International Workshop on SNHC and 3D Imaging, September 5–9, 1997, Rhodes, Greece.Google Scholar
- 5.O.Faugeras: Three-Dimensional Computer Vision — a geometric viewpoint. MIT-Press, 1993.Google Scholar
- 6.O. Faugeras: What can be seen in three dimensions with an uncalibrated stereo rig. Proc. ECCV'92, pp.563–578.Google Scholar
- 7.O. Faugeras, Q.-T. Luong and S. Maybank: Camera self-calibration — Theory and experiments. Proc. ECCV'92, pp.321–334.Google Scholar
- 9.G.L.Gimel'farb and R.M.Haralick: Terrain reconstruction from multiple views. Proc. 7th Int. Conf. on Computer Analysis of Images and Patterns (CAIP'97), Kiel, Germany, Sept. 1997. (G.Sommer, K.Daniilidis, and J.Pauli, Eds.). Lecture Notes in Computer Science 1296, Springer: Berlin e.a., 1997, pp.694–701.Google Scholar
- 10.C.G. Harris and J.M. Pike: 3D Positional Integration from Image Sequences. 3rd Alvey Vision Conf, pp. 233–236, 1987.Google Scholar
- 11.R. Hartley: Estimation of relative camera positions for uncalibrated cameras. Proc. ECCV'92, pp.579–87.Google Scholar
- 12.A. Heyden and K. åström: Euclidean Reconstruction from Image Sequences with Varying and Unknown Focal Length and Principal Point. Proc. CVPR'97.Google Scholar
- 13.M. Irani and S. Peleg: Super resolution from image sequences. Tenth International Conference on Pattern Recognition (Atlantic City, NJ, June 16–21, 1990), IEEE Catalog No. 90CH2898-5, 1990, subconference C, 115–220.Google Scholar
- 14.R. Koch: 3-D Modeling of Human Heads from Stereoscopic Image Sequences. Proc. DAGM'96, Informatik Aktuell, Springer Heidelberg, Germany, Sept. 1996.Google Scholar
- 15.R. Koch: Surface Segmentation and Modeling of 3-D Polygonal Objects from Stereoscopic Image Pairs. Proc. ICPR'96, Vienna 1996.Google Scholar
- 16.R. Koch: Automatische Oberflächenmodellierung starrer dreidimensionaler Objekte aus stereoskopischen Bildfolgen. Ph.D. Thesis, Fortschr.-Ber. 10/499. Düsseldorf: VDI Verlag 1997.Google Scholar
- 17.A. Kochan: Eine Methodenbank zur Evaluierung von Stereo-Vision-Verfahren. Ph.D. Thesis, TU Berlin, June 1991.Google Scholar
- 20.E. Ofek, E. Shilat, A. Rappoport, M. Werman: Highlight and Reflection-Independent Multiresolution Textures from Image Sequences. IEEE Computer Graphics and Applications vol. 17 (2), March–April 1997.Google Scholar
- 21.M. Pollefeys, L. Van Gool and M. Proesmans: Euclidean 3D Reconstruction from Image Sequences with Variable Focal Lengths. Proc. ECCV'96, vol.1, pp. 31–42.Google Scholar
- 22.M. Pollefeys and L. Van Gool: Self-calibration from the absolute conic on the plane at infinity. Proc. CAIP'97.Google Scholar
- 23.M. Pollefeys, R. Koch and L. Van Gool: Self-Calibration and Metric Reconstruction in spite of Varying and Unknown Internal Camera Parameters. Proc. ICCV'98, Bombay, India, 1998.Google Scholar
- 24.P.H.S. Torr: Motion Segmentation and Outlier Detection. PhD thesis, Dept. Eng. Science, University of Oxford, UK, 1995.Google Scholar
- 25.B. Triggs: The Absolute Quadric. Proc. CVPR'97.Google Scholar
- 26.Z. Zhang and O. Faugeras: 3D Dynamic Scene Analysis. Springer Verlag, 1992.Google Scholar