Image-Based 3D Modeling: Modeling from Reality
Increasingly, realistic object, scene, and event modeling is based on image data rather than manual synthesis. The paper describes a system for visits to a virtual, 3D archeological site. One can navigate through this environment, with a virtual guide as companion. One can ask questions using natural, fluent speech. The guide will respond and will bring the visitor to the desired place. Simple answers are given as changes in the orientations of his head, by him raising his eyebrows or by head nodding. In the near future the head will speak.
The idea to model directly from images is applied in three subcomponents of this system. First, there are two systems for 3D modeling. One is a shape-from-video system, that turns multiple, uncalibrated images into realistic 3D models. This system was used to model the landscape and buildings of the site. The second projects a special pattern and was used to model smaller pieces, like statues and ornaments that often had intricate shapes. Secondly, the model of the scene is only as convincing as the texture by which it is covered. As it is impossible to keep images of the texture of a complete landscape, images of the natural surface were used to synthesize more of similar texture, starting from a very compact yet effective texture model. Thirdly, natural lip motions were learned from observed, 3D face dynamics. These will be used to animate the virtual guide in future versions of the system.
KeywordsPairwise Interaction Texture Synthesis Neighborhood System Face Animation Fluent Speech
Unable to display preview. Download preview PDF.
- M. Armstrong, A. Zisserman, and P. Beardsley. Euclidean structure from uncalibrated images. Proc. British Machine Vision Conf., 1994.Google Scholar
- http://www.eyetronics.comGoogle Scholar
- T. Ezzat and T. Poggio. Visual speech synthesis by morphing visemes. AI Memo No. 1658, MIT, May 1999.Google Scholar
- A. Gagalowicz and S. D. Ma. Sequential synthesis of natural textures. CVGIP, vol. 30, pp. 289–315, 1985.Google Scholar
- A. Heyden and K. Aström. Euclidean reconstruction from image sequences with varying and unknown focal length and principal point. Proc. Conf. on Computer Vision and Pattern Recognition, pp. 438–443, 1997.Google Scholar
- R. Koch, M. Pollefeys, and L. Van Gool. Multi viewpoint stereo from uncalibrated video sequences. Proc. Eur. Conf. Computer Vision, Vol I, pages 55–71. LNCS, Springer, 1998.Google Scholar
- M. Pollefeys, R. Koch, and L. Van Gool. Self-calibration and metric reconstruction in spite of varying and unknown internal camera parameters. Int. Conf. on Computer Vision, pages. 90–95, Bombay, India, Jan. 4–7, 1998.Google Scholar
- A. Zalesny. Analysis and Synthesis of Textures With Pairwise Signal Interactions. Techn. rep. KUL/ESAT/PSI/9902, Katholieke Universiteit Leuven, Belgium, 1999.Google Scholar