Advertisement

Image-Based 3D Modeling: Modeling from Reality

  • Luc Van Gool
  • Filip Defoort
  • Johannes Hug
  • Gregor Kalberer
  • Reinhard Koch
  • Danny Martens
  • Marc Pollefeys
  • Marc Proesmans
  • Maarten Vergauwen
  • Alexey Zalesny
Chapter
Part of the NATO Science Series book series (ASHT, volume 84)

Abstract

Increasingly, realistic object, scene, and event modeling is based on image data rather than manual synthesis. The paper describes a system for visits to a virtual, 3D archeological site. One can navigate through this environment, with a virtual guide as companion. One can ask questions using natural, fluent speech. The guide will respond and will bring the visitor to the desired place. Simple answers are given as changes in the orientations of his head, by him raising his eyebrows or by head nodding. In the near future the head will speak.

The idea to model directly from images is applied in three subcomponents of this system. First, there are two systems for 3D modeling. One is a shape-from-video system, that turns multiple, uncalibrated images into realistic 3D models. This system was used to model the landscape and buildings of the site. The second projects a special pattern and was used to model smaller pieces, like statues and ornaments that often had intricate shapes. Secondly, the model of the scene is only as convincing as the texture by which it is covered. As it is impossible to keep images of the texture of a complete landscape, images of the natural surface were used to synthesize more of similar texture, starting from a very compact yet effective texture model. Thirdly, natural lip motions were learned from observed, 3D face dynamics. These will be used to animate the virtual guide in future versions of the system.

Keywords

Pairwise Interaction Texture Synthesis Neighborhood System Face Animation Fluent Speech 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. [1]
    M. Armstrong, A. Zisserman, and P. Beardsley. Euclidean structure from uncalibrated images. Proc. British Machine Vision Conf., 1994.Google Scholar
  2. [2]
    P. Besl. Active Optical Range Imaging Sensors. Machine Vision and Applications, 1(2):127–152, 1988.CrossRefGoogle Scholar
  3. [3]
    K. Boyer and A. Kak. Color-encoded structured light for rapid active ranging. IEEE Transaction on Pattern Analysis and Machine Intelligence, 9(10):14–28, 1987.CrossRefGoogle Scholar
  4. [4]
    http://www.eyetronics.comGoogle Scholar
  5. [5]
    T. Ezzat and T. Poggio. Visual speech synthesis by morphing visemes. AI Memo No. 1658, MIT, May 1999.Google Scholar
  6. [6]
    A. Gagalowicz and S. D. Ma. Sequential synthesis of natural textures. CVGIP, vol. 30, pp. 289–315, 1985.Google Scholar
  7. [7]
    A. Heyden and K. Aström. Euclidean reconstruction from image sequences with varying and unknown focal length and principal point. Proc. Conf. on Computer Vision and Pattern Recognition, pp. 438–443, 1997.Google Scholar
  8. [8]
    R. A. Jarvis. A perspective on range finding techniques for computer vision. IEEE Transaction on Pattern Analysis and Machine Intelligence, 5(2):122–139, 1983.CrossRefGoogle Scholar
  9. [9]
    R. Koch, M. Pollefeys, and L. Van Gool. Multi viewpoint stereo from uncalibrated video sequences. Proc. Eur. Conf. Computer Vision, Vol I, pages 55–71. LNCS, Springer, 1998.Google Scholar
  10. [10]
    M. Maruyama, and S. Abe. Range sensing by projecting multiple slits with random cuts. IEEE Transaction on Pattern Analysis and Machine Intelligence, 15(6):647–650, 1993.CrossRefGoogle Scholar
  11. [11]
    M. Pollefeys, R. Koch, and L. Van Gool. Self-calibration and metric reconstruction in spite of varying and unknown internal camera parameters. Int. Conf. on Computer Vision, pages. 90–95, Bombay, India, Jan. 4–7, 1998.Google Scholar
  12. [12]
    M. Pollefeys, R. Koch, and L. Van Gool. Self-calibration and metric reconstruction inspite of varying and in known intrinsic camera parameters. International Journal of Computer Vision, 32(l):7–25, 1999.CrossRefGoogle Scholar
  13. [13]
    P. Vuylsteke and A. Oosterlinck. Range image acquisition with a single binary-encoded light pattern. IEEE Transaction on Pattern Analysis and Machine Intelligence, 12(2):148–164, 1990.CrossRefGoogle Scholar
  14. [14]
    A. Zalesny. Analysis and Synthesis of Textures With Pairwise Signal Interactions. Techn. rep. KUL/ESAT/PSI/9902, Katholieke Universiteit Leuven, Belgium, 1999.Google Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2000

Authors and Affiliations

  • Luc Van Gool
  • Filip Defoort
  • Johannes Hug
  • Gregor Kalberer
  • Reinhard Koch
  • Danny Martens
  • Marc Pollefeys
  • Marc Proesmans
  • Maarten Vergauwen
  • Alexey Zalesny

There are no affiliations available

Personalised recommendations