Skip to main content
SpringerLink
Log in

Image-based geometrically-correct photorealistic scene/object modeling (IBPhM): A review

  • Session S2A: Computer Vision & Virtual Reality
  • Conference paper
  • First Online:
Book cover Computer Vision — ACCV'98 (ACCV 1998)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 1352))

Included in the following conference series:

Abstract

There are emerging interests from both computer vision and computer graphics communities in obtaining photorealistic modeling of a scene or an object from real images. This paper presents a tentative review of the computer vision techniques used in such modeling which guarantee the generated views to be geometrically correct. The topics covered include mosaicking for building environment maps, CAD-like modeling for building 3D geometric models together with texture maps extracted from real images, image-based rendering for synthesizing new views from uncalibrated images, and techniques for modeling the appearance variation of a scene or an object under different illumination conditions. Major issues and difficulties are addressed.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. S. Chen, “QuickTime VR — an image-based approach to virtual environment navigation,” in Computer Graphics, Annual Conference Series, pp. 29–38, ACM SIGGRAPH, 1995.

    Google Scholar 

  2. S. Chen and L. Williams, “View interpolation for image synthesis,” in Computer Graphics, Annual Conference Series, pp. 279–288, ACM SIGGRAPH, 1993.

    Google Scholar 

  3. T. Werner, R. Hersch, and V Hlavac, “Rendering real-world objects using view intepolation,” in Proc. Fifth International Conference on Computer Vision, (Cambridge, Massachusetts), pp. 957–962, June 1995.

    Google Scholar 

  4. A. Katayama, K. Tanaka, T Oshino, and H. Tamura, “A viewpoint independent stereoscopic display using interpolation of multi-viewpoint images,” in Stereoscopic displays and virtual reality systems II (S. Fisher, J. Merritt, and B. Bolas, eds.), vol. 2409 of Proc. SPIE, pp. 11–20, 1995.

    Google Scholar 

  5. S. Gortler, R. Grzeszczuk, R. Szeliski, and M. Cohen, “The Lumigraph,” in Computer Graphics, Annual Conference Series, pp. 43–54, ACM SIGGRAPH, 1996.

    Google Scholar 

  6. M., Levoy and P Hanraham, “Light field rendering,” in Computer Graphics, Annual Conference Series, pp. 31–42, ACM SIGGRAPH, 1996.

    Google Scholar 

  7. L. McMillan and G. Bishop, “Plenoptic modeling: An image-based rendering system,” in Computer Graphics, Annual Conference Series, pp. 39–46, ACM SIGGRAPH, 1995.

    Google Scholar 

  8. P. Debevec, C. Taylor, and J. Malik, “Modeling and rendering architecture from photographs: A hybrid geometry-and image-based approach,” in Computer Graphics, Annual Conference Series, pp. 11–20, ACM SIGGRAPH, 1996.

    Google Scholar 

  9. O. Faugeras and S. Laveau, “Representing three-dimensional data as a collection of images and fundamental matrices for image synthesis,” in Proc. International Conference on Pattern Recognition, (Jerusalem, Israel), pp. 689–691, Computer Society Press, Oct. 1994.

    Google Scholar 

  10. S. Seitz and C. Dyer, “View morphing,” in Computer Graphics, Annual Conference Series, pp. 21–30, ACM SIGGRAPH, 1996.

    Google Scholar 

  11. S. Kang, “A survey of image-based rendering techniques,” Tech. Rep. CRL 97/4, Digital Equipment Corporation, Cambridge Research Lab, Aug. 1997.

    Google Scholar 

  12. R. Szeliski and H.-Y. Shum, “Creating full view panoramic image mosaics and environment maps,” in Computer Graphics, Annual Conference Series, pp. 251–258, ACM SIGGRAPH, 1997.

    Google Scholar 

  13. I. Zoghlami, O. Faugeras, and R. Deriche, “Using geometric corners to build a 2d mosaic from a set of images,” in Proc. IEEE Conference on Computer Vision and Pattern Recognition, (San Juan, Puerto Rico), pp. 420–425, IEEE Computer Society, June 1997.

    Google Scholar 

  14. S. Peleg and J. Herman, “Panoramic mosaics by manifold projection,” in Proc. IEEE Conference on Computer Vision and Pattern Recognition, (San Juan, Puerto Rico), pp. 338–343, IEEE Computer Society, June 1997.

    Google Scholar 

  15. H.-Y. Shum and R. Szeliski, “Construction and refinement of panoramic mosaics with global and local alignment,” in Proc. 6th International Conference on Computer Vision, (Bombay, India), IEEE Computer Society Press, Jan. 1998.

    Google Scholar 

  16. Y. Yagi and S. Kawato, “Panorama scene analysis with conic projection,” in Proc. IEEE International Workshop on Intelligent Robots and Systems, pp. 181–187, July 1990.

    Google Scholar 

  17. K. Yamazawa, Y. Yagi, and S. Kawato, “Omnidirectional imaging with hyperboloidal projection,” in Proc. IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 1029–1034, July 1993.

    Google Scholar 

  18. S. Nayar, “Catadioptric omnidirectional camera,” in Proc. IEEE Conference on Computer Vision and Pattern Recognition (G. Medioni, R. Nevatia, D. Huttenlocher, and J. Ponce, eds.), (San Juan, Puerto Rico), pp. 482–488, IEEE Computer Society, June 1997.

    Google Scholar 

  19. O. Faugeras, Three-Dimensional Computer Vision: a Geometric Viewpoint. MIT Press, 1993.

    Google Scholar 

  20. J. Aggarwal and N. Nandhakumar, “On the computation of motion from sequences of images — a review,” Proc. IEEE, vol. 76, pp. 917–935, Aug. 1988.

    Google Scholar 

  21. T. Huang and A. Netravali, “Motion and structure from feature correspondences: A review,” Proc. IEEE, vol. 82, pp. 252–268, Feb. 1994.

    Google Scholar 

  22. Z. Zhang, “Motion and structure from two perspective views: From essential parameters to euclidean motion via fundamental matrix,” Journal of the Optical Society of America A, vol. 14, no. 11, 1997. In Press.

    Google Scholar 

  23. Z. Zhang and O. Faugeras, 3D Dynamic Scene Analysis: A Stereo Based Approach. Springer-Verlag, Berlin, New York, 1992.

    Google Scholar 

  24. Z. Zhang, “Iterative point matching for registration of free-form curves and surfaces,” The International Journal of Computer Vision, vol. 13, no. 2, pp. 119–152, 1994. also Research Report No. 1658, INRIA Sophia-Antipolis, 1992.

    Google Scholar 

  25. Z. Zhang, “Motion of a stereo rig: Strong weak and self calibration,” in Recent Developments in Computer Vision (S. Li, D. Mital, E. Teoh, and H. Wang, eds.), vol. 1035 of Lecture Notes in Computer Science, pp. 241–254, Springer-Verlag, Berlin, 1996.

    Google Scholar 

  26. S. J. Maybank and O. D. Faugeras, “A theory of self-calibration of a moving camera,” The International Journal of Computer Vision, vol. 8, pp. 123–152, Aug. 1992.

    Google Scholar 

  27. Q.-T. Luong, Matrice Fondamentale et Calibration Visuelle sur l'Environnement-Vers une plus grande autonomie des systèmes robotiques. PhD thesis, Université de Paris-Sud, Centre d'Orsay, Dec. 1992.

    Google Scholar 

  28. Q.-T. Luong and O. Faugeras, “Self-calibration of a moving camera from point correspondences and fundamental matrices,” The International Journal of Computer Vision, vol. 22, no. 3, pp. 261–289, 1997.

    Google Scholar 

  29. R. Hartley, “An algorithm for self calibration from several views,” in Proc. IEEE Conference on Computer Vision and Pattern Recognition, (Seattle, WA), pp.908–912, 1994.

    Google Scholar 

  30. Z. Zhang, Q.-T. Luong, and O. Faugeras, “Motion of an uncalibrated stereo rig: self-calibration and metric reconstruction,” IEEE Transactions on Robotics and Automation, vol. 12, pp. 103–113, Feb. 1996. Short version appeared in the Proc. International Conference on Pattern Recognition, volume I, pages 695–697, Jerusalem, Israel, Oct. 1994.

    Google Scholar 

  31. A. Zisserman, P A. Beardsley, and 1. D. Reid, “Metric calibration of a stereo rig,” in Proc. Workshop on Visual Scene Representation, (Boston, MA), June 1995.

    Google Scholar 

  32. F. Devemay and O. Faugeras, “From projective to euclidean reconstruction,” in Proc. IEEE Conference on Computer Vision and Pattern Recognition, (San Francisco, CA), pp. 264–269, IEEE, June 1996.

    Google Scholar 

  33. O. Faugeras, “What can be seen in three dimensions with an uncalibrated stereo rig,” in Proc. 2nd European Conference on Computer Vision (G. Sandini, ed.), vol. 588 of Lecture Notes in Computer Science, (Santa Margherita Ligure, Italy), pp. 563–578, Springer-Verlag, May 1992.

    Google Scholar 

  34. R. Hartley, “Projective reconstruction and invariants from multiple images,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 16, no. 10, pp. 1036–1040, 1994.

    Google Scholar 

  35. G. Xu and Z. Zhang, Epipolar Geometry in Stereo, Motion and Object Recognition. Kluwer Academic Publishers, 1996.

    Google Scholar 

  36. R. Hartley, R. Gupta, and T. Chang, “Stereo from uncalibrated cameras,” in Proc. IEEE Conference on Computer Vision and Pattern Recognition, (Urbana Champaign, IL), pp. 761–764, IEEE, June 1992.

    Google Scholar 

  37. R. Mohr, B. Boufama, and P. Brand, “Understanding positioning from multiple images,” Artificial Intelligence, vol. 78, pp. 213–238, 1995.

    Google Scholar 

  38. Z. Zhang, K. Isono, and S. Akamatsu, “Euclidean structure from uncalibrated images using fuzzy domain knowledge: Application to facial images synthesis,” in Proc. 6th International Conference on Computer Vision, (Bombay, India), IEEE Computer Society Press, Jan. 1998.

    Google Scholar 

  39. Q.-T. Luong and O. D. Faugeras, “The fundamental matrix: Theory, algorithms and stability analysis,” The International Journal of Computer Vision, vol. 17, pp. 43–76, Jan. 1996.

    Google Scholar 

  40. A. Shashua, “Algebraic functions for recognition,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 17, no. 8, pp. 779–789, 1995.

    Google Scholar 

  41. Q.-T. Luong and T. Viéville, “Canonical representations for the geometries of multiple projective views,” Computer Vision and Image Understanding, vol. 64, pp. 193–229, Sept. 1996.

    Google Scholar 

  42. Z. Zhang, “Determining the epipolar geometry and its uncertainty: A review,” The International Journal of Computer Vision, 1997. In Press. Updated version of INRIA Research Report No.2927, 1996.

    Google Scholar 

  43. P. Torr and A. Zisserman, “Robust parameterization and computation of the trifocal tensor,” Image and Vision Computing, vol. 15, pp. 591–605, 1997.

    Google Scholar 

  44. S. Laveau, Géométrie d'un système de N caméras. Théorie, estimation et applications. PhD thesis, École Polytechnique, May 1996.

    Google Scholar 

  45. M. Oren and S. Nayar, “Generalization of the lambertian model and implications for machine vision,” The International Journal of Computer Vision, vol. 14, pp. 227–251, Apr. 1995.

    Google Scholar 

  46. L. Wolff, “Generalizing Lambert's law for smooth surfaces,” in Proc. 4th European Conference on Computer Vision (B. Buxton, ed.), vol. II, (Cambridge, UK), pp. 40–53, Apr. 1996.

    Google Scholar 

  47. J. Koenderink, A. van Doorn, and M. Stavridi, “Bidirectional reflection distribution function expressed in terms of surface scattering modes,” Research Report UU-PAhp-046, Utrecht State University, 1995.

    Google Scholar 

  48. A. Shashua, Geometry and Photometry in 3D Visual Recognition. PhD thesis, Massachusetts Institute of Technology, 1992.

    Google Scholar 

  49. S. Nayar and H. Murase, “Dimensionality of illumination in appearance matching,” in Proc. IEEE International Conference on Robotics and Automation, (Minneapolis, Minnesota), pp. 1326–1332, Apr. 1996.

    Google Scholar 

  50. R. Epstein, P. Hallinan, and A. Yuille, “5±2 eigenimages suffice: An empirical investigation of low-dimensional lighting models,” in Proc. IEEE Workshop on Physics Based Modeling in Computer Vision, (Cambridge, Massachusett), pp. 108–116, June 1995.

    Google Scholar 

  51. P. Belhumeur and D. Kriegman, “What is the set of images of an object under all possible lighting conditions?,” in Proc. IEEE Conference on Computer Vision and Pattern Recognition, pp. 270–277, June 1996.

    Google Scholar 

  52. G. Hager and P Belhumeur, “Real-time tracking of image regions with changes in geometry and illumination,” in Proc. IEEE Conference on Computer Vision and Pattern Recognition, June 1996.

    Google Scholar 

  53. G. Golub and C. van Loan, Matrix Computations. The John Hopkins University Press, 1989.

    Google Scholar 

  54. Z. Zhang, “Modeling geometric structure and illumination variation of a scene from real images,” in Proc. 6th International Conference on Computer Vision, (Bombay, India), IEEE Computer Society Press, Jan. 1998.

    Google Scholar 

  55. Y. Sato, M. Wheeler, and K. Ikeuchi, “Object shape and reflectance modeling from observation,” in Computer Graphics, Annual Conference Series, pp. 379–387, ACM SIGGRAPH, 1997.

    Google Scholar 

  56. J. Bergen, P. Anandan, K. Hanna, and R. Hingorani, “Hierarchical model-based motion estimation,” in Proc. 2nd European Conference on Computer Vision (G. Sandini, ed.), vol. 588 of Lecture Notes in Computer Science, (Santa Margherita Ligure, Italy), pp. 237–252, Springer-Verlag, May 1992.

    Google Scholar 

  57. Z. Zhang, R. Deriche, O. Faugeras, and Q.-T. Luong, “A robust technique for matching two uncalibrated images through the recovery of the unknown epipolar geometry,” Artificial Intelligence Journal, vol. 78, pp. 87–119, Oct. 1995.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. INRIA, 2004 route des Lucioles, BP 93, F-06902, Sophia-Antipolis Cedex, France

    Zhengyou Zhang

  2. ATR HIP, 2-2 Hikaridai, Seika-cho Soraku-gun, 619-02, Kyoto, Japan

    Zhengyou Zhang

Authors
  1. Zhengyou Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Roland Chin Ting-Chuen Pong

Rights and permissions

Reprints and permissions

Copyright information

© 1997 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Zhang, Z. (1997). Image-based geometrically-correct photorealistic scene/object modeling (IBPhM): A review. In: Chin, R., Pong, TC. (eds) Computer Vision — ACCV'98. ACCV 1998. Lecture Notes in Computer Science, vol 1352. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-63931-4_235

Download citation

  • DOI: https://doi.org/10.1007/3-540-63931-4_235

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-63931-2

  • Online ISBN: 978-3-540-69670-4

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation