Advertisement

A Survey of Methods for Volumetric Scene Reconstruction from Photographs

  • Greg Slabaugh
  • Ron Schafer
  • Tom Malzbender
  • Bruce Culbertson
Part of the Eurographics book series (EUROGRAPH)

Abstract

Scene reconstruction, the task of generating a 3D model of a scene given multiple 2D photographs taken of the scene, is an old and difficult problem in computer vision. Since its introduction, scene reconstruction has found application in many fields, including robotics, virtual reality, and entertainment. Volumetric models are a natural choice for scene reconstruction. Three broad classes of volumetric reconstruction techniques have been developed based on geometric intersections, color consistency, and pair-wise matching. Some of these techniques have spawned a number of variations and undergone considerable refinement. This paper is a survey of techniques for volumetric scene reconstruction.

Keywords

Visual Hull Reprojection Error Volumetric Model Scene Reconstruction Exterior Space 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. [Chen 99]
    Q. Chen and G. Medioni, “A Volumetric Stereo Matching Method: Application to Image-Based Modeling,” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, June 1999, pp. 29–34.Google Scholar
  2. [Chien 84]
    C. H. Chien and J. K. Aggarwal, “A Volume/Surface Representation,” Proceedings of the International Conference on Pattern Recognition, Montreal, Canada, July 30 — Aug. 2, 1984, pp. 817–820.Google Scholar
  3. [Chien 86]
    C. H. Chien and J. K. Aggarwal, “Volume / Surface Octrees for the Representation of Three-Dimensional Objects,” Computer Vision, Graphics, and Image Processing, Vol. 36, No. 1, Oct. 1986, pp. 100–113.CrossRefGoogle Scholar
  4. [Collins 96]
    R. Collins, “A Space-Sweep Approach to True Multi-Image Matching,” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, June 18–20, 1996, pp. 358–363.Google Scholar
  5. [Culbertson 99]
    W. B. Culbertson, T. Malzbender, and G. Slabaugh, “Generalized Voxel Coloring,” Proceedings of the ICCV Workshop, Vision Algorithms Theory and Practice, Springer-Verlag Lecture Notes in Computer Science 1883, September 1999, pp. 100–115.Google Scholar
  6. [De Bonet 99]
    J. De Bonet and P. Viola, “Roxels: Responsibility Weighted 3D Volume Reconstruction,” Proceedings of the IEEE International Conference on Computer Vision, 1999, Vol. 1, pp. 415–425.Google Scholar
  7. [Dyer 01]
    C. Dyer, “Volumetric Scene Reconstruction from Multiple Views,” Foundations of Image Understanding, L. S. Davis, ed., Kluwer, Boston, 2001, pp. 469–489.CrossRefGoogle Scholar
  8. [Eisert 99]
    P. Eisert, E. Steinbach, and B. Girod, “Multi-Hypothesis, Volumetric Reconstruction of 3-D Objects From Multiple Calibrated Camera Views,” Proceedings of the International Conference on Acoustics, Speech, and Signal Processing, 1999, pp. 3509–3512.Google Scholar
  9. [Faugeras 96]
    O. Faugeras, Three-Dimensional Computer Vision, A Geometrical Viewpoint, MIT Press, 1996.Google Scholar
  10. [Faugeras 98]
    O. Faugeras and R. Keriven, “Variational Principles, Surface Evolution, PDE’s, Level Set Methods, and the Stereo Problem,” IEEE Transactions on Image Processing, Vol. 7, No. 3, March 1998, pp. 336–344.MathSciNetMATHCrossRefGoogle Scholar
  11. [Fromherz 94]
    T. Fromherz and M. Bichsel, “Shape from Contours as Initial Step in Shape from Multiple Cues,” ISPRS Commission III Symposium on Spatial Information from Digital Photogrammetry and Computer Vision, Munich, Germany, 1994, pp. 240–256.Google Scholar
  12. [Fromherz 95]
    T. Fromherz and M. Bichsel, “Shape from Multiple Cues: Integrating Local Brightness Information,” Fourth International Conference for Young Computer Scientist, ICYCS 95, Beijing, P. R. China, 1995, pp. 855–862.Google Scholar
  13. [Garcia 98]
    B. Garcia and P. Brunet, “3D reconstruction with Projective Octrees and Epipolar Geometry,” Proceedings of the IEEE International Conference on Computer Vision, Jan. 4–7, 1998, pp. 1067–1072.Google Scholar
  14. [Gortler 96]
    S. Gortler, R. Grzeszczuk, R. Szeliski, and M. Cohen, “The Lumigraph,” Proceedings of A.C.M. SIGGRAPH, 1996, pp. 43–54.Google Scholar
  15. [Greenleaf 70]
    J. F. Greenleaf, T. S. Tu, and E. H. Wood, “Computer Generated 3-D Oscilloscopic Images and Associated Techniques for Display and Study of the Spatial Distribution of Pulmonary Blood Flow,” IEEE Transactions on Nuclear Science, Vol. 17, No. 3, June 1970, pp. 353–359.CrossRefGoogle Scholar
  16. [Hartley 00]
    R. Hartley and A. Zisserman, Multiple View Geometry, Cambridge University Press, 2000.MATHGoogle Scholar
  17. [Kaufman 91]
    A. Kaufman, Volume Visualization, IEEE Computer Society Press, Los Alamitos, California, 1991.Google Scholar
  18. [Kimura 99]
    M. Kimura, H. Saito, and T. Kanade, “3D Voxel Construction Based on Epipolar Geometry,” Proceedings of the International Conference on Image Processing, 1999, pp. 135–139.Google Scholar
  19. [Kutulakos 00a]
    K. N. Kutulakos, “Approximate N-View Stereo,” Proceedings of the European Conference on Computer Vision, Springer Lecture Notes in Computer Science 1842, June/July 2000, Vol. 1, pp. 67–83.Google Scholar
  20. [Kutulakos 00b]
    K. N. Kutulakos and S. M. Seitz, “A Theory of Shape by Space Carving,” International Journal of Computer Vision, Vol. 38, No. 3, July 2000, pp. 199–218.MATHCrossRefGoogle Scholar
  21. [Kutulakos 98]
    K. N. Kutulakos and S. M. Seitz, “What Do N Photographs Tell Us about 3D Shape?” TR680, Computer Science Dept. U. Rochester, January 1998.Google Scholar
  22. [Laurentini 94]
    A. Laurentini, “The Visual Hull Concept for Silhouette-Based Image Understanding,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 16, No. 2, Feb. 1994.Google Scholar
  23. [Lorenson 87]
    W. Lorenson and H. Cline, “Marching Cubes: A High Resolution 3D Surface Construction Algorithm,” Proceedings of A.C.M. SIGGRAPH, 1987, pp. 163–170.Google Scholar
  24. [Luong 96]
    Q. Luong and O. Faugeras, “The Fundamental Matrix: Theory, Algorithms and Stability Analysis,” International Journal on Computer Vision, Vol. 17, No. 1, Jan. 1996, pp. 43–75.CrossRefGoogle Scholar
  25. [Marr 76]
    D. Marr and T. Poggio, “Cooperative Computation of Stereo Disparity,” Science, Vol. 194, No. 4262, Oct. 1976, pp. 283–287.CrossRefGoogle Scholar
  26. [Martin 83]
    W. Martin and J. K. Aggarwal, “Volumetric Descriptions of Objects from Multiple Views”, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 5, No. 2, March 1983, pp. 150–158.CrossRefGoogle Scholar
  27. [Massone 85]
    L. Massone, P. Morasso, and R. Zaccaria, “Shape from Occluding Contours,” Proceedings of the SPIE Conference on Intelligent Robots and Computer Vision, SPIE Vol. 521, Nov. 1985, pp. 114–120.Google Scholar
  28. [Max 96]
    N. Max, X. Pueyo, and P. Schroder, “Hierarchical Rendering of Trees from Precomputed Multi-Layer Z-Buffers,” Proceedings of the Eurographics Rendering Workshop, 1996, pp. 165–174.Google Scholar
  29. [Moezzi 96]
    S. Moezzi, A. Katkere, D. Kuramura, and R. Jain, “Reality Modeling and Visualization from Multiple Video Sequences,” IEEE Computer Graphics and Applications, Vol. 16, No. 6, Nov. 1996, pp. 58–63.CrossRefGoogle Scholar
  30. [Moezzi 97]
    S. Moezzi, L. Tai, and P. Gerard, “Virtual View Generation for 3D Digital Video,” IEEE Multimedia, Vol. 4, No. 1, Jan.–Mar. 1997, pp. 18–26.CrossRefGoogle Scholar
  31. [Osher 88]
    S. Osher and J. Sethian, “Fronts Propagating with Curvature Dependent Speed: Algorithms Based on Hamilton-Jacobi Formulations,” Journal of Computational Physics, 79, 1988, pp. 12–49.MathSciNetMATHCrossRefGoogle Scholar
  32. [Potmesil 87]
    M. Potmesil, “Generating Octree Models of 3D Objects from Their Silhouettes in a Sequence of Images,” Computer Vision, Graphics and Image Processing, Vol. 40, No. 1, Oct. 1987, pp. 1–29.CrossRefGoogle Scholar
  33. [Prock 98]
    A. Prock and C. Dyer, “Towards Real-Time Voxel Coloring,” Proceedings of the DARPA Image Understanding Workshop, 1998, pp. 315–321.Google Scholar
  34. [Saito 99]
    H. Saito and T. Kanade, “Shape Reconstruction in Projective Grid Space from Large Number of Images”, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, June 23–25, 1999, Vol. 2, pp. 49–54.Google Scholar
  35. [Seitz 95]
    S. Seitz and C. Dyer, “Complete Scene Structure from Four Point Correspondences,” Proceedings of the IEEE International Conference on Computer Vision, June 1995, pp. 330–337.Google Scholar
  36. [Seitz 97]
    S. Seitz and C. Dyer, “Photorealistic Scene Reconstruction by Voxel Coloring,” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, June 1997, pp. 1067–1073.Google Scholar
  37. [Seitz 99]
    S. Seitz and C. Dyer, “Photorealistic Scene Reconstruction by Voxel Coloring,” International Journal of Computer Vision, Vol. 35, No. 2, 1999, pp. 151–173.CrossRefGoogle Scholar
  38. [Sethian 99]
    J. Sethian, Level Set Methods and Fast Marching Methods, Cambridge University Press, Second Edition, 1999.MATHGoogle Scholar
  39. [Shade 98]
    J. Shade, S. Gortler, L. He, and R. Szeliski, “Layered Depth Images,” Proceedings of A.C.M. SIGGRAPH, 1998, pp. 231–242.Google Scholar
  40. [Shashua 95]
    A. Shashua and M. Werman, “On the Trilinear Tensor of Three Perspective Views and its Underlying Geometry,” Proceedings of the IEEE International Conference on Computer Vision, 1995, pp. 920–925.Google Scholar
  41. [Shneier 84]
    M. Shneier, E. Kent, and P. Mansbach, “Representing Workspace and Model Knowledge for a Robot with Mobile Sensors,” Proceedings of the International Conference on Pattern Recognition, Montreal, Canada, July 1984, pp. 199–202.Google Scholar
  42. [Slabaugh 00a]
    G. Slabaugh, T. Malzbender, and W. B. Culbertson, “Volumetric Warping for Voxel Coloring on an Infinite Domain,” Proceedings of the Workshop on 3D Structure from Multiple Images for Large-scale Environments (SMILE), July 2000, pp. 41–50.Google Scholar
  43. [Slabaugh 00b]
    G. Slabaugh, W. B. Culbertson, T. Malzbender, and R. Schafer, “Improved Voxel Coloring Via Volumetric Optimization,” Center for Signal and Image Processing Technical Report TR3, Georgia Institute of Technology, 2000.Google Scholar
  44. [Srivastava 90]
    S. Srivastava and N. Ahuja, “Octree Generation from Object Silhouettes in Perspective Views,” Computer Vision, Graphics and Image Processing, Vol. 49, No. 1, Jan. 1990, pp. 68–84.CrossRefGoogle Scholar
  45. [Steinbach 00a]
    E. Steinbach, B. Girod, P. Eisert, and A. Betz, “3-D Object Reconstruction Using Spatially Extended Voxels and Multi-Hypothesis Voxel Coloring,” Proceedings of the International Conference on Pattern Recognition, 2000, Vol. 1, pp. 774–777.Google Scholar
  46. [Steinbach 00b]
    E. Steinbach, B. Girod, P. Eisert, and A. Betz, “3-D Reconstruction of Real-World Objects Using Extended Voxels,” Proceedings of the International Conference on Image Processing, 2000, Vol. III, pp. 138–141.Google Scholar
  47. [Szeliski 93]
    R. Szeliski, “Rapid Octree Construction from Image Sequences,” Computer Vision, Graphics and Image Processing: Image Understanding, Vol. 58, No. 1, July 1993, pp. 23–32.CrossRefGoogle Scholar
  48. [Szeliski 99]
    R. Szeliski and P. Golland, “Stereo Matching with Transparency and Matting,” International Journal of Computer Vision, Vol. 32, No. 1, 1999, pp. 45–62.CrossRefGoogle Scholar
  49. [Tsai 87]
    R. Tsai, “A Versatile Camera Calibration Technique for High-Accuracy 3D Machine Vision Metrology Using Off-the-Shelf TV Cameras and Lenses,” IEEE Transactions on Robotics and Automation, Vol. 3, No. 4, Aug. 1987, pp. 323–344.CrossRefGoogle Scholar
  50. [Vedula 00]
    S. Vedula, S. Baker, S. Seitz, and T. Kanade, “Shape and Motion Carving in 6D,” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2000, Vol. 2, pp. 592–598.Google Scholar
  51. [Veenstra 86]
    J. Veenstra and N. Ahuja, “Efficient Octree Generation from Silhouettes,” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Miami Beach, Florida, June 1986, pp. 537–542.Google Scholar
  52. [Weghorst 84]
    H. Weghorst, G. Hooper, D. P. Greenberg, “Improving Computational Methods for Ray Tracing”, ACM Transactions on Graphics, Vol. 3, No. 1, January 1984, pp. 52–69.CrossRefGoogle Scholar
  53. [Yang 93]
    Y. Yang, A. Yuille, and J. Lu, “Local, Global, and Multilevel Stereo Matching,” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1993, pp. 274–279.Google Scholar

Copyright information

© Springer-Verlag/Wien 2001

Authors and Affiliations

  • Greg Slabaugh
    • 1
  • Ron Schafer
    • 1
  • Tom Malzbender
    • 2
  • Bruce Culbertson
    • 2
  1. 1.Center for Signal and Image ProcessingGeorgia Institute of TechnologyAtlantaUSA
  2. 2.Visual Computing DepartmentHewlett-Packard LaboratoriesPalo AltoUSA

Personalised recommendations