Advertisement

Shape from Photographs: A Multi-view Stereo Pipeline

  • Carlos Hernández
  • George Vogiatzis
Chapter
Part of the Studies in Computational Intelligence book series (SCI, volume 285)

Abstract

Acquiring 3D shape from images is a classic problem in Computer Vision occupying researchers for at least 20 years. Only recently however have these ideas matured enough to provide highly accurate results. We present a complete algorithm to reconstruct 3D objects from images using the stereo correspondence cue. The technique can be described as a pipeline of four basic building blocks: camera calibration, image segmentation, photo-consistency estimation from images, and surface extraction from photo-consistency. In this Chapter we will put more emphasis on the latter two: namely how to extract geometric information from a set of photographs without explicit camera visibility, and how to combine different geometry estimates in an optimal way.

Keywords

Markov Random Field Camera Calibration Deformable Model Normalize Cross Correlation Visual Hull 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
This is a preview of subscription content, log in to check access.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Baumgart, B.G.: Geometric modelling for computer vision. Ph.D. thesis, Standford University (1974)Google Scholar
  2. 2.
    Blake, A., Rother, C., Brown, M., Perez, P., Torr, P.: Interactive image segmentation using an adaptive GMMRF model. In: Pajdla, T., Matas, J(G.) (eds.) ECCV 2004. LNCS, vol. 3021, pp. 428–441. Springer, Heidelberg (2004)Google Scholar
  3. 3.
    Boissonnat, J.D., Faugeras, O., Lebras, E.: Representing stereo data with the delaunay triangulation. Artificial Intelligence 44, 41–87 (1990)zbMATHCrossRefMathSciNetGoogle Scholar
  4. 4.
    Bolitho, M., Kazhdan, M., Burns, R., Hoppe, H.: Multilevel streaming for out-of-core surface reconstruction. In: Proceedings of the Eurographics Symposium on Geometry Processing, pp. 69–78 (2007)Google Scholar
  5. 5.
    Boykov, Y., Kolmogorov, V.: Computing geodesics and minimal surfaces via graph cuts. In: Proceedings of the International Conference on Computer Vision, pp. 26–33 (2003)Google Scholar
  6. 6.
    Boykov, Y., Lempitsky, V.: From photohulls to photoflux optimization. In: Proceedings of the British Machine Vision Conference, pp. 1149–1158 (2006)Google Scholar
  7. 7.
    Bradley, D., Boubekeur, T., Heidrich, W.: Accurate multi-view reconstruction using robust binocular stereo and surface meshing. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (2008)Google Scholar
  8. 8.
    Campbell, N., Vogiatzis, G., Hernández, C., Cipolla, R.: Using multiple hypotheses to improve depth-maps for multi-view stereo. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part I. LNCS, vol. 5302, pp. 766–779. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  9. 9.
    Cohen, L.D.: On active contour models and balloons. CVGIP: Image Understanding 53(2), 211–218 (1991)zbMATHCrossRefGoogle Scholar
  10. 10.
    Cohen, L.D., Cohen, I.: Finite element methods for active contour models and balloons for 2-D and 3-D images. IEEE Transaction on Pattern Analysis and Machine Intelligence 15(11), 1131–1147 (1993)CrossRefGoogle Scholar
  11. 11.
    Cornelis, N., Leibe, B., Cornelis, K., Gool, L.: 3d urban scene modeling integrating recognition and reconstruction. International Journal of Computer Vision 2-3(78), 121–141 (2008)CrossRefGoogle Scholar
  12. 12.
    Curless, B., Levoy, M.: A volumetric method for building complex models from range images. In: Proceedings of the ACM SIGGRAPH, pp. 303–312 (1996)Google Scholar
  13. 13.
    Faugeras, O., Keriven, R.: Variational principles, surface evolution, pdes, level set methods and the stereo problem. IEEE Transactions on Image Processing 7(3), 335–344 (1998)CrossRefMathSciNetGoogle Scholar
  14. 14.
    Favaro, P., Soatto, S.: 3-D Shape Estimation and Image Restoration: Exploiting Defocus and Motion-Blur. Springer, Heidelberg (2007)zbMATHGoogle Scholar
  15. 15.
    Furukawa, Y., Curless, B., Seitz, S.M., Szeliski, R.: Reconstructing building interiors from images. In: Proceedings of the International Conference on Computer Vision (2009)Google Scholar
  16. 16.
    Furukawa, Y., Ponce, J.: Carved visual hulls for image-based modeling. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3951, pp. 564–577. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  17. 17.
    Furukawa, Y., Ponce, J.: Accurate, dense, and robust multi-view stereopsis. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (2007)Google Scholar
  18. 18.
    Goesele, M., Curless, B., Seitz, S.: Multi-view stereo revisited. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition, pp. 2402–2409 (2006)Google Scholar
  19. 19.
    Habbecke, M., Kobbelt, L.: A surface-growing approach to multi-view stereo reconstruction. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (2007)Google Scholar
  20. 20.
    Hernández, C., Schmitt, F.: Silhouette and stereo fusion for 3d object modeling. Computer Vision and Image Understanding 96(3), 367–392 (2004)CrossRefGoogle Scholar
  21. 21.
    Hernández, C., Schmitt, F., Cipolla, R.: Silhouette coherence for camera calibration under circular motion. IEEE Transaction on Pattern Analysis and Machine Intelligence 29(2), 343–349 (2007)CrossRefGoogle Scholar
  22. 22.
    Hernández, C., Vogiatzis, G., Cipolla, R.: Probabilistic visibility for multi-view stereo. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (2007)Google Scholar
  23. 23.
    Hernández, C., Vogiatzis, G., Cipolla, R.: Multi-view photometric stereo. IEEE Transaction on Pattern Analysis Machine Intelligence 30(1), 548–554 (2008)CrossRefGoogle Scholar
  24. 24.
    Hornung, A., Kobbelt, L.: Hierarchical volumetric multi-view stereo reconstruction of manifold surfaces based on dual graph embedding. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition, pp. 503–510 (2006)Google Scholar
  25. 25.
    Ilic, S., Fua, P.: Implicit meshes for surface reconstruction. IEEE Transactions on Pattern Analysis and Machine Intelligence 28(2), 328–333 (2006)CrossRefGoogle Scholar
  26. 26.
    Jaynes, E.: Probability Theory, The Logic of Science. Cambridge University Press, Cambridge (2003)zbMATHGoogle Scholar
  27. 27.
    Kass, M., Witkin, A., Terzopoulos, D.: Snakes: Active contour models. International Journal of Computer Vision 1, 321–332 (1988)CrossRefGoogle Scholar
  28. 28.
    Kobbelt, L.: \(\sqrt{3}\)-subdivision. In: Proceedings of the ACM SIGGRAPH, pp. 103–112 (2000)Google Scholar
  29. 29.
    Kolev, K., Cremers, D.: Integration of multiview stereo and silhouettes via convex functionals on convex domains. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part I. LNCS, vol. 5302, pp. 752–765. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  30. 30.
    Kolmogorov, V.: Convergent tree-reweighted message passing for energy minimization. IEEE Transaction on Pattern Analysis and Machine Intelligence 28(10), 1568–1583 (2006)CrossRefGoogle Scholar
  31. 31.
    Kolmogorov, V., Zabih, R.: What energy functions can be minimized via graph cuts. IEEE Transaction on Pattern Analysis and Machine Intelligence 26(2), 147–159 (2004)CrossRefGoogle Scholar
  32. 32.
    Kutulakos, K.N., Seitz, S.M.: A theory of shape by space carving. International Journal of Computer Vision 38(3), 199–218 (2000)zbMATHCrossRefGoogle Scholar
  33. 33.
    Lempitsky, V., Boykov, Y., Ivanov, D.: Oriented visibility for multiview reconstruction. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3953, pp. 226–238. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  34. 34.
    Levoy, M., Pulli, K., Curless, B., Rusinkiewicz, S., Koller, D., Pereira, L., Ginzton, M., Anderson, S., Davis, J., Ginsberg, J., Shade, J., Fulk, D.: The digital michelangelo project: 3d scanning of large statues. In: Proceedings of the ACM SIGGRAPH, pp. 15–22 (2000)Google Scholar
  35. 35.
    Lhuillier, M., Quan, L.: A quasi-dense approach to surface reconstruction from uncalibrated images. IEEE Transaction on Pattern Analysis and Machine Intelligence 27(3), 418–433 (2005)CrossRefGoogle Scholar
  36. 36.
    Goesele, M., Snavely, N., Curless, B., Hoppe, H., Seitz, S. M.: Multi-view stereo for community photo collections. In: Proceedings of the International Conference on Computer Vision (2007)Google Scholar
  37. 37.
    Pollefeys, M., Gool, L.J.V., Vergauwen, M., Verbiest, F., Cornelis, K., Tops, J., Koch, R.: Visual modeling with a hand-held camera. International Journal of Computer Vision 59(3), 207–232 (2004)CrossRefGoogle Scholar
  38. 38.
    Pollefeys, M., Nistér, D., Frahm, J.M., Akbarzadeh, A., Mordohai, P., Clipp, B., Engels, C., Gallup, D., Kim, S.J., Merrell, P., Salmi, C., Sinha, S., Talton, B., Wang, L., Yang, Q., Stewénius, H., Yang, R., Welch, G., Towles, H.: Detailed real-time urban 3d reconstruction from video. International Journal of Computer Vision 78(2-3), 143–167 (2008)CrossRefGoogle Scholar
  39. 39.
    Pons, J.P., Keriven, R., Faugeras, O.: Multi-view stereo reconstruction and scene flow estimation with a global image-based matching score. The International Journal of Computer Vision 72(2), 179–193 (2007)CrossRefGoogle Scholar
  40. 40.
    Seitz, S., Curless, B., Diebel, J., Scharstein, D., Szeliski, R.: A comparison and evaluation of multi-view stereo reconstruction algorithms. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition, pp. 519–528 (2006)Google Scholar
  41. 41.
    Sinha, S., Pollefeys, M.: Multi-view reconstruction using photo-consistency and exact silhouette constraints: A maximum-flow formulation. In: Proceedings of the International Conference on Computer Vision, pp. 349–356 (2005)Google Scholar
  42. 42.
    Sinha, S.N., Mordohai, P., Pollefeys, M.: Multi-view stereo via graph cuts on the dual of an adaptive tetrahedral mesh. In: Proceedings of the International Conference on Computer Vision (2007)Google Scholar
  43. 43.
    Snavely, N., Seitz, S.M., Szeliski, R.: Photo tourism: Exploring image collections in 3d. In: Proceedings of the ACM SIGGRAPH (2006)Google Scholar
  44. 44.
    Steger, E., Kutulakos, K.N.: A theory of refractive and specular 3d shape by light-path triangulation. International Journal of Computer Vision 76(1) (2008)Google Scholar
  45. 45.
    Strecha, C., von Hansen, W., Gool, L.V., Fua, P., Thoennessen, U.: On benchmarking camera calibration and multi-view stereo for high resolution imagery. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (2008)Google Scholar
  46. 46.
    Tran, S., Davis, L.: 3d surface reconstruction using graph cuts with surface constraints. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3952, pp. 218–231. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  47. 47.
    Vogiatzis, G., Hernández, C., Torr, P., Cipolla, R.: Multi-view stereo via volumetric graph-cuts and occlusion robust photo-consistency. IEEE Transaction on Pattern Analysis and Machine Intelligence 29(12), 2241–2246 (2007)CrossRefGoogle Scholar
  48. 48.
    Vogiatzis, G., Torr, P., Cipolla, R.: Multi-view stereo via volumetric graph-cuts. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition, pp. 391–398 (2005)Google Scholar
  49. 49.
    Weise, T., Leibe, B., Gool, L.V.: Fast 3d scanning with automatic motion compensation. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (2007)Google Scholar
  50. 50.
    Witkin, A.: Recovering surface shape and orientation from texture. Artificial Intelligence 17(1-3), 17–45 (1981)CrossRefGoogle Scholar
  51. 51.
    Woodham, R.: Photometric method for determining surface orientation from multiple images. Optical Engineering 19(1), 139–144 (1980)Google Scholar
  52. 52.
    Xu, C., Prince, J.L.: Snakes, shapes, and gradient vector flow. IEEE Transactions on Image Processing, 359–369 (1998)Google Scholar
  53. 53.
    Zebedin, L., Bauer, J., Karner, K., Bischof, H.: Fusion of feature- and area-based information for urban buildings modeling from aerial imagery. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition, pp. 873–886 (2008)Google Scholar
  54. 54.
    Zhang, L., Snavely, N., Curless, B., Seitz, S.M.: Spacetime faces: High-resolution capture for modeling and animation. In: ACM Annual Conference on Computer Graphics, pp. 548–558 (2004)Google Scholar
  55. 55.
    Zisserman, A., Hartley, R.: Multiple View Geometry. Springer, Heidelberg (2000)zbMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Carlos Hernández
    • 1
  • George Vogiatzis
    • 2
  1. 1.Toshiba Research CambridgeUK
  2. 2.Aston UniversityBirminghamUK

Personalised recommendations

Cite chapter

Buy options