Skip to main content
SpringerLink
Log in

Shape from Photographs: A Multi-view Stereo Pipeline

  • Chapter
Computer Vision

Part of the book series: Studies in Computational Intelligence ((SCI,volume 285))

Abstract

Acquiring 3D shape from images is a classic problem in Computer Vision occupying researchers for at least 20 years. Only recently however have these ideas matured enough to provide highly accurate results. We present a complete algorithm to reconstruct 3D objects from images using the stereo correspondence cue. The technique can be described as a pipeline of four basic building blocks: camera calibration, image segmentation, photo-consistency estimation from images, and surface extraction from photo-consistency. In this Chapter we will put more emphasis on the latter two: namely how to extract geometric information from a set of photographs without explicit camera visibility, and how to combine different geometry estimates in an optimal way.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 219.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Baumgart, B.G.: Geometric modelling for computer vision. Ph.D. thesis, Standford University (1974)

    Google Scholar 

  2. Blake, A., Rother, C., Brown, M., Perez, P., Torr, P.: Interactive image segmentation using an adaptive GMMRF model. In: Pajdla, T., Matas, J(G.) (eds.) ECCV 2004. LNCS, vol. 3021, pp. 428–441. Springer, Heidelberg (2004)

    Google Scholar 

  3. Boissonnat, J.D., Faugeras, O., Lebras, E.: Representing stereo data with the delaunay triangulation. Artificial Intelligence 44, 41–87 (1990)

    Article  MATH  MathSciNet  Google Scholar 

  4. Bolitho, M., Kazhdan, M., Burns, R., Hoppe, H.: Multilevel streaming for out-of-core surface reconstruction. In: Proceedings of the Eurographics Symposium on Geometry Processing, pp. 69–78 (2007)

    Google Scholar 

  5. Boykov, Y., Kolmogorov, V.: Computing geodesics and minimal surfaces via graph cuts. In: Proceedings of the International Conference on Computer Vision, pp. 26–33 (2003)

    Google Scholar 

  6. Boykov, Y., Lempitsky, V.: From photohulls to photoflux optimization. In: Proceedings of the British Machine Vision Conference, pp. 1149–1158 (2006)

    Google Scholar 

  7. Bradley, D., Boubekeur, T., Heidrich, W.: Accurate multi-view reconstruction using robust binocular stereo and surface meshing. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (2008)

    Google Scholar 

  8. Campbell, N., Vogiatzis, G., Hernández, C., Cipolla, R.: Using multiple hypotheses to improve depth-maps for multi-view stereo. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part I. LNCS, vol. 5302, pp. 766–779. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  9. Cohen, L.D.: On active contour models and balloons. CVGIP: Image Understanding 53(2), 211–218 (1991)

    Article  MATH  Google Scholar 

  10. Cohen, L.D., Cohen, I.: Finite element methods for active contour models and balloons for 2-D and 3-D images. IEEE Transaction on Pattern Analysis and Machine Intelligence 15(11), 1131–1147 (1993)

    Article  Google Scholar 

  11. Cornelis, N., Leibe, B., Cornelis, K., Gool, L.: 3d urban scene modeling integrating recognition and reconstruction. International Journal of Computer Vision 2-3(78), 121–141 (2008)

    Article  Google Scholar 

  12. Curless, B., Levoy, M.: A volumetric method for building complex models from range images. In: Proceedings of the ACM SIGGRAPH, pp. 303–312 (1996)

    Google Scholar 

  13. Faugeras, O., Keriven, R.: Variational principles, surface evolution, pdes, level set methods and the stereo problem. IEEE Transactions on Image Processing 7(3), 335–344 (1998)

    Article  MathSciNet  Google Scholar 

  14. Favaro, P., Soatto, S.: 3-D Shape Estimation and Image Restoration: Exploiting Defocus and Motion-Blur. Springer, Heidelberg (2007)

    MATH  Google Scholar 

  15. Furukawa, Y., Curless, B., Seitz, S.M., Szeliski, R.: Reconstructing building interiors from images. In: Proceedings of the International Conference on Computer Vision (2009)

    Google Scholar 

  16. Furukawa, Y., Ponce, J.: Carved visual hulls for image-based modeling. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3951, pp. 564–577. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  17. Furukawa, Y., Ponce, J.: Accurate, dense, and robust multi-view stereopsis. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (2007)

    Google Scholar 

  18. Goesele, M., Curless, B., Seitz, S.: Multi-view stereo revisited. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition, pp. 2402–2409 (2006)

    Google Scholar 

  19. Habbecke, M., Kobbelt, L.: A surface-growing approach to multi-view stereo reconstruction. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (2007)

    Google Scholar 

  20. Hernández, C., Schmitt, F.: Silhouette and stereo fusion for 3d object modeling. Computer Vision and Image Understanding 96(3), 367–392 (2004)

    Article  Google Scholar 

  21. Hernández, C., Schmitt, F., Cipolla, R.: Silhouette coherence for camera calibration under circular motion. IEEE Transaction on Pattern Analysis and Machine Intelligence 29(2), 343–349 (2007)

    Article  Google Scholar 

  22. Hernández, C., Vogiatzis, G., Cipolla, R.: Probabilistic visibility for multi-view stereo. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (2007)

    Google Scholar 

  23. Hernández, C., Vogiatzis, G., Cipolla, R.: Multi-view photometric stereo. IEEE Transaction on Pattern Analysis Machine Intelligence 30(1), 548–554 (2008)

    Article  Google Scholar 

  24. Hornung, A., Kobbelt, L.: Hierarchical volumetric multi-view stereo reconstruction of manifold surfaces based on dual graph embedding. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition, pp. 503–510 (2006)

    Google Scholar 

  25. Ilic, S., Fua, P.: Implicit meshes for surface reconstruction. IEEE Transactions on Pattern Analysis and Machine Intelligence 28(2), 328–333 (2006)

    Article  Google Scholar 

  26. Jaynes, E.: Probability Theory, The Logic of Science. Cambridge University Press, Cambridge (2003)

    MATH  Google Scholar 

  27. Kass, M., Witkin, A., Terzopoulos, D.: Snakes: Active contour models. International Journal of Computer Vision 1, 321–332 (1988)

    Article  Google Scholar 

  28. Kobbelt, L.: \(\sqrt{3}\)-subdivision. In: Proceedings of the ACM SIGGRAPH, pp. 103–112 (2000)

    Google Scholar 

  29. Kolev, K., Cremers, D.: Integration of multiview stereo and silhouettes via convex functionals on convex domains. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part I. LNCS, vol. 5302, pp. 752–765. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  30. Kolmogorov, V.: Convergent tree-reweighted message passing for energy minimization. IEEE Transaction on Pattern Analysis and Machine Intelligence 28(10), 1568–1583 (2006)

    Article  Google Scholar 

  31. Kolmogorov, V., Zabih, R.: What energy functions can be minimized via graph cuts. IEEE Transaction on Pattern Analysis and Machine Intelligence 26(2), 147–159 (2004)

    Article  Google Scholar 

  32. Kutulakos, K.N., Seitz, S.M.: A theory of shape by space carving. International Journal of Computer Vision 38(3), 199–218 (2000)

    Article  MATH  Google Scholar 

  33. Lempitsky, V., Boykov, Y., Ivanov, D.: Oriented visibility for multiview reconstruction. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3953, pp. 226–238. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  34. Levoy, M., Pulli, K., Curless, B., Rusinkiewicz, S., Koller, D., Pereira, L., Ginzton, M., Anderson, S., Davis, J., Ginsberg, J., Shade, J., Fulk, D.: The digital michelangelo project: 3d scanning of large statues. In: Proceedings of the ACM SIGGRAPH, pp. 15–22 (2000)

    Google Scholar 

  35. Lhuillier, M., Quan, L.: A quasi-dense approach to surface reconstruction from uncalibrated images. IEEE Transaction on Pattern Analysis and Machine Intelligence 27(3), 418–433 (2005)

    Article  Google Scholar 

  36. Goesele, M., Snavely, N., Curless, B., Hoppe, H., Seitz, S. M.: Multi-view stereo for community photo collections. In: Proceedings of the International Conference on Computer Vision (2007)

    Google Scholar 

  37. Pollefeys, M., Gool, L.J.V., Vergauwen, M., Verbiest, F., Cornelis, K., Tops, J., Koch, R.: Visual modeling with a hand-held camera. International Journal of Computer Vision 59(3), 207–232 (2004)

    Article  Google Scholar 

  38. Pollefeys, M., Nistér, D., Frahm, J.M., Akbarzadeh, A., Mordohai, P., Clipp, B., Engels, C., Gallup, D., Kim, S.J., Merrell, P., Salmi, C., Sinha, S., Talton, B., Wang, L., Yang, Q., Stewénius, H., Yang, R., Welch, G., Towles, H.: Detailed real-time urban 3d reconstruction from video. International Journal of Computer Vision 78(2-3), 143–167 (2008)

    Article  Google Scholar 

  39. Pons, J.P., Keriven, R., Faugeras, O.: Multi-view stereo reconstruction and scene flow estimation with a global image-based matching score. The International Journal of Computer Vision 72(2), 179–193 (2007)

    Article  Google Scholar 

  40. Seitz, S., Curless, B., Diebel, J., Scharstein, D., Szeliski, R.: A comparison and evaluation of multi-view stereo reconstruction algorithms. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition, pp. 519–528 (2006)

    Google Scholar 

  41. Sinha, S., Pollefeys, M.: Multi-view reconstruction using photo-consistency and exact silhouette constraints: A maximum-flow formulation. In: Proceedings of the International Conference on Computer Vision, pp. 349–356 (2005)

    Google Scholar 

  42. Sinha, S.N., Mordohai, P., Pollefeys, M.: Multi-view stereo via graph cuts on the dual of an adaptive tetrahedral mesh. In: Proceedings of the International Conference on Computer Vision (2007)

    Google Scholar 

  43. Snavely, N., Seitz, S.M., Szeliski, R.: Photo tourism: Exploring image collections in 3d. In: Proceedings of the ACM SIGGRAPH (2006)

    Google Scholar 

  44. Steger, E., Kutulakos, K.N.: A theory of refractive and specular 3d shape by light-path triangulation. International Journal of Computer Vision 76(1) (2008)

    Google Scholar 

  45. Strecha, C., von Hansen, W., Gool, L.V., Fua, P., Thoennessen, U.: On benchmarking camera calibration and multi-view stereo for high resolution imagery. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (2008)

    Google Scholar 

  46. Tran, S., Davis, L.: 3d surface reconstruction using graph cuts with surface constraints. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3952, pp. 218–231. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  47. Vogiatzis, G., Hernández, C., Torr, P., Cipolla, R.: Multi-view stereo via volumetric graph-cuts and occlusion robust photo-consistency. IEEE Transaction on Pattern Analysis and Machine Intelligence 29(12), 2241–2246 (2007)

    Article  Google Scholar 

  48. Vogiatzis, G., Torr, P., Cipolla, R.: Multi-view stereo via volumetric graph-cuts. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition, pp. 391–398 (2005)

    Google Scholar 

  49. Weise, T., Leibe, B., Gool, L.V.: Fast 3d scanning with automatic motion compensation. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (2007)

    Google Scholar 

  50. Witkin, A.: Recovering surface shape and orientation from texture. Artificial Intelligence 17(1-3), 17–45 (1981)

    Article  Google Scholar 

  51. Woodham, R.: Photometric method for determining surface orientation from multiple images. Optical Engineering 19(1), 139–144 (1980)

    Google Scholar 

  52. Xu, C., Prince, J.L.: Snakes, shapes, and gradient vector flow. IEEE Transactions on Image Processing, 359–369 (1998)

    Google Scholar 

  53. Zebedin, L., Bauer, J., Karner, K., Bischof, H.: Fusion of feature- and area-based information for urban buildings modeling from aerial imagery. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition, pp. 873–886 (2008)

    Google Scholar 

  54. Zhang, L., Snavely, N., Curless, B., Seitz, S.M.: Spacetime faces: High-resolution capture for modeling and animation. In: ACM Annual Conference on Computer Graphics, pp. 548–558 (2004)

    Google Scholar 

  55. Zisserman, A., Hartley, R.: Multiple View Geometry. Springer, Heidelberg (2000)

    MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Toshiba Research Cambridge, UK

    Carlos Hernández

  2. Aston University, Birmingham, UK

    George Vogiatzis

Authors
  1. Carlos Hernández
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. George Vogiatzis
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Engineering, University of Cambridge, CB2 1PZ, Cambridge, UK

    Roberto Cipolla

  2. Dipartimento di Matematica ed Informatica, University of Catania, Viale A. Doria 6, I, 95125, Catania, Italy

    Sebastiano Battiato  & Giovanni Maria Farinella  & 

Rights and permissions

Reprints and permissions

Copyright information

© 2010 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Hernández, C., Vogiatzis, G. (2010). Shape from Photographs: A Multi-view Stereo Pipeline. In: Cipolla, R., Battiato, S., Farinella, G.M. (eds) Computer Vision. Studies in Computational Intelligence, vol 285. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-12848-6_11

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-12848-6_11

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-12847-9

  • Online ISBN: 978-3-642-12848-6

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation