Skip to main content
SpringerLink
Log in

Practical 3D Reconstruction Based on Photometric Stereo

  • Chapter
Computer Vision

Part of the book series: Studies in Computational Intelligence ((SCI,volume 285))

Abstract

Photometric Stereo is a powerful image based 3D reconstruction technique that has recently been used to obtain very high quality reconstructions. However, in its classic form, Photometric Stereo suffers from two main limitations: Firstly, one needs to obtain images of the 3D scene under multiple different illuminations. As a result the 3D scene needs to remain static during illumination changes, which prohibits the reconstruction of deforming objects. Secondly, the images obtained must be from a single viewpoint. This leads to depth-map based 2.5 reconstructions, instead of full 3D surfaces. The aim of this Chapter is to show how these limitations can be alleviated, leading to the derivation of two practical 3D acquisition systems: The first one, based on the powerful Coloured Light Photometric Stereo method can be used to reconstruct moving objects such as cloth or human faces. The second, permits the complete 3D reconstruction of challenging objects such as porcelain vases. In addition to algorithmic details, the Chapter pays attention to practical issues such as setup calibration, detection and correction of self and cast shadows. We provide several evaluation experiments as well as reconstruction results.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 219.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Amit, G.F., Agrawal, A., Raskar, R.: What is the range of surface reconstructions from a gradient field? In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3951, pp. 578–591. Springer, Heidelberg (2006)

    Google Scholar 

  2. Barsky, S., Petrou, M.: The 4-source photometric stereo technique for three-dimensional surfaces in the presence of highlights and shadows. IEEE Transaction on Pattern Analysis and Machine Intelligence 25(10), 1239–1252 (2003)

    Article  Google Scholar 

  3. Bernardini, F., Rushmeier, H., Martin, I., Mittleman, J., Taubin, G.: Building a digital model of michelangelo’s florentine pieta. IEEE Computer Graphics and Applications 22(1), 59–67 (2002)

    Article  Google Scholar 

  4. Bertalmio, M., Sapiro, G., Caselles, V., Ballester, C.: Image inpainting. SIGGRAPH, 417–424 (2000)

    Google Scholar 

  5. Bhat, K.S., Twigg, C.D., Hodgins, J.K., Khosla, P.K., Popović, Z., Seitz, S.M.: Estimating cloth simulation parameters from video. In: Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer animation, pp. 37–51 (2003)

    Google Scholar 

  6. Chandraker, M., Agarwal, S., Kriegman, D.: Shadowcuts: Photometric stereo with shadows. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (2007)

    Google Scholar 

  7. Chen, H., Belhumeur, P., Jacobs, D.: In search of illumination invariants. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition, pp. 254–261 (2000)

    Google Scholar 

  8. Cipolla, R., Giblin, P.: Visual Motion of curves and surfaces. Cambridge University Press, Cambridge (1999)

    Google Scholar 

  9. Davis, T.A.: Algorithm 832: Umfpack, an unsymmetric-pattern multifrontal method. ACM Transactions on Mathematical Software 30(2), 196–199 (2004)

    Article  MATH  Google Scholar 

  10. Dbrohlav, O., Chandler, M.: Can two specular pixels calibrate photometric stereo? In: Proceedings of the International Conference on Computer Vision (2005)

    Google Scholar 

  11. Drew, M.: Reduction of rank-reduced orientation-from-color problem with many unknown lights to two-image known-illuminant photometric stereo. In: Proceedings of the International Symposium on Computer Vision, pp. 419–424 (1995)

    Google Scholar 

  12. Fan, J., Wolff, L.B.: Surface curvature and shape reconstruction from unknown multiple illumination and integrability. Compututer Vision and Image Understanding 65(2), 347–359 (1997)

    Article  Google Scholar 

  13. Fischler, M.A., Bolles, R.C.: Ransac, random sampling consensus: a paradigm for model fitting with applications to image analysis and autoomated cartography. Communications of the ACM 26, 381–395 (1981)

    Article  MathSciNet  Google Scholar 

  14. Goldman, D.B., Curless, B., Hertzmann, A., Seitz, S.M.: Shape and spatially-varying BRDFs from photometric stereo. In: Proceedings of the International Conference on Computer Vision, pp. 341–348 (2005)

    Google Scholar 

  15. Greig, D., Porteous, B., Seheult, A.: Exact maximum a posteriori estimation for binary images. Journal of the Royal Statistical Society 51(2), 271–279 (1989)

    Google Scholar 

  16. Gu, X., Zhang, S., Huang, P., Zhang, L., Yau, S.T., Martin, R.: Holoimages. In: Proceedings of the ACM Symposium on Solid and Physical Modeling, pp. 129–138 (2006)

    Google Scholar 

  17. Hernández, C., Schmitt, F.: Silhouette and stereo fusion for 3d object modeling. Computer Vision and Image Understanding 96(3), 367–392 (2004)

    Article  Google Scholar 

  18. Hernández, C., Schmitt, F., Cipolla, R.: Silhouette coherence for camera calibration under circular motion. IEEE Transaction on Pattern Analysis and Machine Intelligence 29(2), 343–349 (2007)

    Article  Google Scholar 

  19. Hernández, C., Vogiatzis, G., Brostow, G., Stenger, B., Cipolla, R.: Non-rigid photometric stereo with colored lights. In: Proceedings of the International Conference on Computer Vision (2007)

    Google Scholar 

  20. Hertzmann, A., Seitz, S.: Shape and materials by example: a photometric stereo approach. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition, pp. 533–540 (2003)

    Google Scholar 

  21. Hertzmann, A., Seitz, S.: Shape reconstruction with general, varying brdfs. IEEE Transaction on Pattern Analysis and Machine Intelligence 27(8), 1254–1264 (2005)

    Article  Google Scholar 

  22. Horn, B.K.P.: Robot vision. MIT Press, Cambridge (1986)

    Google Scholar 

  23. Jin, H., Cremers, D., Yezzi, A., Soatto, S.: Shedding light in stereoscopic segmentation. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition, pp. 36–42 (2004)

    Google Scholar 

  24. Kolmogorov, V.: Convergent tree-reweighted message passing for energy minimization. IEEE Transaction on Pattern Analysis and Machine Intelligence 28(10), 1568–1583 (2006)

    Article  Google Scholar 

  25. Kontsevich, L., Petrov, A., Vergelskaya, I.: Reconstruction of shape from shading in color images. Journal of the Optical Society of America A 11(3), 1047–1052 (1994)

    Article  Google Scholar 

  26. Laurentini, A.: The visual hull concept for silhouette-based image understanding. IEEE Transaction on Pattern Analysis and Machine Intelligence 16(2) (1994)

    Google Scholar 

  27. Levoy, M.: Why is 3d scanning hard? In: Invited address at 3D Processing, Visualization, Transmission, Padua, Italy (2002)

    Google Scholar 

  28. Levoy, M., Pulli, K., Curless, B., Rusinkiewicz, S., Koller, D., Pereira, L., Ginzton, M., Anderson, S., Davis, J., Ginsberg, J., Shade, J., Fulk, D.: The digital michelangelo project: 3d scanning of large statues. In: Proceedings of the ACM SIGGRAPH, pp. 15–22 (2000)

    Google Scholar 

  29. Lim, J., Ho, J., Yang, M.H., Kriegman, D.: Passive photometric stereo from motion. In: Proceedings of the International Conference on Computer Vision, pp. 1635–1642 (2005)

    Google Scholar 

  30. Lin, S., Lee, S.: Estimation of diffuse and specular appearance. In: Proceedings of the International Conference on Computer Vision, pp. 855–860 (1999)

    Google Scholar 

  31. Malzbender, T., Wilburn, B., Gelb, D., Ambrisco, B.: Surface enhancement using real-time photometric stereo and reflectance transformation. In: Proceedings of the Eurographics Symposium on Rendering (2006)

    Google Scholar 

  32. Nayar, S., Ikeuchi, K., Kanade, T.: Surface reflection: physical and geometrical perspectives. IEEE Transaction on Pattern Analysis and Machine Intelligence 13(7), 611–634 (1991)

    Article  Google Scholar 

  33. Nehab, D., Rusinkiewicz, S., Davis, J., Ramamoorthi, R.: Efficiently combining positions and normals for precise 3d geometry. In: Proceedings of the ACM SIGGRAPH, pp. 536–543 (2005)

    Google Scholar 

  34. North Coleman Jr., E., Jain, R.: Obtaining 3-dimensional shape of textured and specular surfaces using four-source photometry. In: Shape recovery, pp. 180–199. Jones and Bartlett Publishers, Inc., USA (1992)

    Google Scholar 

  35. Onn, R., Bruckstein, A.: Integrability disambiguates surface recovery in two-image photometric stereo. International Journal of Computer Vision 5(1), 105–113 (1990)

    Article  Google Scholar 

  36. Paterson, J., Claus, D., Fitzgibbon, A.: Brdf and geometry capture from extended inhomogeneous samples using flash photography. In: Proceedings of Eurographics (2005)

    Google Scholar 

  37. Petrov, A.: Light, color and shape. Cognitive Processes and their Simulation (in Russian), 350–358 (1987)

    Google Scholar 

  38. Pilet, J., Lepetit, V., Fua, P.: Real-time non-rigid surface detection. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (2005)

    Google Scholar 

  39. Salzmann, M., Ilic, S., Fua, P.: Physically valid shape parameterization for monocular 3-d deformable surface tracking. In: Proceedings of the British Machine Vision Conference (2005)

    Google Scholar 

  40. Sand, P., McMillan, L., Popović, J.: Continuous capture of skin deformation. ACM Transaction on Graphics 22(3), 578–586 (2003)

    Article  Google Scholar 

  41. Scholz, V., Stich, T., Keckeisen, M., Wacker, M., Magnor, M.: Garment motion capture using color-coded patterns. Computer Graphics Forum 24(3), 439–448 (2005)

    Article  Google Scholar 

  42. Seitz, S., Curless, B., Diebel, J., Scharstein, D., Szeliski, R.: A comparison and evaluation of multi-view stereo reconstruction algorithms. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition, pp. 519–528 (2006)

    Google Scholar 

  43. Tankus, A., Kiryati, N.: Photometric stereo under perspective projection. In: Proceedings of the International Conference on Computer Vision (2005)

    Google Scholar 

  44. Treuille, A., Hertzmann, A., Seitz, S.: Example-based stereo with general brdfs. In: Pajdla, T., Matas, J(G.) (eds.) ECCV 2004. LNCS, vol. 3022, pp. 457–469. Springer, Heidelberg (2004)

    Google Scholar 

  45. Vogiatzis, G., Favaro, P., Cipolla, R.: Using frontier points to recover shape, reflectance and illumination. In: Proceedings of the International Conference on Computer Vision, pp. 228–235 (2005)

    Google Scholar 

  46. Vogiatzis, G., Hernández, C., Cipolla, R.: Reconstruction in the round using photometric normals and silhouettes. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition, pp. 1847–1854 (2006)

    Google Scholar 

  47. Weise, T., Leibe, B., Gool, L.V.: Fast 3d scanning with automatic motion compensation. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (2007)

    Google Scholar 

  48. White, R., Forsyth, D.: Combining cues: Shape from shading and texture. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition, pp. 1809–1816 (2006)

    Google Scholar 

  49. White, R., Forsyth, D.: Retexturing single views using texture and shading. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3954, pp. 70–81. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  50. Wolff, L.B., Angelopoulou, E.: 3d stereo using photometric ratios. In: Eklundh, J.-O. (ed.) ECCV 1994. LNCS, vol. 801, pp. 247–258. Springer, Heidelberg (1994)

    Chapter  Google Scholar 

  51. Woodham, R.: Photometric method for determining surface orientation from multiple images. Optical Engineering 19(1), 139–144 (1980)

    Google Scholar 

  52. Yuille, A., Snow, D.: Shape and albedo from multiple images using integrability. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (1997)

    Google Scholar 

  53. Zhang, L., Snavely, N., Curless, B., Seitz, S.M.: Spacetime faces: High-resolution capture for modeling and animation. In: Proceedings of ACM Annual Conference on Computer Graphics, pp. 548–558 (2004)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Aston University, Birmingham, UK

    George Vogiatzis

  2. Toshiba Research Cambridge, UK

    Carlos Hernández

Authors
  1. George Vogiatzis
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Carlos Hernández
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Engineering, University of Cambridge, CB2 1PZ, Cambridge, UK

    Roberto Cipolla

  2. Dipartimento di Matematica ed Informatica, University of Catania, Viale A. Doria 6, I, 95125, Catania, Italy

    Sebastiano Battiato  & Giovanni Maria Farinella  & 

Rights and permissions

Reprints and permissions

Copyright information

© 2010 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Vogiatzis, G., Hernández, C. (2010). Practical 3D Reconstruction Based on Photometric Stereo. In: Cipolla, R., Battiato, S., Farinella, G.M. (eds) Computer Vision. Studies in Computational Intelligence, vol 285. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-12848-6_12

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-12848-6_12

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-12847-9

  • Online ISBN: 978-3-642-12848-6

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation