Advertisement

Practical 3D Reconstruction Based on Photometric Stereo

  • George Vogiatzis
  • Carlos Hernández
Chapter
Part of the Studies in Computational Intelligence book series (SCI, volume 285)

Abstract

Photometric Stereo is a powerful image based 3D reconstruction technique that has recently been used to obtain very high quality reconstructions. However, in its classic form, Photometric Stereo suffers from two main limitations: Firstly, one needs to obtain images of the 3D scene under multiple different illuminations. As a result the 3D scene needs to remain static during illumination changes, which prohibits the reconstruction of deforming objects. Secondly, the images obtained must be from a single viewpoint. This leads to depth-map based 2.5 reconstructions, instead of full 3D surfaces. The aim of this Chapter is to show how these limitations can be alleviated, leading to the derivation of two practical 3D acquisition systems: The first one, based on the powerful Coloured Light Photometric Stereo method can be used to reconstruct moving objects such as cloth or human faces. The second, permits the complete 3D reconstruction of challenging objects such as porcelain vases. In addition to algorithmic details, the Chapter pays attention to practical issues such as setup calibration, detection and correction of self and cast shadows. We provide several evaluation experiments as well as reconstruction results.

Keywords

Light Direction Visual Hull Photometric Stereo Frontier Point High Quality Reconstruction 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Amit, G.F., Agrawal, A., Raskar, R.: What is the range of surface reconstructions from a gradient field? In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3951, pp. 578–591. Springer, Heidelberg (2006)Google Scholar
  2. 2.
    Barsky, S., Petrou, M.: The 4-source photometric stereo technique for three-dimensional surfaces in the presence of highlights and shadows. IEEE Transaction on Pattern Analysis and Machine Intelligence 25(10), 1239–1252 (2003)CrossRefGoogle Scholar
  3. 3.
    Bernardini, F., Rushmeier, H., Martin, I., Mittleman, J., Taubin, G.: Building a digital model of michelangelo’s florentine pieta. IEEE Computer Graphics and Applications 22(1), 59–67 (2002)CrossRefGoogle Scholar
  4. 4.
    Bertalmio, M., Sapiro, G., Caselles, V., Ballester, C.: Image inpainting. SIGGRAPH, 417–424 (2000)Google Scholar
  5. 5.
    Bhat, K.S., Twigg, C.D., Hodgins, J.K., Khosla, P.K., Popović, Z., Seitz, S.M.: Estimating cloth simulation parameters from video. In: Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer animation, pp. 37–51 (2003)Google Scholar
  6. 6.
    Chandraker, M., Agarwal, S., Kriegman, D.: Shadowcuts: Photometric stereo with shadows. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (2007)Google Scholar
  7. 7.
    Chen, H., Belhumeur, P., Jacobs, D.: In search of illumination invariants. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition, pp. 254–261 (2000)Google Scholar
  8. 8.
    Cipolla, R., Giblin, P.: Visual Motion of curves and surfaces. Cambridge University Press, Cambridge (1999)Google Scholar
  9. 9.
    Davis, T.A.: Algorithm 832: Umfpack, an unsymmetric-pattern multifrontal method. ACM Transactions on Mathematical Software 30(2), 196–199 (2004)zbMATHCrossRefGoogle Scholar
  10. 10.
    Dbrohlav, O., Chandler, M.: Can two specular pixels calibrate photometric stereo? In: Proceedings of the International Conference on Computer Vision (2005)Google Scholar
  11. 11.
    Drew, M.: Reduction of rank-reduced orientation-from-color problem with many unknown lights to two-image known-illuminant photometric stereo. In: Proceedings of the International Symposium on Computer Vision, pp. 419–424 (1995)Google Scholar
  12. 12.
    Fan, J., Wolff, L.B.: Surface curvature and shape reconstruction from unknown multiple illumination and integrability. Compututer Vision and Image Understanding 65(2), 347–359 (1997)CrossRefGoogle Scholar
  13. 13.
    Fischler, M.A., Bolles, R.C.: Ransac, random sampling consensus: a paradigm for model fitting with applications to image analysis and autoomated cartography. Communications of the ACM 26, 381–395 (1981)CrossRefMathSciNetGoogle Scholar
  14. 14.
    Goldman, D.B., Curless, B., Hertzmann, A., Seitz, S.M.: Shape and spatially-varying BRDFs from photometric stereo. In: Proceedings of the International Conference on Computer Vision, pp. 341–348 (2005)Google Scholar
  15. 15.
    Greig, D., Porteous, B., Seheult, A.: Exact maximum a posteriori estimation for binary images. Journal of the Royal Statistical Society 51(2), 271–279 (1989)Google Scholar
  16. 16.
    Gu, X., Zhang, S., Huang, P., Zhang, L., Yau, S.T., Martin, R.: Holoimages. In: Proceedings of the ACM Symposium on Solid and Physical Modeling, pp. 129–138 (2006)Google Scholar
  17. 17.
    Hernández, C., Schmitt, F.: Silhouette and stereo fusion for 3d object modeling. Computer Vision and Image Understanding 96(3), 367–392 (2004)CrossRefGoogle Scholar
  18. 18.
    Hernández, C., Schmitt, F., Cipolla, R.: Silhouette coherence for camera calibration under circular motion. IEEE Transaction on Pattern Analysis and Machine Intelligence 29(2), 343–349 (2007)CrossRefGoogle Scholar
  19. 19.
    Hernández, C., Vogiatzis, G., Brostow, G., Stenger, B., Cipolla, R.: Non-rigid photometric stereo with colored lights. In: Proceedings of the International Conference on Computer Vision (2007)Google Scholar
  20. 20.
    Hertzmann, A., Seitz, S.: Shape and materials by example: a photometric stereo approach. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition, pp. 533–540 (2003)Google Scholar
  21. 21.
    Hertzmann, A., Seitz, S.: Shape reconstruction with general, varying brdfs. IEEE Transaction on Pattern Analysis and Machine Intelligence 27(8), 1254–1264 (2005)CrossRefGoogle Scholar
  22. 22.
    Horn, B.K.P.: Robot vision. MIT Press, Cambridge (1986)Google Scholar
  23. 23.
    Jin, H., Cremers, D., Yezzi, A., Soatto, S.: Shedding light in stereoscopic segmentation. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition, pp. 36–42 (2004)Google Scholar
  24. 24.
    Kolmogorov, V.: Convergent tree-reweighted message passing for energy minimization. IEEE Transaction on Pattern Analysis and Machine Intelligence 28(10), 1568–1583 (2006)CrossRefGoogle Scholar
  25. 25.
    Kontsevich, L., Petrov, A., Vergelskaya, I.: Reconstruction of shape from shading in color images. Journal of the Optical Society of America A 11(3), 1047–1052 (1994)CrossRefGoogle Scholar
  26. 26.
    Laurentini, A.: The visual hull concept for silhouette-based image understanding. IEEE Transaction on Pattern Analysis and Machine Intelligence 16(2) (1994)Google Scholar
  27. 27.
    Levoy, M.: Why is 3d scanning hard? In: Invited address at 3D Processing, Visualization, Transmission, Padua, Italy (2002)Google Scholar
  28. 28.
    Levoy, M., Pulli, K., Curless, B., Rusinkiewicz, S., Koller, D., Pereira, L., Ginzton, M., Anderson, S., Davis, J., Ginsberg, J., Shade, J., Fulk, D.: The digital michelangelo project: 3d scanning of large statues. In: Proceedings of the ACM SIGGRAPH, pp. 15–22 (2000)Google Scholar
  29. 29.
    Lim, J., Ho, J., Yang, M.H., Kriegman, D.: Passive photometric stereo from motion. In: Proceedings of the International Conference on Computer Vision, pp. 1635–1642 (2005)Google Scholar
  30. 30.
    Lin, S., Lee, S.: Estimation of diffuse and specular appearance. In: Proceedings of the International Conference on Computer Vision, pp. 855–860 (1999)Google Scholar
  31. 31.
    Malzbender, T., Wilburn, B., Gelb, D., Ambrisco, B.: Surface enhancement using real-time photometric stereo and reflectance transformation. In: Proceedings of the Eurographics Symposium on Rendering (2006)Google Scholar
  32. 32.
    Nayar, S., Ikeuchi, K., Kanade, T.: Surface reflection: physical and geometrical perspectives. IEEE Transaction on Pattern Analysis and Machine Intelligence 13(7), 611–634 (1991)CrossRefGoogle Scholar
  33. 33.
    Nehab, D., Rusinkiewicz, S., Davis, J., Ramamoorthi, R.: Efficiently combining positions and normals for precise 3d geometry. In: Proceedings of the ACM SIGGRAPH, pp. 536–543 (2005)Google Scholar
  34. 34.
    North Coleman Jr., E., Jain, R.: Obtaining 3-dimensional shape of textured and specular surfaces using four-source photometry. In: Shape recovery, pp. 180–199. Jones and Bartlett Publishers, Inc., USA (1992)Google Scholar
  35. 35.
    Onn, R., Bruckstein, A.: Integrability disambiguates surface recovery in two-image photometric stereo. International Journal of Computer Vision 5(1), 105–113 (1990)CrossRefGoogle Scholar
  36. 36.
    Paterson, J., Claus, D., Fitzgibbon, A.: Brdf and geometry capture from extended inhomogeneous samples using flash photography. In: Proceedings of Eurographics (2005)Google Scholar
  37. 37.
    Petrov, A.: Light, color and shape. Cognitive Processes and their Simulation (in Russian), 350–358 (1987)Google Scholar
  38. 38.
    Pilet, J., Lepetit, V., Fua, P.: Real-time non-rigid surface detection. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (2005)Google Scholar
  39. 39.
    Salzmann, M., Ilic, S., Fua, P.: Physically valid shape parameterization for monocular 3-d deformable surface tracking. In: Proceedings of the British Machine Vision Conference (2005)Google Scholar
  40. 40.
    Sand, P., McMillan, L., Popović, J.: Continuous capture of skin deformation. ACM Transaction on Graphics 22(3), 578–586 (2003)CrossRefGoogle Scholar
  41. 41.
    Scholz, V., Stich, T., Keckeisen, M., Wacker, M., Magnor, M.: Garment motion capture using color-coded patterns. Computer Graphics Forum 24(3), 439–448 (2005)CrossRefGoogle Scholar
  42. 42.
    Seitz, S., Curless, B., Diebel, J., Scharstein, D., Szeliski, R.: A comparison and evaluation of multi-view stereo reconstruction algorithms. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition, pp. 519–528 (2006)Google Scholar
  43. 43.
    Tankus, A., Kiryati, N.: Photometric stereo under perspective projection. In: Proceedings of the International Conference on Computer Vision (2005)Google Scholar
  44. 44.
    Treuille, A., Hertzmann, A., Seitz, S.: Example-based stereo with general brdfs. In: Pajdla, T., Matas, J(G.) (eds.) ECCV 2004. LNCS, vol. 3022, pp. 457–469. Springer, Heidelberg (2004)Google Scholar
  45. 45.
    Vogiatzis, G., Favaro, P., Cipolla, R.: Using frontier points to recover shape, reflectance and illumination. In: Proceedings of the International Conference on Computer Vision, pp. 228–235 (2005)Google Scholar
  46. 46.
    Vogiatzis, G., Hernández, C., Cipolla, R.: Reconstruction in the round using photometric normals and silhouettes. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition, pp. 1847–1854 (2006)Google Scholar
  47. 47.
    Weise, T., Leibe, B., Gool, L.V.: Fast 3d scanning with automatic motion compensation. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (2007)Google Scholar
  48. 48.
    White, R., Forsyth, D.: Combining cues: Shape from shading and texture. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition, pp. 1809–1816 (2006)Google Scholar
  49. 49.
    White, R., Forsyth, D.: Retexturing single views using texture and shading. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3954, pp. 70–81. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  50. 50.
    Wolff, L.B., Angelopoulou, E.: 3d stereo using photometric ratios. In: Eklundh, J.-O. (ed.) ECCV 1994. LNCS, vol. 801, pp. 247–258. Springer, Heidelberg (1994)CrossRefGoogle Scholar
  51. 51.
    Woodham, R.: Photometric method for determining surface orientation from multiple images. Optical Engineering 19(1), 139–144 (1980)Google Scholar
  52. 52.
    Yuille, A., Snow, D.: Shape and albedo from multiple images using integrability. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (1997)Google Scholar
  53. 53.
    Zhang, L., Snavely, N., Curless, B., Seitz, S.M.: Spacetime faces: High-resolution capture for modeling and animation. In: Proceedings of ACM Annual Conference on Computer Graphics, pp. 548–558 (2004)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • George Vogiatzis
    • 1
  • Carlos Hernández
    • 2
  1. 1.Aston UniversityBirminghamUK
  2. 2.Toshiba Research CambridgeUK

Personalised recommendations