Visibility Subspaces: Uncalibrated Photometric Stereo with Shadows

  • Kalyan Sunkavalli
  • Todd Zickler
  • Hanspeter Pfister
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6312)


Photometric stereo relies on inverting the image formation process, and doing this accurately requires reasoning about the visibility of light sources with respect to each image point. While simple heuristics for shadow detection suffice in some cases, they are susceptible to error. This paper presents an alternative approach for handling visibility in photometric stereo, one that is suitable for uncalibrated settings where the light directions are not known. A surface imaged under a finite set of light sources can be divided into regions having uniform visibility, and when the surface is Lambertian, these regions generally map to distinct three-dimensional illumination subspaces. We show that by identifying these subspaces, we can locate the regions and their visibilities, and in the process identify shadows. The result is an automatic method for uncalibrated Lambertian photometric stereo in the presence of shadows, both cast and attached.


Subspace Cluster Cast Shadow Motion Segmentation Photometric Stereo Shadow Detection 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Alldrin, N., Kriegman, D.: Toward reconstructing surfaces with arbitrary isotropic reflectance: A stratified photometric stereo approach. In: Proc. IEEE Int. Conf. Computer Vision (2007)Google Scholar
  2. 2.
    Alldrin, N., Mallick, S., Kriegman, D.: Resolving the generalized bas-relief ambiguity by entropy minimization. In: Proc. IEEE Conf. Computer Vision and Pattern Recognition (2007)Google Scholar
  3. 3.
    Barsky, S., Petrou, M.: The 4-source photometric stereo technique for three-dimensional surfaces in the presence of highlights and shadows. IEEE Trans. Pattern Anal. Mach. Intell. 25, 1239–1252 (2003)CrossRefGoogle Scholar
  4. 4.
    Basri, R., Jacobs, D., Kemelmacher, I.: Photometric stereo with general, unknown lighting. Int. Journal of Computer Vision 72(3), 239–257 (2007)CrossRefGoogle Scholar
  5. 5.
    Basri, R., Jacobs, D.W.: Lambertian reflectance and linear subspaces. IEEE Trans. Pattern Anal. Mach. Intell. 25(2), 218–233 (2003)CrossRefGoogle Scholar
  6. 6.
    Belhumeur, P.N., Kriegman, D.J.: What is the set of images of an object under all possible illumination conditions? Int. Journal of Computer Vision 28(3), 245–260 (1998)CrossRefGoogle Scholar
  7. 7.
    Belhumeur, P.N., Kriegman, D.J., Yuille, A.L.: The bas-relief ambiguity. Int. Journal of Computer Vision 35(1), 33–44 (1999)CrossRefGoogle Scholar
  8. 8.
    Chandraker, M., Agarwal, S., Kriegman, D.: Shadowcuts: Photometric stereo with shadows. In: Proc. IEEE Conf. Computer Vision and Pattern Recognition (2007)Google Scholar
  9. 9.
    Chandraker, M., Kahl, F., Kriegman, D.: Reflections on the generalized bas-relief ambiguity. In: Proc. IEEE Conf. Computer Vision and Pattern Recognition (2005)Google Scholar
  10. 10.
    Coleman, E., Jain, R.: Obtaining 3-dimensional shape of textured and specular surfaces using four-source photometry. Computer Graphics and Image Processing 18(4), 309–328 (1982)CrossRefGoogle Scholar
  11. 11.
    Drbohlav, O., Chaniler, M.: Can two specular pixels calibrate photometric stereo? In: Proc. IEEE Int. Conf. Computer Vision (2005)Google Scholar
  12. 12.
    Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. ACM Commun. 24(6), 381–395 (1981)CrossRefMathSciNetGoogle Scholar
  13. 13.
    Garg, R., Du, H., Seitz, S.M., Snavely, N.: The dimensionality of scene appearance. In: Proc. IEEE Int. Conf. Computer Vision (2009)Google Scholar
  14. 14.
    Georghiades, A.: Incorporating the Torrance and Sparrow model of reflectance in uncalibrated photometric stereo. In: Proc. IEEE Int. Conf. Computer Vision (2003)Google Scholar
  15. 15.
    Hayakawa, H.: Photometric stereo under a light source with arbitrary motion. J. Opt. Soc. Am. 11(11) (1994)Google Scholar
  16. 16.
    Hernández, C., Vogiatzis, G., Cipolla, R.: Shadows in three-source photometric stereo. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part I. LNCS, vol. 5302, pp. 290–303. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  17. 17.
    Hernández Esteban, C., Vogiatzis, G., Cipolla, R.: Multiview photometric stereo. IEEE Trans. Pattern Anal. Mach. Intell. 30(3), 548–554 (2008)CrossRefGoogle Scholar
  18. 18.
    Hertzmann, A., Seitz, S.M.: Example-based photometric stereo: Shape reconstruction with general, varying brdfs. IEEE Trans. Pattern Anal. Mach. Intell. 27, 1254–1264 (2005)CrossRefGoogle Scholar
  19. 19.
    Holroyd, M., Lawrence, J., Humphreys, G., Zickler, T.: A photometric approach for estimating normals and tangents. ACM Trans. Graph. 27(5), 1–9 (2008)CrossRefGoogle Scholar
  20. 20.
    Ikeuchi, K.: Determining surface orientations of specular surfaces by using the photometric stereo method. IEEE Trans. Pattern Anal. Mach. Intell. 3(6), 661–669 (1981)CrossRefGoogle Scholar
  21. 21.
    Nayar, S., Ikeuchi, K., Kanade, T.: Determining shape and reflectance of hybrid surfaces by photometric sampling. IEEE T. Robotics Automation 6(4) (1990)Google Scholar
  22. 22.
    Ramamoorthi, R., Hanrahan, P.: On the relationship between radiance and irradiance: determining the illumination from images of a convex Lambertian object. J. Optical Society of America A 18(10), 2448–2458 (2001)CrossRefMathSciNetGoogle Scholar
  23. 23.
    Shashua, A.: On photometric issues in 3D visual recognition from a single 2D image. Int. Journal of Computer Vision 31(1), 99–122 (1997)CrossRefGoogle Scholar
  24. 24.
    Shi, B., Matsushita, Y., Wei, Y., Xu, C., Tan, P.: Self-calibrating photometric stereo. In: Proc. IEEE Conf. Computer Vision and Pattern Recognition (2010)Google Scholar
  25. 25.
    Tan, P., Zickler, T.: A projective framework for radiometric image analysis. In: Proc. IEEE Conf. Computer Vision and Pattern Recognition (2009)Google Scholar
  26. 26.
    Torr, P., Faugeras, O., Kanade, T., Hollinghurst, N., Lasenby, J., Sabin, M., Fitzgibbon, A.: Geometric motion segmentation and model selection (and discussion). Philosophical Transactions: Mathematical, Physical and Engineering Sciences 356(1740) (1998)Google Scholar
  27. 27.
    Tron, R., Vidal, R.: A benchmark for the comparison of 3-D motion segmentation algorithms. In: IEEE Conf. Computer Vision and Pattern Recognition (2007)Google Scholar
  28. 28.
    Vidal, R., Hartley, R.: Motion segmentation with missing data by powerfactorization and generalized PCA. In: IEEE Conf. Computer Vision and Pattern Recognition (2004)Google Scholar
  29. 29.
    Woodham, R.: Photometric stereo: A reflectance map technique for determining surface orientation from image intesity. In: Proc. SPIE, vol. 155, pp. 136–143 (1978)Google Scholar
  30. 30.
    Wu, T.P., Tang, C.K.: Photometric stereo via expectation maximization. IEEE Trans. Pattern Anal. Mach. Intell. 32(3), 546–560 (2010)CrossRefGoogle Scholar
  31. 31.
    Wu, T.P., Tang, K.L., Tang, C.K., Wong, T.T.: Dense photometric stereo: A markov random field approach. IEEE Trans. Pattern Anal. Mach. Intell. 28(11), 1830–1846 (2006)CrossRefGoogle Scholar
  32. 32.
    Yan, J., Pollefeys, M.: A general framework for motion segmentation: Independent, articulated, rigid, non-rigid, degenerate and non-degenerate. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3954, pp. 94–106. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  33. 33.
    Yuille, A., Snow, D.: Shape and albedo from multiple images using integrability. In: Proc. IEEE Conf. Computer Vision and Pattern Recognition (1997)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Kalyan Sunkavalli
    • 1
  • Todd Zickler
    • 1
  • Hanspeter Pfister
    • 1
  1. 1.Harvard UniversityCambridgeUSA

Personalised recommendations