Journal of Mathematical Imaging and Vision

, Volume 60, Issue 4, pp 563–575 | Cite as

Solving Uncalibrated Photometric Stereo Using Fewer Images by Jointly Optimizing Low-rank Matrix Completion and Integrability

  • Soumyadip Sengupta
  • Hao Zhou
  • Walter Forkel
  • Ronen Basri
  • Tom Goldstein
  • David Jacobs
Article
  • 91 Downloads

Abstract

We introduce a new, integrated approach to uncalibrated photometric stereo. We perform 3D reconstruction of Lambertian objects using multiple images produced by unknown, directional light sources. We show how to formulate a single optimization that includes rank and integrability constraints, allowing also for missing data. We then solve this optimization using the Alternating Direction Method of Multipliers (ADMM). We conduct extensive experimental evaluation on real and synthetic data sets. Our integrated approach is particularly valuable when performing photometric stereo using as few as 4–6 images, since the integrability constraint is capable of improving estimation of the linear subspace of possible solutions. We show good improvements over prior work in these cases.

Keywords

Photometric stereo 3D reconstruction Low-rank optimization 

References

  1. 1.
    Ackermann, J., Goesele, M.: A survey of photometric stereo techniques. Found. Trends Comput. Graph. Vis. 9(3–4), 149–254 (2015)CrossRefMATHGoogle Scholar
  2. 2.
    Alldrin, N.G., Mallick, S.P., Kriegman, D.J.: Resolving the generalized bas-relief ambiguity by entropy minimization. In: IEEE Conference on Computer Vision and Pattern Recognition, 2007. CVPR’07, IEEE, pp. 1–7 (2007)Google Scholar
  3. 3.
    Basri, R., Jacobs, D., Kemelmacher, I.: Photometric stereo with general, unknown lighting. Int. J. Comput. Vis. 72(3), 239–257 (2007)CrossRefGoogle Scholar
  4. 4.
    Belhumeur, P.N., Kriegman, D.J., Yuille, A.L.: The bas-relief ambiguity. Int. J. Comput. Vis. 35(1), 33–44 (1999)CrossRefGoogle Scholar
  5. 5.
    Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. 3(1), 1–122 (2011)CrossRefMATHGoogle Scholar
  6. 6.
    Cabral, R.S., Torre, F., Costeira, J.P., Bernardino, A.: Matrix completion for multi-label image classification. In: Advances in Neural Information Processing Systems, pp. 190–198 (2011)Google Scholar
  7. 7.
    Cai, J.F., Candès, E.J., Shen, Z.: A singular value thresholding algorithm for matrix completion. SIAM J. Optim. 20(4), 1956–1982 (2010)MathSciNetCrossRefMATHGoogle Scholar
  8. 8.
    Chandraker, M., Agarwal, S., Kriegman, D.: Shadowcuts: Photometric stereo with shadows. In: IEEE Conference on Computer Vision and Pattern Recognition, 2007. CVPR’07. IEEE, pp. 1–8 (2007)Google Scholar
  9. 9.
    Chandraker, M.K., Kahl, C.F., Kriegman, D.J.: Reflections on the generalized bas-relief ambiguity. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2005. CVPR 2005, IEEE, vol. 1, pp. 788–795 (2005)Google Scholar
  10. 10.
    Drbohlav, O., Chaniler, M.: Can two specular pixels calibrate photometric stereo? In: Tenth IEEE International Conference on Computer Vision, 2005. ICCV 2005, IEEE, vol. 2, pp. 1850–1857 (2005)Google Scholar
  11. 11.
    Favaro, P., Papadhimitri, T.: A closed-form solution to uncalibrated photometric stereo via diffuse maxima. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, pp. 821–828 (2012)Google Scholar
  12. 12.
    Georghiades, A.S.: Incorporating the torrance and sparrow model of reflectance in uncalibrated photometric stereo. In: Proceedings of Ninth IEEE International Conference on Computer Vision, 2003, IEEE, pp. 816–823 (2003)Google Scholar
  13. 13.
    Goldstein, T., O’Donoghue, B., Setzer, S., Baraniuk, R.: Fast alternating direction optimization methods. SIAM J. Imaging Sci. 7(3), 1588–1623 (2014)MathSciNetCrossRefMATHGoogle Scholar
  14. 14.
    Hayakawa, H.: Photometric stereo under a light source with arbitrary motion. JOSA A 11(11), 3079–3089 (1994)CrossRefGoogle Scholar
  15. 15.
    Hu, Y., Zhang, D., Ye, J., Li, X., He, X.: Fast and accurate matrix completion via truncated nuclear norm regularization. IEEE Trans. Pattern Anal. Mach. Intell. 35(9), 2117–2130 (2013)CrossRefGoogle Scholar
  16. 16.
    Joshi, N., Kemelmacher, I., Simon, I.: Photometric stereo dataset. http://courses.cs.washington.edu/courses/cse455/10wi/projects/project4/ (2015)
  17. 17.
    Mecca, R., Tankus, A., Wetzler, A., Bruckstein, A.M.: A direct differential approach to photometric stereo with perspective viewing. SIAM J. Imaging Sci. 7(2), 579–612 (2014)MathSciNetCrossRefMATHGoogle Scholar
  18. 18.
    Oh, T.H., Kim, H., Tai, Y.W., Bazin, J.C., Kweon, I.S.: Partial sum minimization of singular values in RPCA for low-level vision. In: 2013 IEEE International Conference on Computer Vision (ICCV), IEEE, pp. 145–152 (2013)Google Scholar
  19. 19.
    Oh, T.H., Tai, Y.W., Bazin, J.C., Kim, H., Kweon, I.S.: Partial sum minimization of singular values in robust PCA: algorithm and applications. IEEE Trans. Pattern Anal. Mach. Intell. 38(4), 744–758 (2016)CrossRefGoogle Scholar
  20. 20.
    Okabe, T., Sato, I., Sato, Y.: Attached shadow coding: estimating surface normals from shadows under unknown reflectance and lighting conditions. In: 2009 IEEE 12th International Conference on Computer Vision, IEEE, pp. 1693–1700 (2009)Google Scholar
  21. 21.
    Okatani, T., Yoshida, T., Deguchi, K.: Efficient algorithm for low-rank matrix factorization with missing components and performance comparison of latest algorithms. In: 2011 IEEE International Conference on Computer Vision (ICCV), IEEE, pp. 842–849 (2011)Google Scholar
  22. 22.
    Papadhimitri, T., Favaro, P.: A new perspective on uncalibrated photometric stereo. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1474–1481 (2013)Google Scholar
  23. 23.
    Phong, B.T.: Illumination for computer generated pictures. Commun. ACM 18(6), 311–317 (1975)CrossRefGoogle Scholar
  24. 24.
    Quéau, Y., Lauze, F., Durou, J.D.: Solving uncalibrated photometric stereo using total variation. J. Math. Imaging Vis. 52(1), 87–107 (2015)MathSciNetCrossRefMATHGoogle Scholar
  25. 25.
    Shi, B., Matsushita, Y., Wei, Y., Xu, C., Tan, P.: Self-calibrating photometric stereo. In: 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, pp. 1118–1125 (2010)Google Scholar
  26. 26.
    Shi, B., Wu, Z., Mo, Z., Duan, D., Yeung, S.K., Tan, P.: A benchmark dataset and evaluation for non-lambertian and uncalibrated photometric stereo. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3707–3716 (2016)Google Scholar
  27. 27.
    Sunkavalli, K., Zickler, T., Pfister, H.: Visibility subspaces: uncalibrated photometric stereo with shadows. In: European Conference on Computer Vision, Springer, pp. 251–264 (2010)Google Scholar
  28. 28.
    Tan, P.: Phong reflectance model. In: Computer Vision, Springer, pp. 592–594 (2014)Google Scholar
  29. 29.
    Tan, .P, Mallick, S.P., Quan, L., Kriegman, D.J., Zickler, T.: Isotropy, reciprocity and the generalized bas-relief ambiguity. In: IEEE Conference on Computer Vision and Pattern Recognition, 2007. CVPR’07, IEEE, pp. 1–8 (2007)Google Scholar
  30. 30.
    Woodham, R.J.: Photometric method for determining surface orientation from multiple images. Opt. Eng. 19(1), 191 (1980)CrossRefGoogle Scholar
  31. 31.
    Wu, L., Ganesh, A., Shi, B., Matsushita, Y., Wang, Y., Ma, Y.: Robust photometric stereo via low-rank matrix completion and recovery. In: Computer Vision–ACCV 2010, Springer, pp. 703–717 (2011)Google Scholar
  32. 32.
    Xiong, Y., Chakrabarti, A., Basri, R., Gortler, S.J., Jacobs, D.W., Zickler, T.: From shading to local shape. IEEE Trans. Pattern Anal. Mach. Intell. 37(1), 67–79 (2015)CrossRefGoogle Scholar
  33. 33.
    Yuille, A., Snow, D.: Shape and albedo from multiple images using integrability. In: Proceedings of 1997 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 1997. IEEE, pp. 158–164 (1997)Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2017

Authors and Affiliations

  • Soumyadip Sengupta
    • 1
  • Hao Zhou
    • 1
  • Walter Forkel
    • 2
  • Ronen Basri
    • 3
  • Tom Goldstein
    • 4
  • David Jacobs
    • 1
  1. 1.Center for Automation ResearchUniversity of MarylandCollege ParkUSA
  2. 2.TU DresdenDresdenGermany
  3. 3.Department of Computer Science and Applied MathematicsWeizmann Institute of ScienceRehovotIsrael
  4. 4.Department of Computer ScienceUniversity of MarylandCollege ParkUSA

Personalised recommendations