Structure from Motion and Photometric Stereo for Dense 3D Shape Recovery

  • Reza Sabzevari
  • Alessio Del Bue
  • Vittorio Murino
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6978)

Abstract

In this paper we present a dense 3D reconstruction pipeline from monocular video sequences using jointly Photometric Stereo (PS) and Structure from Motion (SfM) approaches. The input videos are completely uncalibrated both from the multi-view geometry and photometric stereo aspects. In particular we make use of the 3D metric information computed with SfM from a set of 2D landmarks in order to solve for the bas-relief ambiguity which is intrinsic from dense PS surface estimation. The algorithm is evaluated over the CMU Multi-Pie database which contains the images of 337 subjects viewed under different lighting conditions and showing various facial expressions.

Keywords

Structure from Motion Photometric Stereo Dense 3D Reconstruction 

References

  1. 1.
    Aliaga, D.G., Xu, Y.: A self-calibrating method for photogeometric acquisition of 3d objects. IEEE Transactions on Pattern Analysis and Machine Intelligence 32(4), 747–754 (2010)CrossRefGoogle Scholar
  2. 2.
    Basri, R., Jacobs, D., Kemelmacher, I.: Photometric stereo with general, unknown lighting. International Journal of Computer Vision 72(3), 239–257 (2007)CrossRefGoogle Scholar
  3. 3.
    Belhumeur, P.N., Kriegman, D.J., Yuille, A.L.: The bas-relief ambiguity. International Journal of Computer Vision 35(1), 33–44 (1999)CrossRefGoogle Scholar
  4. 4.
    Blostein, D., Ahuja, N.: Shape from texture: Integrating texture-element extraction and surface estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence 11(12), 1233–1251 (2002)CrossRefGoogle Scholar
  5. 5.
    Del Bue, A., Xavier, J., Agapito, L., Paladini, M.: Bilinear factorization via augmented lagrange multipliers. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6314, pp. 283–296. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  6. 6.
    Favaro, P., Soatto, S.: A geometric approach to shape from defocus. IEEE Transactions on Pattern Analysis and Machine Intelligence, 406–417 (2005)Google Scholar
  7. 7.
    Gross, R., Matthews, I., Cohn, J., Kanade, T., Baker, S.: Multi-pie. Image and Vision Computing 28(5), 807–813 (2010)CrossRefGoogle Scholar
  8. 8.
    Hartley, R., Zisserman, A.: Multiple view geometry, vol. 642. Cambridge University Press, Cambridge (2000)MATHGoogle Scholar
  9. 9.
    Hernández, C., Vogiatzis, G.: Self-calibrating a real-time monocular 3d facial capture system. In: Proceedings International Symposium on 3D Data Processing, Visualization and Transmission (3DPVT) (2010)Google Scholar
  10. 10.
    Kanatani, K., Sugaya, Y., Ackermann, H.: Uncalibrated factorization using a variable symmetric affine camera. IEICE Transactions on Information and Systems 90(5), 851 (2007)CrossRefGoogle Scholar
  11. 11.
    Lim, J., Ho, J., Yang, M., Kriegman, D.: Passive photometric stereo from motion. In: Proceedings of the Tenth IEEE International Conference on Computer Vision, vol. 2, pp. 1635–1642. IEEE Computer Society, Los Alamitos (2005)Google Scholar
  12. 12.
    Marques, M., Costeira, J.: Estimating 3d shape from degenerate sequences with missing data. Computer Vision and Image Understanding 113(2), 261–272 (2009)CrossRefGoogle Scholar
  13. 13.
    Prados, E., Faugeras, O.: Shape from shading. In: Handbook of Mathematical Models in Computer Vision, pp. 375–388 (2006)Google Scholar
  14. 14.
    Shi, J., Tomasi, C.: Good features to track. In: Proc. IEEE Conference on Computer Vision and Pattern Recognition, pp. 593–600 (1994)Google Scholar
  15. 15.
    Tomasi, C., Kanade, T.: Shape and motion from image streams under orthography: A factorization approach. International Journal of Computer Vision 9(2) (1992)Google Scholar
  16. 16.
    Triggs, B.: Factorization methods for projective structure and motion. In: Proc. IEEE Conference on Computer Vision and Pattern Recognition, San Francisco, pp. 845–851 (1996)Google Scholar
  17. 17.
    Woodham, R.: Photometric method for determining surface orientation from multiple images. Optical Engineering 19(1), 139–144 (1980)CrossRefGoogle Scholar
  18. 18.
    Zhang, L., Curless, B., Hertzmann, A., Seitz, S.M.: Shape and motion under varying illumination: Unifying structure from motion, photometric stereo, and multi-view stereo. In: Proceedings of the Ninth IEEE International Conference on Computer Vision, vol. 2, p. 618. IEEE Computer Society, Los Alamitos (2003)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Reza Sabzevari
    • 1
  • Alessio Del Bue
    • 1
  • Vittorio Murino
    • 1
  1. 1.Istituto Italiano di TecnologiaGenovaItaly

Personalised recommendations