Advertisement

Piecewise Quadratic Reconstruction of Non-Rigid Surfaces from Monocular Sequences

  • João Fayad
  • Lourdes Agapito
  • Alessio Del Bue
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6314)

Abstract

In this paper we present a new method for the 3D reconstruction of highly deforming surfaces (for instance a flag waving in the wind) viewed by a single orthographic camera. We assume that the surface is described by a set of feature points which are tracked along an image sequence. Most non-rigid structure from motion algorithms assume a global deformation model where a rigid mean shape component accounts for most of the motion and the deformation modes are small deviations from it. However, in the case of strongly deforming objects, the deformations become more complex and a global model will often fail to explain the intricate deformations which are no longer small linear deviations from a strong mean component. Our proposed algorithm divides the surface into overlapping patches, reconstructs each of these patches individually using a quadratic deformation model and finally registers them imposing the constraint that points shared by patches must correspond to the same 3D points in space. We show good results on challenging motion capture and real video sequences with strong deformations where global methods fail to achieve good reconstructions.

Keywords

Quadratic Model Bundle Adjustment Shared Point Rigid Factorization Real Video Sequence 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Supplementary material

978-3-642-15561-1_22_MOESM1_ESM.wmv (9 mb)
Electronic Supplementary Material (9,171 KB)

References

  1. 1.
    Bregler, C., Hertzmann, A., Biermann, H.: Recovering non-rigid 3D shape from image streams. In: CVPR (2000)Google Scholar
  2. 2.
    Brand, M.: Morphable models from video. In: CVPR (2001)Google Scholar
  3. 3.
    Brand, M.: A direct method for 3D factorization of nonrigid motion observed in 2D. In: CVPR (2005)Google Scholar
  4. 4.
    Xiao, J., Kanade, T.: Uncalibrated perspective reconstruction of deformable structures. In: ICCV (2005)Google Scholar
  5. 5.
    Xiao, J., Chai, J., Kanade, T.: A closed-form solution to non-rigid shape and motion recovery. IJCV (2006)Google Scholar
  6. 6.
    Del Bue, A., Lladó, X., Agapito, L.: Non-rigid metric shape and motion recovery from uncalibrated images using priors. In: CVPR (2006)Google Scholar
  7. 7.
    Torresani, L., Hertzmann, A., Bregler., C.: Non-rigid structure-from-motion: Estimating shape and motion with hierarchical priors. PAMI (2008)Google Scholar
  8. 8.
    Bartoli, A., Gay-Bellile, V., Castellani, U., Peyras, J., Olsen, S., Sayd, P.: Coarse-to-fine low-rank structure-from-motion. In: CVPR (2008)Google Scholar
  9. 9.
    Hartley, R., Vidal, R.: Perspective nonrigid shape and motion recovery. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part I. LNCS, vol. 5302, pp. 276–289. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  10. 10.
    Fayad, J., Del Bue, A., Agapito, L., Aguiar, P.M.Q.: Non-rigid structure from motion using quadratic deformation models. In: BMVC (2009)Google Scholar
  11. 11.
    Blanz, V., Vetter, T.: A morphable model for the synthesis of 3d faces. In: SIGGRAPH (1999)Google Scholar
  12. 12.
    Aanæs, H., Kahl, F.: Estimation of deformable structure and motion. In: Workshop on Vision and Modelling of Dynamic Scenes, Copenhagen, Denmark (2002)Google Scholar
  13. 13.
    Salzmann, M., Fua, P.: Reconstructing sharply folding surfaces: A convex formulation. In: CVPR (2009)Google Scholar
  14. 14.
    Perriollat, M., Hartley, R., Bartoli, A.: Monocular template-based reconstruction of inextensible surfaces. In: BMVC (2008)Google Scholar
  15. 15.
    Paladini, M., Del Bue, A., Stosic, M., Dodig, M., Xavier, J., Agapito, L.: Factorization for non-rigid and articulated structure using metric projections. In: CVPR (2009)Google Scholar
  16. 16.
    Akhter, I., Sheikh, Y.A., Khan, S., Kanade, T.: Nonrigid structure from motion in trajectory space. In: Neural Information Processing Systems (2008)Google Scholar
  17. 17.
    Rabaud, V., Belongie, S.: Re-thinking non-rigid structure from motion. In: CVPR (2008)Google Scholar
  18. 18.
    Salzmann, M., Urtasun, R.: Combining discriminative and generative methods for 3d deformable surface and articulated pose reconstruction. In: CVPR (2010)Google Scholar
  19. 19.
    Shaji, A., Varol, A., Torresani, L., Fua, P.: Simultaneous Point Matching and 3D Deformable Surface Reconstruction. In: CVPR (2010)Google Scholar
  20. 20.
    Salzmann, M., Urtasun, R., Fua, P.: Local deformation models for monocular 3d shape recovery. In: CVPR (2008)Google Scholar
  21. 21.
    Taylor, J., Jepson, A.D., Kutulakos, K.N.: Non-rigid structure from locally-rigid motion. In: CVPR (2010)Google Scholar
  22. 22.
    Varol, A., Salzmann, M., Tola, E., Fua, P.: Template-free monocular reconstruction of deformable surfaces. In: ICCV (2009)Google Scholar
  23. 23.
    Tenenbaum, J.B., de Silva, V., Langford, J.C.: A global geometric framework for nonlinear dimensionality reduction. Science (2000)Google Scholar
  24. 24.
    Tomasi, C., Kanade, T.: Shape and motion from image streams under orthography: A factorization approach. In: IJCV (1992)Google Scholar
  25. 25.
    Müller, M., Heidelberger, B., Teschner, M., Gross, M.: Meshless deformations based on shape matching. In: SIGGRAPH (2005)Google Scholar
  26. 26.
    Triggs, B., McLauchlan, P.F., Hartley, R.I., Fitzgibbon, A.W.: Bundle adjustment - a modern synthesis. In: ICCV (2000)Google Scholar
  27. 27.
    White, R., Crane, K., Forsyth, D.: Capturing and animating occluded cloth. In: ACM Trans. on Graphics (2007)Google Scholar
  28. 28.
    Del Bue, A., Smeraldi, F., Agapito, L.: Non-rigid structure from motion using ranklet–based tracking and non-linear optimization. In: IVC (2007)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • João Fayad
    • 1
  • Lourdes Agapito
    • 1
  • Alessio Del Bue
    • 2
  1. 1.Queen Mary University of LondonLondonUK
  2. 2.Istituto Italiano di TecnologiaGenovaItaly

Personalised recommendations