Automatic Estimation of the Number of Deformation Modes in Non-rigid SfM with Missing Data

  • Carme Julià
  • Marco Paladini
  • Ravi Garg
  • Domenec Puig
  • Lourdes Agapito
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6688)


This paper proposes a new algorithm to estimate automatically the number of deformation modes needed to describe a non-rigid object with the well-known low-rank shape model, focusing on the missing data case. The 3D shape is assumed to deform as a linear combination of K rigid shape bases according to time varying coefficients. One of the requirements of this formulation is that the number of bases must be known in advance. Most non-rigid structure from motion (NRSfM) approaches based on this model determine the value of K empirically. Our proposed approach is based on the analysis of the frequency spectra of the x and y coordinates corresponding to the individual image trajectories, which are seen as 1D signals. The frequency content of the 2D trajectories is encoded using the modulus of the Discrete Cosine Transform (DCT) of the signals. Our hypothesis is that the value of K that gives the best prediction of the missing data also provides the best 3D reconstruction. Our proposed approach does not assume any prior knowledge and is independent of the 3D reconstruction algorithm used. We validate our approach with experiments on synthetic and real sequences.


non-rigid SfM Discrete Cosine Transform frequency content 


  1. 1.
    Akhter, I., Sheikh, Y., Khan, S., Kanade, T.: Nonrigid structure from motion in trajectory space. In: Neural Information Processing Systems (2008)Google Scholar
  2. 2.
    Bartoli, A., Gay-Bellile, V., Castellani, U., Peyras, J., Olsen, S., Sayd, P.: Coarse-to-fine low-rank structure-from-motion. In: CVPR (2008)Google Scholar
  3. 3.
    Brand, M.: A direct method for 3D factorization of nonrigid motion observed in 2D. In: CVPR, pp. 122–128 (2005)Google Scholar
  4. 4.
    Bregler, C., Hertzmann, A., Biermann, H.: Recovering non-rigid 3D shape from image streams. In: CVPR. pp. 690–696 (2000)Google Scholar
  5. 5.
    Del Bue, A., Lladó, X., Agapito, L.: Non-rigid metric shape and motion recovery from uncalibrated images using priors. In: CVPR, pp. 297–310 (2006)Google Scholar
  6. 6.
    Hartley, R.I., Vidal, R.: Perspective nonrigid shape and motion recovery. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part I. LNCS, vol. 5302, pp. 276–289. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  7. 7.
    Julià, C., Sappa, A.D., Lumbreras, F., Serrat, J., López, A.: Rank estimation in 3D multibody motion segmentation. Electronics Letters 44(4), 279–280 (2008)CrossRefGoogle Scholar
  8. 8.
    Julià, C., Sappa, A.D., Lumbreras, F., Serrat, J., López, A.: Rank estimation in missing data problems. Journal of Mathematical Imaging and Vision 39, 140–160 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  9. 9.
    Paladini, M., Del Bue, A., Stošic, M., Dodig, M., Xavier, J., Agapito, L.: Factorization for non-rigid and articulated structure using metric projections. In: CVPR, pp. 2898–2905 (2009)Google Scholar
  10. 10.
    Roy-Chowdhury, A.K.: Towards a measure of deformability of shape sequences. Pattern Recognition Letters 28, 2164–2172 (2007)CrossRefGoogle Scholar
  11. 11.
    Tomasi, C., Kanade, T.: Shape and motion from image streams under orthography: a factorization method. IJCV 9(2), 137–154 (1992)CrossRefGoogle Scholar
  12. 12.
    Torresani, L., Hertzmann, A., Bregler, C.: Non-rigid structure-from-motion: Estimating shape and motion with hierarchical priors. IEEE Transactions on PAMI 30(5), 878–892 (2008)CrossRefGoogle Scholar
  13. 13.
    Vidal, R., Abretske, D.: Nonrigid shape and motion from multiple perspective views. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3952, pp. 205–218. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  14. 14.
    Xiao, J., Chai, J., Kanade, T.: A closed-form solution to non-rigid shape and motion recovery. In: IJCV, vol. 67(2) (2006)Google Scholar
  15. 15.
    Xiao, J., Kanade, T.: Uncalibrated perspective reconstruction of deformable structures. In: ICCV (2005)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Carme Julià
    • 1
  • Marco Paladini
    • 2
  • Ravi Garg
    • 2
  • Domenec Puig
    • 1
  • Lourdes Agapito
    • 2
  1. 1.Department of Computer Science and MathematicsUniversitat Rovira i VirgiliTarragonaSpain
  2. 2.School of Electronic Engineering and Computer ScienceQueen Mary University of LondonUK

Personalised recommendations