Perspective 3D Reconstruction of Nonrigid Objects

  • Guanghui Wang
  • Q. M. Jonathan Wu
Part of the Advances in Pattern Recognition book series (ACVPR)


The chapter focuses on the problem of nonrigid structure and motion factorization under perspective projection. Many previous methods are based on affine assumption that may be invalid and cause large reconstruction errors when the object is close to the camera. In this chapter, we propose two algorithms to extend these methods to full perspective projection model. The first one is a linear recursive algorithm, which updates the solution from weak-perspective to perspective projection by refining the projective depth scales. The second one is a nonlinear optimization scheme that minimizes the perspective reprojection residuals. Extensive experiments on synthetic and real image sequences are performed to validate the effectiveness of the algorithms.


Perspective Projection Reprojection Error Nonlinear Algorithm Motion Matrice Projective Depth 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Brand, M.: A direct method for 3D factorization of nonrigid motion observed in 2D. In: Proc. of IEEE Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 122–128 (2005) Google Scholar
  2. 2.
    Bregler, C., Hertzmann, A., Biermann, H.: Recovering non-rigid 3D shape from image streams. In: Proc. of IEEE Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 690–696 (2000) Google Scholar
  3. 3.
    Christy, S., Horaud, R.: Euclidean shape and motion from multiple perspective views by affine iterations. IEEE Trans. Pattern Anal. Mach. Intell.18(11), 1098–1104 (1996) CrossRefGoogle Scholar
  4. 4.
    Del Bue, A., Lladó, X., de Agapito, L.: Non-rigid metric shape and motion recovery from uncalibrated images using priors. In: Proc. of IEEE Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 1191–1198 (2006) Google Scholar
  5. 5.
    Dementhon, D.F., Davis, L.S.: Model-based object pose in 25 lines of code. Int. J. Comput. Vis.15(1–2), 123–141 (1995) CrossRefGoogle Scholar
  6. 6.
    Fujiki, J., Kurata, T.: Recursive factorization method for the paraperspective model based on the perspective projection. In: Proc. of International Conference on Pattern Recognition, vol. 1, pp. 406–410 (2000) Google Scholar
  7. 7.
    Hartley, R.I., Zisserman, A.: Multiple View Geometry in Computer Vision, 2nd edn. Cambridge University Press, Cambridge (2004). ISBN: 0521540518 MATHCrossRefGoogle Scholar
  8. 8.
    Li, T., Kallem, V., Singaraju, D., Vidal, R.: Projective factorization of multiple rigid-body motions. In: Proc. of IEEE Conference on Computer Vision and Pattern Recognition (2007) Google Scholar
  9. 9.
    Lladó, X., Del Bue, A., de Agapito, L.: Non-rigid 3D factorization for projective reconstruction. In: Proc. of British Machine Vision Conference (2005) Google Scholar
  10. 10.
    Oliensis, J., Hartley, R.: Iterative extensions of the Sturm/Triggs algorithm: Convergence and nonconvergence. IEEE Trans. Pattern Anal. Mach. Intell.29(12), 2217–2233 (2007) CrossRefGoogle Scholar
  11. 11.
    Ozden, K.E., Schindler, K., Van Gool, L.: Multibody structure-from-motion in practice. IEEE Trans. Pattern Anal. Mach. Intell.32(6), 1134–1141 (2010) CrossRefGoogle Scholar
  12. 12.
    Poelman, C., Kanade, T.: A paraperspective factorization method for shape and motion recovery. IEEE Trans. Pattern Anal. Mach. Intell.19(3), 206–218 (1997) CrossRefGoogle Scholar
  13. 13.
    Pollefeys, M., Koch, R., Van Gool, L.: Self-calibration and metric reconstruction in spite of varying and unknown intrinsic camera parameters. Int. J. Comput. Vis.32(1), 7–25 (1999) CrossRefGoogle Scholar
  14. 14.
    Quan, L.: Self-calibration of an affine camera from multiple views. Int. J. Comput. Vis.19(1), 93–105 (1996) CrossRefGoogle Scholar
  15. 15.
    Tomasi, C., Kanade, T.: Shape and motion from image streams under orthography: a factorization method. Int. J. Comput. Vis.9(2), 137–154 (1992) CrossRefGoogle Scholar
  16. 16.
    Torresani, L., Hertzmann, A., Bregler, C.: Nonrigid structure-from-motion: Estimating shape and motion with hierarchical priors. IEEE Trans. Pattern Anal. Mach. Intell.30(5), 878–892 (2008) CrossRefGoogle Scholar
  17. 17.
    Tresadern, P., Reid, I.: Articulated structure from motion by factorization. In: Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 1110–1115 (2005) Google Scholar
  18. 18.
    Triggs, B.: Factorization methods for projective structure and motion. In: Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 845–851 (1996) Google Scholar
  19. 19.
    Wang, G., Tian, Y., Sun, G.: Modelling nonrigid object from video sequence under perspective projection. In: Lecture Notes in Computer Science, vol. 3784, pp. 64–71 (2005) Google Scholar
  20. 20.
    Wang, G., Tsui, H.T., Hu, Z.: Structure and motion of nonrigid object under perspective projection. Pattern Recogn. Lett.28(4), 507–515 (2007) MATHCrossRefGoogle Scholar
  21. 21.
    Xiao, J., Chai, J.X., Kanade, T.: A closed-form solution to non-rigid shape and motion recovery. In: Proc. of European Conference on Computer Vision, vol. 4, pp. 573–587 (2004) Google Scholar
  22. 22.
    Xiao, J., Kanade, T.: Uncalibrated perspective reconstruction of deformable structures. In: Proc. of the International Conference on Computer Vision, vol. 2, pp. 1075–1082 (2005) Google Scholar
  23. 23.
    Yan, J., Pollefeys, M.: A factorization-based approach for articulated nonrigid shape, motion and kinematic chain recovery from video. IEEE Trans. Pattern Anal. Mach. Intell.30(5), 865–877 (2008) CrossRefGoogle Scholar
  24. 24.
    Yao, J., Cham, W.: Feature matching and scene reconstruction from multiple widely separated views. Tech. Rep., Chinese University of Hong Kong (2005) Google Scholar
  25. 25.
    Zelnik-Manor, L., Machline, M., Irani, M.: Multi-body factorization with uncertainty: Revisiting motion consistency. Int. J. Comput. Vis.68(1), 27–41 (2006) CrossRefGoogle Scholar
  26. 26.
    Zhang, Z.: A flexible new technique for camera calibration. IEEE Trans. Pattern Anal. Mach. Intell.22(11), 1330–1334 (2000) CrossRefGoogle Scholar

Copyright information

© Springer-Verlag London Limited 2011

Authors and Affiliations

There are no affiliations available

Personalised recommendations