Advertisement

Journal of Mathematical Imaging and Vision

, Volume 25, Issue 3, pp 403–421 | Cite as

A Unified Algebraic Approach to 2-D and 3-D Motion Segmentation and Estimation

  • René Vidal
  • Yi Ma
Article

Abstract

In this paper, we present an analytic solution to the problem of estimating an unknown number of 2-D and 3-D motion models from two-view point correspondences or optical flow. The key to our approach is to view the estimation of multiple motion models as the estimation of a single multibody motion model. This is possible thanks to two important algebraic facts. First, we show that all the image measurements, regardless of their associated motion model, can be fit with a single real or complex polynomial. Second, we show that the parameters of the individual motion model associated with an image measurement can be obtained from the derivatives of the polynomial at that measurement. This leads to an algebraic motion segmentation and estimation algorithm that applies to most of the two-view motion models that have been adopted in computer vision. Our experiments show that the proposed algorithm out-performs existing algebraic and factorization-based methods in terms of efficiency and robustness, and provides a good initialization for iterative techniques, such as Expectation Maximization, whose performance strongly depends on good initialization.

Keywords

multibody structure from motion motion segmentation multibody epipolar constraint multibody fundamental matrix multibody homography and Generalized PCA (GPCA) 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    S. Avidan and A. Shashua, “Trajectory triangulation: 3D reconstruction of moving points from a monocular image sequence,” IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol. 22, No. 4, pp. 348–357, 2000.CrossRefGoogle Scholar
  2. 2.
    S. Ayer and H. Sawhney, “Layered representation of motion video using robust maximum-likelihood estimation of mixture models and MDL encoding,” in IEEE International Conference on Computer Vision, 1995, pp. 777–785.Google Scholar
  3. 3.
    M. Black and P. Anandan, “Robust dynamic motion estimation over time,” in IEEE Conference on Computer Vision and Pattern Recognition, 1991, pp. 296–302.Google Scholar
  4. 4.
    T.E. Boult and L.G. Brown, “Factorization-based segmentation of motions,” in Proc. of the IEEE Workshop on Motion Understanding, 1991, pp. 179–186.Google Scholar
  5. 5.
    A. Chiuso, P. Favaro, H. Jin, and S. Soatto, “Motion and structure causally integrated over time,” IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol. 24, No. 4, pp. 523–535, 2002.CrossRefGoogle Scholar
  6. 6.
    T. Darrel and A. Pentland, “Robust estimation of a multi-layered motion representation,” in IEEE Workshop on Visual Motion, 1991, pp. 173–178.Google Scholar
  7. 7.
    X. Feng and P. Perona, “Scene segmentation from 3D motion,” in IEEE Conference on Computer Vision and Pattern Recognition, 1998, pp. 225–231.Google Scholar
  8. 8.
    M.A. Fischler and R. C. Bolles, “RANSAC random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography,” Communications of the ACM, Vol. 26, pp. 381–395, 1981.MathSciNetCrossRefGoogle Scholar
  9. 9.
    A. Fitzgibbon and A. Zisserman, “Multibody structure and motion: 3D reconstruction of independently moving objects,” in European Conference on Computer Vision, 2000, pp. 891–906.Google Scholar
  10. 10.
    M. Han and T. Kanade, “Reconstruction of a scene with multiple linearly moving objects,” in IEEE Conference on Computer Vision and Pattern Recognition, Vol. 2, 2000, pp. 542–549.Google Scholar
  11. 11.
    M. Han and T. Kanade, “Multiple motion scene reconstruction from uncalibrated views,” in IEEE International Conference on Computer Vision, Vol. 1, pp. 163–170, 2001.Google Scholar
  12. 12.
    J. Harris, Algebraic Geometry: A First Course, Springer-Verlag, 1992.Google Scholar
  13. 13.
    R. Hartley and R. Vidal, “The multibody trifocal tensor: Motion segmentation from 3 perspective views,” in IEEE Conference on Computer Vision and Pattern Recognition, Vol. I, pp. 769–775, 2004.Google Scholar
  14. 14.
    R. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision, 2nd edition, Cambridge, 2004.Google Scholar
  15. 15.
    A. Heyden and K. Åström, “Algebraic properties of multilinear constraints,” Mathematical Methods in Applied Sciences, Vol. 20, No. 13, pp. 1135–1162, 1997.MATHCrossRefGoogle Scholar
  16. 16.
    M. Irani, B. Rousso, and S. Peleg, “Detecting and tracking multiple moving objects using temporal integration,” in European Conference on Computer Vision, 1992, pp. 282–287.Google Scholar
  17. 17.
    A. Jepson and M. Black, “ Mixture models for optical flow computation,” in IEEE Conference on Computer Vision and Pattern Recognition, 1993, pp. 760–761.Google Scholar
  18. 18.
    K. Kanatani, “Motion segmentation by subspace separation and model selection,” in IEEE International Conference on Computer Vision, 2001, Vol. 2, pp. 586–591.Google Scholar
  19. 19.
    K. Kanatani, “Evaluation and selection of models for motion segmentation,” in Asian Conference on Computer Vision, 2002, pp. 7–12.Google Scholar
  20. 20.
    K. Kanatani and C. Matsunaga, “Estimating the number of independent motions for multibody motion segmentation,” in European Conference on Computer Vision, 2002, pp. 25–31.Google Scholar
  21. 21.
    K. Kanatani and Y. Sugaya, “Multi-stage optimization for multi-body motion segmentation,” in Australia-Japan Advanced Workshop on Computer Vision, 2003, pp. 335–349.Google Scholar
  22. 22.
    Q. Ke and T. Kanade, “A robust subspace approach to layer extraction,” in IEEE Workshop on Motion and Video Computing, 2002, pp. 37–43.Google Scholar
  23. 23.
    H. C. Longuet-Higgins, “A computer algorithm for reconstructing a scene from two projections,” Nature, Vol. 293, pp. 133–135, 1981CrossRefGoogle Scholar
  24. 24.
    Y. Ma, S. Soatto, J. Kosecka, and S. Sastry, An Invitation to 3D Vision: From Images to Geometric Models, Springer Verlag, 2003.Google Scholar
  25. 25.
    O. Shakernia, R. Vidal, and S. Sastry, “Multi-body motion estimation and segmentation from multiple central panoramic views,” in IEEE International Conference on Robotics and Automation, 2003, Vol. 1, pp. 571–576.Google Scholar
  26. 26.
    A. Shashua and A. Levin, “Multi-frame infinitesimal motion model for the reconstruction of (dynamic) scenes with multiple linearly moving objects,” in IEEE International Conference on Computer Vision, 2001, Vol. 2, pp. 592–599.Google Scholar
  27. 27.
    J. Shi and J. Malik, “ Motion segmentation and tracking using normalized cuts,” in IEEE International Conference on Computer Vision, 1998, pp. 1154–1160.Google Scholar
  28. 28.
    A. Spoerri and S. Ullman, “The early detection of motion boundaries,” in IEEE International Conference on Computer Vision, 1987, pp. 209–218.Google Scholar
  29. 29.
    P. Sturm, “ Structure and motion for dynamic scenes - the case of points moving in planes,” in European Conference on Computer Vision, 2002, pp. 867–882.Google Scholar
  30. 30.
    P. Torr, R. Szeliski, and P. Anandan, “An integrated Bayesian approach to layer extraction from image sequences,” IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol. 23, No. 3, pp. 297–303, 2001.CrossRefGoogle Scholar
  31. 31.
    P.H.S. Torr, “Geometric motion segmentation and model selection,” Phil. Trans. Royal Society of London, Vol. 356, No. 1740, No. 1321–1340, 1998.Google Scholar
  32. 32.
    R. Vidal, Imaging Beyond the Pinhole Camera, chapter Segmentation of Dynamic Scenes Taken by a Central Panoramic Camera, LNCS. Springer Verlag, 2006.Google Scholar
  33. 33.
    R. Vidal and R. Hartley, “Motion segmentation with missing data by PowerFactorization and Generalized PCA,” in IEEE Conference on Computer Vision and Pattern Recognition, 2004, Vol. 2, pp. 310–316.Google Scholar
  34. 34.
    R. Vidal and Y. Ma, “A unified algebraic approach to 2-D and 3-D motion segmentation,” in European Conference on Computer Vision, 2004, pp. 1–15.Google Scholar
  35. 35.
    R. Vidal, Y. Ma, and J. Piazzi, “ A new GPCA algorithm for clustering subspaces by fitting, differentiating and dividing polynomials,” in IEEE Conference on Computer Vision and Pattern Recognition, 2004, Vol. 1, pp. 510–517.Google Scholar
  36. 36.
    R. Vidal, Y. Ma, and S. Sastry, “Generalized Principal Component Analysis (GPCA),” in IEEE Conference on Computer Vision and Pattern Recognition, Vol. I, pp. 621–628, 2003.Google Scholar
  37. 37.
    R. Vidal, Y. Ma, and S. Sastry, “Generalized Principal Component Analysis (GPCA),” IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol. 27, No. 12, pp. 1–15, 2005.CrossRefGoogle Scholar
  38. 38.
    R. Vidal, Y. Ma, S. Soatto, and S. Sastry, “Two-view multibody structure from motion,” International Journal of Computer Vision, Vol. 68, No. 1, 2006.Google Scholar
  39. 39.
    R. Vidal and S. Sastry, “Optimal segmentation of dynamic scenes from two perspective views,” in IEEE Conference on Computer Vision and Pattern Recognition, 2003, Vol. 2, pp. 281–286.Google Scholar
  40. 40.
    R. Vidal, S. Soatto, Y. Ma, and S. Sastry, “Segmentation of dynamic scenes from the multibody fundamental matrix,” in ECCV Workshop on Visual Modeling of Dynamic Scenes, 2002.Google Scholar
  41. 41.
    J. Wang and E. Adelson, “Layered representation for motion analysis,” in IEEE Conference on Computer Vision and Pattern Recognition, 1993, pp. 361–366.Google Scholar
  42. 42.
    Y. Weiss, “A unified mixture framework for motion segmentation: incoprporating spatial coherence and estimating the number of models,” in IEEE Conference on Computer Vision and Pattern Recognition, 1996, pp. 321–326.Google Scholar
  43. 43.
    Y. Weiss, “Smoothness in layers: Motion segmentation using nonparametric mixture estimation,” in IEEE Conference on Computer Vision and Pattern Recognition, 1997, pp. 520–526.Google Scholar
  44. 44.
    L. Wolf and A. Shashua, “Affine 3-D reconstruction from two projective images of independently translating planes,” in IEEE International Conference on Computer Vision, 2001, pp. 238–244.Google Scholar
  45. 45.
    L. Wolf and A. Shashua, “Two-body segmentation from two perspective views,” in IEEE Conference on Computer Vision and Pattern Recognition, 2001, pp. 263–270.Google Scholar
  46. 46.
    Y. Wu, Z. Zhang, T.S. Huang, and J.Y. Lin, “ Multibody grouping via orthogonal subspace decomposition,” in IEEE Conference on Computer Vision and Pattern Recognition, 2001, Vol 2, pp. 252–257.Google Scholar
  47. 47.
    L. Zelnik-Manor and M. Irani, “Degeneracies, dependencies and their implications in multi-body and multi-sequence factorization,” in IEEE Conference on Computer Vision and Pattern Recognition, 2003, Vol. 2, pp. 287–293.Google Scholar

Copyright information

© Springer Science + Business Media, LLC 2006

Authors and Affiliations

  1. 1.Center for Imaging Science, Department of Biomedical EngineeringJohns Hopkins UniversityBaltimoreUSA
  2. 2.Electrical & Computer Engineering DepartmentUniversity of Illinois at Urbana-ChampaignUrbanaUSA

Personalised recommendations