Incorporating Non-motion Cues into 3D Motion Segmentation

  • Amit Gruber
  • Yair Weiss
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3953)


We address the problem of segmenting an image sequence into rigidly moving 3D objects. An elegant solution to this problem is the multibody factorization approach in which the measurement matrix is factored into lower rank matrices. Despite progress in factorization algorithms, the performance is still far from satisfactory and in scenes with missing data and noise, most existing algorithms fail.

In this paper we propose a method for incorporating 2D non-motion cues (such as spatial coherence) into multibody factorization. We formulate the problem in terms of constrained factor analysis and use the EM algorithm to find the segmentation. We show that adding these cues improves performance in real and synthetic sequences.


Measurement Matrix Spatial Coherence Motion Segmentation Inverse Covariance Matrix Loopy Belief Propagation 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Costeira, J., Kanade, T.: A multi-body factorization method for motion analysis. In: ICCV (1995)Google Scholar
  2. 2.
    Gear, C.: Multibody grouping from motion images. IJCV, 133–150 (1998)Google Scholar
  3. 3.
    Zelnik-Manor, L., Machline, M., Irani, M.: Multi-body segmentation: Revisiting motion consistency (2002)Google Scholar
  4. 4.
    Gruber, A., Weiss, Y.: Multibody factorization with uncertainty and missing data using the EM algorithm. In: Computer Vision and Pattern Recognition CVPR (2004) Google Scholar
  5. 5.
    Vidal, R., Hartely, R.: Motion segmentation with missing data using powerfactorization and gpca. In: Computer Vision and Pattern Recognition, CVPR (2004)Google Scholar
  6. 6.
    Kanatani, K.: Evaluation and selection of models for motion segmentation. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2352, pp. 335–349. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  7. 7.
    Vidal, R., Soatto, S., Ma, Y., Sastry, S.: Segmentation of dynamic scenes from the multibody fundamental matrix (2002)Google Scholar
  8. 8.
    Wolf, L., Shashua, A.: Two-body segmentation from two perspective views. In: IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pp. 263–270 (2001)Google Scholar
  9. 9.
    Feng, X., Perona, P.: Scene segmentation from 3D motion. CVPR, 225–231 (1998)Google Scholar
  10. 10.
    MacLean, W.J., Jepson, A.D., Frecker, R.C.: Recovery of egomotion and segmentation of independent object motion using the em algorithm. In: BMVC (1994)Google Scholar
  11. 11.
    Shi, J., Malik, J.: Normalized cuts and image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence 22, 888–905 (2000)CrossRefGoogle Scholar
  12. 12.
    Shi, J., Malik, J.: Motion segmentation and tracking using normalized cuts. In: ICCV, pp. 1154–1160 (1998)Google Scholar
  13. 13.
    Weiss, Y., Adelson, E.: A unified mixture framework for motion segmentation: incorporating spatial coherence and estimating the number of models. In: Proceedings of IEEE conference on Computer Vision and Pattern Recognition, pp. 321–326 (1996)Google Scholar
  14. 14.
    Zabih, R., Kolmogorov, V.: Spatially coherent clustering with graph cuts. In: Computer Vision and Pattern Recognition, CVPR (2004)Google Scholar
  15. 15.
    Gruber, A., Weiss, Y.: Factorization with uncertainty and missing data: Exploiting temporal coherence. In: Neural Information Processing Systems, NIPS (2003)Google Scholar
  16. 16.
    Tomasi, C., Kanade, T.: Shape and motion from image streams under orthography: A factorization method. Int. J. of Computer Vision 9, 137–154 (1992)CrossRefGoogle Scholar
  17. 17.
    Irani, M., Anandan, P.: Factorization with uncertainty. In: Vernon, D. (ed.) ECCV 2000. LNCS, vol. 1842, pp. 539–553. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  18. 18.
    Kolmogorov, V., Zabih, R.: What energy functions can be minimized via graph cuts? Transactions on Pattern Analysis and Machine Intelligence, PAMI (2004)Google Scholar
  19. 19.
    Weiss, Y., Freeman, W.T.: On the optimality of solutions of the max-product belief propagation algorithm in arbitrary graphs. IEEE Transactions on Information Theory 47, 723–735 (2001)CrossRefMathSciNetGoogle Scholar
  20. 20.
    Rubin, D., Thayer, D.: EM algorithms for ML factor analysis. Psychometrika 47(1), 69–76 (1982)zbMATHCrossRefMathSciNetGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Amit Gruber
    • 1
  • Yair Weiss
    • 1
  1. 1.School of Computer Science and EngineeringThe Hebrew University of JerusalemJerusalemIsrael

Personalised recommendations