Robust Motion Segmentation with Unknown Correspondences

  • Pan Ji
  • Hongdong Li
  • Mathieu Salzmann
  • Yuchao Dai
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8694)


Motion segmentation can be addressed as a subspace clustering problem, assuming that the trajectories of interest points are known. However, establishing point correspondences is in itself a challenging task. Most existing approaches tackle the correspondence estimation and motion segmentation problems separately. In this paper, we introduce an approach to performing motion segmentation without any prior knowledge of point correspondences. We formulate this problem in terms of Partial Permutation Matrices (PPMs) and aim to match feature descriptors while simultaneously encouraging point trajectories to satisfy subspace constraints. This lets us handle outliers in both point locations and feature appearance. The resulting optimization problem can be solved via the Alternating Direction Method of Multipliers (ADMM), where each subproblem has an efficient solution. Our experimental evaluation on synthetic and real sequences clearly evidences the benefits of our formulation over the traditional sequential approach that first estimates correspondences and then performs motion segmentation.


Motion segmentation point correspondence subspace clustering partial permutation matrix 


  1. 1.
    Amiaz, T., Kiryati, N.: Piecewise-smooth dense optical flow via level sets. IJCV 68(2), 111–124 (2006)CrossRefGoogle Scholar
  2. 2.
    Brox, T., Malik, J.: Large displacement optical flow: Descriptor matching in variational motion estimation. IEEE TPAMI 33(3), 500–513 (2011)CrossRefGoogle Scholar
  3. 3.
    Cai, J., Candes, E.J., Shen, Z.: A singular value thresholding algorithm for matrix completion. SIAM J. Optimization 20(4), 1956–1982 (2010)CrossRefzbMATHMathSciNetGoogle Scholar
  4. 4.
    Cheriyadat, A.M., Radke, R.J.: Non-negative matrix factorization of partial track data for motion segmentation. In: IEEE ICCV, pp. 865–872 (2009)Google Scholar
  5. 5.
    Cremers, D., Soatto, S.: Motion competition: A variational approach to piecewise parametric motion segmentation. IJCV 62(3), 249–265 (2005)CrossRefGoogle Scholar
  6. 6.
    Dragon, R., Ostermann, J., Van Gool, L.: Robust realtime motion-split-and-merge for motion segmentation. In: Weickert, J., Hein, M., Schiele, B. (eds.) GCPR 2013. LNCS, vol. 8142, pp. 425–434. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  7. 7.
    Duchenne, O., Bach, F., Kweon, I., Ponce, J.: A tensor-based algorithm for high-order graph matching. In: IEEE CVPR, pp. 1980–1987 (2009)Google Scholar
  8. 8.
    Elhamifar, E., Vidal, R.: Sparse subspace clustering: Algorithm, theory, and applications. IEEE TPAMI 35(11), 2765–2781 (2013)CrossRefGoogle Scholar
  9. 9.
    Ji, P., Salzmann, M., Li, H.: Efficient dense subspace clustering. In: IEEE WACV, pp. 461–468 (2014)Google Scholar
  10. 10.
    Jia, K., Chan, T.H., Zeng, Z., Ma, Y.: ROML: A robust feature correspondence approach for matching objects in a set of images. arXiv:1403.7877 (2014)Google Scholar
  11. 11.
    Leordeanu, M., Hebert, M.: A spectral technique for correspondence problems using pairwise constraints. In: IEEE ICCV, pp. 1–8 (2005)Google Scholar
  12. 12.
    Li, H.: Two-view motion segmentation from linear programming relaxation. In: IEEE CVPR, pp. 1–8 (2007)Google Scholar
  13. 13.
    Liu, G., Lin, Z., Yan, S., Sun, J., Yu, Y., Ma, Y.: Robust recovery of subspace structures by low-rank representation. IEEE TPAMI 35(1), 171–184 (2013)CrossRefGoogle Scholar
  14. 14.
    Lowe, D.: Distinctive image features from scale-invariant keypoints. IJCV 60(2), 91–110 (2004)CrossRefGoogle Scholar
  15. 15.
    Lu, C.-Y., Min, H., Zhao, Z.-Q., Zhu, L., Huang, D.-S., Yan, S.: Robust and efficient subspace segmentation via least squares regression. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part VII. LNCS, vol. 7578, pp. 347–360. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  16. 16.
    Mikolajczyk, K., Schmid, C.: Scale and affine invariant interest point detectors. IJCV 60(1), 63–86 (2004)CrossRefGoogle Scholar
  17. 17.
    Munkres, J.: Algorithms for the assignment and transportation problems. Journal of SIAM 5(1), 32–38 (1957)zbMATHMathSciNetGoogle Scholar
  18. 18.
    Ochs, P., Malik, J., Brox, T.: Segmentation of moving objects by long term video analysis. IEEE TPAMI 36(6), 1187–1200 (2014)CrossRefGoogle Scholar
  19. 19.
    Oliveira, R., Costeira, J., Xavier, J.: Optimal point correspondence through the use of rank constraints. In: IEEE CVPR, pp. 1–6 (2005)Google Scholar
  20. 20.
    Shen, Y., Wen, Z., Zhang, Y.: Augmented lagrangian alternating direction method for matrix separation based on low-rank factorization. Optimization Methods and Software 29(2), 239–263 (2014)CrossRefzbMATHMathSciNetGoogle Scholar
  21. 21.
    Shi, J., Malik, J.: Normalized cuts and image segmentation. IEEE TPAMI 22(8), 888–905 (2000)CrossRefGoogle Scholar
  22. 22.
    Shi, J., Tomasi, C.: Good features to track. In: IEEE CVPR, pp. 593–600 (1994)Google Scholar
  23. 23.
    Sivic, J., Schaffalitzky, F., Zisserman, A.: Object level grouping for video shots. IJCV 67(2), 189–210 (2006)CrossRefGoogle Scholar
  24. 24.
    Sun, D., Sudderth, E.B., Black, M.J.: Layered segmentation and optical flow estimation over time. In: IEEE CVPR, pp. 1768–1775 (2012)Google Scholar
  25. 25.
    Volz, S., Bruhn, A., Valgaerts, L., Zimmer, H.: Modeling temporal coherence for optical flow. In: IEEE ICCV, pp. 1116–1123 (2011)Google Scholar
  26. 26.
    Tomasi, C., Kanade, T.: Shape and motion from image streams under orthography: a factorization method. IJCV 9(2), 137–154 (1992)CrossRefGoogle Scholar
  27. 27.
    Torki, M., Elgammal, A.: One-shot multi-set non-rigid feature-spatial matching. In: IEEE CVPR, pp. 3058–3065 (2010)Google Scholar
  28. 28.
    Torresani, L., Kolmogorov, V., Rother, C.: Feature correspondence via graph matching: Models and global optimization. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part II. LNCS, vol. 5303, pp. 596–609. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  29. 29.
    Tron, R., Vidal, R.: A benchmark for the comparison of 3-d motion segmentation algorithms. In: IEEE CVPR, pp. 1–8 (2007)Google Scholar
  30. 30.
    Vidal, R., Favaro, P.: Low rank subspace clustering (LRSC). Pattern Recognition Letters 43, 47–61 (2014)CrossRefGoogle Scholar
  31. 31.
    Yan, J., Pollefeys, M.: A general framework for motion segmentation: Independent, articulated, rigid, non-rigid, degenerate and non-degenerate. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3954, pp. 94–106. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  32. 32.
    Zelnik-Manor, L., Irani, M.: Degeneracies, dependencies and their implications in multi-body and multi-sequence factorizations. In: IEEE CVPR (2003)Google Scholar
  33. 33.
    Zeng, Z., Chan, T.H., Jia, K., Xu, D.: Finding correspondence from multiple images via sparse and low-rank decomposition. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part V. LNCS, vol. 7576, pp. 325–339. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  34. 34.
    Zhang, Y.: An alternating direction algorithm for nonnegative matrix factorization. Tech. rep., Rice University (2010)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Pan Ji
    • 1
  • Hongdong Li
    • 1
  • Mathieu Salzmann
    • 1
    • 2
  • Yuchao Dai
    • 1
  1. 1.Australian National UniversityCanberraAustralia
  2. 2.NICTACanberraAustralia

Personalised recommendations