Inlier Estimation for Moving Camera Motion Segmentation

  • Xuefeng LiangEmail author
  • Cuicui Zhang
  • Takashi Matsuyama
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9006)


In moving camera videos, motion segmentation is often performed on the optical flow. However, there exist two challenges: (1) Camera motions lead to three primary flows in optical flow: translation, rotation, and radial flow. They are not all solved in existing frameworks under Cartesian coordinate system; (2) A moving camera introduces 3D motion, the depth discontinuities cause the motion discontinuities that severely confuse the motion segmentation. Meanwhile, the mixture of the camera motion and moving objects’ motions make indistinctness between foreground and background. In this work, our solution is to find a low order polynomial to model the background flow field due to its coherence. To this end, we first amend the Helmholts-Hodge Decomposition by adding coherence constraints, which can handle translation, rotation, and radial flow fields under a unified framework. Secondly, we introduce an Incoherence Map and a progressive Quad-Tree partition to reject moving objects and motion discontinuities. Finally, the low order polynomial is achieved from the rest flow samples on two potentials in HHD. We present results on more than twenty videos from four benchmarks. Extensive experiments demonstrate a better performance in dealing with challenging scenes with complex backgrounds. Our method improves the segmentation accuracy of state-of-the-art by \(10\%\sim 30\%\).


Optical Flow Camera Motion Radial Flow Motion Segmentation Depth Discontinuity 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



This work is supported by: Japan Society for the Promotion of Science, Scientific Research KAKENHI for Grant-in-Aid for Young Scientists (ID:25730113).

Supplementary material

Supplementary material (avi 1,537 KB)


  1. 1.
    Chen, Y.M., Bajic, I.V.: Motion vector outlier rejection cascade for global motion estimation. IEEE Signal Process. Lett. 17, 197–200 (2010)CrossRefGoogle Scholar
  2. 2.
    Chen, Y.M., Bajic, I.V.: A joint approach to global motion estimation and motion segmentation from a coarsely sampled motion vector field. IEEE Trans. Circ. Syst. Video Technol. 21, 1316–1328 (2011)CrossRefGoogle Scholar
  3. 3.
    Qian, C., Bajic, I.V.: Global motion estimation under translation-zoom ambiguity. In: Proceedings of IEEE PacRim, pp. 46–51 (2013)Google Scholar
  4. 4.
    Narayana, M., Hanson, A., Learned-Miller, E.: Coherent motion segmentation in moving camera videos using optical flow orientations. In: ICCV (2013)Google Scholar
  5. 5.
    Brox, T., Malik, J.: Object segmentation by long term analysis of point trajectories. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part V. LNCS, vol. 6315, pp. 282–295. Springer, Heidelberg (2010) CrossRefGoogle Scholar
  6. 6.
    Ochs, P., Malik, J., Brox, T.: Segmentation of moving objects by long term video analysis. IEEE Trans. Pattern Anal. Mach. Intell. (TPAMI) 36(6), 1187–1200 (2014)CrossRefGoogle Scholar
  7. 7.
    Elqursh, A., Elgammal, A.: Online moving camera background subtraction. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part VI. LNCS, vol. 7577, pp. 228–241. Springer, Heidelberg (2012) CrossRefGoogle Scholar
  8. 8.
    Kwak, S., Lim, T., Nam, W., Han, B., Han, J.H.: Generalized background subtraction based on hybrid inference by belief propagation and bayesian filtering. In: ICCV, pp. 2174–2181 (2011)Google Scholar
  9. 9.
    Irani, M., Rousso, B., Peleg, S.: Recovery of ego-motion using image stabilization. In: CVPR (1994)Google Scholar
  10. 10.
    Helmholtz, H.: On integrals of the hydrodynamical equations, which express vortex-motion. Philos. Mag. J. Sci. 33, 485–512 (1867)Google Scholar
  11. 11.
    Bhatia, H., Norgard, G., Pascucci, V., Bremer, P.T.: The helmholtz-hodge decomposition - a survey. IEEE Trans. Vis. Comput. Graph. (TVCG) 19, 1386–1404 (2013)CrossRefGoogle Scholar
  12. 12.
    Su, Y., Sun, M.T., Hsu, V.: Global motion estimation from coarsely sampled motion vector field and the applications. IEEE Trans. Circuits System and Video Technology 15, 232–242 (2005)CrossRefGoogle Scholar
  13. 13.
    Tron, R., Vidal, R.: A benchmark for the comparison of 3D motion segmentation algorithms. In: CVPR (2007)Google Scholar
  14. 14.
    Tsai, D., Flagg, M., Rehg, J.M.: Motion coherent tracking with multi-label mrf optimization. In: BMVC (2010)Google Scholar
  15. 15.
    Brox, T., Bruhn, A., Papenberg, N., Weickert, J.: High accuracy optical flow estimation based on a theory for warping. In: Pajdla, T., Matas, J.G. (eds.) ECCV 2004. LNCS, vol. 3024, pp. 25–36. Springer, Heidelberg (2004) CrossRefGoogle Scholar
  16. 16.
    Liang, X., McOwan, P., Johnston, A.: A biologically inspired framework for spatial and spectral velocity estimations. J. Opt. Soc. Am. A 28, 713–723 (2011)CrossRefGoogle Scholar
  17. 17.
    Dembczynski, K., Jachnik, A., Kotlowski, W., Waegeman, W., Hullermeier, E.: Optimizing the F-measure in multi-label classication: Plug-in rule approach versus structured loss minimization. In: International Conference on Machine Learning (ICML), pp. 1130–1138 (2013)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Xuefeng Liang
    • 1
    Email author
  • Cuicui Zhang
    • 1
  • Takashi Matsuyama
    • 1
  1. 1.IST, Graduate School of InformaticsKyoto UniversityKyotoJapan

Personalised recommendations