Background Subtraction Using Low Rank and Group Sparsity Constraints

  • Xinyi Cui
  • Junzhou Huang
  • Shaoting Zhang
  • Dimitris N. Metaxas
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7572)


Background subtraction has been widely investigated in recent years. Most previous work has focused on stationary cameras. Recently, moving cameras have also been studied since videos from mobile devices have increased significantly. In this paper, we propose a unified and robust framework to effectively handle diverse types of videos, e.g., videos from stationary or moving cameras. Our model is inspired by two observations: 1) background motion caused by orthographic cameras lies in a low rank subspace, and 2) pixels belonging to one trajectory tend to group together. Based on these two observations, we introduce a new model using both low rank and group sparsity constraints. It is able to robustly decompose a motion trajectory matrix into foreground and background ones. After obtaining foreground and background trajectories, the information gathered on them is used to build a statistical model to further label frames at the pixel level. Extensive experiments demonstrate very competitive performance on both synthetic data and real videos.


Background Subtraction General Rank Foreground Object Pixel Level Sparsity Constraint 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Sheikh, Y., Javed, O., Kanade, T.: Background subtraction for freely moving cameras. In: ICCV (2009)Google Scholar
  2. 2.
    Candes, E.J., Li, X., Ma, Y., Wright, J.: Robust principal component analysis? Journal of ACM (2011)Google Scholar
  3. 3.
    Yuan, M., Lin, Y.: Model selection and estimation in regression with grouped variables. Journals of the Royal Statistical Society (2006)Google Scholar
  4. 4.
    Candes, E., Tao, T.: Near-optimal signal recovery from random projections: Universal encoding strategies? TIT (2006)Google Scholar
  5. 5.
    Starck, J.L., Elad, M., Donoho, D.: Image decomposition via the combination of sparse representations and a variational approach. TIP (2005)Google Scholar
  6. 6.
    Tomasi, C., Kanade, T.: Shape and motion from image streams under orthography: a factorization method. IJCV (1992)Google Scholar
  7. 7.
    Brutzer, S., Hoferlin, B., Heidemann, G.: Evaluation of background subtraction techniques for video surveillance. In: CVPR (2011)Google Scholar
  8. 8.
    Jain, R., Nagel, H.: On the analysis of accumulative difference pictures from image sequences of real world scenes. TPAMI (2009)Google Scholar
  9. 9.
    Haritaoglu, I., Harwood, D., Davis, L.: W4: real-time surveillance of people and their activities. TPAMI (2000)Google Scholar
  10. 10.
    Wren, C., Azarbayejani, A., Darrell, T., Pentland, A.: Pfinder: Real-time tracking of the human body. TPAMI (2002)Google Scholar
  11. 11.
    Stauffer, C., Grimson, W.: Learning patterns of activity using real-time tracking. TPAMI (2000)Google Scholar
  12. 12.
    Elgammal, A., Duraiswami, R., Harwood, D., Davis, L.: Background and foreground modeling using nonparametric kernel density estimation for visual surveillance. Proceedings of the IEEE (2002)Google Scholar
  13. 13.
    Sheikh, Y., Shah, M.: Bayesian object detection in dynamic scenes. In: CVPR (2005)Google Scholar
  14. 14.
    Monnet, A., Mittal, A., Paragios, N., Ramesh, V.: Background modeling and subtraction of dynamic scenes. In: ICCV (2003)Google Scholar
  15. 15.
    Zhong, J., Sclaroff, S.: Segmenting foreground objects from a dynamic textured background via a robust kalman filter. In: ICCV (2003)Google Scholar
  16. 16.
    Liao, V.S., Zhao, G., Kellokumpu, Pietikainen, M., Li, S.Z.: Modeling pixel process with scale invariant local patterns for background subtraction in complex scenes. In: CVPR (2010)Google Scholar
  17. 17.
    Rao, S., Tron, R., Vidal, R., Ma, Y.: Motion segmentation in the presence of outlying, incomplete, or corrupted trajectories. TPAMI (2010)Google Scholar
  18. 18.
    Yan, J., Pollefeys, M.: A General Framework for Motion Segmentation: Independent, Articulated, Rigid, Non-rigid, Degenerate and Non-degenerate. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006, Part IV. LNCS, vol. 3954, pp. 94–106. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  19. 19.
    Hayman, E., Eklundh, J.-O.: Statistical background subtraction for a mobile observer. In: ICCV (2003)Google Scholar
  20. 20.
    Ren, Y., Chua, C., Ho, Y.: Statistical background modeling for non-stationary camera. PR Letters (2003)Google Scholar
  21. 21.
    Yuan, C., Medioni, G., Kang, J., Cohen, I.: Detecting motion regions in the presence of a strong parallax from a moving camera by multiview geometric constraints. TPAMI (2007)Google Scholar
  22. 22.
    Kwak, S., Lim, T., Nam, W., Han, B., Han, J.H.: Generalized background subtraction based on hybrid inference by belief propagation and bayesian filtering. In: ICCV (2011)Google Scholar
  23. 23.
    Huang, J., Zhang, T.: The benefit of group sparsity. The Annals of Statistics (2010)Google Scholar
  24. 24.
    Huang, J., Huang, X., Metaxas, D.: Learning with dynamic group sparsity. In: ICCV, pp. 64–71 (2009)Google Scholar
  25. 25.
    Sundaram, N., Brox, T., Keutzer, K.: Dense Point Trajectories by GPU-Accelerated Large Displacement Optical Flow. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part I. LNCS, vol. 6311, pp. 438–451. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  26. 26.
    Liu, C.: Beyond pixels: Exploring new representations and applications for motion analysis. Doctoral thesis, Massachusetts Institute of Technology (2009)Google Scholar
  27. 27.
    Boykov, Y., Veksler, O., Zabih, R.: Fast approximate energy minimization via graph cuts. TPAMI (2001)Google Scholar
  28. 28.
    Zhang, S., Huang, J., Huang, Y., Yu, Y., Li, H., Metaxas, D.: Automatic image annotation using group sparsity. In: CVPR (2010)Google Scholar
  29. 29.
    Liu, G., Lin, Z., Yu, Y.: Robust subspace segmentation by low-rank representation. In: ICML (2010)Google Scholar
  30. 30.
    Zhang, Z., Liang, X., Ma, Y.: Unwrapping low-rank textures on generalized cylindrical surfaces. In: ICCV (2011)Google Scholar
  31. 31.
    Mu, Y., Dong, J., Yuan, X., Yan, S.: Accelerated low-rank visual recovery by random projection. In: CVPR (2011)Google Scholar
  32. 32.
    Sand, P., Teller, S.: Particle video: Long-range motion estimation using point trajectories. In: CVPR (2006)Google Scholar
  33. 33.
    Tron, R., Vidal, R.: A benchmark for the comparison of 3-D motion segmentation algorithms. In: CVPR (2007)Google Scholar
  34. 34.
    R., Vidal, Y.M., Sastry, S.: Generalized principal component analysis (GPCA). In: CVPR (2003)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Xinyi Cui
    • 1
  • Junzhou Huang
    • 2
  • Shaoting Zhang
    • 1
  • Dimitris N. Metaxas
    • 1
  1. 1.CS Dept.Rutgers UniversityPiscatawayUSA
  2. 2.CSE Dept.Univ. of Texas at ArlingtonArlingtonUSA

Personalised recommendations