Efficient Foreground Layer Extraction in Video

  • Zongmin Li
  • Liangliang Zhong
  • Yujie Liu
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6297)


Extracting foreground moving objects from video sequences is an important task and also a hot topic in computer vision and image processing. Segmentation results can be used in many object-based video applications such as object-based video coding, content-based video retrieval, intelligent video surveillance, video-based human-computer interaction, etc. In this paper, we propose a framework for real-time segmentation of foreground moving objects from monocular video sequences with static background. Our algorithm can extract foreground layers with cast shadow removal accurately and efficiently. To reduce the computation cost, we use Gaussian Mixture Models to model the scene and obtain initial foreground regions. Then we combine the initial foreground mask with shadow detection to generate a quadrant-map for each region. Based on these quadrant-maps, Markov Random Field model is built on each region and the graph cut algorithm is used to get the optimal binary segmentation. To ensure good temporal consistency, we reuse previous segmentation results to build the current foreground model. Experimental results on various videos demonstrate the efficiency of our proposed method.


Video object segmentation Markov random field Shadow removal 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Zhang, D., Lu, G.: Segmentation of moving objects in image sequence: a review. Circuits Systems Signal Processing 20(2), 143–183 (2001)zbMATHCrossRefGoogle Scholar
  2. 2.
    Yilmaz, A., Javed, O., Shah, M.: Object tracking: A survey. ACM Computing Surveys, 1–45 (2006)Google Scholar
  3. 3.
    Turaga, P., Chellappa, R., Subrahmanian, V., Udrea, O.: Machine recognition of human activities: A survey. IEEE Trans. CSVT, 1473–1488 (November 2008)Google Scholar
  4. 4.
    Pritch, Y., Rav-Acha, A., Peleg, S.: Non-chronological video synopsisand indexing. IEEE Trans. Pattern Anal. Machine Intell., 1971–1984 (November 2008)Google Scholar
  5. 5.
    Yuk, J., et al.: Object-Based Surveillance Video Retrieval System with Real-Time Indexing Methodology. Image Analysis and Recognition, 626–637 (2007)Google Scholar
  6. 6.
    Rother, C., Kolmogorov, V., Blake, A.: GrabCut –interactive foreground extraction using iterated graph cuts. In: ACM Trans. on Graphics, SIGGRAPH (2004)Google Scholar
  7. 7.
    Bai, X., Wang, J., Simons, D., Sapiro, G.: Video SnapCut: Robust Video Object Cutout Using Localized Classifiers. In: Proceedings of ACM SIGGRAPH (2009)Google Scholar
  8. 8.
    Kolmogorov, V., Criminisi, A., Blake, A., Cross, G., Rother, C.: Probabilistic fusion of stereo with color and contrast for bi-layer segmentation. IEEE TPAMI (2006)Google Scholar
  9. 9.
    Boykov, Y., Jolly, M.-P.: Interactive graph cuts for optimal boundary & region segmentation of objects in N-D images. In: Proceedings of ICCV (July 2001)Google Scholar
  10. 10.
    Sun, J., Zhang, W., Tang, X., Shum, H.Y.: Background cut. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006,Part II. LNCS, vol. 3952, pp. 628–641. Springer, Heidelberg (2006)Google Scholar
  11. 11.
    Criminisi, A., Cross, G., Blake, A., Kolmogorov, V.: Bilayer Segmenta-tion of Live Video. In: Proc. Int’l. Conf. CVPR, pp. 53–60 (2006)Google Scholar
  12. 12.
    Yin, P., Criminisi, A., Winn, J., Essa, I.: Tree-based Classifiers for Bilayer Video Segmentation. In: Proc. Computer Vision and Pattern Recognition (2007)Google Scholar
  13. 13.
    Wu, X., Wang, Y., Zheng, X.: Monocular video foreground segmentation system. In: ICPR (2008)Google Scholar
  14. 14.
    Stauffer, C., Grimson, W.E.L.: Adaptive background mixture models for real-time tracking. In: Proc. Computer Vision and Pattern Recognition (1999)Google Scholar
  15. 15.
    Elgammal, A., Duraiswami, R., Harwood, D., Davis, L.S.: Background and Foreground Modeling Using Nonparametric Kernel Density Estimation for Visual Surveillance. Proc. of IEEE, 1151–1163 (2002)Google Scholar
  16. 16.
    Salvador, E., Cavallaro, A., Ebrahimi, T.: Cast shadow segmentation using invariant color features. CV IU 95(2), 238–259 (2004)Google Scholar
  17. 17.
    Orchard, M.T., Bouman, C.A.: Color quantization of image. IEEE Trans. Signal Process (1991)Google Scholar
  18. 18.
    Wollborn, M., Mech, R.: Refined procedure for objective evaluation of video object segmentation algorithms. ISO/IEC JTC1/SC29/WG11/ M3448 (March 1998)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Zongmin Li
    • 1
  • Liangliang Zhong
    • 1
  • Yujie Liu
    • 1
  1. 1.College of Computer and Communication EngineeringChina University of PetroleumDongyingChina

Personalised recommendations