Iterative Automated Foreground Segmentation in Video Sequences Using Graph Cuts

  • Tomislav Hrkać
  • Karla Brkić
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9358)


In this paper we propose a method for foreground object segmentation in videos using an improved version of the GrabCut algorithm. Motivated by applications in de-identification, we consider a static camera scenario and take into account common problems with the original algorithm that can result in poor segmentation. Our improvements are as follows: (i) using background subtraction, we build GMM-based segmentation priors; (ii) in building foreground and background GMMs, the contributions of pixels are weighted depending on their distance from the boundary of the object prior; (iii) probabilities of pixels belonging to foreground or background are modified by taking into account the prior pixel classification as well as its estimated confidence; and (iv) the smoothness term of GrabCut is modified by discouraging boundaries further away from the object prior. We perform experiments on CDnet 2014 Pedestrian Dataset and show considerable improvements over a reference implementation of GrabCut.


  1. 1.
    Bai, X., Wang, J., Simons, D., Sapiro, G.: Video snapcut: robust video object cutout using localized classifiers. In: ACM SIGGRAPH 2009 Papers. SIGGRAPH 2009, pp. 70:1–70:11. ACM, New York (2009).
  2. 2.
    Benenson, R., Omran, M., Hosang, J., Schiele, B.: Ten years of pedestrian detection, what have we learned? In: Agapito, L., Bronstein, M.M., Rother, C. (eds.) ECCV 2014 Workshops. LNCS, vol. 8926, pp. 613–627. Springer, Heidelberg (2015) Google Scholar
  3. 3.
    Boykov, Y., Jolly, M.P.: Interactive graph cuts for optimal boundary and region segmentation of objects in n-d images. In: Proceedings of the Eighth IEEE International Conference on Computer Vision. ICCV 2001, vol. 1, pp. 105–112 (2001)Google Scholar
  4. 4.
    Bradski, G.: The OpenCV library. Dr. Dobb’s J. Softw. Tools 25, 120–126 (2000)Google Scholar
  5. 5.
    Brutzer, S., Hoferlin, B., Heidemann, G.: Evaluation of background subtraction techniques for video surveillance. In: Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2011, pp. 1937–1944. IEEE Computer Society, Washington, DC (2011)Google Scholar
  6. 6.
    Cheung, S.C.S., Kamath, C.: Robust techniques for background subtraction in urban traffic video. In: Visual Communications and Image Processing, vol. 5308(1), pp. 881–892 (2004)Google Scholar
  7. 7.
    Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: Proceedings of CVPR, pp. 886–893 (2005)Google Scholar
  8. 8.
    Dollar, P., Tu, Z., Perona, P., Belongie, S.: Integral channel features. In: Proceedings of the British Machine Vision Conference. BMVA Press (2009). doi: 10.5244/C.23.91
  9. 9.
    Dollar, P., Wojek, C., Schiele, B., Perona, P.: Pedestrian detection: an evaluation of the state of the art. IEEE Trans. Pattern Anal. Mach. Intell. 34(4), 743–761 (2012). doi: 10.1109/TPAMI.2011.155CrossRefGoogle Scholar
  10. 10.
    Felzenszwalb, P.F., Girshick, R.B., McAllester, D., Ramanan, D.: Object detection with discriminatively trained part-based models. IEEE Trans. Pattern Anal. Mach. Intell. 32(9), 1627–1645 (2010). doi: 10.1109/TPAMI.2009.167CrossRefGoogle Scholar
  11. 11.
    Harville, M.: A framework for high-level feedback to adaptive, per-pixel, mixture-of-gaussian background models. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002, Part III. LNCS, vol. 2352, pp. 543–560. Springer, Heidelberg (2002) CrossRefGoogle Scholar
  12. 12.
    Hernandez-Vela, A., Reyes, M., Ponce, V., Escalera, S.: Grabcut-based human segmentation in video sequences. Sensors 12, 15376–15393 (2012). doi: 10.3390/s121115376CrossRefGoogle Scholar
  13. 13.
    Herrero, S., Bescós, J.: Background subtraction techniques: systematic evaluation and comparative analysis. In: Blanc-Talon, J., Philips, W., Popescu, D., Scheunders, P. (eds.) ACIVS 2009. LNCS, vol. 5807, pp. 33–42. Springer, Heidelberg (2009) CrossRefGoogle Scholar
  14. 14.
    Kroeger, T., Kappes, J.H., Beier, T., Koethe, U., Hamprecht, F.A.: Asymmetric cuts: joint image labeling and partitioning. In: Jiang, X., Hornegger, J., Koch, R. (eds.) GCPR 2014. LNCS, vol. 8753, pp. 199–211. Springer, Heidelberg (2014) Google Scholar
  15. 15.
    Leibe, B., Seemann, E., Schiele, B.: Pedestrian detection in crowded scenes. In: Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2005), vol. 1, pp. 878–885. IEEE Computer Society, Washington (2005).
  16. 16.
    Ouyang, W., Wang, X.: A discriminative deep model for pedestrian detection with occlusion handling. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3258–3265, June 2012Google Scholar
  17. 17.
    Ouyang, W., Zeng, X., Wang, X.: Modeling mutual visibility relationship in pedestrian detection. In: 2013 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3222–3229, June 2013Google Scholar
  18. 18.
    Poullot, S., Satoh, S.: Vabcut: a video extension of grabcut for unsupervised video foreground object segmentation. In: Proceedings of VISAPP (2014)Google Scholar
  19. 19.
    Rother, C., Vladimir, K., Blake, A.: “GrabCut” - interactive foreground extraction using iterated graph cuts. In: Proceedings of SIGGRAPH (2004)Google Scholar
  20. 20.
    Sermanet, P., Kavukcuoglu, K., Chintala, S., LeCun, Y.: Pedestrian detection with unsupervised multi-stage feature learning. In: Proceedings of CVPR, pp. 3626–3633 (2013)Google Scholar
  21. 21.
    Stauffer, C., Grimson, W.E.L.: Adaptive background mixture models for real-time tracking. In: CVPR, pp. 2246–2252 (1999)Google Scholar
  22. 22.
    Sun, J., Zhang, W., Tang, X., Shum, H.-Y.: Background cut. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3952, pp. 628–641. Springer, Heidelberg (2006) CrossRefGoogle Scholar
  23. 23.
    Viola, P., Jones, M.: Robust real-time object detection. Int. J. Comput. Vis. 57, 137–154 (2001)CrossRefGoogle Scholar
  24. 24.
    Viola, P., Jones, M.J., Snow, D.: Detecting pedestrians using patterns of motion and appearance. Int. J. Comput. Vis. 63(2), 153–161 (2005). doi: 10.1007/s11263-005-6644-8CrossRefGoogle Scholar
  25. 25.
    Wang, Y., Jodoin, P.M., Porikli, F., Konrad, J., Benezeth, Y., Ishwar, P.: CDnet 2014: an expanded change detection benchmark dataset. In: Proceedings of IEEE Workshop on Change Detection (CDW 2014) at CVPR 2014, pp. 387–394 (2014)Google Scholar
  26. 26.
    Wren, C.R., Azarbayejani, A., Darrell, T., Pentland, A.P.: Pfinder: real-time tracking of the human body. IEEE Trans. Pattern Anal. Mach. Intell. 19(7), 780–785 (1997)CrossRefGoogle Scholar
  27. 27.
    Zivkovic, Z.: Improved adaptive gaussian mixture model for background subtraction. In: ICPR (2), pp. 28–31 (2004)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Open Access This chapter is distributed under the terms of the Creative Commons Attribution Noncommercial License, which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.

Authors and Affiliations

  1. 1.Faculty of Electrical Engineering and ComputingUniversity of ZagrebZagrebCroatia

Personalised recommendations