Skip to main content
SpringerLink
Log in

Background modeling and its evaluation for complex scenes

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

We present a new approach for modeling background in complex scenes that contain unpredicted motions caused e.g. by wind over water surface, in tree branches, or over the grass. The background model of each pixel is defined based on the observation of its spatial neighborhood in a recent history, and includes up to \(K \ge 1\) modes, which defines the frequently appeared patterns at the given pixel position in the color space, ranked in decreasing order of occurrence frequency. Foreground regions can then be detected by comparing the intensity of an observed pixel to the high frequency modes of its background model. Experiments show that our spatial-temporal background model is superior to traditional related algorithms in cases for which a pixel encounters modes that are frequent in the spatial neighborhood without being frequent enough in the actual pixel position. As an additional contribution, our paper also proposes an original assessment method, which has the advantage of avoiding the use of costly handmade ground truth sequences of foreground objects silhouettes.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Barnich O, Van Droogenbroeck M (2011) Vibe: a universal background subtraction algorithm for video sequences. Image Process IEEE Trans 20(6):1709–1724

    Article  Google Scholar 

  2. Cho SH, Kang HB (2011) Panoramic background generation using mean-shift in moving camera environment. In: Proceedings of the international conference on image processing, computer vision, and pattern recognition

  3. Cristani M, Farenzena M, Bloisi D, Murino V (2010) Background subtraction for automated multisensor surveillance: a comprehensive review. EURASIP J Adv Sig Process 2010:43

    Google Scholar 

  4. Elgammal A, Harwood D, Davis L (2000) Non-parametric model for background subtraction. In: Computer visionłECCV 2000. Springer, pp 751–767

  5. Hofmann M, Tiefenbacher P, Rigoll G (2012) Background segmentation with feedback: the pixel-based adaptive segmenter. In: IEEE computer society conference on computer vision and pattern recognition workshops (CVPRW) 2012. IEEE, pp 38–43

  6. Kim K, Chalidabhongse TH, Harwood D, Davis L (2005) Real-time foreground–background segmentation using codebook model. Real-Time Imaging 11(3):172–185

    Article  Google Scholar 

  7. Kim W, Jung C, Kim C (2011) Spatiotemporal saliency detection and its applications in static and dynamic scenes. Circ Syst Video Technol IEEE Trans 21(4):446–456

    Article  MathSciNet  Google Scholar 

  8. Ko T, Soatto S, Estrin D (2008) Background subtraction on distributions. In: Computer vision–ECCV 2008. Springer, pp 276–289

  9. Ko T, Soatto S, Estrin D (2010) Warping background subtraction. In: IEEE conference on computer vision and pattern recognition (CVPR) 2010. IEEE, pp 1331–1338

  10. Liu Y, Yao H, Gao W, Chen X, Zhao D (2007) Nonparametric background generation. J Vis Commun Image Represent 18(3):253–263

    Article  Google Scholar 

  11. Mahadevan V, Vasconcelos N (2010) Spatiotemporal saliency in dynamic scenes. Pattern Anal Mach Intell IEEE Trans 32(1):171–177

    Article  Google Scholar 

  12. Mittal A, Paragios N (2004) Motion-based background subtraction using adaptive kernel density estimation. In: Proceedings of the 2004 IEEE computer society conference on computer vision and pattern recognition 2004. CVPR 2004, vol 2. IEEE, pp II–302

  13. Noriega P, Bernier O (2006) Real time illumination invariant background subtraction using local kernel histograms. In: British machine vision association (BMVC), pp 567–580

  14. Pal A, Schaefer G, Celebi ME (2010) Robust codebook-based video background subtraction. In: IEEE international conference on acoustics speech and signal processing (ICASSP) 2010. IEEE, pp 1146–1149

  15. Pless R (2005) Spatio-temporal background models for outdoor surveillance. EURASIP J Appl Sig Process 2005:2281–2291

  16. Sidibé D, Strauss O, Puech W (2009) Automatic background generation from a sequence of images based on robust mode estimation. In: IS&T/SPIE electronic imaging. International Society for Optics and Photonics, pp 72 500P–72 500P

  17. Stauffer C, Grimson WEL (1999) Adaptive background mixture models for real-time tracking. In: IEEE computer society conference on computer vision and pattern recognition 1999, vol 2. IEEE

  18. Tian Y, Wang Y, Hu Z, Huang T (2013) Selective eigenbackground for background modeling and subtraction in crowded scenes. In: IEEE transactions on circuits and systems for video technology

  19. Wang H, Suter D (2006) A novel robust statistical method for background initialization and visual surveillance. In: Computer vision–ACCV 2006. Springer, pp 328–337

  20. Wren CR, Azarbayejani A, Darrell T, Pentland AP (1997) Pfinder: real-time tracking of the human body. IEEE Trans Pattern Anal Mach Intell 19(7):780–785

    Article  Google Scholar 

  21. Wu M, Peng X (2010) Spatio-temporal context for codebook-based dynamic background subtraction. AEU-Int J Electron Commun 64(8):739–747

    Article  Google Scholar 

Download references

Acknowledgments

The authors would like to thank all the anonymous reviewers for their time and valuable comments. This work was supported in part by the National Natural Science Foundation of China, under Project 61302125, and in part by the Fundamental Research Funds for the Central Universities, China.

Author information

Authors and Affiliations

  1. School of Information Science and Technology, East China Normal University, Shanghai, China, 200241

    Li Sun & Yiqing Liu

  2. School of Automation Engineering, Shanghai University of Electric Power, Shanghai, China, 200090

    Wenjuan Sheng

Authors
  1. Li Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wenjuan Sheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yiqing Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Li Sun.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Sun, L., Sheng, W. & Liu, Y. Background modeling and its evaluation for complex scenes. Multimed Tools Appl 74, 3947–3966 (2015). https://doi.org/10.1007/s11042-013-1806-0

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-013-1806-0

Keywords

Search

Navigation