Over-Segmentation Based Background Modeling and Foreground Detection with Shadow Removal by Using Hierarchical MRFs

  • Te-Feng Su
  • Yi-Ling Chen
  • Shang-Hong Lai
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6494)


In this paper, we propose a novel over-segmentation based method for the detection of foreground objects from a surveillance video by integrating techniques of background modeling and Markov Random Fields classification. Firstly, we introduce a fast affinity propagation clustering algorithm to produce the over-segmentation of a reference image by taking into account color difference and spatial relationship between pixels. A background model is learned by using Gaussian Mixture Models with color features of the segments to represent the time-varying background scene. Next, each segment is treated as a node in a Markov Random Field and assigned a state of foreground, shadow and background, which is determined by using hierarchical belief propagation. The relationship between neighboring regions is also considered to ensure spatial coherence of segments. Finally, we demonstrate experimental results on several image sequences to show the effectiveness and robustness of the proposed method.


Local Binary Pattern Background Modeling Foreground Object Shadow Detection Background Scene 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Stauffer, C., Grimson, W.: Adaptive background mixture models for real-time tracking. In: IEEE Conference on CVPR, vol. 2, p. 252 (1999)Google Scholar
  2. 2.
    Paragios, N., Ramesh, V.: A mrf-based approach for real-time subway monitoring. In: IEEE Conference on CVPR, vol. 1, pp. I–1034–I–1040 (2001)Google Scholar
  3. 3.
    Matsuyama, T., Ohya, T., Habe, H.: Background subtraction for nonstationary scenes. In: Proceedings of ACCV, pp. 662–667 (2000)Google Scholar
  4. 4.
    Heikkila, M., Pietikainen, M.: A texture-based method for modeling the background and detecting moving objects. IEEE Transactions on PAMI 28, 657–662 (2006)CrossRefGoogle Scholar
  5. 5.
    Felzenszwalb, P., Huttenlocher, D.: Efficient belief propagation for early vision. International Journal of Computer Vision 70, 41–54 (2006)CrossRefGoogle Scholar
  6. 6.
    Zivkovic, Z., van der Heijden, F.: Recursive unsupervised learning of finite mixture models. IEEE Transactions on PAMI 26, 651–656 (2004)CrossRefGoogle Scholar
  7. 7.
    Elgammal, A., Harwood, D., Davis, L.: Non-parametric model for background subtraction. In: Vernon, D. (ed.) ECCV 2000. LNCS, vol. 1843, pp. 751–767. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  8. 8.
    Migdal, J., Grimson, W.E.L.: Background subtraction using markov thresholds. In: IEEE Workshop on Motion and Video Computing, vol. 2, pp. 58–65 (2005)Google Scholar
  9. 9.
    Wang, Y., Loe, K.F., Wu, J.K.: A dynamic conditional random field model for foreground and shadow segmentation. IEEE Transactions on PAMI 28, 279–289 (2006)CrossRefGoogle Scholar
  10. 10.
    Huang, S.S., Fu, L.C., Hsiao, P.Y.: Region-level motion-based background modeling and subtraction using mrfs. IEEE Transactions on IP 16, 1446–1456 (2007)MathSciNetGoogle Scholar
  11. 11.
    Chen, Y.T., Chen, C.S., Huang, C.R., Hung, Y.P.: Efficient hierarchical method for background subtraction. Pattern Recognition 40, 2706–2715 (2007)CrossRefzbMATHGoogle Scholar
  12. 12.
    Martel-Brisson, N., Zaccarin, A.: Learning and removing cast shadows through a multidistribution approach. IEEE Transactions on PAMI 29, 1133–1146 (2007)CrossRefGoogle Scholar
  13. 13.
    Cucchiara, R., Grana, C., Piccardi, M., Prati, A.: Detecting moving objects, ghosts, and shadows in video streams. IEEE Transactions on PAMI 25, 1337–1342 (2003)CrossRefGoogle Scholar
  14. 14.
    Zeng, H.C., Lai, S.H.: Adaptive foreground object extraction for real-time video surveillance with lighting variations. In: ICASSP, pp. I–1201–I–1204 (2007)Google Scholar
  15. 15.
    Salvador, E., Cavallaro, A., Ebrahimi, T.: Cast shadow segmentation using invariant color features. Computer Visual Image Understand 95, 238–259 (2004)CrossRefGoogle Scholar
  16. 16.
    Zhang, W., Fang, X.Z., Yang, X., Wu, Q.: Moving cast shadows detection using ratio edge. IEEE Transactions on Multimedia 9, 1202–1214 (2007)CrossRefGoogle Scholar
  17. 17.
    Frey, B.J., Dueck, D.: Clustering by passing messages between data points. Science 315, 972–976 (2007)MathSciNetCrossRefzbMATHGoogle Scholar
  18. 18.
    Li, L., Huang, W., Gu, I.Y.H., Tian, Q.: Statistical modeling of complex backgrounds for foreground object detection. IEEE Transactions on IP 13, 1459–1472 (2004)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Te-Feng Su
    • 1
  • Yi-Ling Chen
    • 1
  • Shang-Hong Lai
    • 1
  1. 1.Department of Computer ScienceNational Tsing Hua UniversityHsinchuTaiwan R.O.C.

Personalised recommendations