Advertisement

Figure-Ground Segmentation—Pixel-Based

  • Ahmed Elgammal

Abstract

Background subtraction is a widely used concept to detect moving objects in videos taken from a static camera. In the last two decades several algorithms have been developed for background subtraction and were used in various important applications such as visual surveillance, sports video analysis, motion capture, etc. Various statistical approaches have been proposed to model scene background. In this chapter we review the concept and the practice in background subtraction. We discuss several basic statistical background subtraction models, including parametric Gaussian models and nonparametric models. We discuss the issue of shadow suppression, which is essential for human motion analysis applications. We also discuss approaches and tradeoffs for background maintenance, and point out many of the recent developments within the background subtraction paradigm.

Keywords

Background Subtraction Kernel Density Estimation Dynamic Texture Foreground Region Outdoor Scene 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References

  1. 1.
    Burger, W., Burge, M.: Digital Image Processing, an Algorithmic Introduction Using Java. Springer, New York (2008) Google Scholar
  2. 2.
    Cheng, Y.: Mean shift, mode seeking, and clustering. IEEE Trans. Pattern Anal. Mach. Intell. 17(8), 790–799 (1995) CrossRefGoogle Scholar
  3. 3.
    Comaniciu, D.: Nonparametric robust methods for computer vision. PhD thesis, Rutgers, The State University of New Jersey (January 2000) Google Scholar
  4. 4.
    Comaniciu, D., Meer, P.: Mean shift analysis and applications. In: IEEE International Conference on Computer Vision (September 1999) Google Scholar
  5. 5.
    Cucchiara, R., Grana, C., Piccardi, M., Prati, A.: Detecting moving objects, ghosts, and shadows in video streams. IEEE Trans. Pattern Anal. Mach. Intell. 25, 1337–1342 (2003) CrossRefGoogle Scholar
  6. 6.
    Dempster, A., Laird, N., Rubin, D.: Maximum likelihood from incomplete data via the em algorithm. J. R. Stat. Soc. B 39, 1–38 (1977) MathSciNetMATHGoogle Scholar
  7. 7.
    Duda, R.O., Stork, D.G., Hart, P.E.: Pattern Classification. Wiley, New York (2000) Google Scholar
  8. 8.
    Eghbali, H.J.: K-s test for detecting changes from Landsat imagery data. IEEE Trans. Syst. Man Cybern. 9(1), 17–23 (1979) CrossRefGoogle Scholar
  9. 9.
    Elgammal, A.: Efficient kernel density estimation for realtime computer vision. PhD thesis, University of Maryland (2002) Google Scholar
  10. 10.
    Elgammal, A., Duraiswami, R., Davis, L.S.: Efficient non-parametric adaptive color modeling using fast gauss transform. In: IEEE Conference on Computer Vision and Pattern Recognition (December 2001) Google Scholar
  11. 11.
    Elgammal, A., Harwood, D., Davis, L.S.: Nonparametric background model for background subtraction. In: European Conference of Computer Vision (2000) Google Scholar
  12. 12.
    Elgammal, A., Duraiswami, R., Harwood, D., Davis, L.S.: Background and foreground modeling using non-parametric kernel density estimation for visual surveillance. Proc. IEEE 90(7), 1151–1163 (2002) CrossRefGoogle Scholar
  13. 13.
    Forsyth, D.A., Ponce, J.: Computer Vision a Modern Approach. Prentice Hall, New York (2002) Google Scholar
  14. 14.
    Friedman, N., Russell, S.: Image segmentation in video sequences: A probabilistic approach. In: Conference on Uncertainty in Artificial Intelligence (1997) Google Scholar
  15. 15.
    Fukunaga, K., Hostetler, L.D.: The estimation of the gradient of a density function, with application in pattern recognition. IEEE Trans. Inf. Theory 21, 32–40 (1975) MathSciNetMATHCrossRefGoogle Scholar
  16. 16.
    Gao, X., Boult, T.E.: Error analysis of background adaption. In: IEEE Conference on Computer Vision and Pattern Recognition (2000) Google Scholar
  17. 17.
    Grimson, W.E.L., Stauffer, C., Romano, R., Lee, L.: Using adaptive tracking to classify and monitor activities in a site. In: IEEE Conference on Computer Vision and Pattern Recognition (1998) Google Scholar
  18. 18.
    Hall, E.L.: Computer Image Processing and Recognition. Academic Press, San Diego (1979) MATHGoogle Scholar
  19. 19.
    Han, B., Comaniciu, D., Davis, L.: Sequential kernel density approximation through mode propagation: Applications to background modeling. In: Asian Conference on Computer Vision (2004) Google Scholar
  20. 20.
    Haritaoglu, I., Harwood, D., Davis, L.S.: W4: who? when? where? what? a real time system for detecting and tracking people. In: International Conference on Face and Gesture Recognition (1998) Google Scholar
  21. 21.
    Harville, M.: A framework for high-level feedback to adaptive, per-pixel, mixture-of-gaussian background models. In: European Conference on Computer Vision (2002) Google Scholar
  22. 22.
    Hayman, E., Eklundh, J.O.: Statistical background subtraction for a mobile observer. In: International Conference on Computer Vision (2003) Google Scholar
  23. 23.
    Horprasert, T., Harwood, D., Davis, L.S.: A statistical approach for real-time robust background subtraction and shadow detection. In: IEEE Frame-Rate Applications Workshop (1999) Google Scholar
  24. 24.
    Hsu, Y.Z., Nagel, H.H., Rekers, G.: New likelihood test methods for change detection in image sequences. Comput. Vis. Graph. Image Process. 26, 73–106 (1984) CrossRefGoogle Scholar
  25. 25.
    Huerta, I., Holte, M., Moeslund, T.B., Gonzalez, J.: Detection and removal of chromatic moving shadows in surveillance scenarios. In: International Conference on Computer Vision (2009) Google Scholar
  26. 26.
    Jabri, S., Duric, Z., Wechsler, H., Rosenfeld, A.: Detection and location of people in video images using adaptive fusion of color and edge information. In: International Conference of Pattern Recognition (2000) Google Scholar
  27. 27.
    Jain, R.C., Nagel, H.H.: On the analysis of accumulative difference pictures from image sequences of real world scenes. IEEE Trans. Pattern Anal. Mach. Intell. 1(2), 206–213 (1979) CrossRefGoogle Scholar
  28. 28.
    Javed, O., Shafique, K., Shah, M.: A hierarchical approach to robust background subtraction using color and gradient information. In: IEEE Workshop on Motion and Video Computing (2002) Google Scholar
  29. 29.
    Jin, Y.X., Tao, L.M., Di, H., Rao, N.I., Xu, G.Y.: Background modeling from a free-moving camera by multi-layer homography algorithm. In: International Conference on Image Processing (2008) Google Scholar
  30. 30.
    Karmann, K.P., Brandt, A.V.: Moving object recognition using and adaptive background memory. In: Time-Varying Image Processing and Moving Object Recognition. Elsevier, Amsterdam (1990) Google Scholar
  31. 31.
    Karmann, K.P., Brandt, A.V., Gerl, R.: Moving object segmentation based on adaptive reference images. In: Signal Processing V: Theories and Application. Elsevier, Amsterdam (1990) Google Scholar
  32. 32.
    Kim, K., Chalidabhongse, T.H., Harwood, D., Davis, L.: Background modeling and subtraction by codebook construction. In: International Conference on Image Processing (2004) Google Scholar
  33. 33.
    Kim, K., Harwood, D., Davis, L.S.: Background updating for visual surveillance. In: International Symposium on Visual Computing (2005) Google Scholar
  34. 34.
    Koller, D., Weber, J., Huang, T., Malik, J., Ogasawara, G., Rao, B., Russell, S.: Towards robust automatic traffic scene analysis in real-time. In: International Conference of Pattern Recognition (1994) Google Scholar
  35. 35.
    Levine, M.D.: Vision in Man and Machine. McGraw-Hill, New York (1985) Google Scholar
  36. 36.
    Maddalena, L., Petrosino, A.: A self-organizing approach to background subtraction for visual surveillance applications. IEEE Trans. Image Process. 17(7), 1168–1177 (2008) MathSciNetCrossRefGoogle Scholar
  37. 37.
    Matsuyama, T., Ohya, T., Habe, H.: Background subtraction for nonstationary scenes. In: Asian Conference on Computer Vision (2000) Google Scholar
  38. 38.
    Mckenna, S.J., Jabri, S., Duric, Z., Wechsler, H., Rosenfeld, A.: Tracking groups of people. Comput. Vis. Image Underst. 80, 42–56 (2000) MATHCrossRefGoogle Scholar
  39. 39.
    Mittal, A., Huttenlocher, D.: Scene modeling for wide area surveillance and image synthesis. In: Computer Vision and Pattern Recognition (2000) Google Scholar
  40. 40.
    Mittal, A., Paragios, N.: Motion-based background subtraction using adaptive kernel density estimation. In: Computer Vision and Pattern Recognition (2004) Google Scholar
  41. 41.
    Neal, R.M., Hinton, G.E.: A new view of the em algorithm that justifies incremental and other variants. In: Learning in Graphical Models, pp. 355–368. Kluwer Academic, Dordrecht (1993) Google Scholar
  42. 42.
    Parag, T., Elgammal, A., Mittal, A.: A framework for feature selection for background subtraction. In: IEEE Conference on Computer Vision and Pattern Recognition (June 2006) Google Scholar
  43. 43.
    Patwardhan, K., Sapiro, G., Morellas, V.: Robust foreground detection in video using pixel layers. IEEE Trans. Pattern Anal. Mach. Intell. 30, 746–751 (2008) CrossRefGoogle Scholar
  44. 44.
    Piccardi, M., Jan, T.: Mean-shift background image modelling. In: International Conferencing on Image Processing (2004) Google Scholar
  45. 45.
    Ren, Y., Chua, C.S., Ho, Y.K.: Statistical background modeling for non-stationary camera. Pattern Recognit. Lett. 24(1–3), 183–196 (2003) MATHCrossRefGoogle Scholar
  46. 46.
    Rittscher, J., Kato, J., Joga, S., Blake, A.: A probabilistic background model for tracking. In: European Conference on Computer Vision (2000) Google Scholar
  47. 47.
    Rowe, S., Blake, A.: Statistical mosaics for tracking. Image Vis. Comput. 14(8), 549–564 (1996) CrossRefGoogle Scholar
  48. 48.
    Scott, D.W.: Multivariate Density Estimation. Wiley-Interscience, New York (1992) MATHCrossRefGoogle Scholar
  49. 49.
    Sheikh, Y., Shah, M.: Bayesian modeling of dynamic scenes for object detection. IEEE Trans. Pattern Anal. Mach. Intell. 27, 1778–1792 (2005) CrossRefGoogle Scholar
  50. 50.
    Sheikh, Y., Javed, O., Kanade, T.: Background subtraction for freely moving cameras. In: International Conference on Computer Vision (2009) Google Scholar
  51. 51.
    Soatto, S., Doretto, G., Wu, Y.: Dynamic textures. In: International Conference on Computer Vision (2001) Google Scholar
  52. 52.
    Stauder, J., Mech, R., Ostermann, J.: Detection of moving cast shadows for object segmentation. IEEE Trans. Multimed. 1, 65–76 (1999) CrossRefGoogle Scholar
  53. 53.
    Stauffer, C., Grimson, W.E.L.: Adaptive background mixture models for real-time tracking. In: IEEE Conference on Computer Vision and Pattern Recognition (1999) Google Scholar
  54. 54.
    Stenger, B., Ramesh, V., Paragios, N., Coetzee, F., Bouhman, J.: Topology free hidden Markov models: Application to background modeling. In: IEEE International Conference on Computer Vision (2001) Google Scholar
  55. 55.
    Tomasi, C.: Shape and motion from image streams under orthography: A factorization method. Int. J. Comput. Vis. 9, 137–154 (1992) CrossRefGoogle Scholar
  56. 56.
    Toyama, K., Krumm, J., Brumitt, B., Meyers, B.: Wallflower: Principles and practice of background maintenance. In: IEEE International Conference on Computer Vision (1999) Google Scholar
  57. 57.
    Wada, T., Matsuyama, T.: Appearance sphere: Background model for pan-tilt-zoom camera. In: International Conference on Pattern Recognition (1996) Google Scholar
  58. 58.
    Wern, C.R., Azarbayejani, A., Darrell, T., Pentland, A.: Pfinder: Real-time tracking of human body. IEEE Trans. Pattern Anal. Mach. Intell. (1997) Google Scholar
  59. 59.
    Yang, Y.H., Levine, M.D.: The background primal sketch: An approach for tracking moving objects. Mach. Vis. Appl. 5, 17–34 (1992) CrossRefGoogle Scholar
  60. 60.
    Zappella, L., Lladó, X., Salvi, J.: Motion segmentation: A review. In: Conference on Artificial Intelligence Research and Development, Amsterdam, The Netherlands (2008) Google Scholar
  61. 61.
    Zhang, W., Fang, X.Z., Yang, X.: Moving cast shadows detection based on ratio edge. In: International Conference on Pattern Recognition (2006) Google Scholar
  62. 62.
    Zhong, J., Sclaroff, S.: Segmenting foreground objects from a dynamic textured background via a robust Kalman filter. In: International Conference on Computer Vision, Washington, DC, USA (2003) Google Scholar

Copyright information

© Springer-Verlag London Limited 2011

Authors and Affiliations

  1. 1.Rutgers UniversityNew BrunswickUSA

Personalised recommendations