Spatio-Temporal Optimization for Foreground/Background Segmentation

  • Tobias Feldmann
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6468)


We introduce a procedure for calibrated multi camera setups in which observed persons within a realistic and, thus, difficult surrounding are determined as foreground in image sequences via a fully automatic purely data driven segmentation.

In order to gain an optimal separation of fore- and background for each frame in terms of Expectation Maximization (EM), an algorithm is proposed which utilizes a combination of geometrical constraints of the scene and, additionally, temporal constraints for a optimization over the entire sequence to estimate the background. This background information is then used to determine accurate silhouettes of the foreground.

We demonstrate the effectiveness of our approach based on a qualitative data analysis and compare it to other state of the art approaches.


Gaussian Mixture Model Foreground Pixel Bundle Adjustment Luminance Ratio Foreground Segmentation 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Grauman, K., Shakhnarovich, G., Darrell, T.: A bayesian approach to image-based visual hull reconstruction. In: CVPR 2003, vol. 1, pp. 187–194 (2003)Google Scholar
  2. 2.
    Gordon, G., Darrell, T., Harville, M., Woodfill, J.: Background estimation and removal based on range and color. In: CVPR 1999, vol. 2, pp. 459–464. IEEE Computer Society, Los Alamitos (1999)Google Scholar
  3. 3.
    Cheung, K.M.G.: Visual Hull Construction, Alignment and Refinement for Human Kinematic Modeling, Motion Tracking and Rendering. PhD thesis, Robotics Institute, Carnegie Mellon University, Pittsburgh, Pennsylvania 15213 (2003)Google Scholar
  4. 4.
    Vogiatzis, G., Torr, P.H.S., Cipolla, R.: Multi-view stereo via volumetric graph-cuts. In: CVPR 2005, vol. 2, pp. 391–398 (2005)Google Scholar
  5. 5.
    Mikić, I.: Human Body Model Acquisition and Tracking using Multi-Camera Voxel Data. Electrical and computer engineering (image and signal processing) University of California, San Diego (2002)Google Scholar
  6. 6.
    Rosenhahn, B., Kersting, U.G., Smith, A.W., Gurney, J., Brox, T., Klette, R.: A system for marker-less human motion estimation. In: Kropatsch, W.G., Sablatnig, R., Hanbury, A. (eds.) DAGM 2005. LNCS, vol. 3663, pp. 230–237. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  7. 7.
    Lee, W., Woo, W., Boyer, E.: Identifying foreground from multiple images. In: Yagi, Y., Kang, S.B., Kweon, I.S., Zha, H. (eds.) ACCV 2007, Part II. LNCS, vol. 4844, pp. 580–589. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  8. 8.
    Kim, K., Chalidabhongse, T., Harwood, D., Davis, L.: Background modeling and subtraction by codebook construction. In: ICIP 2004, vol. 5, pp. 3061–3064 (2004)Google Scholar
  9. 9.
    Stauffer, C., Grimson, W.: Adaptive background mixture models for real-time tracking. In: CVPR 1999, vol. 2, pp. 2246–2252. IEEE Computer Society, Los Alamitos (1999)Google Scholar
  10. 10.
    Kaewtrakulpong, P., Bowden, R.: An improved adaptive background mixture model for realtime tracking with shadow detection. In: Proc. 2nd European Workshop on Advanced Video Based Surveillance Systems. Kluwer Academic Publishers, Dordrecht (2001)Google Scholar
  11. 11.
    Russell, D., Gong, S.: Minimum cuts of a time-varying background. In: BMVC 2006, vol. 2, pp. 809–818 (2006)Google Scholar
  12. 12.
    Russell, D.M., Gong, S.G.: Segmenting highly textured nonstationary background. In: BMVC 2007, (2007)Google Scholar
  13. 13.
    Lim, S.N., Mittal, A., Davis, L.S., Paragios, N.: Fast illumination-invariant background subtraction using two views: Error analysis, sensor placement and applications. In: CVPR 2005, pp. 1071–1078. IEEE Computer Society, Washington, DC, USA (2005)Google Scholar
  14. 14.
    Feldmann, T., Dießelberg, L., Wörner, A.: Adaptive foreground/background segmentation using multiview silhouette fusion. In: Denzler, J., Notni, G., Süße, H. (eds.) DAGM. LNCS, vol. 5748, pp. 522–531. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  15. 15.
    Broadhurst, A., Drummond, T.W., Cipolla, R.: A probabilistic framework for space carving. In: ICCV 2001, pp. 388–393 (2001)Google Scholar
  16. 16.
    Franco, J.S., Boyer, E.: Fusion of multi-view silhouette cues using a space occupancy grid. Technical Report 5551, INRIA (2005)Google Scholar
  17. 17.
    Kolev, K., Brox, T., Cremers, D.: Robust variational segmentation of 3d objects from multiple views. In: Franke, K., Müller, K.-R., Nickolay, B., Schäfer, R. (eds.) DAGM 2006. LNCS, vol. 4174, pp. 688–697. Springer, Heidelberg (2006)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Tobias Feldmann
    • 1
  1. 1.Karlsruhe Institute of Technology (KIT)Germany

Personalised recommendations