Unified Crowd Segmentation

  • Peter Tu
  • Thomas Sebastian
  • Gianfranco Doretto
  • Nils Krahnstoever
  • Jens Rittscher
  • Ting Yu
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5305)


This paper presents a unified approach to crowd segmentation. A global solution is generated using an Expectation Maximization framework. Initially, a head and shoulder detector is used to nominate an exhaustive set of person locations and these form the person hypotheses. The image is then partitioned into a grid of small patches which are each assigned to one of the person hypotheses. A key idea of this paper is that while whole body monolithic person detectors can fail due to occlusion, a partial response to such a detector can be used to evaluate the likelihood of a single patch being assigned to a hypothesis. This captures local appearance information without having to learn specific appearance models. The likelihood of a pair of patches being assigned to a person hypothesis is evaluated based on low level image features such as uniform motion fields and color constancy. During the E-step, the single and pairwise likelihoods are used to compute a globally optimal set of assignments of patches to hypotheses. In the M-step, parameters which enforce global consistency of assignments are estimated. This can be viewed as a form of occlusion reasoning. The final assignment of patches to hypotheses constitutes a segmentation of the crowd. The resulting system provides a global solution that does not require background modeling and is robust with respect to clutter and partial occlusion.


Partial Occlusion Color Constancy Crowded Scene Pairwise Likelihood IEEE Computer Vision 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Munder, S., Gavrila, D.: An experimental study on pedestrian classification. IEEE Trans. on Pattern Analysis and Machine Intelligence 28(11), 1863–1868 (2006)CrossRefGoogle Scholar
  2. 2.
    Leibe, B., Seemann, E., Schiele, B.: Pedestrian detection in crowded scenes. IEEE Computer Vision and Pattern Recognition, 878–885 (2005)Google Scholar
  3. 3.
    Leibe, B., Cornelis, N., Cornelis, K., Gool, L.V.: Dynamic 3d scene analysis from a moving vehicle. IEEE Computer Vision and Pattern Recognition, 1–8 (2007)Google Scholar
  4. 4.
    Rittscher, J., Tu, P.H., Krahnstoever, N.: Simultaneous estimation of segmentation and shape. IEEE Computer Vision and Pattern Recognition 2, 486–493 (2005)Google Scholar
  5. 5.
    Brostow, G.J., Cipolla, R.: Unsupervised bayesian detection of independent motion in crowds. IEEE Computer Vision and Pattern Recognition I, 594–601 (2006)Google Scholar
  6. 6.
    Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. IEEE Computer Vision and Pattern Recognition, 886–893 (2005)Google Scholar
  7. 7.
    Tuzel, O., Porikli, F., Meer, P.: Pedestrian detection via classification on riemannian manifolds. IEEE Computer Vision and Pattern Recognition (2007)Google Scholar
  8. 8.
    Stauffer, C., Grimson, W.: Adaptive background mixture models for real-time tracking. IEEE Computer Vision and Pattern Recognition 2, 246–252 (1998)Google Scholar
  9. 9.
    Rabaud, V., Belongie, S.: Counting crowded moving objects. IEEE Computer Vision and Pattern Recognition, 705–711 (2006)Google Scholar
  10. 10.
    Viola, P., Jones, M., Snow, D.: Detecting pedestrians using patterns of motion and appearance. International Journal of Computer Vision 2, 734–741 (2003)Google Scholar
  11. 11.
    Fergus, R., Perona, P., Zisserman, A.: A visual category filter for Google images. In: European Conference on Computer Vision, vol. 1, pp. 242–256 (2004)Google Scholar
  12. 12.
    Mikolajczyk, K., Schmid, C., Zisserman, A.: Human detection based on a probabilistic assembly of robust part detectors. In: European Conference on Computer Vision (2004)Google Scholar
  13. 13.
    Wu, B., Nevatia, R.: Detection and tracking of multiple partially occluded humans by bayesian combination of edgelet based part detectors. International Journal of Computer Vision 75(2), 247–266 (2007)CrossRefGoogle Scholar
  14. 14.
    Zhao, T., Nevatia, R.R.: Bayesian human segmentation in crowded situations. IEEE Computer Vision and Pattern Recognition 2, 459–466 (2003)Google Scholar
  15. 15.
    Chui, H., Rangarajan, A.: A new point matching algorithm for non-rigid registration. Computer Vision and Image Understanding 89(3), 114–141 (2003)zbMATHCrossRefGoogle Scholar
  16. 16.
    Viola, P., Jones, M.J.: Robust real-time face detection. International Journal of Computer Vision 57(2), 137–154 (2004)CrossRefGoogle Scholar
  17. 17.
    Krahnstoever, N., Tu, P., Sebastian, T., Perera, A., Collins, R.: Multi-view detection and tracking of travelers and luggage in mass transit environments. In: Proc. Ninth IEEE International Workshop on Performance Evaluation of Tracking and Surveillance (PETS) (2006)Google Scholar
  18. 18.
    Leibe, B., Schindler, K., Gool, L.V.: Coupled detection and trajectory estimation for multi-object tracking. In: International Conference on Computer Vision (ICCV 2007), Rio de Janeiro, Brasil (October 2007)Google Scholar
  19. 19.
    Blackman, S., Popoli, R.: Design and Analysis of Modern Tracking Systems. Artech House Publishers (1999)Google Scholar
  20. 20.
    Rasmussen, C., Hager, G.: Joint probabilistic techniques for tracking multi-part objects. In: Proc. IEEE Conference on Computer Vision and Pattern Recognition, pp. 16–21 (1998)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Peter Tu
    • 1
  • Thomas Sebastian
    • 1
  • Gianfranco Doretto
    • 1
  • Nils Krahnstoever
    • 1
  • Jens Rittscher
    • 1
  • Ting Yu
    • 1
  1. 1.GE Global ResearchNiskayuna, NYUSA

Personalised recommendations