Improving Data Association by Joint Modeling of Pedestrian Trajectories and Groupings

  • Stefano Pellegrini
  • Andreas Ess
  • Luc Van Gool
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6311)


We consider the problem of data association in a multi-person tracking context. In semi-crowded environments, people are still discernible as individually moving entities, that undergo many interactions with other people in their direct surrounding. Finding the correct association is therefore difficult, but higher-order social factors, such as group membership, are expected to ease the problem. However, estimating group membership is a chicken-and-egg problem: knowing pedestrian trajectories, it is rather easy to find out possible groupings in the data, but in crowded scenes, it is often difficult to estimate closely interacting trajectories without further knowledge about groups. To this end, we propose a third-order graphical model that is able to jointly estimate correct trajectories and group memberships over a short time window. A set of experiments on challenging data underline the importance of joint reasoning for data association in crowded scenarios.


Data Association Short Time Window Space Syntax Social Force Model Crowded Scene 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Hall, E.T.: The Hidden Dimension. Garden City (1966)Google Scholar
  2. 2.
    Helbing, D., Molnár, P.: Social force model for pedestrian dynamics. Physical Review 51(5) (1995)Google Scholar
  3. 3.
    Penn, A., Turner, A.: Space syntax based agent simulation. In: Pedestrian and Evacuation dynamics (2002)Google Scholar
  4. 4.
    Schadschneider, A.: Cellular automaton approach to pedestrian dynamics - theory. In: Pedestrian and Evacuation Dynamics (2001)Google Scholar
  5. 5.
    Lerner, A., Chrysanthou, Y., Lischinski, D.: Crowds by example. In: EUROGRAPHICS (2007)Google Scholar
  6. 6.
    Massive Software: Massive (2010)Google Scholar
  7. 7.
    Ali, S., Shah, M.: Floor fields for tracking in high density crowd scenes. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part II. LNCS, vol. 5303, pp. 1–14. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  8. 8.
    Antonini, G., Martinez, S.V., Bierlaire, M., Thiran, J.: Behavioral priors for detection and tracking of pedestrians in video sequences. IJCV 69, 159–180 (2006)CrossRefGoogle Scholar
  9. 9.
    Choi, W., Shahid, K., Savarese, S.: What are they doing? collective activity classification using spatio-temporal relationship among people. In: Workshop on Visual Surveillance (VSWS ’09) in conjuction with ICCV’09 (2009)Google Scholar
  10. 10.
    Mehran, R., Oyama, A., Shah, M.: Abnormal crowd behavior detection using Social Force model. In: CVPR (2009)Google Scholar
  11. 11.
    Pellegrini, S., Ess, A., Schindler, K., Gool, L.V.: You’ll never walk alone: Modeling social behavior for multi-target tracking. In: ICCV (2009)Google Scholar
  12. 12.
    Scovanner, P., Tappen, M.: Learning pedestrian dynamics from the real world. In: ICCV (2009)Google Scholar
  13. 13.
    Ge, W., Collins, R., Ruback, B.: Automatically detecting the small group structure of a crowd. In: IEEE Workshop on Applications of Computer Vision, WACV (2009)Google Scholar
  14. 14.
    French, A.: Visual Tracking: From An Individual To Groups of Animals. PhD thesis (2006)Google Scholar
  15. 15.
    Breitenstein, M., Reichlin, F., Leibe, B., Koller-Meier, E., Gool, L.V.: Robust tracking-by-detection using a detector confidence particle filter. In: ICCV (2009)Google Scholar
  16. 16.
    Ess, A., Leibe, B., Schindler, K., Gool, L.V.: A mobile vision system for robust multi-person tracking. In: CVPR (2008)Google Scholar
  17. 17.
    Li, Y., Huang, C., Nevatia, R.: Learning to associate: HybridBoosted multi-target tracker for crowded scene. In: CVPR (2009)Google Scholar
  18. 18.
    Okuma, K., Taleghani, A., de Freitas, N., Little, J., Lowe, D.: A boosted particle filter: Multitarget detection and tracking. In: Pajdla, T., Matas, J(G.) (eds.) ECCV 2004. LNCS, vol. 3021, pp. 28–39. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  19. 19.
    Wu, B., Nevatia, R.: Detection and tracking of multiple, partially occluded humans by bayesian combination of edgelet part detectors. IJCV 75, 247–266 (2007)CrossRefGoogle Scholar
  20. 20.
    Zhang, L., Li, Y., Nevatia, R.: Global data association for multi-object tracking using network flows. In: CVPR (2008)Google Scholar
  21. 21.
    Khan, Z., Balch, T., Dellaert, F.: MCMC-based particle filtering for tracking a variable number of interacting targets. PAMI 27(11), 1805–1819 (2005)Google Scholar
  22. 22.
    Sutton, C., McCallum, A.: Piecewise training of undirected models. In: Conference on Uncertainty in Artificial Intelligence, UAI (2005)Google Scholar
  23. 23.
    Komodakis, N., Paragios, N., Tziritas, G.: MRF optimization via dual decomposition: Message-passing revisited. In: ICCV (2007)Google Scholar
  24. 24.
    Torresani, L., Kolmogorov, V., Rother, C.: Feature correspondence via graph matching: Models and global optimization. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part II. LNCS, vol. 5303, pp. 596–609. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  25. 25.
    Mooij, J.M., et al.: libDAI 0.2.5: A free/open source C++ library for Discrete Approximate Inference (2010),
  26. 26.
    Gall, J., Lempitsky, V.: Class-specic hough forests for object detection. In: CVPR (2009)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Stefano Pellegrini
    • 1
  • Andreas Ess
    • 1
  • Luc Van Gool
    • 1
    • 2
  1. 1.Computer Vision LaboratoryETH Zurich 
  2. 2.ESAT-PSI / IBBTKU Leuven 

Personalised recommendations