Segmentation Based Particle Filtering for Real-Time 2D Object Tracking

  • Vasileios Belagiannis
  • Falk Schubert
  • Nassir Navab
  • Slobodan Ilic
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7575)


We address the problem of visual tracking of arbitrary objects that undergo significant scale and appearance changes. The classical tracking methods rely on the bounding box surrounding the target object. Regardless of the tracking approach, the use of bounding box quite often introduces background information. This information propagates in time and its accumulation quite often results in drift and tracking failure. This is particularly the case with the particle filtering approach that is often used for visual tracking. However, it always uses a bounding box around the object to compute features of the particle samples. Since this causes the drift, we propose to use segmentation for sampling. Relying on segmentation and computing the colour and gradient orientation histograms from these segmented particle samples allows the tracker to easily adapt to the object’s deformations, occlusions, orientation, scale and appearance changes. We propose two particle sampling strategies based on segmentation. In the first, segmentation is done for every propagated particle sample, while in the second only the strongest particle sample is segmented. Depending on this decision there is obviously a trade-off between speed and performance.

We perform an exhaustive quantitative evaluation on a number of challenging sequences and compare our method with the number of state-of-the-art methods previously evaluated on those sequences. The results we obtain outperform majority of the related work, both in terms of the performance and speed.


Online Learning Visual Tracking Particle Sample Foreground Object Appearance Change 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Breitenstein, M., Reichlin, F., Leibe, B., Koller-Meier, E., Van Gool, L.: Online multiperson tracking-by-detection from a single, uncalibrated camera. IEEE Trans. on PAMI (2011)Google Scholar
  2. 2.
    Ikizler, N., Forsyth, D.: Searching video for complex activities with finite state models. In: CVPR (2007)Google Scholar
  3. 3.
    Wagner, D., Langlotz, T., Schmalstieg, D.: Robust and unobtrusive marker tracking on mobile phones. In: ISMAR (2008)Google Scholar
  4. 4.
    Pérez, P., Hue, C., Vermaak, J., Gangnet, M.: Color-Based Probabilistic Tracking. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002, Part I. LNCS, vol. 2350, pp. 661–675. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  5. 5.
    Lu, W., Okuma, K., Little, J.: Tracking and recognizing actions of multiple hockey players using the boosted particle filter. Image and Vision Computing (2009)Google Scholar
  6. 6.
    Avidan, S.: Ensemble tracking. In: CVPR (2005)Google Scholar
  7. 7.
    Godec, M., Roth, P., Bischof, H.: Hough-based tracking of non-rigid objects. In: ICCV (2011)Google Scholar
  8. 8.
    Isard, M., Blake, A.: Condensation-conditional density propagation for visual tracking. IJCV (1998)Google Scholar
  9. 9.
    Nummiaro, K., Koller-Meier, E., Van Gool, L.: An adaptive color-based particle filter. Image and Vision Computing (2003)Google Scholar
  10. 10.
    Doucet, A., De Freitas, N., Gordon, N.: Sequential Monte Carlo methods in practice. Springer (2001)Google Scholar
  11. 11.
    Okuma, K., Taleghani, A., de Freitas, N., Little, J.J., Lowe, D.G.: A Boosted Particle Filter: Multitarget Detection and Tracking. In: Pajdla, T., Matas, J. (eds.) ECCV 2004. LNCS, vol. 3021, pp. 28–39. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  12. 12.
    Bibby, C., Reid, I.: Robust Real-Time Visual Tracking Using Pixel-Wise Posteriors. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part II. LNCS, vol. 5303, pp. 831–844. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  13. 13.
    Chockalingam, P., Pradeep, N., Birchfield, S.: Adaptive fragments-based tracking of non-rigid objects using level sets. In: ICCV (2009)Google Scholar
  14. 14.
    Tsai, D., Flagg, M., Rehg, J.: Motion coherent tracking with multi-label mrf optimization. Algorithms (2010)Google Scholar
  15. 15.
    Shahed Nejhum, S., Ho, J., Yang, M.: Visual tracking with histograms and articulating blocks. In: CVPR (2008)Google Scholar
  16. 16.
    Javed, O., Ali, S., Shah, M.: Online detection and classification of moving objects using progressively improving detectors. In: CVPR (2005)Google Scholar
  17. 17.
    Grabner, H., Leistner, C., Bischof, H.: Semi-supervised On-Line Boosting for Robust Tracking. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part I. LNCS, vol. 5302, pp. 234–247. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  18. 18.
    Babenko, B., Yang, M., Belongie, S.: Visual tracking with online multiple instance learning. In: CVPR (2009)Google Scholar
  19. 19.
    Kalal, Z., Matas, J., Mikolajczyk, K.: Pn learning: Bootstrapping binary classifiers by structural constraints. In: CVPR (2010)Google Scholar
  20. 20.
    Lucas, B., Kanade, T.: With an application to stereo vision. In: Proceedings DARPA Image Understanding Workrhop (1998)Google Scholar
  21. 21.
    Kwon, J., Lee, K.: Tracking of a non-rigid object via patch-based dynamic appearance modeling and adaptive basin hopping monte carlo sampling. In: CVPR (2009)Google Scholar
  22. 22.
    Gall, J., Lempitsky, V.: Class-specific hough forests for object detection. In: CVPR (2009)Google Scholar
  23. 23.
    Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: CVPR (2005)Google Scholar
  24. 24.
    Stoica, P., Moses, R.: Introduction to spectral analysis, vol. 51. Prentice Hall, Upper Saddle River (1997)zbMATHGoogle Scholar
  25. 25.
    Rother, C., Kolmogorov, V., Blake, A.: Grabcut: Interactive foreground extraction using iterated graph cuts. ACM Transactions on Graphics, TOG (2004)Google Scholar
  26. 26.
    Pellegrini, S., Ess, A., Schindler, K., Van Gool, L.: You’ll never walk alone: Modeling social behavior for multi-target tracking. In: ICCV (2009)Google Scholar
  27. 27.
    Leibe, B., Schindler, K., Van Gool, L.: Coupled detection and trajectory estimation for multi-object tracking. In: ICCV (2007)Google Scholar
  28. 28.
    Ollero, A., Lacroix, S., Merino, L., Gancet, J., Wiklund, J., Remuss, V., Perez, I., Gutierrez, L., Viegas, D., Benitez, M., et al.: Multiple eyes in the skies: architecture and perception issues in the comets unmanned air vehicles project. IEEE Robotics & Automation Magazine (2005)Google Scholar
  29. 29.
    Lockheed-Martin: Ucf lockheed-martin uav dataset (2009),
  30. 30.
    Everingham, M., Van Gool, L., Williams, C.K.I., Winn, J., Zisserman, A.: The pascal visual object classes (voc) challenge. IJCV (2010)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Vasileios Belagiannis
    • 1
  • Falk Schubert
    • 2
  • Nassir Navab
    • 1
  • Slobodan Ilic
    • 1
  1. 1.Computer Aided Medical ProceduresTechnische Universität MünchenGermany
  2. 2.EADS Innovation WorksGermany

Personalised recommendations