Online, Real-Time Tracking Using a Category-to-Individual Detector

  • David Hall
  • Pietro Perona
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8689)


A method for online, real-time tracking of objects is presented. Tracking is treated as a repeated detection problem where potential target objects are identified with a pre-trained category detector and object identity across frames is established by individual-specific detectors. The individual detectors are (re-)trained online from a single positive example whenever there is a coincident category detection. This ensures that the tracker is robust to drift. Real-time operation is possible since an individual-object detector is obtained through elementary manipulations of the thresholds of the category detector and therefore only minimal additional computations are required. Our tracking algorithm is benchmarked against nine state-of-the-art trackers on two large, publicly available and challenging video datasets. We find that our algorithm is 10% more accurate and nearly as fast as the fastest of the competing algorithms, and it is as accurate but 20 times faster than the most accurate of the competing algorithms.


Target Object Tracking Algorithm Appearance Model Pedestrian Detection Individual Detector 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Supplementary material

978-3-319-10590-1_24_MOESM1_ESM.pdf (133 kb)
Electronic Supplementary Material (PDF 134 KB)


  1. 1.
    Adam, A., Rivlin, E., Shimshoni, I.: Robust Fragments-Based Tracking Using the Integral Histogram. In: CVPR (2006)Google Scholar
  2. 2.
    Andriyenko, A., Schindler, K.: Globally Optimal Multi-target Tracking on a Hexagonal Lattice. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part I. LNCS, vol. 6311, pp. 466–479. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  3. 3.
    Avidan, S.: Ensemble Tracking. PAMI 29(2), 431–435 (2007)CrossRefGoogle Scholar
  4. 4.
    Babenko, B., Belongie, S., Yang, M.H.: Visual Tracking with Online Multiple Instance Learning. In: CVPR (2009)Google Scholar
  5. 5.
    Bao, C., Wu, Y., Ling, H., Ji, H.: Real Time Robust L1 Tracker using Accelerated Proximal Gradient Approach. In: CVPR (2012)Google Scholar
  6. 6.
    Bauml, M., Tapaswi, M., Stiefelhagen, R.: Semi-Supervised Learning with Constraints for Person Identification in Multimedia Data. In: CVPR (2013)Google Scholar
  7. 7.
    Benenson, R., Mathias, M., Timofte, R., Van Gool, L.: Pedestrian Detection at 100 Frames per Second. In: CVPR (2013)Google Scholar
  8. 8.
    Berclaz, J., Fleuret, F., Fua, P.: Multiple Object Tracking using Flow Linear Programming. In: PETS (2009)Google Scholar
  9. 9.
    Berclaz, J., Türetken, E., Fleuret, F., Fua, P.: Multiple Object Tracking using K-Shortest Paths Optimization. PAMI 33(9), 1806–1819 (2011)CrossRefGoogle Scholar
  10. 10.
    Bernardin, K., Stiefelhagen, R.: Evaluating Multiple Object Tracking Performance: The CLEAR MOT Metrics. EURASIP JIVP (1), 1–10 (2008)Google Scholar
  11. 11.
    Black, M.J., Jepson, A.D.: EigenTracking: Robust Matching and Tracking of Articulated Objects Using a View-Based Representation. IJCV 26(1), 63–84 (1996)CrossRefGoogle Scholar
  12. 12.
    Burgos-Artizzu, X., Hall, D., Perona, P., Dollár, P.: Merging Pose Estimates across Space and Time. In: BMVC (2013)Google Scholar
  13. 13.
    Comaniciu, D., Ramesh, V., Meer, P.: Kernel-Based Object Tracking. PAMI 25(5), 564–577 (2003)CrossRefGoogle Scholar
  14. 14.
    Dalal, N., Triggs, B.: Histograms of Oriented Gradients for Human Detection. In: CVPR (2005)Google Scholar
  15. 15.
    Dollár, P., Appel, R., Belongie, S., Perona, P.: Fast Feature Pyramids for Object Detection. PAMI (2014)Google Scholar
  16. 16.
    Dollár, P., Tu, Z., Perona, P., Belongie, S.: Integral Channel Features. In: BMVC (2009)Google Scholar
  17. 17.
    Dollár, P., Wojek, C., Schiele, B., Perona, P.: Pedestrian Detection: A Benchmark. In: CVPRGoogle Scholar
  18. 18.
    Dollár, P., Belongie, S., Perona, P.: The Fastest Pedestrian Detector in the West. In: BMVC (2010)Google Scholar
  19. 19.
    Dollár, P., Wojek, C., Schiele, B., Perona, P.: Pedestrian Detection: an Evaluation of the State of the Art. PAMI 34(4), 743–761 (2012)CrossRefGoogle Scholar
  20. 20.
    Everingham, M., Van Gool, L., Williams, C.K.I., Winn, J., Zisserman, A.: The PASCAL Visual Object Classes Challenge 2009 (VOC) Results (2009)Google Scholar
  21. 21.
    Everingham, M.R., Sivic, J., Zisserman, A.: Hello! My Name is.... Buffy – Automatic Naming of Characters in TV Video. In: BMVC (2006)Google Scholar
  22. 22.
    Freund, Y., Schapire, R.E.: A Decision-Theoretic Generalization of On-Line Learning and an Application to Boosting. Journal of Computer and System Sciences 55(1), 119–139 (1997)CrossRefzbMATHMathSciNetGoogle Scholar
  23. 23.
    Fukunaga, K., Hostetler, L.: The Estimation of the Gradient of a Density Function, with Applications in Pattern Recognition. IEEE Transactions on Information Theory 21 (1975)Google Scholar
  24. 24.
    Grabner, H., Grabner, M., Bischof, H.: Real-Time Tracking Via On-line Boosting. In: BMVC (2006)Google Scholar
  25. 25.
    Grabner, H., Leistner, C., Bischof, H.: Semi-supervised On-Line Boosting for Robust Tracking. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part I. LNCS, vol. 5302, pp. 234–247. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  26. 26.
    Gray, D., Tao, H.: Viewpoint Invariant Pedestrian Recognition with an Ensemble of Localized Features. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part I. LNCS, vol. 5302, pp. 262–275. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  27. 27.
    Hall, D., Perona, P.: From Categories to Individuals in Real Time - A Unified Boosting Approach. In: CVPR (2014)Google Scholar
  28. 28.
    Henriques, J.F., Caseiro, R., Martins, P., Batista, J.: Exploiting the Circulant Structure of Tracking-by-Detection with Kernels. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part IV. LNCS, vol. 7575, pp. 702–715. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  29. 29.
    Huang, G.B., Ramesh, M., Berg, T., Learned-Miller, E.: Labeled Faces in the Wild: A Database for Studying Face Recognition in Unconstrained Environments. Tech. Rep. 07-49, University of Massachusetts, Amherst (October 2007)Google Scholar
  30. 30.
    Jiang, H., Fels, S., Little, J.J.: A Linear Programming Approach for Multiple Object Tracking. In: CVPR (2007)Google Scholar
  31. 31.
    Kalal, Z., Mikolajczyk, K., Matas, J.: Tracking-Learning-Detection. PAMI 6(1), 1–14 (2011)Google Scholar
  32. 32.
    Li, Y., Huang, C., Nevatia, R.: Learning to Associate: HybridBoosted Multi-Target Tracker for Crowded Scene. In: CVPR (2009)Google Scholar
  33. 33.
    Ma, Y., Yu, Q., Cohen, I.: Target Tracking with Incomplete Detection. Computer Vision and Image Understanding 113(4), 580–587 (2009)CrossRefGoogle Scholar
  34. 34.
    Nevatia, R.: Global Data Association for Multi-Object Tracking using Network Flows. In: CVPR (2008)Google Scholar
  35. 35.
    Oron, S., Bar-Hillel, A., Levi, D., Avidan, S.: Locally Orderless Tracking. In: CVPR (2012)Google Scholar
  36. 36.
    Pellegrini, S., Ess, A., Van Gool, L.: Improving Data Association by Joint Modeling of Pedestrian Trajectories and Groupings. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part I. LNCS, vol. 6311, pp. 452–465. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  37. 37.
    Pirsiavash, H., Ramanan, D., Fowlkes, C.C.: Globally-Optimal Greedy Algorithms for Tracking a Variable Number of Objects. In: CVPR (2011)Google Scholar
  38. 38.
    Ross, D.A., Lim, J., Lin, R.S., Yang, M.H.: Incremental Learning for Robust Visual Tracking. IJCV 77, 125–141 (2007)CrossRefGoogle Scholar
  39. 39.
    Sevilla-Lara, L., Learned-Miller, E.: Distribution Fields for Tracking. In: CVPR (2012)Google Scholar
  40. 40.
    Sivic, J., Everingham, M., Zisserman, A.: Who are You? - Learning Person Specific Classifiers from Video. In: CVPR (2009)Google Scholar
  41. 41.
    Stalder, S., Grabner, H., Van Gool, L.: Beyond Semi-Supervised Tracking: Tracking Should be as Simple as Detection, but not Simpler than Recognition. In: ICCV Workshops (2009)Google Scholar
  42. 42.
    Viola, P., Jones, M.: Rapid Object Detection Using a Boosted Cascade of Simple Features. In: CVPR (2001)Google Scholar
  43. 43.
    Viola, P., Jones, M.J.: Robust Real-Time Face Detection. IJCV 57(2), 137–154 (2004)CrossRefGoogle Scholar
  44. 44.
    Wu, Y., Lim, J., Yang, M.H.: Online Object Tracking: A Benchmark. In: CVPR (2013)Google Scholar
  45. 45.
    Wu, Y., Shen, B.: Online Robust Image Alignment Via Iterative Convex Optimization. In: CVPR (2012)Google Scholar
  46. 46.
    Zhang, K., Zhang, L., Yang, M.-H.: Real-Time Compressive Tracking. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part III. LNCS, vol. 7574, pp. 864–877. Springer, Heidelberg (2012)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • David Hall
    • 1
  • Pietro Perona
    • 1
  1. 1.California Institute of TechnologyUSA

Personalised recommendations