Improving Multi-frame Data Association with Sparse Representations for Robust Near-online Multi-object Tracking

  • Loïc Fagot-BouquetEmail author
  • Romaric Audigier
  • Yoann Dhome
  • Frédéric Lerasle
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9912)


Multiple Object Tracking still remains a difficult problem due to appearance variations and occlusions of the targets or detection failures. Using sophisticated appearance models or performing data association over multiple frames are two common approaches that lead to gain in performances. Inspired by the success of sparse representations in Single Object Tracking, we propose to formulate the multi-frame data association step as an energy minimization problem, designing an energy that efficiently exploits sparse representations of all detections. Furthermore, we propose to use a structured sparsity-inducing norm to compute representations more suited to the tracking context. We perform extensive experiments to demonstrate the effectiveness of the proposed formulation, and evaluate our approach on two public authoritative benchmarks in order to compare it with several state-of-the-art methods.


Multiple Object Tracking Tracking by detection Multiple frame data association Sparse representation MCMC sampling 

Supplementary material

Supplementary material 1 (avi 16472 KB)

Supplementary material 2 (avi 6925 KB)

419983_1_En_47_MOESM3_ESM.pdf (92 kb)
Supplementary material 3 (pdf 91 KB)
419983_1_En_47_MOESM4_ESM.pdf (166 kb)
Supplementary material 4 (pdf 166 KB)


  1. 1.
    Xiang, Y., Alahi, A., Savarese, S.: Learning to track: online multi-object tracking by decision making. In: ICCV (2015)Google Scholar
  2. 2.
    Yoon, J.H., Yang, M.H., Lim, J., Yoon, K.J.: Bayesian multi-object tracking using motion context from multiple objects. In: WACV (2015)Google Scholar
  3. 3.
    Fagot-Bouquet, L., Audigier, R., Dhome, Y., Lerasle, F.: Online multi-person tracking based on global sparse collaborative representations. In: ICIP (2015)Google Scholar
  4. 4.
    Bae, S.H., Yoon, K.J.: Robust online multi-object tracking based on tracklet confidence and online discriminative appearance learning. In: CVPR (2014)Google Scholar
  5. 5.
    Wang, S., Fowlkes, C.C.: Learning optimal parameters for multi-target tracking. In: BMVC (2015)Google Scholar
  6. 6.
    McLaughlin, N., Del Rincon, J.M., Miller, P.: Enhancing linear programming with motion modeling for multi-target tracking. In: WACV (2015)Google Scholar
  7. 7.
    Leal-Taix, L., Fenzi, M., Kuznetsova, A., Rosenhahn, B., Savarese, S.: Learning an image-based motion context for multiple people tracking. In: CVPR (2014)Google Scholar
  8. 8.
    Milan, A., Schindler, K., Roth, S.: Multi-target tracking by discrete-continuous energy minimization. TPAMI (2016). doi: 10.1109/TPAMI.2015.2505309 Google Scholar
  9. 9.
    Milan, A., Roth, S., Schindler, K.: Continuous energy minimization for multitarget tracking. TPAMI 36(1), 58–72 (2014)CrossRefGoogle Scholar
  10. 10.
    Dicle, C., Sznaier, M., Camps, O.: The way they move: tracking targets with similar appearance. In: ICCV (2013)Google Scholar
  11. 11.
    Geiger, A., Lauer, M., Wojek, C., Stiller, C., Urtasun, R.: 3D traffic scene understanding from movable platforms. TPAMI 36(5), 1012–1025 (2014)CrossRefGoogle Scholar
  12. 12.
    Pirsiavash, H., Ramanan, D., Fowlkes, C.C.: Globally-optimal greedy algorithms for tracking a variable number of objects. In: CVPR (2011)Google Scholar
  13. 13.
    Zamir, A.R., Dehghan, A., Shah, M.: GMCP-tracker: global multi-object tracking using generalized minimum clique graphs. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part II. LNCS, vol. 7573, pp. 343–356. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  14. 14.
    Dehghan, A., Assari, S.M., Shah, M.: GMMCP-tracker: globally optimal generalized maximum multi clique problem for multiple object tracking. In: CVPR (2015)Google Scholar
  15. 15.
    Brendel, W., Amer, M.R., Todorovic, S.: Multiobject tracking as maximum weight independent set. In: CVPR (2011)Google Scholar
  16. 16.
    Choi, W.: Near-online multi-target tracking with aggregated local flow descriptor. In: ICCV (2015)Google Scholar
  17. 17.
    Kim, C., Li, F., Ciptadi, A., Rehg, J.M.: Multiple hypothesis tracking revisited. In: ICCV (2015)Google Scholar
  18. 18.
    Rezatofighi, S.H., Milan, A., Zhang, Z., Shi, Q., Dick, A.R., Reid, I.D.: Joint probabilistic data association revisited. In: ICCV (2015)Google Scholar
  19. 19.
    Milan, A., Leal-Taix, L., Schindler, K., Reid, I.: Joint tracking and segmentation of multiple targets. In: CVPR (2015)Google Scholar
  20. 20.
    Benfold, B., Reid, I.: Stable multi-target tracking in real-time surveillance video. In: CVPR (2011)Google Scholar
  21. 21.
    Mei, X., Ling, H.: Robust visual tracking and vehicle classification via sparse representation. TPAMI 33(11), 2259–2272 (2011)CrossRefGoogle Scholar
  22. 22.
    Bao, C., Wu, Y., Ling, H., Ji, H.: Real time robust L1 tracker using accelerated proximal gradient approach. In: CVPR (2012)Google Scholar
  23. 23.
    Jia, X., Lu, H., Yang, M.: Visual tracking via adaptive structural local sparse appearance model. In: CVPR (2012)Google Scholar
  24. 24.
    Zhong, W., Lu, H., Yang, M.: Robust object tracking via sparsity-based collaborative model. In: CVPR (2012)Google Scholar
  25. 25.
    Hong, Z., Mei, X., Prokhorov, D., Tao, D.: Tracking via robust multi-task multi-view joint sparse representation. In: ICCV (2013)Google Scholar
  26. 26.
    Zhang, S., Yao, H., Sun, X., Lu, X.: Sparse coding based visual tracking: review and experimental comparison. Pattern Recogn. 46(7), 1772–1788 (2013)CrossRefGoogle Scholar
  27. 27.
    Fagot-Bouquet, L., Audigier, R., Dhome, Y., Lerasle, F.: Collaboration and spatialization for an efficient multi-person tracking via sparse representations. In: AVSS (2015)Google Scholar
  28. 28.
    Naiel, M.A., Ahmad, M.O., Swamy, M.N.S., Wu, Y., Yang, M.: Online multi-person tracking via robust collaborative model. In: ICIP (2014)Google Scholar
  29. 29.
    Oh, S., Russell, S.J., Sastry, S.: Markov chain Monte Carlo data association for multi-target tracking. Trans. Autom. Control 54(3), 481–497 (2009)CrossRefMathSciNetGoogle Scholar
  30. 30.
    Wright, J., Yang, A., Ganesh, A., Sastry, S., Ma, Y.: Robust face recognition via sparse representation. TPAMI 31(2), 210–227 (2009)CrossRefGoogle Scholar
  31. 31.
    Parikh, N., Boyd, S.: Proximal algorithms. Found. Trends Optim. 1(3), 123–231 (2013)Google Scholar
  32. 32.
    Quattoni, A., Carreras, X., Collins, M., Darrell, T.: An efficient projection for l1, infinity regularization. In: ICML (2009)Google Scholar
  33. 33.
    Bach, F., Jenatton, R., Mairal, J., Obozinski, G.: Optimization with sparsity-inducing penalties. Found. Trends Mach. Learn. 4(1), 1–106 (2012)CrossRefzbMATHGoogle Scholar
  34. 34.
    Leal-Taixé, L., Milan, A., Reid, I., Roth, S., Schindler, K.: MOTChallenge 2015: Towards a Benchmark for Multi-Target Tracking. arXiv:1504.01942 [cs] (2015)
  35. 35.
    Milan, A., Leal-Taixé, L., Reid, I., Roth, S., Schindler, K.: MOT16: A Benchmark for Multi-Object Tracking. arXiv:1603.00831 [cs] (2016)
  36. 36.
    Ess, A., Leibe, B., Gool, L.V.: Depth and appearance for mobile scene analysis. In: ICCV (2007)Google Scholar
  37. 37.
    Andriluka, M., Roth, S., Schiele, B.: Monocular 3D pose estimation and tracking by detection. In: CVPR (2010)Google Scholar
  38. 38.
    Andriluka, M., Roth, S., Schiele, B.: People-tracking-by-detection and people-detection-by-tracking. In: CVPR (2008)Google Scholar
  39. 39.
    Ferryman, J., Shahrokni, A.: Pets 2009: dataset and challenge. In: Performance Evaluation of Tracking and Surveillance (PETS-Winter) (2009)Google Scholar
  40. 40.
    Geiger, A., Lenz, P., Urtasun, R.: Are we ready for autonomous driving? The KITTI vision benchmark suite. In: CVPR (2012)Google Scholar
  41. 41.
    Benfold, B., Reid, I.: Guiding visual surveillance by tracking human attention. In: BMVC (2009)Google Scholar
  42. 42.
    Dollar, P., Appel, R., Belongie, S., Perona, P.: Fast feature pyramids for object detection. TPAMI 36(8), 1532–1545 (2014)CrossRefGoogle Scholar
  43. 43.
    Felzenszwalb, P.F., Girshick, R.B., McAllester, D., Ramanan, D.: Object detection with discriminatively trained part-based models. TPAMI 32(9), 1627–1645 (2010)CrossRefGoogle Scholar
  44. 44.
    Bernardin, K., Stiefelhagen, R.: Evaluating multiple object tracking performance: the CLEAR MOT metrics. EURASIP J. Image Video Process. 2008(1), 1–10 (2008). doi: 10.1155/2008/246309 CrossRefGoogle Scholar
  45. 45.
    Hutter, F., Hoos, H.H., Leyton-Brown, K.: Sequential model-based optimization for general algorithm configuration. In: Coello, C.A.C. (ed.) LION 2011. LNCS, vol. 6683, pp. 507–523. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  46. 46.
    Bewley, A., Ott, L., Ramos, F., Upcroft, B.: ALExTRAC: affinity learning by exploring temporal reinforcement within association chains. In: ICRA (2016)Google Scholar

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  • Loïc Fagot-Bouquet
    • 1
    Email author
  • Romaric Audigier
    • 1
  • Yoann Dhome
    • 1
  • Frédéric Lerasle
    • 2
    • 3
  1. 1.CEA, LIST, Vision and Content Engineering LaboratoryGif-sur-YvetteFrance
  2. 2.CNRS, LAASToulouseFrance
  3. 3.Université de Toulouse, UPS, LAASToulouseFrance

Personalised recommendations