A Robust Multi-Athlete Tracking Algorithm by Exploiting Discriminant Features and Long-Term Dependencies

  • Nan Ran
  • Longteng Kong
  • Yunhong Wang
  • Qingjie LiuEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11295)


This paper addresses multiple athletes tracking problem. Athletes tracking is the key to whether sports video analysis can be more effective and practical or not. One great challenge faced by multi-athlete tracking is that athletes, especially the athletes in the same team, share very similar appearance, thus, most existing MOT approaches are hardly applicable in this task. To address this problem, we put forward a novel triple-stream network which could capture long-term dependencies by exploiting pose information to better distinguish different athletes. The method is motivated by the fact that poses of athletes are distinct from each other in a period of time because they play different roles in the team thus could be used as a strong feature to match the correct athletes. We design our Multi-Athlete Tracking (MAT) model on top of the online tracking-by-detection paradigm whereby bounding boxes from the output of a detector are connected across video frames, and improve it from two aspects. Firstly, we propose a Pose-based Triple Stream Networks (PTSN) based on Long Short-Term Memory (LSTM) networks, which are capable of modeling and capturing more subtle differences between athletes. Secondly, based on PTSN, we propose a multi-athlete tracking algorithm that is robust to noisy detection and occlusion. We demonstrate the effectiveness of our method on a collection of volleyball videos by comparing it with recent advanced multi-object trackers.


Sports video analysis Multi-Athlete Tracking (MAT) Long Short-Term Memory (LSTM) networks 



This work was supported by the National Natural Science Foundation of China (61573045).


  1. 1.
    Adam, A., Rivlin, E., Shimshoni, I.: Robust fragments-based tracking using the integral histogram. In: CVPR, vol. 1, pp. 798–805. IEEE (2006)Google Scholar
  2. 2.
    Bernardin, K., Stiefelhagen, R.: Evaluating multiple object tracking performance: the CLEAR MOT metrics. J. Image Video Process. 1 (2008)CrossRefGoogle Scholar
  3. 3.
    Dicle, C., Camps, O.I., Sznaier, M.: The way they move: tracking multiple targets with similar appearance. In: ICCV, pp. 2304–2311. IEEE (2013)Google Scholar
  4. 4.
    Fang, H., Xie, S., Tai, Y.W., Lu, C.: RMPE: regional multi-person pose estimation. In: ICCV, vol. 2 (2017)Google Scholar
  5. 5.
    Gomez, G., López, P.H., Link, D., Eskofier, B.: Tracking of ball and players in beach volleyball videos. PLoS ONE 9, e111730 (2014)CrossRefGoogle Scholar
  6. 6.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: CVPR, pp. 770–778 (2016)Google Scholar
  7. 7.
    Henschel, R., Leal-Taixé, L., Cremers, D., Rosenhahn, B.: Improvements to Frank-Wolfe optimization for multi-detector multi-object tracking. CoRR (2017)Google Scholar
  8. 8.
    Kim, C., Li, F., Ciptadi, A., Rehg, J.M.: Multiple hypothesis tracking revisited. In: ICCV, pp. 4696–4704. IEEE (2015)Google Scholar
  9. 9.
    Kuo, C.H., Nevatia, R.: How does person identity recognition help multi-person tracking? In: CVPR, pp. 1217–1224. IEEE (2011)Google Scholar
  10. 10.
    Leal-Taixe, L., Canton-Ferrer, C., Schindler, K.: Learning by tracking: siamese CNN for robust target association. In: CVPR Workshop. IEEE, June 2016Google Scholar
  11. 11.
    Leal-Taixé, L., Milan, A., Reid, I., Roth, S., Schindler, K.: Motchallenge 2015: towards a benchmark for multi-target tracking. arXiv preprint arXiv:1504.01942 (2015)
  12. 12.
    Li, Y., Huang, C., Nevatia, R.: Learning to associate: hybridboosted multi-target tracker for crowded scene. In: CVPR (2009)Google Scholar
  13. 13.
    Liu, J., Carr, P., Collins, R.T., Liu, Y.: Tracking sports players with context-conditioned motion models. In: CVPR, pp. 1830–1837 (2013)Google Scholar
  14. 14.
    Lu, J., Huang, D., Wang, Y., Kong, L.: Scaling and occlusion robust athlete tracking in sports videos. In: 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1526–1530. IEEE (2016)Google Scholar
  15. 15.
    Mauthner, T., Koch, C., Tilp, M., Bischof, H.: Visual tracking of athletes in beach volleyball using a single camera. Int. J. Comput. Sci. Sport 6(2), 21–34 (2007)Google Scholar
  16. 16.
    Milan, A., Leal-Taixé, L., Reid, I., Roth, S., Schindler, K.: MOT16: a benchmark for multi-object tracking. arXiv preprint arXiv:1603.00831 (2016)
  17. 17.
    Perazzi, F., Pont-Tuset, J., McWilliams, B., Van Gool, L., Gross, M., Sorkine-Hornung, A.: A benchmark dataset and evaluation methodology for video object segmentation. In: CVPR, pp. 724–732 (2016)Google Scholar
  18. 18.
    Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. In: NIPS, pp. 91–99. MIT Press (2015)Google Scholar
  19. 19.
    Sadeghian, A., Alahi, A., Savarese, S.: Tracking the untrackable: learning to track multiple cues with long-term dependencies. In: ICCV (2017)Google Scholar
  20. 20.
    Shu, G., Dehghan, A., Oreifej, O., Hand, E., Shah, M.: Part-based multiple-person tracking with partial occlusion handling. In: CVPR, pp. 1815–1821. IEEE (2012)Google Scholar
  21. 21.
    Yamaguchi, K., Berg, A.C., Ortiz, L.E., Berg, T.L.: Who are you with and where are you going? In: CVPR, pp. 1345–1352. IEEE (2011)Google Scholar
  22. 22.
    Yoon, J.H., Yang, M.H., Lim, J., Yoon, K.J.: Bayesian multi-object tracking using motion context from multiple objects. In: 2015 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 33–40. IEEE (2015)Google Scholar
  23. 23.
    Yu, F., Li, W., Li, Q., Liu, Y., Shi, X., Yan, J.: POI: multiple object tracking with high performance detection and appearance feature. In: Hua, G., Jégou, H. (eds.) ECCV 2016. LNCS, vol. 9914, pp. 36–42. Springer, Cham (2016). Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Nan Ran
    • 1
  • Longteng Kong
    • 1
  • Yunhong Wang
    • 1
  • Qingjie Liu
    • 1
    • 2
    Email author
  1. 1.The State Key Laboratory of Virtual Reality Technology and SystemsBeihang UniversityBeijingChina
  2. 2.Beijing Key Laboratory of Digital Media, School of Computer Science and EngineeringBeihang UniversityBeijingChina

Personalised recommendations