Advertisement

International Journal of Computer Vision

, Volume 127, Issue 1, pp 1–21 | Cite as

Structural Constraint Data Association for Online Multi-object Tracking

  • Ju Hong Yoon
  • Chang-Ryeol Lee
  • Ming-Hsuan Yang
  • Kuk-Jin YoonEmail author
Article
  • 536 Downloads

Abstract

Online two-dimensional (2D) multi-object tracking (MOT) is a challenging task when the objects of interest have similar appearances. In that case, the motion of objects is another helpful cue for tracking and discriminating multiple objects. However, when using a single moving camera for online 2D MOT, observable motion cues are contaminated by global camera movements and, thus, are not always predictable. To deal with unexpected camera motion, we propose a new data association method that effectively exploits structural constraints in the presence of large camera motion. In addition, to reduce incorrect associations with mis-detections and false positives, we develop a novel event aggregation method to integrate assignment costs computed by structural constraints. We also utilize structural constraints to track missing objects when they are re-detected again. By doing this, identities of the missing objects can be retained continuously. Experimental results validated the effectiveness of the proposed data association algorithm under unexpected camera motions. In addition, tracking results on a large number of benchmark datasets demonstrated that the proposed MOT algorithm performs robustly and favorably against various online methods in terms of several quantitative metrics, and that its performance is comparable to offline methods.

Keywords

Multi-object tracking Data association Structural constraint 

Notes

Acknowledgements

This work was partially supported by ‘The Cross-Ministry Giga KOREA Project’ grant funded by the Korea government (MSIT) (No. GK18P0200, Development of 4D reconstruction and dynamic deformable action model based hyper-realistic service technology and No. GK18P0300, Real-time 4D reconstruction of dynamic objects for ultra-realistic service). The work was also partially supported by IITP grant funded by the Korea government (MSIP) (2014-0-00059) and Samsung Research Funding Center of Samsung Electronics under Project Number SRFC-TC1603-05. M.-H. Yang is supported in part by the the NSF CAREER Grant #1149783.

References

  1. Bae, S. H., Yoon, K. J. (2014). Robust online multi-object tracking based on tracklet confidence and online discriminative appearance learning. InProceedings of IEEE conference on computer vision and pattern recognition (pp. 1218–1225).Google Scholar
  2. Bar-Shalom, Y., & Li, X. R. (1995). Multitarget-multisensor tracking: Principles and techniques. Storrs, CT: YBS Publishing.Google Scholar
  3. Bernardin, K., & Stiefelhagen, R. (2008). Evaluating multiple object tracking performance: The clear mot metrics. EURASIP Journal on Image and Video Processing, 1(1–1), 10.Google Scholar
  4. Betke, M., & Wu, Z. (2016). Data Association for multi-object visual tracking. Synthesis lectures on computer vision. Morgan & Claypool. https://books.google.co.kr/books?id=tn0cvgAACAAJ.
  5. Blackman, S., & Popoli, R. (1999). Design and analysis of modern tracking systems. Norwood: Artech House Radar Library, Artech House.zbMATHGoogle Scholar
  6. Breitenstein, M. D., Reichlin, F., Leibe, B., Koller-Meier, E., & Van Gool, L. (2011). Online multiperson tracking-by-detection from a single, uncalibrated camera. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(9), 1820–1833.CrossRefGoogle Scholar
  7. Choi, W. (2015). Near-online multi-target tracking with aggregated local flow descriptor. In Proceedings of IEEE international conference on computer vision (pp. 3029–3037).Google Scholar
  8. Dollar, P., Appel, R., Belongie, S., & Perona, P. (2014). Fast feature pyramids for object detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 36(8), 1532–1545.CrossRefGoogle Scholar
  9. Duan, G., Ai, H., Cao, S., & Lao, S. (2012). Group tracking: Exploring mutual relations for multiple object tracking. In Proceedings of European conference on computer vision (pp. 129–143).Google Scholar
  10. Ess, A., Leibe, B., Schindler, K., & Gool, L. V. (2008). A mobile vision system for robust multi-person tracking. In Proceedings of IEEE conference on computer vision and pattern recognition.Google Scholar
  11. Everingham, M., Gool, L., Williams, C. K., Winn, J., & Zisserman, A. (2010). The pascal visual object classes (VOC) challenge. International Journal of Computer Vision, 88(2), 303–338.CrossRefGoogle Scholar
  12. Felzenszwalb, P. F., Girshick, R. B., McAllester, D., & Ramanan, D. (2010). Object detection with discriminatively trained part based models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(9), 1627–1645.CrossRefGoogle Scholar
  13. Geiger, A. (2013). Probabilistic models for 3D urban scene understanding from movable platforms. Ph.D. thesis, KIT.Google Scholar
  14. Geiger, A., Lauer, M., Wojek, C., Stiller, C., & Urtasun, R. (2014). 3D traffic scene understanding from movable platforms. IEEE Transactions on Pattern Analysis and Machine Intelligence, 36(5), 1012–1025.CrossRefGoogle Scholar
  15. Geiger, A., Lenz, P., Stiller, C., & Urtasun, R. (2013). Vision meets robotics: The kitti dataset. The International Journal of Robotics Research, 32(11), 1231–1237.CrossRefGoogle Scholar
  16. Grabner, H., Matas, J., Gool, L. J. V., & Cattin, P. C. (2010). Tracking the invisible: Learning where the object might be. In Proceedings of IEEE conference on computer vision and pattern recognition (pp. 1285–1292).Google Scholar
  17. Kim, C., Li, F., Ciptadi, A., & Rehg, J. M. (2015). Multiple hypothesis tracking revisited. In Proceedings of IEEE international conference on computer vision (pp. 4696–4704).Google Scholar
  18. Kim, S., Kwak, S., Feyereisl, J., & Han, B. (2012). Online multi-target tracking by large margin structured learning. In Proceedings of Asian conference on computer vision (pp. 98–111).Google Scholar
  19. Kuhn, H. W. (1955). The hungarian method for the assignment problem. Naval Research Logistics, 2(1–2), 83–97.MathSciNetCrossRefzbMATHGoogle Scholar
  20. Leal-Taixe, L., Canton-Ferrer, C., & Schindler, K. (2016). Learning by tracking: Siamese CNN for robust target association. In Proceedings of IEEE conference on computer vision and pattern recognition workshops (pp. 33–40).Google Scholar
  21. Leal-Taixé, L., Fenzi, M., Kuznetsova, A., Rosenhahn, B., & Savarese, S. (2014). Learning an image-based motion context for multiple people tracking. In Proceedings of IEEE conference on computer vision and pattern recognition (pp. 3542–3549).Google Scholar
  22. Leal-Taixé, L., Pons-Moll, G., & Rosenhahn, B. (2011). Everybody needs somebody: Modeling social and grouping behavior on a linear programming multiple people tracker. In Proceedings of IEEE international conference on computer vision workshop Google Scholar
  23. Lenz, P., Geiger, A., & Urtasun, R. (2015). Followme: Efficient online min-cost flow tracking with bounded memory and computation. In Proceedings of IEEE international conference on computer vision (pp. 4364–4372).Google Scholar
  24. Li, X. R., & Jilkov, V. P. (2003). Survey of maneuvering target tracking. Part i. Dynamic models. IEEE Transactions on Aerospace and Electronic Systems, 39(4), 1333–1364.CrossRefGoogle Scholar
  25. Li, Y., Huang, C., & Nevatia, R. (2009). Learning to associate: Hybridboosted multi-target tracker for crowded scene. In Proceedings of IEEE conference on computer vision and pattern recognition (pp. 2953–2960).Google Scholar
  26. McLaughlin, N., del Rincón, J. M., & Miller, P. C. (2015). Enhancing linear programming with motion modeling for multi-target tracking. In IEEE winter conference on applications of computer vision (pp. 71–77).Google Scholar
  27. Milan, A., Leal-Taixé, L., Schindler, K., & Reid, I. (2015). Joint tracking and segmentation of multiple targets. In Proceedings of IEEE conference on computer vision and pattern recognition.Google Scholar
  28. Milan, A., Roth, S., & Schindler, K. (2014). Continuous energy minimization for multitarget tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence, 36(1), 58–72.CrossRefGoogle Scholar
  29. Milan, A., Schindler, K., & Roth, S. (2013). Detection- and trajectory-level exclusion in multiple object tracking. In Proceedings of IEEE conference on computer vision and pattern recognition.Google Scholar
  30. Pellegrini, S., Ess, A., Schindler, K., & Gool, L. V. (2009). You’ll never walk alone: Modeling social behavior for multi-target tracking. In Proceedings of IEEE international conference on computer vision (pp. 261–268).Google Scholar
  31. Pirsiavash, H., Ramanan, D., & Fowlkes, C. C. (2011). Globally-optimal greedy algorithms for tracking a variable number of objects. In Proceedings of IEEE conference on computer vision and pattern recognition (pp. 1201–1208).Google Scholar
  32. Poiesi, F., Mazzon, R., & Cavallaro, A. (2013). Multi-target tracking on confidence maps: An applicaion to people tracking. In CVIU (pp. 257–1272).Google Scholar
  33. Rezatofighi, H., Milan, A., Zhang, Z., Shi, Q., Dick, A., & Reid, I. (2015). Joint probabilistic data association revisited. In Proceedings of IEEE international conference on computer vision (pp. 3047–3055).Google Scholar
  34. Rezatofighi, S. H., Milan, A., Zhang, Z., Shi, Q., Dick, A., & Reid, I. (2016). Joint probabilistic matching using m-Best solutions. In Proceedings of IEEE conference on computer vision and pattern recognition (pp. 136–145).Google Scholar
  35. Takala, V., Pietikäinen, M. (2007). Multi-object tracking using color, texture and motion. In Proceedings of IEEE conference on computer vision and pattern recognition.Google Scholar
  36. Wang, S., & Fowlkes, C. (2015). Learning optimal parameters for multi-target tracking. In Proceedings of the British machine vision conference (pp. 4.1–4.13).Google Scholar
  37. Wang, X., Yang, M., Zhu, S., & Lin, Y. (2015). Regionlets for generic object detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 37(10), 2071–2084.CrossRefGoogle Scholar
  38. Wu, Z., & Betke, M. (2016). Global optimization for coupled detection and data association in multiple object tracking. Computer Vision and Image Understanding, 143, 25–37.CrossRefGoogle Scholar
  39. Xiang, Y., Alahi, A., & Savarese, S. (2015). Learning to track: Online multi-object tracking by decision making. In Proceedings of IEEE international conference on computer vision (pp. 4705–4713).Google Scholar
  40. Xing, J., Ai, H., & Lao, S. (2009). Multi-object tracking through occlusions by local tracklets filtering and global tracklets association with detection responses. In Proceedings of IEEE conference on computer vision and pattern recognition (pp. 1200–1207).Google Scholar
  41. Yang, B., & Nevatia, R. (2012). Multi-target tracking by online learning of non-linear motion patterns and robust appearance models. In Proceedings of IEEE conference on computer vision and pattern recognition (pp. 1918–1925).Google Scholar
  42. Yang, B., & Nevatia, R. (2014). Multi-target tracking by online learning a CRF model of appearance and motion patterns. International Journal of Computer Vision, 107(2), 203–217.MathSciNetCrossRefzbMATHGoogle Scholar
  43. Yoon, J. H., Lee, C. R., Yang, M. H., & Yoon, K. J. (2016). Online multi-object tracking via structural constraint event aggregation. In Proceedings of IEEE conference on computer vision and pattern recognition (pp. 1392–1400).Google Scholar
  44. Yoon, J. H., Yang, M. H., Lim, J., & Yoon, K. J. (2015). Bayesian multi-object tracking using motion context from multiple objects. In Proceedings of IEEE winter conference on applications of computer vision (pp. 33–40).Google Scholar
  45. Zhang, L., & van der Maaten, L. (2013). Structure preserving object tracking. In Proceedings of IEEE conference on computer vision and pattern recognition (pp. 1838–1845).Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Korea Electronics Technology InstituteSeongnamRepublic of Korea
  2. 2.Electrical Engineering and Computer ScienceGwangju Institute of Science and TechnologyGwangjuRepublic of Korea
  3. 3.Electrical Engineering and Computer ScienceUniversity of California at MercedMercedUSA
  4. 4.Department of Mechanical EngineeringKorea Advanced Institute of Science and TechnologyDaejeonRepublic of Korea

Personalised recommendations