Advertisement

Learning a Robust Society of Tracking Parts Using Co-occurrence Constraints

  • Elena BurceanuEmail author
  • Marius Leordeanu
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11129)

Abstract

Object tracking is an essential problem in computer vision that has been researched for several decades. One of the main challenges in tracking is to adapt to object appearance changes over time and avoiding drifting to background clutter. We address this challenge by proposing a deep neural network composed of different parts, which functions as a society of tracking parts. They work in conjunction according to a certain policy and learn from each other in a robust manner, using co-occurrence constraints that ensure robust inference and learning. From a structural point of view, our network is composed of two main pathways. One pathway is more conservative. It carefully monitors a large set of simple tracker parts learned as linear filters over deep feature activation maps. It assigns the parts different roles. It promotes the reliable ones and removes the inconsistent ones. We learn these filters simultaneously in an efficient way, with a single closed-form formulation, for which we propose novel theoretical properties. The second pathway is more progressive. It is learned completely online and thus it is able to better model object appearance changes. In order to adapt in a robust manner, it is learned only on highly confident frames, which are decided using co-occurrences with the first pathway. Thus, our system has the full benefit of two main approaches in tracking. The larger set of simpler filter parts offers robustness, while the full deep network learned online provides adaptability to change. As shown in the experimental section, our approach achieves state of the art performance on the challenging VOT17 benchmark, outperforming the published methods both on the general EAO metric and in the number of fails, by a significant margin.

Keywords

Unsupervised tracking Co-occurrences Part-based tracker 

Notes

Acknowledgements

This work was supported in part by UEFISCDI, under projects PN-III-P4-ID-ERC-2016-0007 and PN-III-P1-1.2-PCCDI-2017-0734.

Supplementary material

478770_1_En_9_MOESM1_ESM.zip (95.4 mb)
Supplementary material 1 (zip 97699 KB)

References

  1. 1.
    Visual object tracking using adaptive correlation filters (2010).  https://doi.org/10.1109/CVPR.2010.5539960
  2. 2.
    Agapito, L., Bronstein, M.M., Rother, C. (eds.): Computer Vision - ECCV 2014 Workshops - Zurich, Switzerland, September 6–7 and 12, 2014, Proceedings, Part II. LNCS, vol. 8926. Springer, Cham (2015).  https://doi.org/10.1007/978-3-319-16181-5CrossRefGoogle Scholar
  3. 3.
    Akin, O., Erdem, E., Erdem, A., Mikolajczyk, K.: Deformable part-based tracking by coupled global and local correlation filters. J. Vis. Commun. Image Represent. 38, 763–774 (2016).  https://doi.org/10.1016/j.jvcir.2016.04.018CrossRefGoogle Scholar
  4. 4.
    Bertinetto, L., Valmadre, J., Golodetz, S., Miksik, O., Torr, P.H.S.: Staple: complementary learners for real-time tracking. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, 27–30 June 2016 (2016).  https://doi.org/10.1109/CVPR.2016.156
  5. 5.
    Cehovin, L., Leonardis, A., Kristan, M.: Robust visual tracking using template anchors. In: 2016 IEEE Winter Conference on Applications of Computer Vision, WACV 2016, Lake Placid, NY, USA, 7–10 March 2016 (2016).  https://doi.org/10.1109/WACV.2016.7477570
  6. 6.
    Chen, D., Yuan, Z., Wu, Y., Zhang, G., Zheng, N.: Constructing adaptive complex cells for robust visual tracking. In: IEEE International Conference on Computer Vision, ICCV 2013, Sydney, Australia, 1–8 December 2013 (2013).  https://doi.org/10.1109/ICCV.2013.142
  7. 7.
    Danelljan, M., Bhat, G., Khan, F.S., Felsberg, M.: ECO: efficient convolution operators for tracking. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, 21–26 July 2017 (2017).  https://doi.org/10.1109/CVPR.2017.733
  8. 8.
    Danelljan, M., Häger, G., Khan, F.S., Felsberg, M.: Discriminative scale space tracking. IEEE Trans. Pattern Anal. Mach. Intell. 39(8), 1561–1575 (2017).  https://doi.org/10.1109/TPAMI.2016.2609928CrossRefGoogle Scholar
  9. 9.
    Danelljan, M., Robinson, A., Shahbaz Khan, F., Felsberg, M.: Beyond correlation filters: learning continuous convolution operators for visual tracking. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9909, pp. 472–488. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-46454-1_29CrossRefGoogle Scholar
  10. 10.
    Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: ImageNet: a large-scale hierarchical image database. In: CVPR (2009)Google Scholar
  11. 11.
    Du, D., Qi, H., Li, W., Wen, L., Huang, Q., Lyu, S.: Online deformable object tracking based on structure-aware hyper-graph. IEEE Trans. Image Process. 25(8), 3572–3584 (2016).  https://doi.org/10.1109/TIP.2016.2570556MathSciNetCrossRefGoogle Scholar
  12. 12.
    Forsyth, D.A.: Object detection with discriminatively trained part-based models. IEEE Comput. 47, 1–7 (2010)Google Scholar
  13. 13.
    Godec, M., Roth, P.M., Bischof, H.: Hough-based tracking of non-rigid objects. Comput. Vis. Image Underst. 117, 1245–1256 (2011)CrossRefGoogle Scholar
  14. 14.
    Haller, E., Leordeanu, M.: Unsupervised object segmentation in video by efficient selection of highly probable positive features. In: IEEE International Conference on Computer Vision, ICCV 2017, Venice, Italy, 22–29 October 2017 (2017).  https://doi.org/10.1109/ICCV.2017.544
  15. 15.
    He, Z., Fan, Y., Zhuang, J., Dong, Y., Bai, H.: Correlation filters with weighted convolution responses. In: The IEEE International Conference on Computer Vision (ICCV) Workshops, October 2017Google Scholar
  16. 16.
    Henriques, J.F., Caseiro, R., Martins, P., Batista, J.: High-speed tracking with kernelized correlation filters. CoRR abs/1404.7584 (2014). http://arxiv.org/abs/1404.7584
  17. 17.
    Kristan, M., et al.: The visual object tracking VOT2016 challenge results. In: Hua, G., Jégou, H. (eds.) ECCV 2016. LNCS, vol. 9914, pp. 777–823. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-48881-3_54CrossRefGoogle Scholar
  18. 18.
    Kristan, M., et al.: The visual object tracking vot2017 challenge results. In: The IEEE International Conference on Computer Vision (ICCV) Workshops, October 2017Google Scholar
  19. 19.
    Kuo, C.H., Nevatia, R.: How does person identity recognition help multi-person tracking? In: CVPR (2011)Google Scholar
  20. 20.
    Kwon, J., Lee, K.M.: Tracking of a non-rigid object via patch-based dynamic appearance modeling and adaptive basin hopping Monte Carlo sampling. In: 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2009), 20–25 June 2009, Miami, Florida, USA (2009).  https://doi.org/10.1109/CVPRW.2009.5206502
  21. 21.
    Lukezic, A., Cehovin, L., Kristan, M.: Deformable parts correlation filters for robust visual tracking. CoRR abs/1605.03720 (2016). http://arxiv.org/abs/1605.03720
  22. 22.
    Lukezic, A., Vojir, T., Zajc, L.C., Matas, J., Kristan, M.: Discriminative correlation filter with channel and spatial reliability. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, 21–26 July 2017 (2017).  https://doi.org/10.1109/CVPR.2017.515
  23. 23.
    Luo, W., Zhao, X., Kim, T.: Multiple object tracking: a review. CoRR abs/1409.7618 (2014). http://arxiv.org/abs/1409.7618
  24. 24.
    Malisiewicz, T., Gupta, A., Efros, A.A.: Ensemble of exemplar-SVMs for object detection and beyond. In: 2011 IEEE International Conference on Computer Vision (ICCV). IEEE (2011)Google Scholar
  25. 25.
    Murphy, K.P.: Machine Learning - A Probabilistic Perspective. Adaptive Computation and Machine Learning Series. MIT Press (2012)Google Scholar
  26. 26.
    Nam, H., Baek, M., Han, B.: Modeling and propagating CNNs in a tree structure for visual tracking. CoRR abs/1608.07242 (2016). http://arxiv.org/abs/1608.07242
  27. 27.
    Nam, H., Han, B.: Learning multi-domain convolutional neural networks for visual tracking. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, 27–30 June 2016 (2016).  https://doi.org/10.1109/CVPR.2016.465
  28. 28.
    Nebehay, G., Pflugfelder, R.P.: Clustering of static-adaptive correspondences for deformable object tracking. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2015, Boston, MA, USA, 7–12 June 2015 (2015).  https://doi.org/10.1109/CVPR.2015.7298895
  29. 29.
    Paszke, A., Gross, S., Chintala, S., Chanan, G.: Pytorch (2017)Google Scholar
  30. 30.
    Shu, G., Dehghan, A., Oreifej, O., Hand, E., Shah, M.: Part-based multiple-person tracking with partial occlusion handling. In: CVPR (2012)Google Scholar
  31. 31.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. CoRR abs/1409.1556 (2014). http://arxiv.org/abs/1409.1556
  32. 32.
    Smeulders, A.W.M., Chu, D.M., Cucchiara, R., Calderara, S., Dehghan, A., Shah, M.: Visual tracking: an experimental survey. IEEE Trans. Pattern Anal. Mach. Intell. 36, 1442–1468 (2014)CrossRefGoogle Scholar
  33. 33.
    Valstar, M.F., French, A.P., Pridmore, T.P. (eds.): British Machine Vision Conference, BMVC 2014, Nottingham, UK, 1–5 September 2014. BMVA Press (2014)Google Scholar
  34. 34.
    Vojir, T., Noskova, J., Matas, J.: Robust scale-adaptive mean-shift for tracking. Pattern Recognit. Lett. 49, 250–258 (2014).  https://doi.org/10.1016/j.patrec.2014.03.025CrossRefGoogle Scholar
  35. 35.
    Wang, S., Lu, H., Yang, F., Yang, M.H.: Superpixel tracking. In: ICCV (2011)Google Scholar
  36. 36.
    Wu, Y., Lim, J., Yang, M.H.: Object tracking benchmark. IEEE Trans. Pattern Anal. Mach. Intell. 37, 1834–1848 (2015)CrossRefGoogle Scholar
  37. 37.
    Zhang, T., Xu, C., Yang, M.: Multi-task correlation particle filter for robust object tracking. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, 21–26 July 2017 (2017).  https://doi.org/10.1109/CVPR.2017.512
  38. 38.
    Zhu, G., Porikli, F., Li, H.: Beyond local search: tracking objects everywhere with instance-specific proposals. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, 27–30 June 2016 (2016).  https://doi.org/10.1109/CVPR.2016.108

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.BitdefenderBucharestRomania
  2. 2.Institute of Mathematics of the Romanian AcademyBucharestRomania
  3. 3.Mathematics and Computer ScienceUniversity of BucharestBucharestRomania
  4. 4.Automatic Control and Computer ScienceUniversity Politehnica of BucharestBucharestRomania

Personalised recommendations