Object tracking with particles weighted by region proposal network

  • Qidan Zhu
  • Yanke WangEmail author
  • Yunqian He
  • Xiao Hong


Most of existing particle filters suffer from computation of weights and almost all object detection networks have the risk of missing objects. Therefore, we propose a novel way to calculate the weight for each particle using the anchor scores output from region proposal network (RPN) in Faster RCNN. We first change the original anchor style in RPN slightly while training and then cast particles in feature space of VGG16 to do filtering both for center and scale. Without fully connected layers, it can lower the computational cost to a large extent and it can effectively maintain an accurate prediction of the posterior density using less than 30 particles. When increasing the number of particles, it is still capable to stay in a stable operating speed as there is no need to compute weights for particles a second time. Extensive experimental results on parts of OTB datasets and comparison with other methods demonstrate that the proposed tracker performs favorably both in location of object and the decision of scale.


Object tracking Particle filter Region proposal network Anchor score 



  1. 1.
    Comaniciu D, Ramesh V, Meer P (2003) Kernel-based object tracking. IEEE Trans Pattern Anal Mach Intell 25(5):564–577CrossRefGoogle Scholar
  2. 2.
  3. 3.
    Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: IEEE computer society conference on computer vision and pattern recognition (CVPR)Google Scholar
  4. 4.
    Deng J, Dong W, Girshick R, Socher R (2009) ImageNet: a large-scale hierarchical image database. In: IEEE Computer Society conference on computer vision and pattern recognition (CVPR)Google Scholar
  5. 5.
    Derpanis KG (2004) The Harris corner detector,
  6. 6.
    Ding J, Tang Y, Liu W et al (2015) Tracking by local structural manifold learning in a new SSIR particle filter. Neurocomputing 161(5):277–289CrossRefGoogle Scholar
  7. 7.
    Fang J, Wang Q, Yuan Y (2014) Part-based online tracking with geometry constraint and attention selection. IEEE Trans Circ Syst Video Technol 24(5):854–864CrossRefGoogle Scholar
  8. 8.
  9. 9.
    Geng Y, Liang R-Z, Li W, Wang J, Liang G, Xu C, Wang J-Y (2016) Learning convolutional neural network to maximize pos@ top performance measure. In: ESANN 2017 - ProceedingsGoogle Scholar
  10. 10.
    Geng Y, Zhang G, Li W, Gu Y, Liang R-Z, Liang G, Wang J, Wu Y, Patil N, Wang J-Y (2017) A novel image tag completion method based on convolutional neural transformation. In: International conference on artificial neural networks, pp 539–546Google Scholar
  11. 11.
    Girshick R (2015) Fast R-CNN. In: International conference on computer vision (ICCV)Google Scholar
  12. 12.
    Girshick R, Donahue J, Darrell T, Malik J (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. In: IEEE Computer Society conference on computer vision and pattern recognition (CVPR)Google Scholar
  13. 13.
    He K, Zhang X, Ren S, Sun J (2015) Deep residual learning for image recognition. arXiv:1512.03385v1
  14. 14.
    He K, Gkioxari G, Dollar P, Girshick R (2017) Mask R-CNN. In: IEEE international conference on computer vision (ICCV)Google Scholar
  15. 15.
    Jenkins MD, Barrie P, Buggy T, Morison G (2018) Selective sampling importance resampling particle filter tracking with multibag subspace restoration. IEEE Trans Cybern 48:264–276CrossRefGoogle Scholar
  16. 16.
    Levi DM (2008) Crowding—an essential bottleneck for object recognition: a mini-review. Vis Res 48(5):635–654CrossRefGoogle Scholar
  17. 17.
    Li X, Hu W, Shen CA et al (2013) A survey of appearance models in visual object tracking. ACM Trans Intell Syst Technol (TIST) 4:58Google Scholar
  18. 18.
    Lienhart R, Maydt J (2002) An extended set of haar-like features for rapid object detection. In: International conference on image processing (ICIP)Google Scholar
  19. 19.
    Lin T-Y, Dollar P, Girshick R, He K, Hariharan B, Belongie S (2017) Feature pyramid networks for object detection. arXiv:1612.03144v2
  20. 20.
    Liu W, Anguelov D, Erhan D et al (2016) SSD: single shot multibox detector. In: European conference on computer vision (ECCV), vol 9905, pp 21–37CrossRefGoogle Scholar
  21. 21.
    Lucas BD, Kanade T (1981) An iterative image registration technique with an application to stereo vision. In: Proceedings of the 7th international joint conference on artificial intelligence, vol 4, pp 674–679Google Scholar
  22. 22.
    Ma C, Huang J-B, Yang X, Yang M-H (2015) Hierarchical convolutional features for visual tracking. In: International conference on computer vision (ICCV)Google Scholar
  23. 23.
    Mehdipour M, Ekenel HK (2016) A comprehensive analysis of deep learning based representation for face recognition. In: IEEE conference on computer vision and pattern recognition (CVPR) workshops, pp 34–41Google Scholar
  24. 24.
    Okuma K, Taleghani A, de Freitas N et al (2004) A boosted particle filter: multitarget detection and tracking. In: European conference on computer vision (ECCV), vol 3021, pp 28–39Google Scholar
  25. 25.
    Pancham A, Withey D, Bright G (2015) Tracking image features with PCA-SURF descriptors. In: 14th IAPR international conference on machine vision applications (MVA)Google Scholar
  26. 26.
    Porikli F, Tuzel O, Meer P (2006) Covariance tracking using model update based on lie algebra. In: IEEE Computer Society conference on computer vision and pattern recognition (CVPR)Google Scholar
  27. 27.
    Ray S, Turi RH (1999) Determination of number of clusters in -means clustering and application in colour image segmentation. In: Proceedings of the 4th international conference on advances in pattern recognition and digital techniques (ICAPRDT’99)Google Scholar
  28. 28.
    Redmon J, Farhadi A (2017) YOLO9000: better, faster, stronger. In: IEEE Computer Society conference on computer vision and pattern recognition (CVPRGoogle Scholar
  29. 29.
    Redmon J, Divvala S, Girshick R, Farhadi A (2016) You only look once: unified, real-time object detection. In: IEEE Computer Society conference on computer vision and pattern recognition (CVPR)Google Scholar
  30. 30.
    Ren S, He K, Girshick R, Sun J (2016) Faster R-CNN: Towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell 39:1137–1149CrossRefGoogle Scholar
  31. 31.
    Ross DA, Lim J, Lin R-S et al (2008) Incremental learning for robust visual tracking. Int J Comput Vis 77:125–141CrossRefGoogle Scholar
  32. 32.
    Rout RK (2015) A survey on object detection and tracking algorithms, pp 1–75Google Scholar
  33. 33.
    Smeulders AWM, Chu DM, Cucchiara R et al (2013) Visual tracking: an experimental survey. IEEE Trans Pattern Anal Mach Intell 36:1442–1468Google Scholar
  34. 34.
    Tang F, Brennan S, Zhao Q, Tao H (2007) Co-tracking using semi-supervised support vector machines. In: IEEE 11th international conference on computer vision (ICCV)Google Scholar
  35. 35.
    Trzcinski T, Christoudias M, Lepetit V, Fua P (2012) Learning image descriptors with the boosting-trick. In: 25th international conference on neural information processing systems (NIPS), vol 1, pp 269–277Google Scholar
  36. 36.
    Tuzel O, Porikli F, Meer P (2006) Region covariance: a fast descriptor for detection and classification. In: European conference on computer vision (ECCV), vol 3952, pp 589–600CrossRefGoogle Scholar
  37. 37.
    Vedaldi A (2007) An open implementation of the SIFT detector and descriptor,
  38. 38.
    Wang Q, Fang J, Yuan Y (2013) Multi-cue based tracking. Neurocomputing 131:227–236CrossRefGoogle Scholar
  39. 39.
    Wu Y, Lim J, Yang M-H (2013) Online object tracking: a benchmark. In: The IEEE conference on computer vision and pattern recognition (CVPR), pp 2411–2418Google Scholar
  40. 40.
    Yilmaz A, Javed O, Shah M (2006) Object tTracking: a survey. ACM Comput 38(4):1–45Google Scholar
  41. 41.
    Yuan Y, Lu Y, Wang Q (2017) Tracking as a whole: multi-target tracking by modeling group behavior with sequential detection. IEEE Trans Intell Transp Syst 18 (12):3339–3349CrossRefGoogle Scholar
  42. 42.
    Zhang G, Liang G, Li W, Fang J, Wang J, Geng Y, Wang J-Y (2017) Learning convolutional ranking-score function by query preference regularization. In: International conference on intelligent data engineering and automated learning, pp 1–8Google Scholar
  43. 43.
    Zhang G, Liang G, Su F, Qu F, Wang J-Y (2018) Learning convolutional attribute embedding for domain-transfer learning. Lecture Notes in Artificial IntelligenceGoogle Scholar
  44. 44.
    Zhou Z, Zhou M, Li J (2017) Object tracking method based on hybrid particle filter and sparse representation. Multimed Tools Appl 76(2):2979–299CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Institute of Intelligent Control, College of AutomationHarbin Engineering UniversityHarbinChina

Personalised recommendations