Advertisement

International Journal of Computer Vision

, Volume 67, Issue 3, pp 343–363 | Cite as

A General Framework for Combining Visual Trackers – The "Black Boxes" Approach

  • IDO LeichterEmail author
  • MICHAEL LINDENBAUM
  • EHUD RIVLIN
Article

Abstract.

Over the past few years researchers have been investigating the enhancement of visual tracking performance by devising trackers that simultaneously make use of several different features. In this paper we investigate the combination of synchronous visual trackers that use different features while treating the trackers as “black boxes”. That is, instead of fusing the usage of the different types of data as has been performed in previous work, the combination here is allowed to use only the trackers' output estimates, which may be modified before their propagation to the next time step. We propose a probabilistic framework for combining multiple synchronous trackers, where each separate tracker outputs a probability density function of the tracked state, sequentially for each image. The trackers may output either an explicit probability density function, or a sample-set of it via Condensation. Unlike previous tracker combinations, the proposed framework is fairly general and allows the combination of any set of trackers of this kind, even in different state-spaces of different dimensionality, under a few reasonable assumptions. The combination may consist of different trackers that track a common object, as well as trackers that track separate, albeit related objects, thus improving the tracking performance of each object. The benefits of merely using the final estimates of the separate trackers in the combination are twofold. Firstly, the framework for the combination is fairly general and may be easily used from the software aspects. Secondly, the combination may be performed in a distributed setting, where each separate tracker runs on a different site and uses different data, while avoiding the need to share the data. The suggested framework was successfully tested using various state-spaces and datasets, demonstrating that fusing the trackers' final distribution estimates may indeed be applicable.

Keywords:

visual tracking tracker combination Kalman filter Condensation 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Supplementary material

Supplementary material (5.00 MB)

Supplementary material (5.00 MB)

experiment_ii_withou_1CBB2B.m1v (311 kb)
Supplementary material (5.00 MB)
experiment_ii_with_c_1CBB2A.m1v (410 kb)
Supplementary material (5.00 MB)

Supplementary material (5.00 MB)

Supplementary material (5.00 MB)

Supplementary material (5.00 MB)

Supplementary material (5.00 MB)

Supplementary material (5.00 MB)

experiment_vi_withou_1CBB32.m1v (103 kb)
Supplementary material (5.00 MB)
experiment_vi_with_c_1CBB31.m1v (103 kb)
Supplementary material (5.00 MB)

References

  1. Baker, T. and Strens, M. 1998. Representation of uncertainty in spatial target tracking. In Proceedings of the 14th International Conference on Pattern Recognition, pp. 1339–1342.Google Scholar
  2. Bar-Shalom, Y. and Fortmann, T. 1988. Tracking and Data Association. Academic Press.Google Scholar
  3. Bar-Shalom, Y. ed. 1992. Multitarget-multisensor tracking. Artech House.Google Scholar
  4. Beal, M.J., Jojic, N., and Attias, H. 2003. A graphical model for audiovisual object tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(7):828–836.Google Scholar
  5. Blake, A., Curwen, R., and Zisserman, A. 1993. A framework for spatio-temporal control in the tracking of visual contours. International Journal of Computer Vision, 11(2):127–145.Google Scholar
  6. Blatt, D. and Hero, A. 2004. Distributed maximum likelihood estimation for sensor networks. In Proceedings of the 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing, vol. 3, pp. 929–932.Google Scholar
  7. Collins, R. and Liu, Y. 2003. On-line selection of discriminative tracking features. In Proceedings of the 9th IEEE International Conference on Computer Vision, pp. 346–352.Google Scholar
  8. Darrell, T., Gordon, G., Harville, M., and Woodfill, J. 2000. Integrated person tracking using stereo, color, and pattern detection. International Journal of Computer Vision, 37(2):175–185.Google Scholar
  9. Gil, S., Milanese, R., and Pun, T. 1996. Combining multiple motion estimates for vehicle tracking. In Proceedings of the 4th European Conference on Computer Vision, vol. 2, pp. 307–320.Google Scholar
  10. Hue, C., Le Cadre, J.P., and Pérez, P. 2001. A particle filter to track multiple objects. In Proceedings of the 2001 IEEE Workshop on Multi-Object Tracking. Google Scholar
  11. Isard, M. and Blake, A. 1998. Condensation -- conditional density propagation for visual tracking. International Journal of Computer Vision, 29(1):5–28.Google Scholar
  12. Isard, M. and MacCormick, J. 2001. BraMBLe: A bayesian multiple-blob tracker. In Proceedings of the 8th IEEE International Conference on Computer Vision, pp. 34–41.Google Scholar
  13. Isard, M. 2003. PAMPAS: Real-valued graphical models for computer vision. In Proceedings of the 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 613–620.Google Scholar
  14. Jepson, A.D., Fleet, D.J., and El-Maraghi, T. F. 2001. Robust online appearance models for visual tracking. In Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 415–422.Google Scholar
  15. Leichter, I., Lindenbaum, M., and Rivlin. E. 2004a. A probabilistic cooperation between trackers of coupled objects. In Proceedings of the 2004 IEEE International Conference on Image Processing, vol. 2, pp. 1045–1048.Google Scholar
  16. Leichter, I., Lindenbaum, M., and Rivlin. E. 2004b. A probabilsitic framework for combining tracking algorithms. In Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 445–451.Google Scholar
  17. MacCormick, J. and Blake, A. 2000b. A probabilistic exclusion principle for tracking multiple objects. International Journal of Computer Vision, 39(1):57–71.Google Scholar
  18. McCane, B., Galvin, B., and Novins, K. 2002. Algorithmic fusion for more rubust feature tracking. International Journal of Computer Vision, 49(1):79–89.Google Scholar
  19. Okuma, K., Taleghani, A., de Freitas, N., Little, J.J., and Lowe, S.G. 2004. A boosted particle filter: Multitarget detection and tracking. In Proceedings of the 8th European Conference on Computer Vision, vol. 1, pp. 28–39.Google Scholar
  20. Papoulis, A. 1991. Probability, Random Variables, and Stochastic Processes. McGraw-Hill, 3rd edition.Google Scholar
  21. Pérez, P., Hue, C., Vermaak, J., and Gangnet, M. 2002. Color-based probabilistic tracking. In Proceedings of the 7th European Conference on Computer Vision, pp. 661–675.Google Scholar
  22. Pérez, P., Vermaak, J., and Blake, A. 2004. Data fusion for visual tracking with particles. Proceedings of the IEEE, 92(3):495–513.Google Scholar
  23. Rasmussen, C. and Hager, G.D. 2001. Probabilistic data association methods for tracking complex visual objects. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(6):560–576.Google Scholar
  24. Rosenberg, Y., and Werman, M. 1997a. A general filter for measurements with any probability distribution. In Proceedings of the 1997 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 654–659.Google Scholar
  25. Rosenberg, Y. and Werman, M. 1997b. Representing local motion as a probability distribution matrix applied to object tracking. In Proceedings of the 1997 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 106–111.Google Scholar
  26. Scott, D.W. 1992. Multivariate Density Estimation. New York: Wiley.Google Scholar
  27. Shearer, K., Wong, K.D., and Venkatesh, S. 2001. Combining multiple tracking algorithms for improved general performance. Pattern Recognition, 34(6):1257–1269.Google Scholar
  28. Sidenbladh, H. and Wirkander, S.L. 2003. Tracking random sets of vehicles in terrain. In Proceedings of the 2003 IEEE Workshop on Multi-Object Tracking. Google Scholar
  29. Siebel, N.T. and Maybank, S. 2002. Fusion of multiple tracking algorithms for robust people tracking. In Proceedings of the 7th European Conference on Computer Vision, vol. 4, pp. 373–387.Google Scholar
  30. Spengler, M. and Schiele, B. 2003. Towards robust multi-cue integration for visual tracking. Machine Vision and Applications, 14:50–58.Google Scholar
  31. Sudderth, E.B., Ihler, A.T., Freeman, W.T., and Willsky, A.S. 2003. Nonparametric belief propagation. In Proceedings of the 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 605–612.Google Scholar
  32. Toyama, K. and Blake, A. 2002. Probabilistic tracking with exemplars in a metric space. International Journal of Computer Vision, 48(1):9–19.Google Scholar
  33. Toyama, A. and Hager, G.D. 1995. Tracker fusion for robustness in visual feature tracking. SPIE -- The International Society for Optical Engineering, 2569:38–49.Google Scholar
  34. Triesch, J. and von der Malsburg, C. 2000. Self-organized integration of adaptive visual cues for face tracking. In Proceedings of the 4th International Conference on Automatic Face and Gesture Recognition. Google Scholar
  35. Vermaak, J., Doucet, A., and Pérez, P. 2003. Maintaining multi-modality through mixture tracking. In Proceedings of the 9th IEEE International Conference on Computer Vision, vol. 2, pp. 1110–1116.Google Scholar
  36. Viola, P. and Jones, M. 2001. Rapid object detection using a boosted cascade of simple features. In Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 511–518.Google Scholar
  37. Wu, Y. and Huang, T.S. 2004. Robust visual tracking by integrating multiple cues based on co-inference learning. International Journal of Computer Vision, 58(1):55–71.Google Scholar
  38. Proceedings of the 2001 IEEE Workshop on Multi-Object Tracking, Proceedings of the 2003 IEEE Workshop on Multi-Object Tracking. Google Scholar
  39. http://www.visualsurveillance.org and related links.Google Scholar
  40. Proceedings of the IEEE International Workshop Series on Performance Evaluation of Tracking and Surveillance. Google Scholar

Copyright information

© Springer-Science + Business Media, Inc 2006

Authors and Affiliations

  1. 1.Department of Computer ScienceTechnion – Israel Institute of TechnologyTechnion City, Haifa 32000Israel

Personalised recommendations