Extended Lucas-Kanade Tracking

  • Shaul Oron
  • Aharon Bar-Hille
  • Shai Avidan
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8693)


The Lucas-Kanade (LK) method is a classic tracking algorithm exploiting target structural constraints thorough template matching. Extended Lucas Kanade or ELK casts the original LK algorithm as a maximum likelihood optimization and then extends it by considering pixel object / background likelihoods in the optimization. Template matching and pixel-based object / background segregation are tied together by a unified Bayesian framework. In this framework two log-likelihood terms related to pixel object / background affiliation are introduced in addition to the standard LK template matching term. Tracking is performed using an EM algorithm, in which the E-step corresponds to pixel object/background inference, and the M-step to parameter optimization. The final algorithm, implemented using a classifier for object / background modeling and equipped with simple template update and occlusion handling logic, is evaluated on two challenging data-sets containing 50 sequences each. The first is a recently published benchmark where ELK ranks 3rd among 30 tracking methods evaluated. On the second data-set of vehicles undergoing severe view point changes ELK ranks in 1st place outperforming state-of-the-art methods.


  1. 1.
    Appel, R., Fuchs, T., Dollar, P., Perona, P.: Quickly boosting decision trees a pruning underachieving features early. In: ICML (2013)Google Scholar
  2. 2.
    Avidan, S.: Support vector tracking. PAMI (2004)Google Scholar
  3. 3.
    Avidan, S.: Ensemble tracking. In: CVPR (2005)Google Scholar
  4. 4.
    Babenko, B., Yang, M.H., Belongie, S.: Visual tracking with online multiple instance learning. In: CVPR (2009)Google Scholar
  5. 5.
    Bao, C., Wu, Y., Ling, H., Ji, H.: Real time robust l1 tracker using accelerated proximal gradient approach. In: CVPR (2012)Google Scholar
  6. 6.
    Comaniciu, D., Ramesh, V., Meer, P.: Real-time tracking of non-rigid objects using mean shift. In: CVPR (2000)Google Scholar
  7. 7.
    Cootes, T., Edwards, G., Taylor, C.: Active appearance models. TPAMI (2001)Google Scholar
  8. 8.
    DeGroot, M.: Optimal Statistical Decisions. McGraw-Hill, New York (1970)Google Scholar
  9. 9.
    Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum likelihood from incomplete data via the em algorithm. Journal of the Royal Statistical Society 39(1), 1–38 (1977)zbMATHMathSciNetGoogle Scholar
  10. 10.
    Dinh, T.B., Vo, N., Medioni., G.: Context tracker: Exploring supporters and distracters in unconstrained environments. In: CVPR (2011)Google Scholar
  11. 11.
    Grabner, H., Grabner, M., Bischof, H.: Real-time tracking via online boosting. In: BMVC (2006)Google Scholar
  12. 12.
    Hare, S., Saffari, A., Torr., P.H.S.: Struck: Structured output tracking with kernels. In: ICCV (2011)Google Scholar
  13. 13.
    Henriques, J.F., Caseiro, R., Martins, P., Batista, J.: Exploiting the circulant structure of tracking-by-detection with kernels. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part IV. LNCS, vol. 7575, pp. 702–715. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  14. 14.
    Jia, X., Lu, H., Yang, M.H.: Visual tracking via adaptive structural local sparse appearance model. In: CVPR (2012)Google Scholar
  15. 15.
    Kalal, Z., Mikolajczyk, K., Matas, J.: Tracking-learning-detection. TPAMI (2010)Google Scholar
  16. 16.
    Kwon, J., Lee, K.M.: Visual tracking decomposition. In: CVPR (2010)Google Scholar
  17. 17.
    Kwon, J., Lee, K.M.: Tracking by sampling trackers. In: ICCV (2011)Google Scholar
  18. 18.
    Liu, B., Huang, J., Yang, L., Kulikowsk, C.: Robust tracking using local sparse appearance model and k-selection. In: CVPR (2011)Google Scholar
  19. 19.
    Lucas, B.D., Kanade, T.: An iterative image registration technique with an application to stereo vision. In: Proccedings of Imageing Understanding Workshop (1981)Google Scholar
  20. 20.
    Matthews, I., Baker, S.: Lucas-kanade 20 years on: A unifying framework. IJCV (2004)Google Scholar
  21. 21.
    Matthews, I., Ishikawa, T., Baker, S.: The template update problem. TPAMI (2004)Google Scholar
  22. 22.
    Oron, S., Bar-Hillel, A., Avidan, S.: Real time tracking-with-detection. Submitted to Machine Vision and Applications (2014)Google Scholar
  23. 23.
    Oron, S., Hillel, A.B., Levi, D., Avidan, S.: Locally orderless tracking. In: CVPR (2012)Google Scholar
  24. 24.
    Ross, D., Lim, J., Lin, R.S., Yang, M.H.: Incremental learning for robust visual tracking. IJCV (2007)Google Scholar
  25. 25.
    Stauffer, C., Grimson, E.: Learning patterns of activity using real-time tracking. PAMI (2000)Google Scholar
  26. 26.
    Wu, Y., Lim, J., Yang, M.H.: Online object tracking: A benchmark. In: CVPR (2013)Google Scholar
  27. 27.
    Xiang, Y., Song, C., Mottaghi, R., Savarese, S.: Monocular multiview object tracking with 3d aspect parts. In: European Conference on Computer Vision, ECCV (2014)Google Scholar
  28. 28.
    Yilmaz, A., Javed, O., Shah, M.: Object tracking: A survey. ACM. Comp. Survey 38(4) (2006)Google Scholar
  29. 29.
    Zhang, K., Zhang, L., Yang, M.-H.: Real-time compressive tracking. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part III. LNCS, vol. 7574, pp. 864–877. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  30. 30.
    Zhong, W., Lu, H., Yang, M.H.: Robust object tracking via sparsity-based collaborative model. In: CVPR (2012)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Shaul Oron
    • 1
  • Aharon Bar-Hille
    • 2
  • Shai Avidan
    • 1
  1. 1.Tel Aviv UniversityIsrael
  2. 2.Microsoft ResearchUSA

Personalised recommendations