Automatic Facial Landmark Tracking in Video Sequences Using Kalman Filter Assisted Active Shape Models

  • Utsav Prabhu
  • Keshav Seshadri
  • Marios Savvides
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6553)


In this paper we address the problem of automatically locating the facial landmarks of a single person across frames of a video sequence. We propose two methods that utilize Kalman filter based approaches to assist an Active Shape Model (ASM) in achieving this goal. The use of Kalman filtering not only aids in better initialization of the ASM by predicting landmark locations in the next frame but also helps in refining its search results and hence in producing improved fitting accuracy. We evaluate our tracking methods on frames from three video sequences and quantitatively demonstrate their reliability and accuracy.


Kalman Filter Video Sequence Tracking Method Face Detection Previous Frame 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Cootes, T.F., Hill, A., Taylor, C.J., Haslam, J.: The Use of Active Shape Models for Locating Structures in Medical Images. Image and Vision Computing 12, 355–366 (1994)CrossRefGoogle Scholar
  2. 2.
    de Bruijne, M., van Ginneken, B., Viergever, M.A., Niessen, W.J.: Adapting Active Shape Models for 3D Segmentation of Tubular Structures in Medical Images. In: Taylor, C.J., Noble, J.A. (eds.) IPMI 2003. LNCS, vol. 2732, pp. 136–147. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  3. 3.
    Cootes, T.F., Taylor, C.J., Lanitis, A.: Active Shape Models: Evaluation of a Multi-Resolution Method for Improving Image Search. In: Proceedings of the 5th British Machine Vision Conference, pp. 327–336 (1994)Google Scholar
  4. 4.
    Milborrow, S., Nicolls, F.: Locating Facial Features with an Extended Active Shape Model. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part IV. LNCS, vol. 5305, pp. 504–513. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  5. 5.
    Seshadri, K., Savvides, M.: Robust Modified Active Shape Model for Automatic Facial Landmark Annotation of Frontal Faces. In: BTAS 2009: Proceedings of the 3rd IEEE International Conference on Biometrics: Theory, Applications and Systems, pp. 319–326. IEEE Press, Piscataway (2009)Google Scholar
  6. 6.
    Gu, L., Kanade, T.: A Generative Shape Regularization Model for Robust Face Alignment. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part I. LNCS, vol. 5302, pp. 413–426. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  7. 7.
    Edwards, G.J., Cootes, T.F., Taylor, C.J.: Face Recognition Using Active Appearance Models. In: Burkhardt, H., Neumann, B. (eds.) ECCV 1998. LNCS, vol. 1407, pp. 581–595. Springer, Heidelberg (1998)CrossRefGoogle Scholar
  8. 8.
    Choi, H., Oh, S.: Real-time Recognition of Facial Expression Using Active Appearance Model with Second Order Minimization and Neural Network. In: Proceedings of the 2006 IEEE International Conference on Systems, Man, and Cybernetics, pp. 1559–1564 (2006)Google Scholar
  9. 9.
    Zhao, S., Gao, Y.: Automated Face Pose Estimation Using Elastic Energy Models. In: ICPR 2006: Proceedings of the 18th International Conference on Pattern Recognition, vol. 04, pp. 618–621. IEEE Computer Society, Washington, DC (2006)CrossRefGoogle Scholar
  10. 10.
    Kalman, R.E.: A New Approach to Linear Filtering and Prediction Problems. Transactions of the ASME – Journal of Basic Engineering 82, 35–45 (1960)CrossRefGoogle Scholar
  11. 11.
    Gower, J.C.: Generalized Procrustes Analysis. Psychometrika 40, 33–51 (1975)MathSciNetzbMATHCrossRefGoogle Scholar
  12. 12.
    Cootes, T.F., Taylor, C.J.: Statistical Models of Appearance for Computer Vision. Technical report, Imaging Science and Biomedical Engineering, University of Manchester (2004)Google Scholar
  13. 13.
    Cootes, T.F., Taylor, C.J., Cooper, D.H., Graham, J.: Active Shape Models - Their Training and Application. Computer Vision and Image Understanding 61, 38–59 (1995)CrossRefGoogle Scholar
  14. 14.
    Intel: Open Source Computer Vision Library (2007)Google Scholar
  15. 15.
    Viola, P., Jones, M.J.: Robust Real-Time Face Detection. International Journal of Computer Vision 57, 137–154 (2004)CrossRefGoogle Scholar
  16. 16.
    Phillips, P.J., Flynn, P.J., Beveridge, J.R., Scruggs, W.T., O’Toole, A.J., Bolme, D., Bowyer, K.W., Draper, B.A., Givens, G.H., Lui, Y.M., Sahibzada, H., Scallan III, J.A., Weimer, S.: Overview of the Multiple Biometrics Grand Challenge. In: Tistarelli, M., Nixon, M.S. (eds.) ICB 2009. LNCS, vol. 5558, pp. 705–714. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  17. 17.
    National Institute of Standards and Technology (NIST): Multiple Biometric Grand Challenge (2008),
  18. 18.
    Baumberg, A.M., Hogg, D.C.: An Efficient Method for Contour Tracking using Active Shape Models. In: Proceedings of the 1994 IEEE Workshop on Motion of Nonrigid and Articulated Objects, pp. 194–199 (1994)Google Scholar
  19. 19.
    Baumberg, A.: Hierarchical shape fitting using an iterated linear filter. Image and Vision Computing 16, 329–335 (1998)CrossRefGoogle Scholar
  20. 20.
    Lee, S.W., Kang, J., Shin, Paik, J.: Hierarchical active shape model with motion prediction for real-time tracking of non-rigid objects. IET Computer Vision 1, 17–24 (2007)MathSciNetCrossRefGoogle Scholar
  21. 21.
    Pu, B., Liang, S., Xie, Y., Yi, Z., Heng, P.: Video Facial Feature Tracking with Enhanced ASM and Predicted Meanshift. In: Proceedings of the 2nd International Conference on Computer Modeling and Simulation, vol. 2, pp. 151–155. IEEE Computer Society, Los Alamitos (2010)CrossRefGoogle Scholar
  22. 22.
    Comaniciu, D., Ramesh, V., Meer, P.: Real-Time Tracking of Non-Rigid Objects using Mean Shift. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 142–149. IEEE Computer Society, Los Alamitos (2000)Google Scholar
  23. 23.
    Ahlberg, J.: An Active Model for Facial Feature Tracking. EURASIP Journal on Applied Signal Processing 2002, 566–571 (2002)CrossRefGoogle Scholar
  24. 24.
    Cootes, T.F., Edwards, G.J., Taylor, C.J.: Active Appearance Models. In: Burkhardt, H., Neumann, B. (eds.) ECCV 1998. LNCS, vol. 1407, pp. 484–498. Springer, Heidelberg (1998)CrossRefGoogle Scholar
  25. 25.
    Bar-Shalom, Y., Kirubarajan, T., Li, X.R.: Estimation with Applications to Tracking and Navigation. John Wiley & Sons, Inc., New York (2002)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Utsav Prabhu
    • 1
  • Keshav Seshadri
    • 1
  • Marios Savvides
    • 1
  1. 1.Department of Electrical and Computer EngineeringCarnegie Mellon UniversityPittsburghUSA

Personalised recommendations