Upper Facial Action Unit Recognition

  • Cemre Zor
  • Terry Windeatt
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5558)


This paper concentrates on the comparisons of systems that are used for the recognition of expressions generated by six upper face action units (AUs) by using Facial Action Coding System (FACS). Haar wavelet, Haar-Like and Gabor wavelet coefficients are compared, using Adaboost for feature selection. The binary classification results by using Support Vector Machines (SVM) for the upper face AUs have been observed to be better than the current results in the literature, for example 96.5% for AU2 and 97.6% for AU5. In multi-class classification case, the Error Correcting Output Coding (ECOC) has been applied. Although for a large number of classes, the results are not as accurate as the binary case, ECOC has the advantage of solving all problems simultaneously; and for large numbers of training samples and small number of classes, error rates are improved.


FACS ECOC Adaboost 


  1. 1.
    Ekman, P., Friesen, W.V.: Pictures of Facial Affect. Consulting Psychologist Press, Palo Alto (1976)Google Scholar
  2. 2.
    Izard, C., Dougherty, L., Hembree, E.A.: A System for Identifying Affect Expressions by Holistic Judgements. Univ. Of Delaware (unpublished manuscript) (1983)Google Scholar
  3. 3.
    Bartlett, M.S., Hager, J., Ekman, P., Sejnowski, T.: Measuring Facial Expressions by Computer Image Analysis. J. Psychophysiology 36, 253–263 (1999)Google Scholar
  4. 4.
    Bartlett, M.S., Littlewort, G., Lainscsek, C., Fasel, I., Movellan, J.: Machine Learning Methods for Fully Automatic Recognition of Facial Expressions and Facial Actions. In: IEEE International Conference on Systems, Men and Cybernetics, Netherlands, pp. 592–597 (2004)Google Scholar
  5. 5.
    Ekman, P., Friesen, W.V.: Facial Action Coding System: A Technique for the Measurement of Facial Movement. Consulting Psychologist Press, Palo Alto (1978)Google Scholar
  6. 6.
    Mase, K.: Recognition of Facial Expression from Optical Flow. IEICE Trans.  E74(10), 3474–3483 (1991)Google Scholar
  7. 7.
    Yacoob, Y., Davis, L.S.: Recognizing Human Facial Expression from Long Image Sequences Using Optical Flow. IEEE Trans. Pattern Analysis and Machine Intelligence 18(6), 636–642 (1996)Google Scholar
  8. 8.
    Suwa, M., Sugie, N., Fujimora, K.A.: Preliminary Note on Pattern Recognition of Human Emotional Expression. In: Proc. International Joint Conf. Pattern Recognition, pp. 408–410 (1978)Google Scholar
  9. 9.
    Lanitis, A., Taylor, C., Cootes, T.: Automatic Interpretation and Coding of Face Images Using Flexible Models. IEEE Trans. Pattern Analysis and Machine Intelligence 19(7), 743–756 (1997)Google Scholar
  10. 10.
    Zhang, Z.: Feature-Based Facial Expression Recognition: Sensitivity Analysis and Experiments with a Multilayer Perceptron. Int’l. J. Pattern Recognition and Artificial Intelligence 13(6), 893–911 (1999)Google Scholar
  11. 11.
    Whitehill, J., Omlin, C.W.: Haar Features for FACS AU Recognition. In: Proceedings of the IEEE International Conference on Automatic Face and Gesture Recognition (2006)Google Scholar
  12. 12.
    Donato, G., Bartlett, M.S., Hager, J., Ekman, P., Sejnowski, T.J.: Classifying Facial Actions. IEEE Transactions on Pattern Analysis and Machine Intelligence 21(10), 974–988 (1999)Google Scholar
  13. 13.
    Viola, P., Jones, M.J.: Robust Real-Time Face Detection. International J. of Computer Vision 57(2), 137–154 (2004)Google Scholar
  14. 14.
    Freund, Y., Schapire, R.: A Decision-Theoretic Generalization of On-Line Learning and an Application to Boosting. Computer and System Sciences 55, 119–139 (1997)Google Scholar
  15. 15.
    Shen, L., Bai, L., Fairhurst, M.: Gabor Wavelets and General Discriminant Analysis for Face Identification and Verification. Image Vision Computing 25(5), 553–563 (2007)Google Scholar
  16. 16.
    Jain, A.K., Farrokhnia, F.: Unsupervised Texture Segmentation Using Gabor Filters. Pattern Recognition 24(12), 1167–1186 (1991)Google Scholar
  17. 17.
    Lee, C.J., Wang, S.D.: Fingerprint Feature Extraction Using Gabor Filters. Electronics Letters 35(4), 288–290 (1999)Google Scholar
  18. 18.
    Zhan, Y., Niu, D., Cao, P.: Facial Expression Recognition Based on Gabor Wavelet Transformation and Elastic Templates Matching. In: Third International Conference on Image and Graphics (ICIG 2004), pp. 254–257 (2004)Google Scholar
  19. 19.
    Dietterich, T.G., Bakiri, G.: Solving Multi-class Learning Problems via Error-Correcting Output Codes. J. Artificial Intelligence Research 2, 263–286 (1995)Google Scholar
  20. 20.
    Tian, Y., Kanade, T., Cohn, J.F.: Recognizing Action Units for Facial Expression Analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence 23(2), 97–115 (2001)Google Scholar
  21. 21.
    Efron, B.: Bootstrap methods: Another Look at the Jackknife. The Annals of Statistics 7(1), 1–26 (1979)Google Scholar
  22. 22.
    Asuncion, A., Newman, D.J.: UCI Machine Learning Repository. School of Information and Computer Science, University of California, Irvine (2007),
  23. 23.
    Boser, B.E., Guyon, I.M., Vapnik, V.N.: A Training Algorithm for Optimal Margin Classifiers. In: COLT 1992: Proceedings of the Fifth Annual Workshop on Computational Learning Theory, pp. 144–152. ACM Press, New York (1992)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Cemre Zor
    • 1
  • Terry Windeatt
    • 1
  1. 1.Centre for Vision, Speech and Signal Processing (CVSSP)University of SurreyGuildford, SurreyUnited Kingdom

Personalised recommendations