Advertisement

Multi-view Facial Expression Recognition Analysis with Generic Sparse Coding Feature

  • Usman Tariq
  • Jianchao Yang
  • Thomas S. Huang
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7585)

Abstract

Expression recognition from non-frontal faces is a challenging research area with growing interest. This paper works with a generic sparse coding feature, inspired from object recognition, for multi-view facial expression recognition. Our extensive experiments on face images with seven pan angles and five tilt angles, rendered from the BU-3DFE database, achieve state-of-the-art results. We achieve a recognition rate of 69.1% on all images with four expression intensity levels, and a recognition performance of 76.1% on images with the strongest expression intensity. We then also present detailed analysis of the variations in expression recognition performance for various pose changes.

Keywords

Tilt Angle Recognition Rate Emotion Recognition Recognition Performance Sparse Code 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References

  1. 1.
    Zeng, Z., Pantic, M., Roisman, G.I., Huang, T.S.: A survey of affect recognition methods: Audio, visual, and spontaneous expressions. IEEE Transactions on Pattern Analysis and Machine Intelligence 31, 39–58 (2009)CrossRefGoogle Scholar
  2. 2.
    Zheng, W., Tang, H., Huang, T.S.: Emotion recognition from non-frontal facial images. In: Konar, A., Chakraborty, A. (eds.) Advances in Emotion Recognition. Wiley (in press, 2012)Google Scholar
  3. 3.
    Bruce, V., Valentine, T., Baddeley, A.: The basis of the 3/4 view advantage in face recognition. Applied Cognitive Psychology 1, 109–120 (1987)CrossRefGoogle Scholar
  4. 4.
    Liu, X., Chen, T., Rittscher, J.: Optimal pose for face recognition. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 1439–1446 (2006)Google Scholar
  5. 5.
    Pantic, M., Bartlett, M.S.: Machine analysis of facial expressions. In: Delac, K., Grgic, M. (eds.) Face Recognition, pp. 377–416. I-Tech Education and Publishing (2007)Google Scholar
  6. 6.
    Lucey, S., Ashraf, A.B., Cohen, J.F.: Investigating spontaneous facial action recognition through aam representations of the face. In: Delac, K., Grgic, M. (eds.) Face Recognition, pp. 275–286. I-Tech Education and Publishing (2007)Google Scholar
  7. 7.
    Chang, Y., Hu, C., Feris, R., Turk, M.: Manifold based analysis of facial expression. Image and Vision Computing 24, 605–614 (2006)CrossRefGoogle Scholar
  8. 8.
    Valstar, M.F., Gunes, H., Pantic, M.: How to distinguish posed from spontaneous smiles using geometric features. In: Proceedings of the 9th International Conference on Multimodal Interfaces, ICMI 2007, pp. 38–45 (2007)Google Scholar
  9. 9.
    Anderson, K., McOwan, P.W.: A real-time automated system for the recognition of human facial expressions. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics 36, 96–105 (2006)CrossRefGoogle Scholar
  10. 10.
    Valstar, M., Pantic, M., Patras, I.: Motion history for facial action detection in video. In: Conference Proceedings - IEEE International Conference on Systems, Man and Cybernetics, vol. 1, pp. 635–640 (2004)Google Scholar
  11. 11.
    Dhall, A., Asthana, A., Goecke, R., Gedeon, T.: Emotion recognition using phog and lpq features. In: 2011 IEEE International Conference on Automatic Face and Gesture Recognition and Workshops, FG 2011, pp. 878–883 (2011)Google Scholar
  12. 12.
    Zeng, Z., Tu, J., Pianfetti Jr., B.M., Huang, T.S.: Audio-visual affective expression recognition through multistream fused hmm. IEEE Transactions on Multimedia 10, 570–577 (2008)CrossRefGoogle Scholar
  13. 13.
    Tian, Y., Kanade, T., Cohn, J.F.: Recognizing action units for facial expression analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence 23, 97–115 (2001)CrossRefGoogle Scholar
  14. 14.
    Fasel, B., Luettin, J.: Automatic facial expression analysis: A survey. Pattern Recognition 36, 259–275 (2003)zbMATHCrossRefGoogle Scholar
  15. 15.
    Hu, Y., Zeng, Z., Yin, L., Wei, X., Tu, J., Huang, T.S.: A study of non-frontal-view facial expressions recognition. In: Proceedings - International Conference on Pattern Recognition (2008)Google Scholar
  16. 16.
    Rudovic, O., Patras, I., Pantic, M.: Regression-based multi-view facial expression recognition. In: Proceedings - International Conference on Pattern Recognition, pp. 4121–4124 (2010)Google Scholar
  17. 17.
    Rudovic, O., Patras, I., Pantic, M.: Coupled Gaussian Process Regression for Pose-Invariant Facial Expression Recognition. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part II. LNCS, vol. 6312, pp. 350–363. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  18. 18.
    Hu, Y., Zeng, Z., Yin, L., Wei, X., Zhou, X., Huang, T.S.: Multi-view facial expression recognition. In: 2008 8th IEEE International Conference on Automatic Face and Gesture Recognition, FG 2008 (2008)Google Scholar
  19. 19.
    Zheng, W., Tang, H., Lin, Z., Huang, T.S.: A novel approach to expression recognition from non-frontal face images. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1901–1908 (2009)Google Scholar
  20. 20.
    Moore, S., Bowden, R.: The effects of pose on facial expression recognition. In: British Machine Vision Conference (2009)Google Scholar
  21. 21.
    Moore, S., Bowden, R.: Local binary patterns for multi-view facial expression recognition. Computer Vision and Image Understanding 115, 541–558 (2011)CrossRefGoogle Scholar
  22. 22.
    Zheng, W., Tang, H., Lin, Z., Huang, T.S.: Emotion Recognition from Arbitrary View Facial Images. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part VI. LNCS, vol. 6316, pp. 490–503. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  23. 23.
    Tang, H., Hasegawa-Johnson, M., Huang, T.: Non-frontal view facial expression recognition based on ergodic hidden markov model supervectors. In: 2010 IEEE International Conference on Multimedia and Expo., ICME 2010, pp. 1202–1207 (2010)Google Scholar
  24. 24.
    Yin, L., Wei, X., Sun, Y., Wang, J., Rosato, M.J.: A 3D facial expression database for facial behavior research. In: FGR 2006: Proceedings of the 7th International Conference on Automatic Face and Gesture Recognition, vol. 2006, pp. 211–216 (2006)Google Scholar
  25. 25.
    Lazebnik, S., Schmid, C., Ponce, J.: Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 2169–2178 (2006)Google Scholar
  26. 26.
    Yang, J., Yu, K., Gong, Y., Huang, T.: Linear spatial pyramid matching using sparse coding for image classification. In: 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, CVPR Workshops 2009, pp. 1794–1801 (2009)Google Scholar
  27. 27.
    Wang, J., Yang, J., Yu, K., Gong, Y., Huang, T.: Locality-constrained linear coding for image classification. In: IEEE Conference on Computer Vision and Pattern Recognition (2009)Google Scholar
  28. 28.
    Boureau, Y.L., Bach, F., LeCun, Y., Ponce, J.: Learning mid-level features for recognition. In: IEEE Conference on Computer Vision and Pattern Recognition (2010)Google Scholar
  29. 29.
    Fei-Fei, L., Fergus, R., Perona, P.: Learning generative visual models from few training examples: An incremental bayesian approach tested on 101 object categories. Computer Vision and Image Understanding 106, 59–70 (2007)CrossRefGoogle Scholar
  30. 30.
    Griffin, G., Holub, A., Perona, P.: Caltech-256 object category dataset. Technical Report 7694, California Institute of Technology (2007)Google Scholar
  31. 31.
    Lee, H., Battle, A., Raina, R., Ng, A.Y.: Efficient sparse coding algorithms. In: NIPS, pp. 801–808 (2007)Google Scholar
  32. 32.
    Serre, T., Wolf, L., Poggio, T.: Object recognition with features inspired by visual cortex. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 994–1000 (2005)Google Scholar
  33. 33.
    Fan, R.E., Chang, K.W., Hsieh, C.J., Wang, X.R., Lin, C.J.: LIBLINEAR: A library for large linear classification. Journal of Machine Learning Research 9, 1871–1874 (2008)zbMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Usman Tariq
    • 1
    • 2
  • Jianchao Yang
    • 1
    • 2
    • 3
  • Thomas S. Huang
    • 1
    • 2
  1. 1.Department of Electrical and Computer Engineering, Coordinated Science LaboratoryUniversity of Illinois at Urbana-ChampaignUrbanaUSA
  2. 2.Beckman Institute for Advanced Science and TechnologyUniversity of Illinois at Urbana-ChampaignUrbanaUSA
  3. 3.Adobe Systems IncorporatedSan JoseUSA

Personalised recommendations