Skip to main content
SpringerLink
Log in

Monte Carlo Based Importance Estimation of Localized Feature Descriptors for the Recognition of Facial Expressions

  • Conference paper
  • First Online:
Multimodal Pattern Recognition of Social Signals in Human-Computer-Interaction (MPRSS 2014)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 8869))

  • 777 Accesses

Abstract

The automated and exact identification of facial expressions in human computer interaction scenarios is a challenging but necessary task to recognize human emotions by a machine learning system. The human face consists of regions whose elements contribute to single expressions in a different manner. This work aims to shed light onto the importance of specific facial regions to provide information which can be used to discriminate between different facial expressions from a statistical pattern recognition perspective. A sampling based classification approach is used to reveal informative locations in the face. The results are expression-sensitive importance maps that indicate regions of high discriminative power which can be used for various applications.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 34.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 44.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Bosch, A., Zisserman, A., Munoz, X.: Representing shape with a spatial pyramid kernel. In: Proceedings of the 6th ACM International Conference on Image and Video retrieval, CIVR 2007, pp. 401–408. ACM, New York (2007)

    Google Scholar 

  2. Ekman, P., Friesen, W.V.: Facial Action Coding System (FACS): A technique for the measurement of facial action. Consulting, Palo Alto (1978)

    Google Scholar 

  3. Ekman, P., Sorenson, E.R., Friesen, W.V.: Pan-cultural elements in facial displays of emotion. Science 164(3875), 86–88 (1969)

    Article  Google Scholar 

  4. Glodek, M., Schels, M., Schwenker, F., Palm, G.: Combination of sequential class distributions from multiple channels using Markov fusion networks. J. Multimodal User Interfaces 8, 257–272 (2014)

    Article  Google Scholar 

  5. Guoying, Z., Pietikäinen, M.: Dynamic texture recognition using local binary patterns with an application to facial expressions. IEEE Trans. Pattern Anal. Mach. Intell. 29(6), 915–928 (2007)

    Article  Google Scholar 

  6. Kächele, M., Glodek, M., Zharkov, D., Meudt, S., Schwenker, F.: Fusion of audio-visual features using hierarchical classifier systems for the recognition of affective states and the state of depression. In: Proceedings of the International Conference on Pattern Recognition Applications and Methods (ICPRAM), pp. 671–678. SciTePress (2014)

    Google Scholar 

  7. Kächele, M., Schels, M., Schwenker, F.: Inferring depression and affect from application dependent meta knowledge. In: Proceedings of AVEC, AVEC 2014, pp. 41–48. ACM, New York (2014)

    Google Scholar 

  8. Kächele, M., Schwenker, F.: Cascaded fusion of dynamic, spatial, and textural feature sets for person-independent facial emotion recognition. In: Proceedings of the International Conference on Pattern Recognition (ICPR), pp. 4660–4665 (2014)

    Google Scholar 

  9. Kächele, M., Zharkov, D., Meudt, S., Schwenker, F.: Prosodic, spectral and voice quality feature selection using a long-term stopping criterion for audio-based emotion recognition. In: Proceedings of the International Conference on Pattern Recognition (ICPR), pp. 803–808 (2014)

    Google Scholar 

  10. Kanade, T., Cohn, J., Tian, Y.: Comprehensive database for facial expression analysis. Autom. Face Gesture Recogn. 2000, 46–53 (2000)

    Article  Google Scholar 

  11. Kim, J., André, E.: Emotion recognition based on physiological changes in music listening. IEEE Trans. Pattern Anal. Mach. Intell. 30(12), 2067–2083 (2008)

    Article  Google Scholar 

  12. Liu, M., Li, S., Shan, S., Chen, X.: Au-aware deep networks for facial expression recognition. In: 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG), pp. 1–6, April 2013

    Google Scholar 

  13. Meng, H., Romera-Paredes, B., Bianchi-Berthouze, N.: Emotion recognition by two view SVM-2K classifier on dynamic facial expression features. In: Proceedings of Automatic Face Gesture Recognition and Workshops (FG 2011), pp. 854–859 (2011)

    Google Scholar 

  14. Meudt, S., Zharkov, D., Kächele, M., Schwenker, F.: Multi classifier systems and forward backward feature selection algorithms to classify emotional coloured speech. In: Proceedings of the International Conference on Multimodal Interaction, ICMI 2013, pp. 551–556. ACM, New York (2013)

    Google Scholar 

  15. Ojala, T., Pietikainen, M., Maenpaa, T.: Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans. Pattern Anal. Mach. Intell. 24(7), 971–987 (2002)

    Article  Google Scholar 

  16. Palm, G., Glodek, M.: Towards emotion recognition in human computer interaction. In: Apolloni, B., Bassis, S., Esposito, A., Morabito, F.C. (eds.) Neural Nets and Surroundings. SIST, vol. 19, pp. 323–336. Springer, Heidelberg (2013)

    Chapter  Google Scholar 

  17. Russell, J.A., Mehrabian, A.: Evidence for a three-factor theory of emotions. J. Res. Pers. 11(3), 273–294 (1977)

    Article  Google Scholar 

  18. Saragih, J.M., Lucey, S., Cohn, J.F.: Deformable model fitting by regularized landmark mean-shift. Int. J. Comput. Vis. 91(2), 200–215 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  19. Schels, M., Glodek, M., Schwenker, F., Palm, G.: Revisiting AVEC 2011 – an information fusion architecture. In: Apolloni, B., Bassis, S., Esposito, A., Morabito, F.C. (eds.) Neural Nets and Surroundings. SIST, vol. 19, pp. 385–393. Springer, Heidelberg (2013)

    Chapter  Google Scholar 

  20. Shen, L.L., Bai, L., Bardsley, D., Wang, Y.: Gabor feature selection for face recognition using improved adaboost learning. In: Li, S.Z., Sun, Z., Tan, T., Pankanti, S., Chollet, G., Zhang, D. (eds.) IWBRS 2005. LNCS, vol. 3781, pp. 39–49. Springer, Heidelberg (2005)

    Chapter  Google Scholar 

  21. Valstar, M., Pantic, M.: Fully automatic facial action unit detection and temporal analysis. In: Conference on Computer Vision and Pattern Recognition Workshop, CVPRW 2006, pp. 149–149, June 2006

    Google Scholar 

  22. Valstar, M.F., Pantic, M.: Biologically vs. logic inspired encoding of facial actions and emotions in video. In: Proceedings of ICME, pp. 325–328. IEEE (2006)

    Google Scholar 

  23. Vapnik, V.N.: Statistical Learning Theory, vol. 2. Wiley, New York (1998)

    MATH  Google Scholar 

  24. Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: Proceedings of Computer Vision and Pattern Recognition (CVPR), vol. 1, pp. I-511–I-518 (2001)

    Google Scholar 

  25. Werner, P., Al-Hamadi, A., Niese, R., Walter, S., Gruss, S., Traue, H.C.: Automatic pain recognition from video and biomedical signals. In: International Conference on Pattern Recognition, pp. 4582–4587 (2014)

    Google Scholar 

  26. Zeng, Z., Pantic, M., Roisman, G., Huang, T.: A survey of affect recognition methods: audio, visual, and spontaneous expressions. IEEE Trans. Pattern Anal. Mach. Intell. 31(1), 39–58 (2009)

    Article  Google Scholar 

  27. Zhong, L., Liu, Q., Yang, P., Liu, B., Huang, J., Metaxas, D.: Learning active facial patches for expression analysis. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2562–2569, June 2012

    Google Scholar 

Download references

Acknowledgements

This paper is based on work done within the Transregional Collaborative Research Centre SFB/TRR 62 Companion-Technology for Cognitive Technical Systems funded by the German Research Foundation (DFG). Markus Kächele is supported by a scholarship of the Landesgraduiertenförderung Baden-Württemberg at Ulm University.

Author information

Authors and Affiliations

  1. Institute of Neural Information Processing, Ulm University, 89069, Ulm, Germany

    Markus Kächele, Günther Palm & Friedhelm Schwenker

Authors
  1. Markus Kächele
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Günther Palm
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Friedhelm Schwenker
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Markus Kächele .

Editor information

Editors and Affiliations

  1. University of Ulm, Universität Ulm, Ulm, Germany

    Friedhelm Schwenker

  2. University of Southern California, Playa Vista, California, USA

    Stefan Scherer

  3. University of Southern California, Playa Vista, California, USA

    Louis-Philippe Morency

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this paper

Cite this paper

Kächele, M., Palm, G., Schwenker, F. (2015). Monte Carlo Based Importance Estimation of Localized Feature Descriptors for the Recognition of Facial Expressions. In: Schwenker, F., Scherer, S., Morency, LP. (eds) Multimodal Pattern Recognition of Social Signals in Human-Computer-Interaction. MPRSS 2014. Lecture Notes in Computer Science(), vol 8869. Springer, Cham. https://doi.org/10.1007/978-3-319-14899-1_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-14899-1_4

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-14898-4

  • Online ISBN: 978-3-319-14899-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation