Abstract
An automatic system for facial expression recognition should be able to recognize on-line multiple facial expressions (i.e. “emotional segments”) without interruption. The current paper proposes a new method for the automatic segmentation of “emotional segments” and the dynamic recognition of the corresponding facial expressions in video sequences. First, a new spatial filtering method based on Log-Normal filters is introduced for the analysis of the whole face towards the automatic segmentation of the “emotional segments”. Secondly, a similar filtering-based method is applied to the automatic and precise segmentation of the transient facial features (such as nasal root wrinkles and nasolabial furrows) and the estimation of their orientation. Finally, a dynamic and progressive fusion process of the permanent and transient facial feature deformations is made inside each “emotional segment” for a temporal recognition of the corresponding facial expression. When tested for automatic detection of “emotional segment” in 96 sequences from the MMI and Hammal-Caplier facial expression databases, the proposed method achieved an accuracy of 89%. Tested on 1655 images the automatic detection of transient features achieved a mean precision of 70 % with an error of 2.5 for the estimation of the corresponding orientation. Finally compared to the original model for static facial expression classification, the introduction of transient features and the temporal information increases the precision of the classification of facial expression by 12% and compare favorably to human observers’ performances.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Hammal, Z., Couvreur, L., Caplier, A., Rombaut, M.: Facial expressions classification: A new approach based on transferable belief model. International Journal of Approximate Reasoning 46(3), 542–567 (2007)
Zeng, Z., Pantic, M., Roisman, G.I., Huang, T.S.: A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions. IEEE Transactions on Pattern Analysis and Machine Intelligence 31(1), 39–58 (2009)
Pantic, M., Patras, I.: Dynamics of Facial Expression: Recognition of Facial Actions and Their Temporal Segments from Face Profile Image Sequences. IEEE Trans. SMC- Part B 36(2), 433–449 (2006)
Valstar, M.F., Pantic, M.: Combined Support Vector Machines and Hidden Markov Models for Modeling Facial Action Temporal Dynamics. In: Proc. IEEE Workshop on Human Computer Interaction, Rio de Janeiro, Brazil, pp. 118–127 (2007)
Koelstra, S., Pantic, M.: Non-rigid registration using free-form deformations for recognition of facial actions and their temporal dynamics. In: Proc. IEEE Int’l Conf. Automatic Face and Gesture Recognition, Amsterdam, Netherlands (September 2008)
Tong, Y., Liao, W., Ji, Q.: Facial action unit recognition by exploiting their dynamics and semantic relationships. IEEE Trans. PAMI 29, 1683–1699 (2007)
Zhang, Y., Qiang, J.: Active and dynamic information fusion for facial expression understanding from image sequences. IEEE Trans. PAMI 27(5), 699–714 (2005)
Gralewski, L., Campbell, N., Voak, I.P.: Using a tensor framework for the analysis of facial dynamics. In: Proc. IEEE Int. Conf. FG, pp. 217–222 (2006)
Littlewort, G., Bartlett, M.S., Fasel, I., Susskind, J., Movellan, J.: Dynamics of facial expression extracted automatically from video. J. Image Vis. Comput. 24, 615–625 (2006)
Kaiser, M.D., Le Grand, R., Jim, W.: Tanaka, On holistic processing of facial expressions Journal of Vision 6, 685 (2006)
Hammal, Z., Eveno, N., Caplier, A., Coulon, P.-Y.: Parametric models for facial features segmentation. Signal Processing 86, 399–413 (2006)
Tian, Y., Kanade, T., Cohn, J.F.: Recognizing Action Units for Facial Expression Analysis. IEEE Trans. PAMI 23(2), 97–115 (2001)
Smets, P., Kruse, R.: The transferable belief model. Artificial Intelligence 66, 191–234 (1994)
Tian, Y.L., Kanade, T., Cohn, J.F.: Facial expression analysis. In: Li, S.Z., Jain, A.K. (eds.) Handbook of Face Recognition, pp. 247–276. Springer, NY (2005)
Fasel, I., Fortenberry, B., Movellan, J.: A generative framework for real time object detection and classification. Computer Vision and Image Understanding 98, 182–210 (2005)
Beaudot, W.: Le traitement neuronal de l’information dans la rétine des vertébrés: Un creuset d’idées pour la vision artificielle, Thèse de Doctorat INPG, Laboratoire TIRF, Grenoble, France (1994)
Massot, C., Herault, J.: Model of Frequency Analysis in the Visual Cortex and the Shape from Texture Problem. Int. Journal of Computer Vision 76(2) (2008)
Ekman, P., Friesen, W.V.: The facial action coding system (facs): A technique for the measurement of facial action. Consulting Psychologists Press, Palo Alto (1978)
Pantic, M., Valstar, M.F., Rademaker, R., Maat, L.: Web-based database for facial expression analysis. In: Proc. IEEE Int. Conf. ICME 2005, Amsterdam, The Netherlands (July 2005)
Smith, M., Cottrell, G., Gosselin, F., Schyns, P.G.: Transmitting and decoding facial expressions of emotions. Psychological Science 16, 184–189 (2005)
Denoeux, T.: Conjunctive and disjunctive combination of belief functions induced by non-distinct bodies of evidence. Artificial Intelligence 172, 234–264 (2008)
Wehrle, T., Kaiser, S., Schmidt, S., Scherer, K.R.: Studying the dynamics of emotional expression using synthesized facial muscle movements. Journal of Personality and Social Psychology 78(1), 105–119 (2000)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Hammal, Z., Massot, C. (2011). Gabor-Like Image Filtering for Transient Feature Detection and Global Energy Estimation Applied to Multi-expression Classification. In: Richard, P., Braz, J. (eds) Computer Vision, Imaging and Computer Graphics. Theory and Applications. VISIGRAPP 2010. Communications in Computer and Information Science, vol 229. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-25382-9_10
Download citation
DOI: https://doi.org/10.1007/978-3-642-25382-9_10
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-25381-2
Online ISBN: 978-3-642-25382-9
eBook Packages: Computer ScienceComputer Science (R0)