Abstract
In this paper we present a user independent real-time capable automatic method for recognition of facial expressions related to basic emotions from stereo image sequences. The method automatically detects faces in unconstraint pose based on depth and color information. In order to overcome difficulties caused by increasing change in pose, lighting transitions, or complicated background, we introduce a face normalization algorithm based on an Iterative Closest Point algorithm. In normalized face images we defined a set of physiologically motivated face regions related to a subset of facial muscles which are apt to automatically detect the six well-known basis emotions. Visual facial expression analysis takes place by an optical flow based feature extraction and a nearest neighbor classification, which uses a distance measure, i.e. the current flow vector pattern is matched against empirically determined ground truth data.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Li, S.Z., Jain, A.K.: Handbook of Face Recognition (2005), ISBN: 0-387-40595-X
Fasel, B., Luettin, J.: Automatic facial expression analysis: a survey. Pattern Recog. 36, 259–275 (2003)
Pantic, M., Rothkrantz, L.J.M.: Automatic analysis of facial expressions: the state of the art. IEEE Trans. Pattern Anal. Mach. Int. 22(12), 1424–1445 (2000)
Essa, I.A., Pentland, A.P.: Coding, analysis, interpretation, and recognition of facial expressions. IEEE Trans. Pattern Anal.Mach. Intell. 19(7), 757–763 (1997)
Rosenblum, M., Yacoob, Y., Davis, L.S.: Human expression recognition from motion using a radial basis function network architecture. IEEE Trans. Neural Netw. 7(5), 1121–1138 (1996)
Cohen, I., Sebe, N., Garg, A., Chen, L., Huang, T.: Facial expression recognition from video sequences: Temporal and static modeling. CVIU 91(1-2), 160–187 (2003)
Black, M.J., Yacoob, Y.: Tracking and recognizing rigid and nonrigid facial motions using local parametric models of image motion. In: Proceedings of the International Conference on Computer Vision, pp. 374–381 (1995)
Oliver, N., Pentland, A., Berard, F., LAFTER,: a real-time face lips tracker with facial expression recognition. Pattern Recog. 33, 1369–1382 (2000)
Lucas, B., Kanade, T.: An Iterative Image Registration Technique with an Application to Stereo Vision. In: Proc. of 7th International Joint Conference on Artificial Intelligence (IJCAI), pp. 674–679 (1981)
Ekman, P.: Strong evidence for universals in facial expressions: a reply to Russell’s mistaken critique. Psychol. Bull. 115(2), 268–287 (1994)
Rusinkiewicz, S., Levoy, M.: Efficient variants of the ICP algorithm. In: Proc. of the 3rd Int. Conf. on 3D Digital Imaging & Modeling, pp. 145–152 (2001)
Niese, R., Al-Hamadi, A., Michaelis, B.: A Stereo and Color-based Method for Face Pose Estimation and Facial Feature Extraction. In: ICPR, pp. 299–302 (2006)
Al-Hamadi, A., Panning, A., Niese, R., Michaelis, B.: A Model-based Image Analysis Method for Extraction and Tracking of Facial Features in Video Sequences. In: CSIT, Amman, pp. 502–512 (2006)
Valstar, M.F., Pantic, M.: Fully automatic facial action unit detection and temporal analysis. In: CVPR’06. Proceedings of IEEE Int’l Conf. Computer Vision and Pattern Recognition, New York, USA, June 2006, vol. 3, p. 149. IEEE Computer Society Press, Los Alamitos (2006)
Woo, M., Neider, J., Davis, T.: OpenGL Programming Guide, 2nd edn. (1997)
Tian, Y.L., Kanade, T., Cohn, J.F.: Facial Expression Analysis. In: Li, S.Z., Jain, A.K. (eds.) Handbook of Face Recognition, Springer, New York (2005)
Bartlett, M.S., Littlewort, G., Frank, M.G., Lainscsek, C., Fasel, I., Movellan, J.: Fully automatic facial action recognition in spontaneous behavior. In: Proc. Conf. Automatic Face & Gesture Recognition, pp. 223–230 (2006)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Niese, R., Al-Hamadi, A., Panning, A., Michaelis, B. (2007). Real-Time Capable Method for Facial Expression Recognition in Color and Stereo Vision. In: Gervasi, O., Gavrilova, M.L. (eds) Computational Science and Its Applications – ICCSA 2007. ICCSA 2007. Lecture Notes in Computer Science, vol 4705. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-74472-6_32
Download citation
DOI: https://doi.org/10.1007/978-3-540-74472-6_32
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-74468-9
Online ISBN: 978-3-540-74472-6
eBook Packages: Computer ScienceComputer Science (R0)