Abstract
In this paper an accurate real-time sequence-based system for representation, recognition, interpretation, and analysis of the facial action units (AUs) and expressions is presented. Our system has the following characteristics: 1) employing adaptive-network-based fuzzy inference systems (ANFIS) and temporal information, we developed a classification scheme based on neuro-fuzzy modeling of the AU intensity, which is robust to intensity variations, 2) using both geometric and appearance-based features, and applying efficient dimension reduction techniques, our system is robust to illumination changes and it can represent the subtle changes as well as temporal information involved in formation of the facial expressions, and 3) by continuous values of intensity and employing top-down hierarchical rule-based classifiers, we can develop accurate human-interpretable AU-to-expression converters. Extensive experiments on Cohn-Kanade database show the superiority of the proposed method, in comparison with support vector machines, hidden Markov models, and neural network classifiers.
Chapter PDF
Similar content being viewed by others
Keywords
References
Mehrabian, A.: Communication without words. Psychology Today 2(4), 53–56 (1968)
Patnic, M., Rothkrantz, J.: Automatic analysis of facial expressions: the state of art. IEEE Transactions on PAMIÂ 22(12) (2000)
Fasel, B., Luettin, J.: Automatic facial expression analysis: a survey. Pattern Recognition 36(1), 259–275 (2003)
Lyons, M., Akamatsu, S., Kamachi, M., Gyoba, J.: Coding facial expressions with Gabor wavelets. In: 3rd IEEE Int. Conf. on Automatic Face and Gesture Recognition, pp. 200–205 (1998)
Cohen, I., Sebe, N., Cozman, F., Cirelo, M., Huang, T.: Coding, analysis, interpretation, and recognition of facial expressions. Journal of Computer Vision and Image Understanding Special Issue on Face Recognition (2003)
Rosenblum, M., Yacoob, Y., Davis, L.: Human expression recognition from motion using a radial basis function network architecture. IEEE Transactions on Neural Network 7(5), 1121–1138 (1996)
Cohn, J., Kanade, T., Moriyama, T., Ambadar, Z., Xiao, J., Gao, J., Imamura, H.: A comparative study of alternative faces coding algorithms, Technical Report CMU-RI-TR-02-06, Robotics Institute, Carnegie Mellon University, Pittsburgh (2001)
Ekman, P., Friesen, W.: The facial action coding system: A technique for the measurment of facial movement. Consulting Psychologist Press, San Francisco (1978)
Tian, Y., Kanade, T., Cohn, F.: Recognizing action units for facial expression analysis. IEEE Transactions on PAMIÂ 23(2) (2001)
Sean, Z., Huang, T.: Small sample learning during multimedia retrieval using bias map. In: IEEE Int. Conf. on Computer Vision and Pattern Recognition, Hawaii (2001)
Lu, Y., Yu, J., Sebe, N., Tian, Q.: Two-dimensional adaptive discriminant analysis. In: IEEE International Conf. on Acoustics, Speech and Signal Processing, vol. 1, pp. 985–988 (2007)
Yang, D., Frangi, A., Yang, J.: Two-dimensional PCA: A new approach to appearance-based face representation and recognition. IEEE Transactions on PAMI 26(1), 131–137 (2004)
Kotsia, I., Pitas, I.: Facial expression recognition in image sequences using geometric deformation features and support vector machines. IEEE Transactions on Image Processing 16(1) (2007)
Wiskott, L., Fellous, K.N., Malsburg, C.: Face recognition by elastic bunch graph matching. IEEE Transactions on PAMI 19(7), 775–779 (1997)
Bouguet, J.: Pyramidal implementation of the Lucas Kanade feature tracker description of the algorithm, Technical Report, Intel Corporation, Microprocessor Research Labs (1999)
Jang, R.: ANFIS: Adaptive-network-based fuzzy inference systems. IEEE Transactions on Systems, Man and Cybernetics 23(3), 665–685 (1993)
Quinlan, R.: C4.5: Programs for machine learning. Morgan Kaufmann Publishers, San Mateo (1993)
Kanade, T., Tian, Y.: Comprehensive database for facial expression analysis. In: IEEE In. Conf. on Face and Gesture Recognition, pp. 46–53 (2000)
Bartlett, M., Braathen, B., Littlewort-Ford, G., Hershey, J., Fasel, I., Marks, T., Smith, E., Sejnowski, T.: Movellan. J.:Automatic analysis of spontaneous facial behavior: A final project report, Technical Report INC-MPLab-TR-2001.08, UCSD (2001)
Tian, Y., Kanade, T., Cohn., J.: Evaluation of gabor-wavelet-based facial action unit recognition in image sequences of increasing complexity. In: IEEE Int. Conf. on Automatic Face and Gesture Recognition (2002)
Takagi, T., Sugeno, M.: Fuzzy identification of systems and its applications to modeling and control. IEEE Transactions on systems, Man and Cybernetics 15(1) (1985)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Khademi, M., Kiapour, M.H., Manzuri-Shalmani, M.T., Kiaei, A.A. (2010). Analysis, Interpretation, and Recognition of Facial Action Units and Expressions Using Neuro-Fuzzy Modeling. In: Schwenker, F., El Gayar, N. (eds) Artificial Neural Networks in Pattern Recognition. ANNPR 2010. Lecture Notes in Computer Science(), vol 5998. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-12159-3_15
Download citation
DOI: https://doi.org/10.1007/978-3-642-12159-3_15
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-12158-6
Online ISBN: 978-3-642-12159-3
eBook Packages: Computer ScienceComputer Science (R0)