Abstract
Face-to-face communications between humans involve emotions, which often are unconsciously conveyed by facial expressions and body gestures. Intelligent human-machine interfaces, for example in cognitive robotics, need to recognize emotions. This paper addresses facial expressions and their neural correlates on the basis of a model of the visual cortex: the multi-scale line and edge coding. The recognition model links the cortical representation with Paul Ekman’s Action Units which are related to the different facial muscles. The model applies a top-down categorization with trends and magnitudes of displacements of the mouth and eyebrows based on expected displacements relative to a neutral expression. The happy vs. not-happy categorization yielded a correct recognition rate of 91%, whereas final recognition of the six expressions happy, anger, disgust, fear, sadness and surprise resulted in a rate of 78%.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Oliva, A., et al.: Top-down control of visual attention in object detection. In: IEEE Proc. Int. Conf. Image Processing, vol. 1, pp. 253–256 (2003)
Berson, D.: Strange vision: ganglion cells as circadian photoreceptors. TRENDS in Neurosciences 26(6), 314–320 (2003)
Cunha, J., Rodrigues, J., du Buf, J.M.H.: Face normalization using multi-scale cortical keypoints. In: Proc. 13rd Portuguese Conf. on Pattern Recogn., p. 2 (2007)
Ekman, P., Friesen, W.: Facial action coding system (FACS): Manual. Consulting Psychologists Press, Palo Alto (1978)
Heitger, F., et al.: Simulation of neural contour mechanisms: from simple to end-stopped cells. Vision Res. 32(5), 963–981 (1992)
Fleet, D.J., Jepson, A.D., Jenkin, M.R.M.: Phase-based disparity measurement. CVGIP: Image Understanding 53(2), 198–210 (1991)
Gama, S.: Facial emoticons: Reprodução de informação associada a expressões faciais por via do seu reconhecimento. Master Thesis, IST Lisbon, 100 p. (2009)
Grigorescu, C., Petkov, N., Westenberg, M.A.: Contour detection based on nonclassical receptive field inhibition. IEEE Tr. IP 12(7), 729–739 (2003)
Hubel, D.H.: Eye, brain and vision. Scientific American Library (1995)
Kamachi, M., Lyons, M., Gyoba, J.: Facial expression database: Japanese female facial expression (JAFFE) database (February 2009), http://kasrl.org/jaffe.html
Massimo, T., Manuele, B., Enrico, G.: Dynamic face recognition: From human to machine vision. Image Vision Comput. 27(3), 222–232 (2009)
Matsumoto, D., Ekman, P.: Facial expression analysis. Scholarpedia 3(5), 4237 (2008)
Moshe, B.: A cortical mechanism for triggering top-down facilitation in visual object recognition. J. Cognitive Neuroscience 15(4), 600–609 (2003)
Bartlett, M.S., et al.: Fully automatic facial action recognition in spontaneous behavior. In: IEEE Proc. 7th Int. Conf. on Automatic Face and Gesture Recognition, pp. 223–230 (2006)
Neth, D., Martinez, A.M.: Emotion perception in emotionless face images suggests a norm-based representation. Journal of Vision 9(1-5), 1–11 (2006)
Pantic, M., Rothkrantz, L.J.M.: Facial action recognition for facial expression analysis from static face images. IEEE Tr. on Systems, Man, and Cybernetics 34(3), 1449–1461 (2004)
Rodrigues, J., du Buf, J.M.H.: Visual cortex frontend: integrating lines, edges, keypoints and disparity. In: Campilho, A.C., Kamel, M.S. (eds.) ICIAR 2004. LNCS, vol. 3211, pp. 664–671. Springer, Heidelberg (2004)
Rodrigues, J., du Buf, J.M.H.: Face recognition by cortical multi-scale line and edge representations. In: Campilho, A., Kamel, M.S. (eds.) ICIAR 2006. LNCS, vol. 4142, pp. 329–340. Springer, Heidelberg (2006)
Rodrigues, J., du Buf, J.M.H.: A cortical framework for invariant object categorization and recognition. Cognitive Processing 10(3), 243–261 (2009)
Rodrigues, J., du Buf, J.M.H.: Multi-scale lines and edges in v1 and beyond: brightness, object categorization and recognition, and consciousness. BioSystems 95, 206–226 (2009)
Feitosa, R.Q., et al.: Facial expression classification using RBF and back-propagation neural networks. In: Proc. 6th Int. Conf. on Information Systems Analysis and Synthesis, pp. 73–77 (2000)
Kumano, S., et al.: Pose-invariant facial expression recognition using variable-intensity templates. Int. J. Comput. Vision 83(2), 178–194 (2009)
Zhao, W., et al.: Face recognition: A literature survey. ACM Comput. Surv. 35(4), 399–458 (2003)
Zhang, Y., Ji, Q.: Facial expression understanding in image sequences using dynamic and active visual information fusion. In: IEEE Proc. 9th Int. Conf. on Computer Vision, 8 p. (2003)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
de Sousa, R.J.R., Rodrigues, J.M.F., du Buf, J.M.H. (2010). Recognition of Facial Expressions by Cortical Multi-scale Line and Edge Coding. In: Campilho, A., Kamel, M. (eds) Image Analysis and Recognition. ICIAR 2010. Lecture Notes in Computer Science, vol 6111. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-13772-3_42
Download citation
DOI: https://doi.org/10.1007/978-3-642-13772-3_42
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-13771-6
Online ISBN: 978-3-642-13772-3
eBook Packages: Computer ScienceComputer Science (R0)