Abstract
Facial emotion expressions are an important gateway for studying human emotions. For many decades, this research was limited to human ratings of arousal and valence of emotional expressions. Such ratings are very time-consuming and have limited objectivity due to rater biases. By exploiting improvements in machine learning, the demand for a swifter and more objective method to assess facial emotional expressions was met by a plethora of software. These novel approaches are based on theories of human perception and emotion and their algorithms are often trained with massive and almost-generalizable data bases. However, they still face limitations such as 2D recognition and cultural biases. Nevertheless, the accuracy of computerized emotion recognition software has surpassed human raters in many cases. Consequently, such software has become instrumental in psychological research and has delivered remarkable findings, e.g. on human emotional abilities and dynamic expressions. Furthermore, recent developments for mobile devices have introduced such software into daily life, allowing for the immediate and ambulatory assessment of facial emotion expression. These trends provide intriguing new opportunities for studying human emotions, such as photograph-based experience sampling, incidental or implicit data recording in interventions, and many more.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Affectiva (2018) Affectiva Affdex. https://www.affectiva.com/
Baltrusaitis T, Robinson P, Morency L-P (2016) OpenFace: an open source facial behavior analysis toolkit. https://doi.org/10.1109/wacv.2016.7477553
Bruce V, Young A (1986) Understanding face recognition. Br J Psychol 77:305–327. https://doi.org/10.1111/j.2044-8295.1986.tb02199.x
Calvo MG, Fernández-MartÃn A, Recio G, Lundqvist D (2018) Human observers and automated assessment of dynamic emotional facial expressions: KDEF-dyn database validation. Front Psych 9:2052
Cannon WB (1927) The James-Lange theory of emotions: a critical examination and an alternative theory. Am J Psychol 39:106–124. https://doi.org/10.2307/1415404
Chen C, Crivelli C, Garrod OGB et al (2018) Distinct facial expressions represent pain and pleasure across cultures. Proc Natl Acad Sci U S A 115:E10013–E10021. https://doi.org/10.1073/pnas.1807862115
Corneanu C, Simón MO, Cohn JF, Guerrero SE (2016) Survey on RGB, 3D, thermal, and multimodal approaches for facial expression recognition: history, trends, and affect-related applications. IEEE Trans Pattern Anal Mach Intell 38(8):1548–1568. https://doi.org/10.1109/TPAMI.2016.2515606
Darwin C (1987) Expression of the emotions in man and animal. Appleton and Company, New York
De la Torre F, Chu WS, Xiong X et al (2015) IntraFace. Presented at 11th IEEE international conference and workshops on automatic face and gesture recognition (FG), Ljubljana, Slovenia, 4–8 May 2015
Del LÃbano M, Calvo MG, Fernández-MartÃn A, Recio G (2018) Discrimination between smiling faces: human observers vs. automated face analysis. Acta Psychol 187:19–29. https://doi.org/10.1016/j.actpsy.2018.04.019
Ekman P (1992) An argument for basic emotions. Cogn Emot 6:169–200. https://doi.org/10.1080/02699939208411068
Ekman P, Friesen WV (1978) Facial action coding system: a technique for the measurement of facial action. Manual for the Facial Action Coding System
Ekman P, Rosenberg E, Hager J (1998) Facial action coding system affect interpretation database (FACSAID). http://face-and-emotion.com/dataface/facsaid/description.jsp
Emotient (2016a) Facet Emotient Inc. www.emotient.com
Emotient (2016b) FACET 2.0 performance evaluation. https://imotions.com
Emotient (2016c) Emotient SDK 4.1 performance evaluation. https://imotions.com
Hildebrandt A, Olderbak S, Wilhelm O (2015) Facial emotion expression, individual differences. In: Wright JD (ed) International encyclopedia of the social & behavioral sciences, 2nd edn. Elsevier, Oxford, pp 667–675
Hinton GE, Sejnowski TJ, Poggio TA (1999) Unsupervised learning: foundations of neural computation. MIT Press, Cambridge
Jack RE, Blais C, Scheepers C et al (2009) Cultural confusions show that facial expressions are not universal. Curr Biol 19:1543–1548. https://doi.org/10.1016/j.cub.2009.07.051
Jaswal VK, Akhtar N (2018) Being vs. appearing socially uninterested: challenging assumptions about social motivation in autism. Behav Brain Sci 1–84. https://doi.org/10.1017/s0140525x18001826
Jeni LA, Cohn JF, Kanade T (2015) Dense 3D face alignment from 2D videos in real-time. Presented at 2015 11th IEEE international conference and workshops on automatic face and gesture recognition (FG), Ljubljana, Slovenia, 4–8 May 2015
Kulke L, Feyerabend D, Schacht A (2018) Comparing the Affectiva iMotions facial expression analysis software with EMG. PsyArXiv. https://doi.org/10.31234/osf.io/6c58y
Marĉelja S (1980) Mathematical description of the responses of simple cortical cells*. J Opt Soc Am, JOSA 70:1297–1300. https://doi.org/10.1364/JOSA.70.001297
Mayer JD, Salovey P (1997) What is emotional intelligence? In: Salovey P, Sluyter D (eds) Emotional development and emotional intelligence: implications for educators. Basic Books, New York, pp 3–31
Noldus Information Technology (2018) FaceReader. www.noldus.com
Olderbak S, Geiger M, Wilhelm O (2019) A call for revamping socio-emotional ability research in autism. Behav Brain Sci 42
Olderbak S, Hildebrandt A, Pinkpank T et al (2014) Psychometric challenges and proposed solutions when scoring facial emotion expression codes. Behav Res Methods 46:992–1006. https://doi.org/10.3758/s13428-013-0421-3
Reznick JS (1997) Intelligence, language, nature, and nurture in young twins. In: Sternberg RJ, Grigorenko E (eds) Intelligence, heredity, and environment. Cambridge University Press, New York
Schachter S, Singer J (1962) Cognitive, social, and physiological determinants of emotional state. Psychol Rev 69:379–399
Scherer KR (2005) What are emotions? And how can they be measured? Soc Sci Inf 44:695–729. https://doi.org/10.1177/0539018405058216
Tcherkassof A, Bollon T, Dubois M et al (2007) Facial expressions of emotions: a methodological contribution to the study of spontaneous and dynamic emotional faces. Eur J Soc Psychol 37:1325–1345. https://doi.org/10.1002/ejsp.427
Trampe D, Quoidbach J, Taquet M (2015) Emotions in everyday life. PLoS ONE 10:e0145450. https://doi.org/10.1371/journal.pone.0145450
van de Ven R (2016a) Emotion Hero. https://emotionhero.com/
van de Ven R (2016b) Emotion Hero. https://rubenvandeven.com/. Accessed 4 Oct 2018
Werner P, Al-Hamadi A, Walter S (2017) Analysis of facial expressiveness during experimentally induced heat pain. Presented at 2017 seventh IEEE international conference on affective computing and intelligent interaction workshops and demos (ACIIW). San Antonio, TX, USA, 23–26 October 2017
Wilhelm O, Hildebrandt A, Manske K et al (2014) Test battery for measuring the perception and recognition of facial expressions of emotion. Front Psychol 5. https://doi.org/10.3389/fpsyg.2014.00404
Williams AC de C (2002) Facial expression of pain: an evolutionary account. Behav Brain Sci 25. https://doi.org/10.1017/s0140525x02000080
Wilt J, Funkhouser K, Revelle W (2011) The dynamic relationships of affective synchrony to perceptions of situations. J Res Pers 45:309–321. https://doi.org/10.1016/j.jrp.2011.03.005
Zhang C, Zhang Z (2010) A survey of recent advances in face detection
Zinkernagel A, Alexandrowicz RW, Lischetzke T, Schmitt M (2018) The blenderFace method: video-based measurement of raw movement data during facial expressions of emotion using open-source software. Behav Res Methods. https://doi.org/10.3758/s13428-018-1085-9
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Geiger, M., Wilhelm, O. (2019). Computerized Facial Emotion Expression Recognition. In: Baumeister, H., Montag, C. (eds) Digital Phenotyping and Mobile Sensing. Studies in Neuroscience, Psychology and Behavioral Economics. Springer, Cham. https://doi.org/10.1007/978-3-030-31620-4_3
Download citation
DOI: https://doi.org/10.1007/978-3-030-31620-4_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-31619-8
Online ISBN: 978-3-030-31620-4
eBook Packages: Biomedical and Life SciencesBiomedical and Life Sciences (R0)