The emotion–facial expression link: evidence from human and automatic expression recognition

Abstract

While it has been taken for granted in the development of several automatic facial expression recognition tools, the question of the coherence between subjective feelings and facial expressions is still a subject of debate. On one hand, the “Basic Emotion View” conceives emotions as genetically hardwired and, therefore, being genuinely displayed through facial expressions. Consequently, emotion recognition is perceiver independent. On the other hand, the constructivist approach conceives emotions as socially constructed, the emotional meaning of a facial expression being inferred by the perceiver. Hence, emotion recognition is perceiver dependent. In order (1) to evaluate the coherence between the subjective feeling of emotions and their spontaneous facial displays, and (2) to compare the recognition of such displays by human perceivers and by an automatic facial expression classifier, 232 videos of expressers recruited to carry out an emotion elicitation task were annotated by 1383 human perceivers as well as by Affdex, an automatic classifier. Results show a weak consistency between self-reported emotional states by expressers and their facial emotional displays. They also show low accuracy both of human perceivers and of the automatic classifier to infer the subjective feeling from the spontaneous facial expressions displayed by expressers. However, the results are more in favor of a perceiver-dependent view. Based on these results, the hypothesis of genetically hardwired emotion genuinely displayed is difficult to support, whereas the idea of emotion and facial expression as being socially constructed appears to be more likely. Accordingly, automatic emotion recognition tools based on facial expressions should be questioned.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3

Data availability

The R code and relevant data for statistical computing are available on the “The emotion–facial expression link (data and code)” repository from the Center for Open Science at https://osf.io/erua5/ (Dupré & Tcherkassof, 2020). The data that support the findings of this study are available from the DynEmo database but restrictions apply to the availability of these data. DynEmo is devoted to academic research to the exclusion of any other purpose, and so the data are not publicly available. Data are however available from the authors upon reasonable request and with permission of DynEmo.

References

  1. Aust, F., & Barth, M. (2018). Papaja: Create APA manuscripts with R Markdown. https://github.com/crsh/papaja. Accessed 6 Nov 2020.

  2. Aviezer, H., Ensenberg, N., & Hassin, R. R. (2017). The inherently contextualized nature of facial emotion perception. Current Opinion in Psychology, 17, 47–54. https://doi.org/10.1016/j.copsyc.2017.06.006.

    Article  PubMed  Google Scholar 

  3. Aviezer, H., Hassin, R. R., Ryan, J., Grady, C., Susskind, J., Anderson, A., et al. (2008). Angry, disgusted, or afraid? Studies on the malleability of emotion perception. Psychological Science, 19(7), 724–732. https://doi.org/10.1111/j.1467-9280.2008.02148.x.

    Article  PubMed  Google Scholar 

  4. Baltrušaitis, T., Robinson, P., & Morency, L.-P. (2016). Openface: An open source facial behavior analysis toolkit. Winter Conference on Applications of Computer Vision. https://doi.org/10.1109/WACV.2016.7477553.

    Article  Google Scholar 

  5. Barrett, L. F. (2006). Are emotions natural kinds? Perspectives on Psychological Science, 1(1), 28–58. https://doi.org/10.1111/j.1745-6916.2006.00003.x.

    Article  PubMed  Google Scholar 

  6. Barrett, L. F., Adolphs, R., Marsella, S., Martinez, A. M., & Pollak, S. D. (2019). Emotional expressions reconsidered: Challenges to inferring emotion from human facial movements. Psychological Science in the Public Interest, 20(1), 1–68. https://doi.org/10.1177/1529100619832930.

    Article  PubMed  PubMed Central  Google Scholar 

  7. Barrett, L. F., Mesquita, B., & Gendron, M. (2011). Context in emotion perception. Current Directions in Psychological Science, 20(5), 286–290. https://doi.org/10.1177/0963721411422522.

    Article  Google Scholar 

  8. Bennett, D. S., Bendersky, M., & Lewis, M. (2002). Facial expressivity at 4 months: A context by expression analysis. Infancy, 3(1), 97–113. https://doi.org/10.1207/S15327078IN0301_5.

    Article  PubMed  PubMed Central  Google Scholar 

  9. Bonanno, G., & Keltner, D. (2004). The coherence of emotion systems: Comparing “on-line” measures of appraisal and facial expressions, and self-report. Cognition and Emotion, 18(3), 431–444. https://doi.org/10.1080/02699930341000149.

    Article  Google Scholar 

  10. Buck, R. (1985). Prime theory: An integrated view of motivation and emotion. Psychological Review, 92(3), 389–413. https://doi.org/10.1037/0033-295X.92.3.389.

    Article  Google Scholar 

  11. Camras, L. A., Malatesta, C., & Izard, C. E. (1991). The development of facial expressions in infancy. In R. S. Feldman & B. Rimé (Eds.), Studies in emotion & social interaction. Fundamentals of nonverbal behavior (pp. 31–54). New York: Cambridge University Press.

    Google Scholar 

  12. Camras, L. A., Oster, H., Bakeman, R., Meng, Z., Ujiie, T., & Campos, J. J. (2007). Do infants show distinct negative facial expressions for fear and anger? Emotional expression in 11-month-old european american, chinese, and japanese infants. Infancy, 11(2), 131–155. https://doi.org/10.1111/j.1532-7078.2007.tb00219.x.

    Article  Google Scholar 

  13. Cohn, J. F., & Schmidt, K. (2004). The timing of facial motion in posed and spontaneous smiles. International Journal of Wavelets, Multiresolution and Information Processing, 2(2), 121–132. https://doi.org/10.1142/S021969130400041X.

    Article  Google Scholar 

  14. Crivelli, C., Carrera, P., & Fernández-Dols, J.-M. (2015). Are smiles a sign of happiness? Spontaneous expressions of judo winners. Evolution and Human Behavior, 36(1), 52–58. https://doi.org/10.1016/j.evolhumbehav.2014.08.009.

    Article  Google Scholar 

  15. Crivelli, C., & Fridlund, A. J. (2018). Facial displays are tools for social influence. Trends in Cognitive Sciences, 22(5), 388–399. https://doi.org/10.1016/j.tics.2018.02.006.

    Article  PubMed  Google Scholar 

  16. Crivelli, C., Russell, J. A., Jarillo, S., & Fernández-Dols, J.-M. (2017). Recognizing spontaneous facial expressions of emotion in a small-scale society of papua new guinea. Emotion, 17(2), 337–347. https://doi.org/10.1037/emo0000236.

    Article  PubMed  Google Scholar 

  17. Darwin, C. (1872). The expression of the emotions in man and animals. London: John Murray.

    Google Scholar 

  18. de Waal, F. (2019). Mama’s last hug: Animal emotions and what they teach us about ourselves. London: Granta Books.

    Google Scholar 

  19. Den Uyl, M., & Van Kuilenburg, H. (2005). The facereader: Online facial expression recognition. Proceedings of Measuring Behavior, 30, 589–590.

    Google Scholar 

  20. Dente, P., Küster, D., Skora, L., & Krumhuber, E. (2017). Measures and metrics for automatic emotion classification via facet. In: Proceedings of the Conference on the Study of Artificial Intelligence and Simulation of Behaviour, pp 160–163.

  21. DiGirolamo, M. A., & Russell, J. A. (2017). The emotion seen in a face can be a methodological artifact: The process of elimination hypothesis. Emotion, 17(3), 538–546. https://doi.org/10.1037/emo0000247.

    Article  PubMed  Google Scholar 

  22. Duchenne, G.-B. (1876). Mécanisme de la physionomie humaine: Où, analyse électro-physiologique de l’expression des passions. Paris: J.-B. Baillière.

    Google Scholar 

  23. Dupré, D., Akpan, D., Elias, E., Adam, J.-M., Meillon, B., Bonnefond, N., et al. (2015). Oudjat: A configurable and usable annotation tool for the study of facial expressions of emotion. International Journal of Human-Computer Studies, 83, 51–61. https://doi.org/10.1016/j.ijhcs.2015.05.010.

    Article  Google Scholar 

  24. Dupré, D., Andelic, N., Morrison, G., & McKeown, G. (2018). Accuracy of three commercial automatic emotion recognition systems across different individuals and their facial expressions. Proceedings of the International Conference on Pervasive Computing and Communications. https://doi.org/10.1109/PERCOMW.2018.8480127.

    Article  Google Scholar 

  25. Dupré, D., & Tcherkassof, A. (2020). The emotion–facial expression link (data and code). OSF, https://doi.org/10.17605/OSF.IO/ERUA5.

    Article  Google Scholar 

  26. Dupré, D., Krumhuber, E. G., Küster, D., & McKeown, G. J. (2020). A performance comparison of eight commercially available automatic classifiers for facial affect recognition. PLoS ONE, 15(4), e0231968. https://doi.org/10.1371/journal.pone.0231968.

    Article  PubMed  PubMed Central  Google Scholar 

  27. Durán, J. I., Reisenzein, R., & Fernández-Dols, J.-M. (2017). Coherence between emotions and facial expressions. In J.-M. Fernández-Dols & J. A. Russell (Eds.), The science of facial expression (pp. 107–129). New York: Oxford University Press.

    Google Scholar 

  28. Ekman, P. (1992). An argument for basic emotions. Cognition and Emotion, 6(3–4), 169–200. https://doi.org/10.1080/02699939208411068.

    Article  Google Scholar 

  29. Ekman, P. (2007). The directed facial action task. In J. A. Coan & J. J. B. Allen (Eds.), Handbook of emotion elicitation and assessment (pp. 47–53). New York: Oxford University Press.

    Google Scholar 

  30. Ekman, P. (2017). Facial expressions. In J.-M. Fernández-Dols & J. A. Russell (Eds.), The science of facial expression (pp. 39–56). New York: Oxford University Press.

    Google Scholar 

  31. Ekman, P., & Cordaro, D. (2011). What is meant by calling emotions basic. Emotion Review, 3(4), 364–370. https://doi.org/10.1177/1754073911410740.

    Article  Google Scholar 

  32. Ekman, P., Freisen, W. V., & Ancoli, S. (1980). Facial signs of emotional experience. Journal of Personality and Social Psychology, 39(6), 1125–1134. https://doi.org/10.1037/h0077722.

    Article  Google Scholar 

  33. Ekman, P., Friesen, W. V., O’sullivan, M., Chan, A., Diacoyanni-Tarlatzis, I., Heider, K., et al. (1987). Universals and cultural differences in the judgments of facial expressions of emotion. Journal of Personality and Social Psychology, 53(4), 712–717. https://doi.org/10.1037/0022-3514.53.4.712.

    Article  PubMed  Google Scholar 

  34. Ekman, P., & Heider, K. G. (1988). The universality of a contempt expression: A replication. Motivation and Emotion, 12(3), 303–308. https://doi.org/10.1007/BF00993116.

    Article  Google Scholar 

  35. Elfenbein, H. A., & Ambady, N. (2002). On the universality and cultural specificity of emotion recognition: A meta-analysis. Psychological Bulletin, 128(2), 203–235. https://doi.org/10.1037/0033-2909.128.2.203.

    Article  PubMed  Google Scholar 

  36. Fernández-Dols, J. M. (2017). Natural facial expression: A view from psychological constructionism and pragmatics. In J. M. Fernández-Dols & J. A. Russell (Eds.), The science of facial expression (pp. 457–478). New York: Oxford University Press.

    Google Scholar 

  37. Fernández-Dols, J.-M., & Crivelli, C. (2013). Emotion and expression: Naturalistic studies. Emotion Review, 5(1), 24–29. https://doi.org/10.1177/1754073912457229.

    Article  Google Scholar 

  38. Fernández-Dols, J.-M., & Ruiz-Belda, M.-A. (1995). Are smiles a sign of happiness? Gold medal winners at the olympic games. Journal of Personality and Social Psychology, 69(6), 1113–1119. https://doi.org/10.1037/0022-3514.69.6.1113.

    Article  Google Scholar 

  39. Fernandez-Dols, J.-M., Sanchez, F., Carrera, P., & Ruiz-Belda, M.-A. (1997). Are spontaneous expressions and emotions linked? An experimental test of coherence. Journal of Nonverbal Behavior, 21(3), 163–177. https://doi.org/10.1023/A:1024917530100.

    Article  Google Scholar 

  40. Firke, S. (2019). Janitor: Simple tools for examining and cleaning dirty data. https://CRAN.R-project.org/package=janitor. Accessed 6 Nov 2020.

  41. Fridlund, A. J. (1994). Human facial expression: An evolutionary view. San Diego: Academic Press.

    Google Scholar 

  42. Frijda, N. H. (2012). Recognition of emotion in others. In F. Paglieri (Ed.), Consciousness in interaction. The role of the natural and social context in shaping consciousness (pp. 237–256). Amsterdam: J. Benjamins.

    Google Scholar 

  43. Frijda, N. H., & Tcherkassof, A. (1997). Facial expressions as modes of action readiness. In J. A. Russell & J.-M. Fernández-Dols (Eds.), The psychology of facial expression (pp. 78–102). New York: Cambridge University Press.

    Google Scholar 

  44. Gendron, M., & Barrett, L. F. (2017). Facing the past: A history of the face in psychological research on emotion perception. In J.-M. Fernández-Dols & J. A. Russell (Eds.), The science of facial expression (pp. 15–36). Oxford: Oxford University Press.

    Google Scholar 

  45. Gendron, M., Hoemann, K., Crittenden, A. N., Mangola, S. M., Ruark, G. A., & Barrett, L. F. (2020). emotion perception in Hadza Hunter-Gatherers.Scientific Reports, 10(1), 1-17. https://doi.org/10.1038/s41598-020-60257-2.

    Article  PubMed  PubMed Central  Google Scholar 

  46. Gendron, M., Roberson, D., van der Vyver, J. M., & Barrett, L. F. (2014). Perceptions of emotion from facial expressions are not culturally universal: Evidence from a remote culture. Emotion, 14(2), 251–262. https://doi.org/10.1037/a0036052.

    Article  PubMed  PubMed Central  Google Scholar 

  47. Hassin, R. R., Aviezer, H., & Bentin, S. (2013). Inherently ambiguous: Facial expressions of emotions, in context. Emotion Review, 5(1), 60–65. https://doi.org/10.1177/1754073912451331.

    Article  Google Scholar 

  48. Hess, U., & Fischer, A. H. (2016). Emotional mimicry in social context. New York: Cambridge University Press.

    Google Scholar 

  49. Hess, U., & Kleck, R. E. (1990). Differentiating emotion elicited and deliberate emotional facial expressions. European Journal of Social Psychology, 20(5), 369–385. https://doi.org/10.1002/ejsp.2420200502.

    Article  Google Scholar 

  50. Jakobs, E., Manstead, A. S., & Fischer, A. H. (1999). Social motives and emotional feelings as determinants of facial displays: The case of smiling. Personality and Social Psychology Bulletin, 25(4), 424–435. https://doi.org/10.1177/0146167299025004003.

    Article  Google Scholar 

  51. Jed Wing, M. K. C., Weston, S., Williams, A., Keefer, C., Engelhardt, A., Cooper, T., Mayer, Z., Kenkel, B., R Core Team, Benesty, M., Lescarbeau, R., Ziem, A., Scrucca, L., Tang, Y., Candan, C., & Hunt., T. (2019). Caret: Classification and regression training. https://CRAN.R-project.org/package=caret. Accessed 6 Nov 2020.

  52. Johnson, K. J., Waugh, C. E., & Fredrickson, B. L. (2010). Smile to see the forest: Facially expressed positive emotions broaden cognition. Cognition and Emotion, 24(2), 299–321. https://doi.org/10.1080/02699930903384667.

    Article  PubMed  Google Scholar 

  53. Kamachi, M., Bruce, V., Mukaida, S., Gyoba, J., Yoshikawa, S., & Akamatsu, S. (2013). Dynamic properties influence the perception of facial expressions. Perception, 42(11), 1266–1278. https://doi.org/10.1068/p3131n.

    Article  PubMed  Google Scholar 

  54. Kanade, T., Cohn, J. F., & Tian, Y. (2000). Comprehensive database for facial expression analysis. In: Proceedings Fourth International Conference on Automatic Face and Gesture Recognition, pp 46–53.

  55. Kappas, A. (2003). What facial activity can and cannot tell us about emotions. In M. Katsikitis (Ed.), The human face (pp. 215–234). Boston: Springer.

  56. Keltner, D., Sauter, D., Tracy, J., & Cowen, A. (2019). Emotional expression: Advances in basic emotion theory. Journal of Nonverbal Behavior, 43, 133–160. https://doi.org/10.1007/s10919-019-00293-3.

    Article  PubMed  PubMed Central  Google Scholar 

  57. Krause, R., Steimer-Krause, E., Merten, J., & Ullrich, B. (1998). Dyadic interaction regulation, emotion, and psychopathology. In W. F. J. Flack & J. D. Laird (Eds.), Emotions and psychopathology: Theory and research (pp. 70–80). New York: Oxford University Press.

    Google Scholar 

  58. Kraut, R. E., & Johnston, R. E. (1979). Social and emotional messages of smiling: An ethological approach. Journal of Personality and Social Psychology, 37(9), 1539–1553. https://doi.org/10.1037/0022-3514.37.9.1539.

    Article  Google Scholar 

  59. Krumhuber, E. G., Kappas, A., & Manstead, A. S. R. (2013). Effects of dynamic aspects of facial expressions: A review. Emotion Review, 5(1), 41–46. https://doi.org/10.1177/1754073912451349.

    Article  Google Scholar 

  60. Krumhuber, E. G., Küster, D., Namba, S., & Skora, L. (2020). Human and machine validation of 14 databases of dynamic facial expressions. Behavior Research Methods. https://doi.org/10.3758/s13428-020-01443-y.

    Article  PubMed  Google Scholar 

  61. Lee, V., & Wagner, H. (2002). The effect of social presence on the facial and verbal expression of emotion and the interrelationships among emotion components. Journal of Nonverbal Behavior, 26(1), 3–25. https://doi.org/10.1023/A:1014479919684.

    Article  Google Scholar 

  62. Lindquist, K. A., Barrett, L. F., Bliss-Moreau, E., & Russell, J. A. (2006). Language and the perception of emotion. Emotion, 6(1), 125–138. https://doi.org/10.1037/1528-3542.6.1.125.

    Article  PubMed  Google Scholar 

  63. Lindquist, K. A., & Gendron, M. (2013). What’s in a word? Language constructs emotion perception. Emotion Review, 5(1), 66–71. https://doi.org/10.1177/1754073912451351.

    Article  Google Scholar 

  64. Lottridge, D., & Chignell, M. (2009). Emotional bandwidth: Information theory analysis of affective response ratings using a continuous slider. In: IFIP Conference on Human-Computer Interaction, 111–114.

  65. McDuff, D. (2016). Discovering facial expressions for states of amused, persuaded, informed, sentimental and inspired. Proceedings of the International Conference on Multimodal Interaction. https://doi.org/10.1145/2993148.2993192.

    Article  Google Scholar 

  66. McDuff, D., Mahmoud, A., Mavadati, M., Amr, M., Turcot, J., & el Kaliouby, R. (2016). AFFDEX sdk: A cross-platform real-time multi-face expression recognition toolkit. Proceedings of the Conference on Human Factors in Computing Systems. https://doi.org/10.1145/2851581.2890247.

    Article  Google Scholar 

  67. McKeown, G. J. (2013). The analogical peacock hypothesis: The sexual selection of mind-reading and relational cognition in human communication. Review of General Psychology, 17(3), 267–287. https://doi.org/10.1037/a0032631.

    Article  Google Scholar 

  68. Mehu, M., Grammer, K., & Dunbar, R. I. (2007). Smiles when sharing. Evolution and Human Behavior, 28(6), 415–422. https://doi.org/10.1016/j.evolhumbehav.2007.05.010.

    Article  Google Scholar 

  69. Meillon, B., Tcherkassof, A., Mandran, N., Adam, J.-M., Dubois, M., Dupré, D., Benoit, A.-M., Guérin-Dugué, A., & Caplier, A. (2010). DynEmo: A corpus of dynamic and spontaneous emotional facial expressions. In: Proceedings of International Workshop Series on Multimodal Corpora, Tools and Resources. Multimodal Corpora: Advances in Capturing, Coding and Analyzing Multimodality, pp 31–36.

  70. Müller, K. (2017). Here: A simpler way to find your files. https://CRAN.R-project.org/package=here. Accessed 6 Nov 2020.

  71. Niedenthal, P. M., Wood, A., Rychlowska, M., & Korb, S. (2017). Embodied simulation in decoding facial expression. In J.-M. Fernández-Dols & J. A. Russell (Eds.), The science of facial expression (pp. 397–414). New York: Oxford University Press.

    Google Scholar 

  72. Nielsen, L., & Kaszniak, A. W. (2007). Conceptual, theoretical, and methodological issues in inferring subjective emotion experience. In J. A. Coan & J. J. B. Allen (Eds.), Handbook of emotion elicitation and assessment (pp. 361–375). New York: Oxford University Press.

    Google Scholar 

  73. Pacherie, E. (2003). Modes de structuration des contenus perceptifs visuels. In J. Bouveresse & J.-J. Rosat (Eds.), Philosophies de la perception. Phénoménologie, grammaire et sciences cognitives (pp. 263–289). Paris: Odile Jacob.

    Google Scholar 

  74. R Core Team. (2019). R: A language and environment for statistical computing. R Foundation for Statistical Computing. https://www.R-project.org/. Accessed 6 Nov 2020.

  75. Reisenzein, R. (2000). Exploring the strength of association between the components of emotion syndromes: The case of surprise. Cognition and Emotion, 14(1), 1–38. https://doi.org/10.1080/026999300378978.

    Article  Google Scholar 

  76. Reisenzein, R., Bördgen, S., Holtbernd, T., & Matz, D. (2006). Evidence for strong dissociation between emotion and facial displays: The case of surprise. Journal of Personality and Social Psychology, 91(2), 295–315. https://doi.org/10.1037/0022-3514.91.2.295.

    Article  PubMed  Google Scholar 

  77. Reisenzein, R., Studtmann, M., & Horstmann, G. (2013). Coherence between emotion and facial expression: Evidence from laboratory experiments. Emotion Review, 5(1), 16–23. https://doi.org/10.1177/1754073912457228.

    Article  Google Scholar 

  78. Rosenberg, E. L., & Ekman, P. (1994). Coherence between expressive and experiential systems in emotion. Cognition and Emotion, 8(3), 201–229. https://doi.org/10.1080/02699939408408938.

    Article  Google Scholar 

  79. Ruiz-Belda, M.-A., Fernández-Dols, J.-M., Carrera, P., & Barchard, K. (2003). Spontaneous facial expressions of happy bowlers and soccer fans. Cognition and Emotion, 17(2), 315–326. https://doi.org/10.1080/02699930302288.

    Article  PubMed  Google Scholar 

  80. Russell, J. A. (1993). Forced-choice response format in the study of facial expression. Motivation and Emotion, 17(1), 41–51. https://doi.org/10.1007/BF00995206.

    Article  Google Scholar 

  81. Russell, J. A. (1994). Is there universal recognition of emotion from facial expression? A review of the cross-cultural studies. Psychological Bulletin, 115(1), 102–141. https://doi.org/10.1037/0033-2909.115.1.102.

    Article  PubMed  Google Scholar 

  82. Russell, J. A., Bachorowski, J.-A., & Fernández-Dols, J.-M. (2003). Facial and vocal expressions of emotion. Annual Review of Psychology, 54(1), 329–349. https://doi.org/10.1146/annurev.psych.54.101601.145102.

    Article  PubMed  Google Scholar 

  83. Russell, J. A., & Fehr, B. (1987). Relativity in the perception of emotion in facial expressions. Journal of Experimental Psychology: General, 116(3), 223–237. https://doi.org/10.1037/0096-3445.116.3.223.

    Article  Google Scholar 

  84. Sander, D., & Scherer, K. (2014). Oxford companion to emotion and the affective sciences. New York: Oxford University Press.

    Google Scholar 

  85. Scherer, K. R., & Ceschi, G. (1997). Lost luggage: A field study of emotion–antecedent appraisal. Motivation and Emotion, 21(3), 211–235. https://doi.org/10.1023/A:1024498629430.

    Article  Google Scholar 

  86. Schützwohl, A., & Reisenzein, R. (2012). Facial expressions in response to a highly surprising event exceeding the field of vision: A test of darwin’s theory of surprise. Evolution and Human Behavior, 33(6), 657–664. https://doi.org/10.1016/j.evolhumbehav.2012.04.003.

    Article  Google Scholar 

  87. Tcherkassof, A., Bollon, T., Dubois, M., Pansu, P., & Adam, J.-M. (2007). Facial expressions of emotions: A methodological contribution to the study of spontaneous and dynamic emotional faces. European Journal of Social Psychology, 37(6), 1325–1345. https://doi.org/10.1002/ejsp.427.

    Article  Google Scholar 

  88. Tcherkassof, A., Dupré, D., Meillon, B., Mandran, N., Dubois, M., & Adam, J.-M. (2013). DynEmo: A video database of natural facial expressions of emotions. The International Journal of Multimedia & Its Applications, 5(5), 61–80. https://doi.org/10.5121/ijma.2013.5505.

    Article  Google Scholar 

  89. Tcherkassof, A., & Frijda, N. H. (2014). Les émotions: Une conception relationnelle. L’Année Psychologique, 114(3), 501–535.

    Article  Google Scholar 

  90. Tsai, J. L., Chentsova-Dutton, Y., Freire-Bebeau, L., & Przymus, D. E. (2002). Emotional expression and physiology in european americans and hmong americans. Emotion, 2(4), 380–397. https://doi.org/10.1037/1528-3542.2.4.380.

    Article  PubMed  Google Scholar 

  91. Vernon, L. L., & Berenbaum, H. (2002). Disgust and fear in response to spiders. Cognition and Emotion, 16(6), 809–830. https://doi.org/10.1080/02699930143000464.

    Article  Google Scholar 

  92. Wagner, H. L. (1997). Methods for the study of facial behavior. In J. A. Russell & J.-M. Fernández-Dols (Eds.), The psychology of facial expression (pp. 31–54). Cambridge: Cambridge University Press.

    Google Scholar 

  93. Wagner, H. L., MacDonald, C. J., & Manstead, A. S. (1986). Communication of individual emotions by spontaneous facial expressions. Journal of Personality and Social Psychology, 50(4), 737–743. https://doi.org/10.1037/0022-3514.50.4.737.

    Article  Google Scholar 

  94. Wickham, H. (2017). Tidyverse: Easily install and load the ’tidyverse’. https://CRAN.R-project.org/package=tidyverse. Accessed 6 Nov 2020.

  95. Zhu, H. (2019). KableExtra: Construct complex table with ’kable’ and pipe syntax. https://CRAN.R-project.org/package=kableExtra. Accessed 6 Nov 2020.

Download references

Acknowledgements

The authors would like to thank the developers of the following R packages used to process, analyze, display and report data: R (Version 4.0.2; R Core Team, 2019) and the R-packages caret (Version 6.0.86; Jed Wing et al., 2019), here (Version 0.1; Müller, 2017), janitor (Version 2.0.1; Firke, 2019), kableExtra (Version 1.2.1; Zhu, 2019), papaja (Version 0.1.0.9997; Aust & Barth, 2018), and tidyverse (Version 1.3.0; Wickham, 2017).

Author information

Affiliations

Authors

Contributions

The authors made the following contributions. AT: Conceptualization, Project administration, Validation, Writing—Original Draft Preparation, Writing—Review and Editing; DD: Investigation, Data curation, Formal Analysis, Software, Visualization, Writing—Original Draft Preparation, Writing—Review and Editing.

Corresponding author

Correspondence to Anna Tcherkassof.

Ethics declarations

Conflicts of interest

No potential conflicts of interests regarding the research reported in the manuscript exist.

Ethical approval

This is an observational study. The CERGA Research Ethics Committee has confirmed that no ethical approval is required.

Informed consent

Written informed consent was obtained from all individual participants included in the original study.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Tcherkassof, A., Dupré, D. The emotion–facial expression link: evidence from human and automatic expression recognition. Psychological Research (2020). https://doi.org/10.1007/s00426-020-01448-4

Download citation