Across Cultures: A Cognitive and Computational Analysis of Emotional and Conversational Facial Expressions in Germany and Korea

  • Christian WallravenEmail author
  • Dong-Cheol Hur
  • Ahyoung Shin
Part of the Trends in Augmentation of Human Performance book series (TAHP, volume 5)


Humans use a wide variety of communicative signals – among those, facial expressions play a key role in communicating not only emotional, but also more general, non-verbal signals. Here, we present results from a combined cognitive and computational analysis of emotional and conversational facial expressions in the context of cross-cultural research. Using two large databases of dynamic facial expressions, we show that both Western and Asian observers structure the interpretation space of a large range of facial expressions using the same two evaluative dimensions (valence and arousal). In addition, several computational experiments show the advantage of using graph-models for automatic recognition of facial expressions, since these models are able to capture the complex dynamics and inter-dependence of the movements of facial features in the face.


Facial expressions Cross-cultural psychology Emotions Conversational expressions Graph models 


  1. 1.
    Ekman P (1994) Strong evidence for universals in facial expressions. Psychol Bull 115(2):268–287CrossRefPubMedGoogle Scholar
  2. 2.
    Izard CE (1994) Innate and universal facial expressions: evidence from developmental and cross-cultural research. Psychol Bull 115(2):288–299CrossRefPubMedGoogle Scholar
  3. 3.
    Russell JA (1994) Is there universal recognition of emotion from facial expression? A review of the cross-cultural studies. Psychol Bull 115(1):102–141CrossRefPubMedGoogle Scholar
  4. 4.
    Nelson NL, Russell JA (2013) Universality revisited. Emot Rev 5(1):8–15CrossRefGoogle Scholar
  5. 5.
    Jack RE, Garrod OGB, Schyns PG (2013) Dynamic facial expressions of emotion transmit an evolving hierarchy of signals over time. Curr Biol 24(2):187–192CrossRefGoogle Scholar
  6. 6.
    Lee K-U, Khang HS, Kim K-T, Kim Y-J, Kweon Y-S, Shin Y-W, Liberzon I (2008) Distinct processing of facial emotion of own-race versus other-race. NeuroReport 19(10):1021–1025CrossRefPubMedGoogle Scholar
  7. 7.
    Matsumoto D, Nakagawa S, Estrada A (2009) The role of dispositional traits in accounting for country and ethnic group differences on adjustment. J Pers 77(1):177–211CrossRefPubMedGoogle Scholar
  8. 8.
    Jack RE, Blais C, Scheepers C, Schyns PG, Caldara R (2009) Cultural confusions show that facial expressions are not universal. Curr Biol 19(18):1543–1548CrossRefPubMedGoogle Scholar
  9. 9.
    Jack RE, Garrod OGB, Yu H, Caldara R, Schyns PG (2012) Facial expressions of emotion are not culturally universal. Proc Natl Acad Sci 109(19):4–7CrossRefGoogle Scholar
  10. 10.
    Schmidt KL, Cohn JF (2001) Human facial expressions as adaptations: evolutionary questions in facial expression research. Am J Phys Anthropol 33(S33):3–24CrossRefPubMedGoogle Scholar
  11. 11.
    Elfenbein HA, Beaupré M, Lévesque M, Hess U (2007) Toward a dialect theory: cultural differences in the expression and recognition of posed facial expressions. Emotion 7(1):131–146CrossRefPubMedGoogle Scholar
  12. 12.
    McCarthy A, Lee K, Itakura S, Muir DW (2008) Gaze display when thinking depends on culture and context. J Cross Cult Psychol 39(6):716–729CrossRefGoogle Scholar
  13. 13.
    Gatica-Perez D (2009) Automatic nonverbal analysis of social interaction in small groups: a review. Image Vis Comput 27(12):1775–1787CrossRefGoogle Scholar
  14. 14.
    Vinciarelli A, Pantic M, Heylen D, Pelachaud C, Poggi I, D’Errico F, Schröder M (2012) Bridging the gap between social animal and unsocial machine: a survey of social signal processing. IEEE Trans Affect Comput 3(1):69–87CrossRefGoogle Scholar
  15. 15.
    Kanaujia A, Metaxas D (2006) Recognizing facial expressions by tracking feature shapes. In: Proceedings – International conference on pattern recognition. Hongkong, vol 2, pp 33–38Google Scholar
  16. 16.
    Rivera J, Kreuz T (2009) Reading faces with conditional random fields, Technical report. Robotics Institute, Carnegie Mellon University, PittsburghGoogle Scholar
  17. 17.
    Chang KY, Liu TL, Lai SH (2009) Learning partially-observed hidden conditional random fields for facial expression recognition. In: 2009 IEEE computer society conference on computer vision and pattern recognition workshops. Miami, pp 533–540Google Scholar
  18. 18.
    Bousmalis K, Morency LP, Pantic M (2011) Modeling hidden dynamics of multimodal cues for spontaneous agreement and disagreement recognition. In: 2011 IEEE International conference on automatic face and gesture recognition and workshops. Santa Barbara, pp 746–752Google Scholar
  19. 19.
    McDuff D, El Kaliouby R, Kassam K, Picard R (2010) Affect valence inference from facial action unit spectrograms. In: 2010 IEEE computer society conference on computer vision and pattern recognition – workshops. San Francisco, pp 17–24Google Scholar
  20. 20.
    Cunningham DW, Wallraven C (2009) Dynamic information for the recognition of conversational expressions. J Vis 9(13):7.1–17CrossRefGoogle Scholar
  21. 21.
    Bülthoff HH, Cunningham DW, Wallraven C (2011) Dynamic aspects of face processing in humans. In: Li SZ, Jain KA (eds) Handbook of face recognition. Springer, London, pp 571–576Google Scholar
  22. 22.
    Kaulard K, Cunningham DW, Bülthoff HH, Wallraven C (2012) The MPI facial expression database – a validated database of emotional and conversational facial expressions. PLoS One 7(3):e32321PubMedCentralCrossRefPubMedGoogle Scholar
  23. 23.
    Lee H, Shin A, Kim B, Wallraven C (2012) The KU facial expression database: a validated database of emotional and conversational expressions. In: Proceedings of Asian Pacific conference on vision. IncheonGoogle Scholar
  24. 24.
    Bartlett M, Littlewort G, Wu T, Movellan J (2008) Computer expression recognition toolbox (CERT). In: 2008 8th IEEE International conference on automatic face and gesture recognition. AmsterdamGoogle Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2015

Authors and Affiliations

  • Christian Wallraven
    • 1
    Email author
  • Dong-Cheol Hur
    • 2
  • Ahyoung Shin
    • 1
  1. 1.Department of Brain and Cognitive EngineeringKorea UniversitySeoulRepublic of Korea
  2. 2.Department of Computer Science and EngineeringKorea UniversitySeoulRepublic of Korea

Personalised recommendations