Advertisement

Perceptual Study on Facial Expressions

  • Eva G. Krumhuber
  • Lina Skora
Reference work entry

Abstract

Facial expressions play a paramount role in character animation since they reveal much of a person’s emotions and intentions. Although animation techniques have become more sophisticated over time, there is still need for knowledge in terms of what behavior appears emotionally convincing and believable. The present chapter examines how motion contributes to the perception and interpretation of facial expressions. This includes a description of the early beginnings in research on facial motion and more recent work, pointing toward a dynamic advantage in facial expression recognition. Attention is further drawn to the potential characteristics (i.e., directionality and speed) that facilitate such dynamic advantage. This is followed by a review on how facial motion affects perception and behavior more generally, with the neural systems that underlie the processing of dynamic emotions. The chapter concludes by discussing remaining challenges and future directions for the animation of natural occurring emotional expressions in dynamic faces.

Keywords

Motion Dynamic Facial expression Emotion Perception 

References

  1. Adolphs R, Tranel D, Damasio AR (2003) Dissociable neural systems for recognizing emotions. Brain Cogn 52:61–69.  https://doi.org/10.1016/S0278-2626(03)00009-5CrossRefGoogle Scholar
  2. Ambadar Z, Cohn JF, Reed LI (2009) All smiles are not created equal: morphology and timing of smiles perceived as amused, polite, and embarrassed/nervous. J Nonverbal Behav 33:17–34.  https://doi.org/10.1007/s10919-008-0059-5CrossRefGoogle Scholar
  3. Ambadar Z, Schooler JW, Cohn JF (2005) Deciphering the enigmatic face. The importance of facial dynamics in interpreting subtle facial expressions. Psychol Sci 16:403–410.  https://doi.org/10.1111/j.0956-7976.2005.01548.xCrossRefGoogle Scholar
  4. Arsalidou M, Morris D, Taylor MJ (2011) Converging evidence for the advantage of dynamic facial expressions. Brain Topogr 24:149–163.  https://doi.org/10.1007/s10548-011-0171-4CrossRefGoogle Scholar
  5. Bassili JN (1978) Facial motion in the perception of faces and of emotional expression. J Exp Psychol Hum Percept Perform 4:373–379.  https://doi.org/10.1037/0096-1523.4.3.373CrossRefGoogle Scholar
  6. Bassili JN (1979) Emotion recognition: the role of facial movement and the relative importance of upper and lower face. J Pers Soc Psychol 37:2049–2058.  https://doi.org/10.1037/0022-3514.37.11.2049CrossRefGoogle Scholar
  7. Biele C, Grabowska A (2006) Sex differences in perception of emotion intensity in dynamic and static facial expressions. Exp Brain Res 171:1–6.  https://doi.org/10.1007/s00221-005-0254-0CrossRefGoogle Scholar
  8. Botvinick M, Jha AP, Bylsma LM, Fabian SA, Solomon PE, Prkachin KM (2005) Viewing facial expressions of pain engages cortical areas involved in the direct experience of pain. Neuroimage 25:312–319.  https://doi.org/10.1016/j.neuroimage.2004.11.043CrossRefGoogle Scholar
  9. Bould E, Morris N (2008) Role of motion signals in recognizing subtle facial expressions of emotion. Br J Psychol 99:167–189.  https://doi.org/10.1348/000712607X206702CrossRefGoogle Scholar
  10. Bould E, Morris N, Wink B (2008) Recognising subtle emotional expressions: the role of facial movements. Cognit Emot 22:1569–1587.  https://doi.org/10.1080/02699930801921156CrossRefGoogle Scholar
  11. Bruce V, Valentine T (1988) When a nod’s as good as a wink: the role of dynamic information in facial recognition. In: Gruneberg MM, Morris PE, Sykes RN (eds) Practical aspects of memory: current research and issues, vol 1. John Wiley and Sons, New York, pp 169–174Google Scholar
  12. Bugental DB (1986) Unmasking the “polite smile”. Situational and personal determinants of managed affect in adult-child interaction. Pers Soc Psychol Bull 12:7–16.  https://doi.org/10.1177/0146167286121001CrossRefGoogle Scholar
  13. Cosker D, Krumhuber EG, Hilton A (2010) Perception of linear and nonlinear motion properties using a FACS validated 3D facial model. In: Proceedings of the symposium on applied Perception in graphics and visualization (APGV), Los AngelesGoogle Scholar
  14. Cunningham DW, Wallraven C (2009a) The interaction between motion and form in expression recognition. In: Bodenheimer B, O’Sullivan C (eds) proceedings of the 6th symposium on applied perception in graphics and visualization (APGV2009), New YorkGoogle Scholar
  15. Cunningham DW, Wallraven C (2009b) Dynamic information for the recognition of conversational expressions. J Vis 9:1–17.  https://doi.org/10.1167/9.13.7CrossRefGoogle Scholar
  16. Dapretto M, Davies MS, Pfeifer JH, Scott AA, Sigman M, Bookheimer SY, Iacoboni M (2006) Understanding emotions in others: mirror neuron dysfunction in children with autism spectrum disorders. Nat Neurosci 9:28–30.  https://doi.org/10.1038/nn1611CrossRefGoogle Scholar
  17. Dimberg U (1988) Facial electromyography and the experience of emotion. J Psychophysiol 2:277–282Google Scholar
  18. Edwards K (1998) The face of time: temporal cues in facial expressions of emotion. Psychol Sci 9:270–276.  https://doi.org/10.1111/1467-9280.00054CrossRefGoogle Scholar
  19. Ehrlich SM, Schiano DJ, Sheridan K (2000) Communicating facial affect: it’s not the realism, it’s the motion. In: Proceedings of ACM CHI 2000 conference on human factors in computing systems, New YorkGoogle Scholar
  20. Ekman P (1985) Telling lies. Norton, New YorkGoogle Scholar
  21. Ekman P, Friesen WV (1982) Felt, false, and miserable smiles. J Nonverbal Behav 6:238–252.  https://doi.org/10.1007/BF00987191CrossRefGoogle Scholar
  22. Ekman P, Friesen WV (1978) Facial action coding system: a technique for the measurement of facial movement. Consulting Psychologists Press, Palo AltoGoogle Scholar
  23. Ekman P, Davidson RJ, Friesen WV (1990) The Duchenne smile: emotional expression and brain physiology: II. J Pers Soc Psychol 58:342–353.  https://doi.org/10.1037/0022-3514.58.2.342CrossRefGoogle Scholar
  24. Fiorentini C, Viviani P (2011) Is there a dynamic advantage for facial expressions. J Vis 11:1–15.  https://doi.org/10.1167/11.3.17CrossRefGoogle Scholar
  25. Frank MG, Ekman P, Friesen WV (1993) Behavioral markers and recognizability of the smile of enjoyment. J Pers Soc Psychol 64:83–93.  https://doi.org/10.1037/0022-3514.64.1.83CrossRefGoogle Scholar
  26. Gallese V (2001) The ‘shared manifold’ hypothesis. From mirror neurons to empathy. J Conscious Stud 8:33–50Google Scholar
  27. Geller T (2008) Overcoming the uncanny valley. IEEE Comput Graph Appl 28:11–17.  https://doi.org/10.1109/MCG.2008.79CrossRefGoogle Scholar
  28. Gibson JJ (1966) The senses considered as perceptual systems. Houghton Mifflin, Boston  https://doi.org/10.1080/00043079.1969.10790296
  29. Hehman E, Flake JK, Freeman JB (2015) Static and dynamic facial cues differentially affect the consistency of social evaluations. Pers Soc Psychol Bull 41:1123–1134.  https://doi.org/10.1177/0146167215591495CrossRefGoogle Scholar
  30. Hess U, Kleck RE (1990) Differentiating emotion elicited and deliberate emotional facial expressions. Eur J Soc Psychol 20:369–385.  https://doi.org/10.1002/ejsp.2420200502CrossRefGoogle Scholar
  31. Humphreys GW, Donnely N, Riddoch MJ (1993) Expression is computed separately from facial identity, and is computed separately for moving and static faces: neuropsychological evidence. Neuropsychologia 31:173–181.  https://doi.org/10.1016/0028-3932(93)90045-2CrossRefGoogle Scholar
  32. Iacoboni M (2009) Imitation, empathy, and mirror neurons. Annu Rev Psychol 60:653–670.  https://doi.org/10.1146/annurev.psych.60.110707.163604CrossRefGoogle Scholar
  33. Iacoboni M, Dapretto M (2006) The mirror neuron system and the consequences of its dysfunction. Nat Rev Neurosci 7:942–951.  https://doi.org/10.1038/nrn2024CrossRefGoogle Scholar
  34. Johansson G (1973) Visual perception of biological motion and a model for its analysis. Percept Psychophys 14:201–211.  https://doi.org/10.3758/BF03212378CrossRefGoogle Scholar
  35. Kamachi M, Bruce V, Mukaida S, Gyoba J, Yoshikawa S, Akamatsu S (2001) Dynamic properties influence the perception of facial expressions. Perception 30:875–887.  https://doi.org/10.1068/p3131CrossRefGoogle Scholar
  36. Kappas A, Krumhuber EG, Küster D (2013) Facial behavior. In: Hall JA, Knapp ML (eds) Nonverbal communication (Handbooks of Communication Science, HOCS 2). Mouton de Gruyter, Berlin, pp 131–165Google Scholar
  37. Kätsyri J, Sams M (2008) The effect of dynamics on identifying basic emotions from synthetic and natural faces. Int J Hum Comput Stud 66:233–242.  https://doi.org/10.1016/j.ijhcs.2007.10.001CrossRefGoogle Scholar
  38. Kätsyri J, Förger K, Mäkäräinen M, Takala T (2015) A review of empirical evidence on different uncanny valley hypotheses: support for perceptual mismatch as one road to the valley of eeriness. Front Psychol 6:390.  https://doi.org/10.3389/fpsyg.2015.00390CrossRefGoogle Scholar
  39. Kerlow IV (2004) The art of 3D computer animation and effects, 3rd edn. John Wiley and Sons, HobokenGoogle Scholar
  40. Kessler H, Doyen-Waldecker C, Hofer C, Hoffmann H, Traue HC, Abler B (2011) Neural correlates of the perception of dynamic versus static facial expressions of emotion. GMS Psychosoc Med 8:1–8.  https://doi.org/10.3205/psm000072Google Scholar
  41. Kilts CD, Egan G, Gideon DA, Ely TD, Hoffmann JM (2003) Dissociable neural pathways are involved in the recognition of emotion in static and dynamic facial expressions. Neuroimage 18:156–168.  https://doi.org/10.1006/nimg.2002.1323CrossRefGoogle Scholar
  42. Korb S, With S, Niedenthal PM, Kaiser S, Grandjean D (2014) The perception and mimicry of facial movements predict judgments of smile authenticity. PLoS One 9:e99194.  https://doi.org/10.1371/journal.pone.0099194CrossRefGoogle Scholar
  43. Krumhuber EG, Tamarit L, Roesch EB, Scherer KR (2012) FACSGen 2.0 animation software: generating three-dimensional FACS-valid facial expressions for emotion research. Emotion 12:351–363.  https://doi.org/10.1037/a0026632CrossRefGoogle Scholar
  44. Krumhuber EG, Kappas (2005) Moving smiles: the role of dynamic components for the perception of the genuineness of smiles. J Nonverbal Behav 29:3–24.  https://doi.org/10.1007/s10919-004-0887-xCrossRefGoogle Scholar
  45. Krumhuber EG, Manstead ASR (2009) Can Duchenne smiles be feigned? New evidence on felt and false smiles. Emotion 9:807–820.  https://doi.org/10.1037/a0017844CrossRefGoogle Scholar
  46. Krumhuber EG, Manstead ASR, Cosker D, Marshall D, Rosin PL (2009) Effects of dynamic attributes of smiles in human and synthetic faces: a simulated job interview setting. J Nonverbal Behav 33:1–15.  https://doi.org/10.1007/s10919-008-0056-8CrossRefGoogle Scholar
  47. Krumhuber EG, Manstead ASR, Cosker D, Marshall D, Rosin PL, Kappas A (2007a) Facial dynamics as indicators of trustworthiness and cooperative behavior. Emotion 7:730–735.  https://doi.org/10.1037/1528-3542.7.4.730CrossRefGoogle Scholar
  48. Krumhuber EG, Manstead ASR, Kappas A (2007b) Temporal aspects of facial displays in person and expression perception: the effects of smile dynamics, head-tilt and gender. J Nonverbal Behav 31:39–56.  https://doi.org/10.1007/s10919-006-0019-xCrossRefGoogle Scholar
  49. Krumhuber EG, Skora P, Küster D, Fou L (in press) A review of dynamic datasets for facial expression research. Emotion RevGoogle Scholar
  50. Küster D, Krumhuber EG, Kappas A (2014) Nonverbal behavior online: a focus on interactions with and via artificial agents and avatars. In: Kostic A, Chadee D (eds) Social psychology of nonverbal communications. Palgrave MacMillan, New York, pp 272–302Google Scholar
  51. LaBar KS, Crupain MJ, Vovodic JT, McCarthy G (2003) Dynamic perception of facial affect and identity in the human brain. Cereb Cortex 13:1023–1033.  https://doi.org/10.1093/cercor/13.10.1023CrossRefGoogle Scholar
  52. Lee TW, Josephs O, Dolan RJ, Critchley HD (2006) Imitating expressions: emotion specific neural substrates in facial mimicry. Soc Cogn Affect Neurosci 1:122–135.  https://doi.org/10.1093/scan/nsl012CrossRefGoogle Scholar
  53. Lundqvist L, Dimberg U (1995) Facial expressions are contagious. J Psychophysiol 9:203–211Google Scholar
  54. Maringer M, Krumhuber EG, Fischer AH, Niedenthal P (2011) Beyond smile dynamics: mimicry and beliefs in judgments of smiles. Emotion 11:181–187.  https://doi.org/10.1037/a0022596CrossRefGoogle Scholar
  55. McDonnnell R, Breidt M, Buelthoff HH (2012) Render me real? Investigating the effect of render style on the perception of animated virtual humans. ACM Trans Graph 31:1–11.  https://doi.org/10.1145/2185520.2185587CrossRefGoogle Scholar
  56. Mori M (1970) Bukimi No Tani. The Uncanny Valley (MacDorman KF and Minato T, Trans). Energy 7:33–35Google Scholar
  57. Niedenthal P, Brauer M, Halberstadt JB, Innes-Ker AH (2001) When did her smile drop? Facial mimicry and the influences of emotional state on the detection of change in emotional expression. Cognit Emot 15:853–864.  https://doi.org/10.1080/02699930143000194CrossRefGoogle Scholar
  58. Oberman LM, Winkielman P, Ramachandran VS (2007) Face to face: blocking facial mimicry can selectively impair recognition of emotional expressions. Soc Neurosci 2:167–178.  https://doi.org/10.1080/17470910701391943CrossRefGoogle Scholar
  59. Piwek L, McKay LS, Pollick FE (2014) Empirical evaluation of the uncanny valley hypothesis fails to confirm the predicted effect of motion. Cognition 130:271–277.  https://doi.org/10.1016/j.cognition.2013.11.001CrossRefGoogle Scholar
  60. Ponari M, Conson M, D’Amico NP, Grossi D, Trojano L (2012) Mapping correspondence between facial mimicry and emotion recognition in healthy subjects. Emotion 12:1398–1403.  https://doi.org/10.1037/a0028588CrossRefGoogle Scholar
  61. Recio G, Schacht A, Sommer W (2013) Classification of dynamic facial expressions of emotion presented briefly. Cognit Emot 27:1486–1494.  https://doi.org/10.1080/02699931.2013.794128CrossRefGoogle Scholar
  62. Recio G, Sommer W, Schacht A (2011) Electrophysiological correlates of perceiving and evaluating static and dynamic facial emotional expressions. Brain Res 1376:66–75.  https://doi.org/10.1016/j.brainres.2010.12.041CrossRefGoogle Scholar
  63. Rizzolatti G, Craighero L (2004) The mirror-neuron system. Annu Rev Neurosci 27:169–192.  https://doi.org/10.1146/annurev.neuro.27.070203.144230CrossRefGoogle Scholar
  64. Rizzolatti G, Fadiga L, Gallese V, Fogassi L (1996) Premotor cortex and the recognition of motor actions. Cogn Brain Res 3:131–141.  https://doi.org/10.1016/0926-6410(95)00038-0CrossRefGoogle Scholar
  65. Rychlowska M, Canadas E, Wood A, Krumhuber EG, Niedenthal P (2014) Blocking mimicry makes true and false smiles look the same. PLoS One 9:e90876.  https://doi.org/10.1371/journal.pone.0090876CrossRefGoogle Scholar
  66. Rymarczyk K, Biele C, Grabowska A, Majczynski H (2011) EMG activity in response to static and dynamic facial expressions. Int J Psychophysiol 79:330–333.  https://doi.org/10.1016/j.ijpsycho.2010.11.001CrossRefGoogle Scholar
  67. Sato W, Yoshikawa S (2004) Brief report. The dynamic aspects of emotional facial expressions. Cognit Emot 18:701–710.  https://doi.org/10.1080/02699930341000176CrossRefGoogle Scholar
  68. Sato W, Yoshikawa S (2007a) Spontaneous facial mimicry in response to dynamic facial expressions. Cognition 104:1–18.  https://doi.org/10.1109/DEVLRN.2005.1490936vCrossRefGoogle Scholar
  69. Sato W, Yoshikawa S (2007b) Enhanced experience of emotional arousal in response to dynamic facial expressions. J Nonverbal Behav 31:119–135.  https://doi.org/10.1007/s10919-007-0025-7CrossRefGoogle Scholar
  70. Sato W, Fujimura T, Suzuki N (2008) Enhanced facial EMG activity in response to dynamic facial expressions. Int J Psychophysiol 70:70–74.  https://doi.org/10.1016/j.ijpsycho.2008.06.001CrossRefGoogle Scholar
  71. Sato W, Fujimura T, Kochiyama T, Suzuki N (2013) Relationships among facial mimicry, emotional experience, and emotion recognition. PLoS One 8:e57889.  https://doi.org/10.1371/journal.pone.0057889CrossRefGoogle Scholar
  72. Sato W, Kochiyama T, Yoshikawa S, Naito E, Matsumura M (2004) Enhanced neural activity in response to dynamic facial expressions of emotion: an fMRI study. Cogn Brain Res 20:81–91.  https://doi.org/10.1016/S0926-6410(04)00039-4CrossRefGoogle Scholar
  73. Saygin AP, Chaminade T, Ishiguro H, Driver J, Frith C (2012) The thing that should not be: predictive coding and the uncanny valley in perceiving human and humanoid robot actions. Soc Cogn Affect Neurosci 7:413–422.  https://doi.org/10.1093/scan/nsr025CrossRefGoogle Scholar
  74. Schmidt KL, Ambadar Z, Cohn J, Reed LI (2006) Movement differences between deliberate and spontaneous facial expressions: zygomaticus major action in smiling. J Nonverbal Behav 301:37–52.  https://doi.org/10.1007/s10919-005-0003-xCrossRefGoogle Scholar
  75. Schmidt KL, Bhattacharya S, Delinger R (2009) Comparison of deliberate and spontaneous facial movement in smiles and eyebrow raises. J Nonverbal Behav 33:35–45.  https://doi.org/10.1007/s10919-008-0058-6CrossRefGoogle Scholar
  76. Schulte-Rüther M, Markowitsch HJ, Fink GR, Piefke M (2007) Mirror neuron and theory of mind mechanisms involved in face-to-face interactions: a functional magnetic resonance imaging approach to empathy. J Cogn Neurosci 19:1354–1372.  https://doi.org/10.1162/jocn.2007.19.8.1354CrossRefGoogle Scholar
  77. Singer T, Seymour B, O’Doherty JP, Frith CD (2004) Empathy for pain involves the affective but not the sensory components of pain. Science 303:1157–1162.  https://doi.org/10.1126/science.1093535CrossRefGoogle Scholar
  78. Stel M, Van Baaren RB, Vonk R (2008) Effects of mimicking: acting prosocially by being emotionally moved. Eur J Soc Psychol 38:965–976.  https://doi.org/10.1002/ejsp.472CrossRefGoogle Scholar
  79. Thompson JC, Trafton JG, McKnight P (2011) The perception of humanness from the movements of synthetic agents. Perception 40:695–704.  https://doi.org/10.1068/p6900CrossRefGoogle Scholar
  80. Tinwell A, Grimshaw M, Nabi DA, Williams A (2011) Facial expression of emotion and perception of the Uncanny Valley in virtual characters. Comput Hum Behav 27:741–749.  https://doi.org/10.1016/j.chb.2010.10.018CrossRefGoogle Scholar
  81. Trautmann SA, Fehr T, Hermann M (2009) Emotions in motion: dynamic compared to static facial expressions of disgust and happiness reveal more widespread emotion-specific activations. Brain Res 1284:100–115.  https://doi.org/10.1016/j.brainres.2009.05.075CrossRefGoogle Scholar
  82. Wallraven C, Breidt M, Cunningham DW, Bülthoff H (2008) Evaluating the perceptual realism of animated facial expressions. ACM Trans Appl Percept 4:1–20.  https://doi.org/10.1145/1278760.1278764CrossRefGoogle Scholar
  83. Wehrle T, Kaiser S, Schmidt S, Scherer K (2000) Studying the dynamics of emotional expression using synthetized facial muscle movement. J Pers Soc Psychol 78:105–119.  https://doi.org/10.1037/0022-3514.78.1.105CrossRefGoogle Scholar
  84. Weiss F, Blum GS, Gleberman L (1987) Anatomically based measurement of facial expressions in simulated versus hypnotically induced affect. Motiv Emot 11:67–81.  https://doi.org/10.1007/BF00992214CrossRefGoogle Scholar
  85. Weyers P, Mühlberger A, Hefele C, Pauli P (2006) Electromyographic responses to static and dynamic avatar emotional facial expressions. Psychophysiology 43:45–453.  https://doi.org/10.1111/j.1469-8986.2006.00451.xCrossRefGoogle Scholar
  86. Yang M, Wang K, Zhang L (2013) Realistic real-time facial expression animation via 3D morphing target. J Softw 8:418–425.  https://doi.org/10.4304/jsw.8.2.418-425Google Scholar
  87. Yoshikawa S, Sato W (2008) Dynamic facial expressions of emotion induce representational momentum. Cogn Affect Behav Neurosci 8:25–31.  https://doi.org/10.3758/CABN.8.1.25CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.University College LondonLondonUK

Section editors and affiliations

  • Zhigang Deng
    • 1
  1. 1.Department of Computer Science,University of HoustonHoustonUSA

Personalised recommendations