Abstract
Facial expressions play a paramount role in character animation since they reveal much of a person’s emotions and intentions. Although animation techniques have become more sophisticated over time, there is still need for knowledge in terms of what behavior appears emotionally convincing and believable. The present chapter examines how motion contributes to the perception and interpretation of facial expressions. This includes a description of the early beginnings in research on facial motion and more recent work, pointing toward a dynamic advantage in facial expression recognition. Attention is further drawn to the potential characteristics (i.e., directionality and speed) that facilitate such dynamic advantage. This is followed by a review on how facial motion affects perception and behavior more generally, with the neural systems that underlie the processing of dynamic emotions. The chapter concludes by discussing remaining challenges and future directions for the animation of natural occurring emotional expressions in dynamic faces.
This is a preview of subscription content, log in via an institution to check access.
References
Adolphs R, Tranel D, Damasio AR (2003) Dissociable neural systems for recognizing emotions. Brain Cogn 52:61–69. https://doi.org/10.1016/S0278-2626(03)00009-5
Ambadar Z, Cohn JF, Reed LI (2009) All smiles are not created equal: morphology and timing of smiles perceived as amused, polite, and embarrassed/nervous. J Nonverbal Behav 33:17–34. https://doi.org/10.1007/s10919-008-0059-5
Ambadar Z, Schooler JW, Cohn JF (2005) Deciphering the enigmatic face. The importance of facial dynamics in interpreting subtle facial expressions. Psychol Sci 16:403–410. https://doi.org/10.1111/j.0956-7976.2005.01548.x
Arsalidou M, Morris D, Taylor MJ (2011) Converging evidence for the advantage of dynamic facial expressions. Brain Topogr 24:149–163. https://doi.org/10.1007/s10548-011-0171-4
Bassili JN (1978) Facial motion in the perception of faces and of emotional expression. J Exp Psychol Hum Percept Perform 4:373–379. https://doi.org/10.1037/0096-1523.4.3.373
Bassili JN (1979) Emotion recognition: the role of facial movement and the relative importance of upper and lower face. J Pers Soc Psychol 37:2049–2058. https://doi.org/10.1037/0022-3514.37.11.2049
Biele C, Grabowska A (2006) Sex differences in perception of emotion intensity in dynamic and static facial expressions. Exp Brain Res 171:1–6. https://doi.org/10.1007/s00221-005-0254-0
Botvinick M, Jha AP, Bylsma LM, Fabian SA, Solomon PE, Prkachin KM (2005) Viewing facial expressions of pain engages cortical areas involved in the direct experience of pain. Neuroimage 25:312–319. https://doi.org/10.1016/j.neuroimage.2004.11.043
Bould E, Morris N (2008) Role of motion signals in recognizing subtle facial expressions of emotion. Br J Psychol 99:167–189. https://doi.org/10.1348/000712607X206702
Bould E, Morris N, Wink B (2008) Recognising subtle emotional expressions: the role of facial movements. Cognit Emot 22:1569–1587. https://doi.org/10.1080/02699930801921156
Bruce V, Valentine T (1988) When a nod’s as good as a wink: the role of dynamic information in facial recognition. In: Gruneberg MM, Morris PE, Sykes RN (eds) Practical aspects of memory: current research and issues, vol 1. John Wiley and Sons, New York, pp 169–174
Bugental DB (1986) Unmasking the “polite smile”. Situational and personal determinants of managed affect in adult-child interaction. Pers Soc Psychol Bull 12:7–16. https://doi.org/10.1177/0146167286121001
Cosker D, Krumhuber EG, Hilton A (2010) Perception of linear and nonlinear motion properties using a FACS validated 3D facial model. In: Proceedings of the symposium on applied Perception in graphics and visualization (APGV), Los Angeles
Cunningham DW, Wallraven C (2009a) The interaction between motion and form in expression recognition. In: Bodenheimer B, O’Sullivan C (eds) proceedings of the 6th symposium on applied perception in graphics and visualization (APGV2009), New York
Cunningham DW, Wallraven C (2009b) Dynamic information for the recognition of conversational expressions. J Vis 9:1–17. https://doi.org/10.1167/9.13.7
Dapretto M, Davies MS, Pfeifer JH, Scott AA, Sigman M, Bookheimer SY, Iacoboni M (2006) Understanding emotions in others: mirror neuron dysfunction in children with autism spectrum disorders. Nat Neurosci 9:28–30. https://doi.org/10.1038/nn1611
Dimberg U (1988) Facial electromyography and the experience of emotion. J Psychophysiol 2:277–282
Edwards K (1998) The face of time: temporal cues in facial expressions of emotion. Psychol Sci 9:270–276. https://doi.org/10.1111/1467-9280.00054
Ehrlich SM, Schiano DJ, Sheridan K (2000) Communicating facial affect: it’s not the realism, it’s the motion. In: Proceedings of ACM CHI 2000 conference on human factors in computing systems, New York
Ekman P (1985) Telling lies. Norton, New York
Ekman P, Friesen WV (1982) Felt, false, and miserable smiles. J Nonverbal Behav 6:238–252. https://doi.org/10.1007/BF00987191
Ekman P, Friesen WV (1978) Facial action coding system: a technique for the measurement of facial movement. Consulting Psychologists Press, Palo Alto
Ekman P, Davidson RJ, Friesen WV (1990) The Duchenne smile: emotional expression and brain physiology: II. J Pers Soc Psychol 58:342–353. https://doi.org/10.1037/0022-3514.58.2.342
Fiorentini C, Viviani P (2011) Is there a dynamic advantage for facial expressions. J Vis 11:1–15. https://doi.org/10.1167/11.3.17
Frank MG, Ekman P, Friesen WV (1993) Behavioral markers and recognizability of the smile of enjoyment. J Pers Soc Psychol 64:83–93. https://doi.org/10.1037/0022-3514.64.1.83
Gallese V (2001) The ‘shared manifold’ hypothesis. From mirror neurons to empathy. J Conscious Stud 8:33–50
Geller T (2008) Overcoming the uncanny valley. IEEE Comput Graph Appl 28:11–17. https://doi.org/10.1109/MCG.2008.79
Gibson JJ (1966) The senses considered as perceptual systems. Houghton Mifflin, Boston https://doi.org/10.1080/00043079.1969.10790296
Hehman E, Flake JK, Freeman JB (2015) Static and dynamic facial cues differentially affect the consistency of social evaluations. Pers Soc Psychol Bull 41:1123–1134. https://doi.org/10.1177/0146167215591495
Hess U, Kleck RE (1990) Differentiating emotion elicited and deliberate emotional facial expressions. Eur J Soc Psychol 20:369–385. https://doi.org/10.1002/ejsp.2420200502
Humphreys GW, Donnely N, Riddoch MJ (1993) Expression is computed separately from facial identity, and is computed separately for moving and static faces: neuropsychological evidence. Neuropsychologia 31:173–181. https://doi.org/10.1016/0028-3932(93)90045-2
Iacoboni M (2009) Imitation, empathy, and mirror neurons. Annu Rev Psychol 60:653–670. https://doi.org/10.1146/annurev.psych.60.110707.163604
Iacoboni M, Dapretto M (2006) The mirror neuron system and the consequences of its dysfunction. Nat Rev Neurosci 7:942–951. https://doi.org/10.1038/nrn2024
Johansson G (1973) Visual perception of biological motion and a model for its analysis. Percept Psychophys 14:201–211. https://doi.org/10.3758/BF03212378
Kamachi M, Bruce V, Mukaida S, Gyoba J, Yoshikawa S, Akamatsu S (2001) Dynamic properties influence the perception of facial expressions. Perception 30:875–887. https://doi.org/10.1068/p3131
Kappas A, Krumhuber EG, Küster D (2013) Facial behavior. In: Hall JA, Knapp ML (eds) Nonverbal communication (Handbooks of Communication Science, HOCS 2). Mouton de Gruyter, Berlin, pp 131–165
Kätsyri J, Sams M (2008) The effect of dynamics on identifying basic emotions from synthetic and natural faces. Int J Hum Comput Stud 66:233–242. https://doi.org/10.1016/j.ijhcs.2007.10.001
Kätsyri J, Förger K, Mäkäräinen M, Takala T (2015) A review of empirical evidence on different uncanny valley hypotheses: support for perceptual mismatch as one road to the valley of eeriness. Front Psychol 6:390. https://doi.org/10.3389/fpsyg.2015.00390
Kerlow IV (2004) The art of 3D computer animation and effects, 3rd edn. John Wiley and Sons, Hoboken
Kessler H, Doyen-Waldecker C, Hofer C, Hoffmann H, Traue HC, Abler B (2011) Neural correlates of the perception of dynamic versus static facial expressions of emotion. GMS Psychosoc Med 8:1–8. https://doi.org/10.3205/psm000072
Kilts CD, Egan G, Gideon DA, Ely TD, Hoffmann JM (2003) Dissociable neural pathways are involved in the recognition of emotion in static and dynamic facial expressions. Neuroimage 18:156–168. https://doi.org/10.1006/nimg.2002.1323
Korb S, With S, Niedenthal PM, Kaiser S, Grandjean D (2014) The perception and mimicry of facial movements predict judgments of smile authenticity. PLoS One 9:e99194. https://doi.org/10.1371/journal.pone.0099194
Krumhuber EG, Tamarit L, Roesch EB, Scherer KR (2012) FACSGen 2.0 animation software: generating three-dimensional FACS-valid facial expressions for emotion research. Emotion 12:351–363. https://doi.org/10.1037/a0026632
Krumhuber EG, Kappas (2005) Moving smiles: the role of dynamic components for the perception of the genuineness of smiles. J Nonverbal Behav 29:3–24. https://doi.org/10.1007/s10919-004-0887-x
Krumhuber EG, Manstead ASR (2009) Can Duchenne smiles be feigned? New evidence on felt and false smiles. Emotion 9:807–820. https://doi.org/10.1037/a0017844
Krumhuber EG, Manstead ASR, Cosker D, Marshall D, Rosin PL (2009) Effects of dynamic attributes of smiles in human and synthetic faces: a simulated job interview setting. J Nonverbal Behav 33:1–15. https://doi.org/10.1007/s10919-008-0056-8
Krumhuber EG, Manstead ASR, Cosker D, Marshall D, Rosin PL, Kappas A (2007a) Facial dynamics as indicators of trustworthiness and cooperative behavior. Emotion 7:730–735. https://doi.org/10.1037/1528-3542.7.4.730
Krumhuber EG, Manstead ASR, Kappas A (2007b) Temporal aspects of facial displays in person and expression perception: the effects of smile dynamics, head-tilt and gender. J Nonverbal Behav 31:39–56. https://doi.org/10.1007/s10919-006-0019-x
Krumhuber EG, Skora P, Küster D, Fou L (in press) A review of dynamic datasets for facial expression research. Emotion Rev
Küster D, Krumhuber EG, Kappas A (2014) Nonverbal behavior online: a focus on interactions with and via artificial agents and avatars. In: Kostic A, Chadee D (eds) Social psychology of nonverbal communications. Palgrave MacMillan, New York, pp 272–302
LaBar KS, Crupain MJ, Vovodic JT, McCarthy G (2003) Dynamic perception of facial affect and identity in the human brain. Cereb Cortex 13:1023–1033. https://doi.org/10.1093/cercor/13.10.1023
Lee TW, Josephs O, Dolan RJ, Critchley HD (2006) Imitating expressions: emotion specific neural substrates in facial mimicry. Soc Cogn Affect Neurosci 1:122–135. https://doi.org/10.1093/scan/nsl012
Lundqvist L, Dimberg U (1995) Facial expressions are contagious. J Psychophysiol 9:203–211
Maringer M, Krumhuber EG, Fischer AH, Niedenthal P (2011) Beyond smile dynamics: mimicry and beliefs in judgments of smiles. Emotion 11:181–187. https://doi.org/10.1037/a0022596
McDonnnell R, Breidt M, Buelthoff HH (2012) Render me real? Investigating the effect of render style on the perception of animated virtual humans. ACM Trans Graph 31:1–11. https://doi.org/10.1145/2185520.2185587
Mori M (1970) Bukimi No Tani. The Uncanny Valley (MacDorman KF and Minato T, Trans). Energy 7:33–35
Niedenthal P, Brauer M, Halberstadt JB, Innes-Ker AH (2001) When did her smile drop? Facial mimicry and the influences of emotional state on the detection of change in emotional expression. Cognit Emot 15:853–864. https://doi.org/10.1080/02699930143000194
Oberman LM, Winkielman P, Ramachandran VS (2007) Face to face: blocking facial mimicry can selectively impair recognition of emotional expressions. Soc Neurosci 2:167–178. https://doi.org/10.1080/17470910701391943
Piwek L, McKay LS, Pollick FE (2014) Empirical evaluation of the uncanny valley hypothesis fails to confirm the predicted effect of motion. Cognition 130:271–277. https://doi.org/10.1016/j.cognition.2013.11.001
Ponari M, Conson M, D’Amico NP, Grossi D, Trojano L (2012) Mapping correspondence between facial mimicry and emotion recognition in healthy subjects. Emotion 12:1398–1403. https://doi.org/10.1037/a0028588
Recio G, Schacht A, Sommer W (2013) Classification of dynamic facial expressions of emotion presented briefly. Cognit Emot 27:1486–1494. https://doi.org/10.1080/02699931.2013.794128
Recio G, Sommer W, Schacht A (2011) Electrophysiological correlates of perceiving and evaluating static and dynamic facial emotional expressions. Brain Res 1376:66–75. https://doi.org/10.1016/j.brainres.2010.12.041
Rizzolatti G, Craighero L (2004) The mirror-neuron system. Annu Rev Neurosci 27:169–192. https://doi.org/10.1146/annurev.neuro.27.070203.144230
Rizzolatti G, Fadiga L, Gallese V, Fogassi L (1996) Premotor cortex and the recognition of motor actions. Cogn Brain Res 3:131–141. https://doi.org/10.1016/0926-6410(95)00038-0
Rychlowska M, Canadas E, Wood A, Krumhuber EG, Niedenthal P (2014) Blocking mimicry makes true and false smiles look the same. PLoS One 9:e90876. https://doi.org/10.1371/journal.pone.0090876
Rymarczyk K, Biele C, Grabowska A, Majczynski H (2011) EMG activity in response to static and dynamic facial expressions. Int J Psychophysiol 79:330–333. https://doi.org/10.1016/j.ijpsycho.2010.11.001
Sato W, Yoshikawa S (2004) Brief report. The dynamic aspects of emotional facial expressions. Cognit Emot 18:701–710. https://doi.org/10.1080/02699930341000176
Sato W, Yoshikawa S (2007a) Spontaneous facial mimicry in response to dynamic facial expressions. Cognition 104:1–18. https://doi.org/10.1109/DEVLRN.2005.1490936v
Sato W, Yoshikawa S (2007b) Enhanced experience of emotional arousal in response to dynamic facial expressions. J Nonverbal Behav 31:119–135. https://doi.org/10.1007/s10919-007-0025-7
Sato W, Fujimura T, Suzuki N (2008) Enhanced facial EMG activity in response to dynamic facial expressions. Int J Psychophysiol 70:70–74. https://doi.org/10.1016/j.ijpsycho.2008.06.001
Sato W, Fujimura T, Kochiyama T, Suzuki N (2013) Relationships among facial mimicry, emotional experience, and emotion recognition. PLoS One 8:e57889. https://doi.org/10.1371/journal.pone.0057889
Sato W, Kochiyama T, Yoshikawa S, Naito E, Matsumura M (2004) Enhanced neural activity in response to dynamic facial expressions of emotion: an fMRI study. Cogn Brain Res 20:81–91. https://doi.org/10.1016/S0926-6410(04)00039-4
Saygin AP, Chaminade T, Ishiguro H, Driver J, Frith C (2012) The thing that should not be: predictive coding and the uncanny valley in perceiving human and humanoid robot actions. Soc Cogn Affect Neurosci 7:413–422. https://doi.org/10.1093/scan/nsr025
Schmidt KL, Ambadar Z, Cohn J, Reed LI (2006) Movement differences between deliberate and spontaneous facial expressions: zygomaticus major action in smiling. J Nonverbal Behav 301:37–52. https://doi.org/10.1007/s10919-005-0003-x
Schmidt KL, Bhattacharya S, Delinger R (2009) Comparison of deliberate and spontaneous facial movement in smiles and eyebrow raises. J Nonverbal Behav 33:35–45. https://doi.org/10.1007/s10919-008-0058-6
Schulte-Rüther M, Markowitsch HJ, Fink GR, Piefke M (2007) Mirror neuron and theory of mind mechanisms involved in face-to-face interactions: a functional magnetic resonance imaging approach to empathy. J Cogn Neurosci 19:1354–1372. https://doi.org/10.1162/jocn.2007.19.8.1354
Singer T, Seymour B, O’Doherty JP, Frith CD (2004) Empathy for pain involves the affective but not the sensory components of pain. Science 303:1157–1162. https://doi.org/10.1126/science.1093535
Stel M, Van Baaren RB, Vonk R (2008) Effects of mimicking: acting prosocially by being emotionally moved. Eur J Soc Psychol 38:965–976. https://doi.org/10.1002/ejsp.472
Thompson JC, Trafton JG, McKnight P (2011) The perception of humanness from the movements of synthetic agents. Perception 40:695–704. https://doi.org/10.1068/p6900
Tinwell A, Grimshaw M, Nabi DA, Williams A (2011) Facial expression of emotion and perception of the Uncanny Valley in virtual characters. Comput Hum Behav 27:741–749. https://doi.org/10.1016/j.chb.2010.10.018
Trautmann SA, Fehr T, Hermann M (2009) Emotions in motion: dynamic compared to static facial expressions of disgust and happiness reveal more widespread emotion-specific activations. Brain Res 1284:100–115. https://doi.org/10.1016/j.brainres.2009.05.075
Wallraven C, Breidt M, Cunningham DW, Bülthoff H (2008) Evaluating the perceptual realism of animated facial expressions. ACM Trans Appl Percept 4:1–20. https://doi.org/10.1145/1278760.1278764
Wehrle T, Kaiser S, Schmidt S, Scherer K (2000) Studying the dynamics of emotional expression using synthetized facial muscle movement. J Pers Soc Psychol 78:105–119. https://doi.org/10.1037/0022-3514.78.1.105
Weiss F, Blum GS, Gleberman L (1987) Anatomically based measurement of facial expressions in simulated versus hypnotically induced affect. Motiv Emot 11:67–81. https://doi.org/10.1007/BF00992214
Weyers P, Mühlberger A, Hefele C, Pauli P (2006) Electromyographic responses to static and dynamic avatar emotional facial expressions. Psychophysiology 43:45–453. https://doi.org/10.1111/j.1469-8986.2006.00451.x
Yang M, Wang K, Zhang L (2013) Realistic real-time facial expression animation via 3D morphing target. J Softw 8:418–425. https://doi.org/10.4304/jsw.8.2.418-425
Yoshikawa S, Sato W (2008) Dynamic facial expressions of emotion induce representational momentum. Cogn Affect Behav Neurosci 8:25–31. https://doi.org/10.3758/CABN.8.1.25
Author information
Authors and Affiliations
Corresponding author
Section Editor information
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this entry
Cite this entry
Krumhuber, E.G., Skora, L. (2018). Perceptual Study on Facial Expressions. In: Handbook of Human Motion. Springer, Cham. https://doi.org/10.1007/978-3-319-14418-4_18
Download citation
DOI: https://doi.org/10.1007/978-3-319-14418-4_18
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-14417-7
Online ISBN: 978-3-319-14418-4
eBook Packages: EngineeringReference Module Computer Science and Engineering