Journal of Nonverbal Behavior

, Volume 33, Issue 1, pp 17–34 | Cite as

All Smiles are Not Created Equal: Morphology and Timing of Smiles Perceived as Amused, Polite, and Embarrassed/Nervous

  • Zara Ambadar
  • Jeffrey F. Cohn
  • Lawrence Ian Reed
Original Paper


We investigated the correspondence between perceived meanings of smiles and their morphological and dynamic characteristics. Morphological characteristics included co-activation of Orbicularis oculi (AU 6), smile controls, mouth opening, amplitude, and asymmetry of amplitude. Dynamic characteristics included duration, onset and offset velocity, asymmetry of velocity, and head movements. Smile characteristics were measured using the Facial Action Coding System (Ekman et al. 2002) and Automated Facial Image Analysis (Cohn and Kanade 2007). Observers judged 122 smiles as amused, embarrassed, nervous, polite, or other. Fifty-three smiles met criteria for classification as perceived amused, embarrassed/nervous, or polite. In comparison with perceived polite, perceived amused more often included AU 6, open mouth, smile controls, larger amplitude, larger maximum onset and offset velocity, and longer duration. In comparison with perceived embarrassed/nervous, perceived amused more often included AU 6, lower maximum offset velocity, and smaller forward head pitch. In comparison with perceived polite, perceived embarrassed/nervous more often included mouth opening and smile controls, larger amplitude, and greater forward head pitch. Occurrence of the AU 6 in perceived embarrassed/nervous and polite smiles questions the assumption that AU 6 with a smile is sufficient to communicate felt enjoyment. By comparing three perceptually distinct types of smiles, we found that perceived smile meanings were related to specific variation in smile morphological and dynamic characteristics.


Dynamic characteristics Morphological characteristics Smile interpretation FACS Automatic facial image analysis 



Portions of this work were supported by NIMH grant MH51435.


  1. Abe, J. A. A., Beetham, M., & Izard, C. E. (2002). What do smiles mean? An analysis in terms of differential emotions theory. In M. H. Abel (Ed.), An empirical reflection on the smile (Vol. 4, pp. 83–109). Lewiston, NY: The Edwin Mellen Press.Google Scholar
  2. Bonanno, G. A., Keltner, D., Noll, J. G., Putnam, F. W., Trickett, P. K., LeJeune, J., et al. (2002). When the face reveals what words do not: Facial expressions of emotion, smiling, and the willingness to disclose childhood sexual abuse. Journal of Personality and Social Psychology, 83, 94–110.PubMedCrossRefGoogle Scholar
  3. Brannigan, C. R., & Humphries, D. A. (1972). Human non-verbal behaviour, a means of communication. In N. B. Jones (Ed.), Ethological studies of child behaviour (pp. 37–64). Oxford, UK: Cambridge University Press.Google Scholar
  4. Brown, W. M., & Moore, C. (2002). Smile asymmetries and reputation as reliable indicators of likelihood to cooperate: An evolutionary analysis. In S. P. Shohov (Ed.), Advances in psychology research (Vol. 11, pp. 59–78). Huntington, NY: Nova Science.Google Scholar
  5. Carroll, J. M., & Russell, J. A. (1996). Do facial expressions signal specific emotions? Judging emotion from the face in context. Journal of Personality and Social Psychology, 70, 205–218.PubMedCrossRefGoogle Scholar
  6. Cheyne, J. A. (1976). Development of forms and functions of smiling in preschoolers. Child Development, 47, 820–823.CrossRefGoogle Scholar
  7. Cohn, J. F., & Kanade, T. (2007). Use of automated facial image analysis for measurement of emotion expression. In J. A. Coan & J. B. Allen (Eds.), The handbook of emotion elicitation and assessment (pp. 222–238). New York: Oxford University Press.Google Scholar
  8. Cohn, J. F., Reed, L. I., Moriyama, T., Xiao, J., Schmidt, K. L., & Ambadar, Z. (2004). Multimodal coordination of facial action, head rotation, and eye motion during spontaneous smiles and gesture recognition. Paper Presented at the Sixth IEEE International Conference on Automatic Face and Gesture Recognition.Google Scholar
  9. Cohn, J. F., Zlochower, A. J., Lien, J., & Kanade, T. (1999). Automated face analysis by feature point tracking has high concurrent validity with manual FACS coding. Psychophysiology, 36, 35–43.PubMedCrossRefGoogle Scholar
  10. Costa, M., Dinsbach, W., Manstead, A. S. R., & Ricci Bitti, P. E. (2001). Social presence, embarrassment and nonverbal behavior. Journal of Nonverbal Behavior, 25, 225–240.CrossRefGoogle Scholar
  11. Dinehart, L. H. B., Messinger, D. S., Acosta, S. I., Cassel, T., Ambadar, Z., & Cohn, J. (2005). Adult perceptions of positive and negative infant emotional expressions. Infancy, 8, 279–303.CrossRefGoogle Scholar
  12. Ekman, P. (1992). Telling lies. Cues to deceit in the marketplace, politics, and marriage. New York: WW. Norton & Company.Google Scholar
  13. Ekman, P. (2007). The directed facial action task: Emotional responses without appraisal. In J. A. Coan & J. B. Allen (Eds.), The handbook of emotion elicitation and assessment (pp. 47–53). New York: Oxford University Press.Google Scholar
  14. Ekman, P., Davidson, R. J., & Friesen, W. V. (1990). The Duchenne smile: Emotional expression and brain physiology II. Journal of Personality and Social Psychology, 58, 342–353.PubMedCrossRefGoogle Scholar
  15. Ekman, P., & Friesen, W. V. (1982). Felt, false, and miserable smiles. Journal of Nonverbal Behavior, 6, 238–252.CrossRefGoogle Scholar
  16. Ekman, P., Friesen, W. V., & Hager, J. (2002). Facial action coding system (FACS): A technique for the measurement of facial movement. Salt Lake City, UT: Research Nexus.Google Scholar
  17. Ekman, P., Friesen, W. V., & O’Sullivan, M. (1988). Smiles when lying. Journal of Personality and Social Psychology, 54, 414–420.PubMedCrossRefGoogle Scholar
  18. Ekman, P., Hager, J. C., & Friesen, W. V. (1981). The symmetry of emotional and deliberate facial actions. Psychophysiology, 18, 101–106.PubMedCrossRefGoogle Scholar
  19. Fernández-Dols, J. M., Wallbott, H., & Sanchez, F. (1991). Emotion category accessibility and the decoding of emotion from facial expression and context. Journal of Nonverbal Behavior, 15, 107–123.CrossRefGoogle Scholar
  20. Frank, M. G., Ekman, P., & Friesen, W. V. (1993). Behavioral markers and recognizability of the smile of enjoyment. Journal of Personality and Social Psychology, 64, 83–93.PubMedCrossRefGoogle Scholar
  21. Fridlund, A. J. (1991). Sociality of solitary smiling: Potentiation by an implicit audience. Journal of Personality and Social Psychology, 60, 229–240.CrossRefGoogle Scholar
  22. Garotti, P. L., Caterina, R., Brighetti, G., Giberti, E., & Ricci-Bitti, P. E. (1993). The role of the smile as an indicator of different emotional states. Rassegna di Psicologia, 10, 29–53.Google Scholar
  23. Gosselin, P., Perron, M., Legault, M., & Campanella, P. (2002). Children’s and adults’ knowledge of the distinction between enjoyment and nonenjoyment smiles. Journal of Nonverbal Behavior, 26, 83–108.CrossRefGoogle Scholar
  24. Haidt, J., & Keltner, D. (1999). Culture and facial expression: Open ended methods find more expressions and a gradient of recognition. Cognition and Emotion, 13, 225–266.CrossRefGoogle Scholar
  25. Harrigan, J. A., & Taing, K. T. (1997). Fooled by a smile: Detecting anxiety in others. Journal of Nonverbal Behavior, 21, 203–221.CrossRefGoogle Scholar
  26. Harris, C. R. (2001). Cardiovascular responses of embarrassment and effects of emotional suppression in a social setting. Journal of Personality and Social Psychology, 81, 886–897.PubMedCrossRefGoogle Scholar
  27. Harris, C. R., & Alvarado, N. (2005). Facial expressions, smile types, and self-report during humour, tickle, and pain. Cognition and Emotion, 19, 655–669.CrossRefGoogle Scholar
  28. Hess, U., Adams, R. B., & Kleck, R. E. (2007). Looking at you or looking elsewhere: The influence of head orientation on the signal value of emotional facial expressions. Motivation and Emotion, 31, 137–144.CrossRefGoogle Scholar
  29. Hess, U., Banse, R., & Kappas, A. (1995). The intensity of facial expression is determined by underlying affective state and social situation. Journal of Personality and Social Psychology, 69, 280–288.CrossRefGoogle Scholar
  30. Hess, U., Beaupré, M., & Cheung, N. (2002). Who to whom and why––cultural differences and similarities in the function of smiles. In M. H. Abel (Ed.), An empirical reflection on the smile (pp. 187–216). Lewiston, NY: The Edwin Mellen Press.Google Scholar
  31. Hess, U., Kappas, A., McHugo, G. J., Kleck, R. E., & Lanzetta, J. T. (1989). An analysis of the encoding and decoding of spontaneous and posed smiles: The use of facial electromyography. Journal of Nonverbal Behavior, 13, 121–137.CrossRefGoogle Scholar
  32. Hess, U., & Kleck, R. E. (1990). Differentiating emotion elicited and deliberate emotional facial expressions. European Journal of Social Psychology, 20, 369–385.CrossRefGoogle Scholar
  33. Hess, U., & Kleck, R. E. (1994). The cues decoders use in attempting to differentiate emotion-elicited and posed facial expressions. European Journal of Social Psychology, 24, 367–381.CrossRefGoogle Scholar
  34. Kanade, T., Cohn, J. F., & Tian, Y. (2000). Comprehensive database for facial expression analysis. Proceedings of the Fourth IEEE International Conference on Automatic Face and Gesture Recognition (FG’00) (pp. 46–53). Grenoble, France.Google Scholar
  35. Kappas, A., Hess, U., Barr, C. L., & Kleck, R. E. (1994). Angle of regard: The effect of vertical viewing angle on the perception of facial expressions. Journal of Nonverbal Behavior, 18, 263–280.CrossRefGoogle Scholar
  36. Keltner, D. (1995). Signs of appeasement: Evidence for the distinct displays of embarrassment, amusement, and shame. Journal of Personality and Social Psychology, 68, 441–454.CrossRefGoogle Scholar
  37. Kraut, R. E., & Johnston, R. E. (1979). Social and emotional messages of smiling: An ethological approach. Journal of Personality and Social Psychology, 37, 1539–1553.CrossRefGoogle Scholar
  38. Krull, D. S., & Dill, J. C. (1998). Do smiles elicit more inferences than do frowns? The effect of emotional valence on the production of spontaneous inferences. Personality and Social Psychology Bulletin, 24, 289–300.CrossRefGoogle Scholar
  39. Krumhuber, E., & Kappas, A. (2005). Moving smiles: The role of dynamic components for the perception of the genuineness of smiles. Journal of Nonverbal Behavior, 29, 3–24.CrossRefGoogle Scholar
  40. Krumhuber, E., Manstead, A. R., & Kappas, A. (2007). Temporal aspects of facial displays in person and expression perception: The effects of smile dynamics, head-tilt, and gender. Journal of Nonverbal Behavior, 29, 3–24.CrossRefGoogle Scholar
  41. Liu, Y., Schmidt, K. L., Cohn, J. F., & Mitra, S. (2003). Facial asymmetry quantification for expression invariant human identification. Computer Vision and Image Understanding, 91, 138–159.CrossRefGoogle Scholar
  42. Lyons, M. J., Campbell, R., Plante, A., Coleman, M., Kamachi, M., & Akamatsu, S. (2000). Viewpoint dependent facial expression recognition. Japanese Noh mask and human face. Paper Presented at the 22nd Annual Conference of the Cognitive Science Society, Philadelphia.Google Scholar
  43. McGee, A. M., & Skinner, M. (1987). Facial asymmetry and the attribution of personality traits. British Journal of Social Psychology, 26, 181–184.PubMedGoogle Scholar
  44. Merkel, K., Schmidt, K., Levenstein, R., VanSwearingen, J., & Bentley, B. (2007). Positive affect predicts improved lip movement in facial movement disorder. Otolaryngology - Head and Neck Surgery, 137, 100–104.PubMedCrossRefGoogle Scholar
  45. Messinger, D. S. (2002). Positive and negative: Infant facial expressions and emotions. Current Directions in Psychological Science, 11, 1–6.CrossRefGoogle Scholar
  46. Messinger, D. S., Fogel, A., & Dickson, K. L. (1999). What’s in a smile? Developmental Psychology, 35, 701–708.PubMedCrossRefGoogle Scholar
  47. Messinger, D. S., Fogel, A., & Dickson, K. L. (2001). All smiles are positive, but some smiles are more positive than others. Developmental Psychology, 37, 642–653.PubMedCrossRefGoogle Scholar
  48. Mignault, A., & Chaudhuri, A. (2003). The many faces of a neutral face: Head tilt and perception of dominance and emotion. Journal of Nonverbal Behavior, 27, 111–132.CrossRefGoogle Scholar
  49. Miles, L., & Johnston, L. (2007). Detecting happiness: Perceiver sensitivity to enjoyment and non-enjoyment smiles. Journal of Nonverbal Behavior, 31, 259–275.CrossRefGoogle Scholar
  50. Moriyama, T., Xiao, J., Cohn, J. F., & Kanade, T. (2006). Meticulously detailed eye region model and its application to analysis of facial images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28, 738–752.PubMedCrossRefGoogle Scholar
  51. Motley, M., & Camden, C. T. (1988). Facial expression of emotion: A comparison of posed expressions versus spontaneous expressions in an interpersonal communication setting. Western Journal of Speech Communication, 52, 1–22.Google Scholar
  52. Nijholt, A. (2002). Embodied agents: A new impetus to humor research. Paper Presented at the April Fools’ Day Workshop on Computational Humour, Trento, Italy.Google Scholar
  53. Otta, E., Folladore Abrosio, F., & Hoshino, R. L. (1996). Reading a smiling face: Messages conveyed by various forms of smiling. Perceptual and Motor Skills, 82, 1111–1121.PubMedGoogle Scholar
  54. Pollak, S. D., Cicchetti, D., Hornung, K., & Reed, A. (2000). Recognizing emotion in faces: Developmental effects of child abuse and neglect. Developmental Psychology, 36, 679–688.PubMedCrossRefGoogle Scholar
  55. Prkachin, K. M. (1992). The consistency of facial expressions of pain: A comparison across modalities. Pain, 51, 297–306.PubMedCrossRefGoogle Scholar
  56. Russell, J. A. (1993). Forced-choice response format in the study of facial expression. Motivation and Emotion, 17, 41–51.CrossRefGoogle Scholar
  57. Scharlemann, J. P. W., Eckel, C. C., Kacelnik, A., & Wilson, R. K. (2001). The value of a smile: Game theory with a human face. Journal of Economic Psychology, 22, 617–640.CrossRefGoogle Scholar
  58. Scheff, T. J. (1983). Toward integration in the social psychology of emotions. Annual Reviews of Sociology, 9, 333–354.CrossRefGoogle Scholar
  59. Schmidt, K. L., Ambadar, Z., & Cohn, J. F. (2005). Timing of lip corner movement affects perceived genuineness of smiles. University of Pittsburgh.Google Scholar
  60. Schmidt, K. L., Ambadar, Z., Cohn, J. F., & Reed, L. I. (2006a). Movement differences between deliberate and spontaneous facial expressions: Zygomaticus major action in smiling. Journal of Nonverbal Behavior, 30, 37–52.CrossRefGoogle Scholar
  61. Schmidt, K. L., & Cohn, J. F. (2001). Human facial expressions as adaptations: Evolutionary questions in facial expression research. American Journal of Physical Anthropology, 116, 3–24.CrossRefGoogle Scholar
  62. Schmidt, K. L., Liu, Y., & Cohn, J. F. (2006b). The role of structural facial asymmetry in asymmetry of peak facial expressions. Laterality, 11, 40–51.Google Scholar
  63. Skinner, M., & Mullen, B. (1991). Facial asymmetry in emotional expression: A meta-analysis of research. British Journal of Social Psychology, 30, 113–124.Google Scholar
  64. Van Swearingen, J. M., Cohn, J. F., & Bajaj-Luthra, A. (1999). Specific impairment of smiling increases severity of depressive symptoms in patients with facial neuromuscular disorders. Journal of Aesthetic Plastic Surgery, 23, 416–423.CrossRefGoogle Scholar
  65. Vrugt, A., Duijnhouwer, H., & Stam, O. (2004). The interpretation of smiling in different situations. Nederlands Tijdschrift voor de Psychologie en haar Grensgebieden, 59, 55–64.Google Scholar
  66. Wallbott, H. G. (1991). The robustness of communication of emotion via facial expression: Emotion recognition from photographs with deteriorated pictorial quality. European Journal of Social Psychology, 21, 89–98.CrossRefGoogle Scholar
  67. Williams, L. M., Senior, C., David, A. S., Loughland, C. M., & Gordon, E. (2001). In search of the “Duchenne Smile”: Evidence from eye movements. Journal of Psychophysiology, 15, 122–127.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2008

Authors and Affiliations

  • Zara Ambadar
    • 1
  • Jeffrey F. Cohn
    • 1
  • Lawrence Ian Reed
    • 1
  1. 1.Department of PsychologyUniversity of PittsburghPittsburghUSA

Personalised recommendations