Front View vs. Side View of Facial and Postural Expressions of Emotions in a Virtual Character

  • Matthieu Courgeon
  • Céline Clavel
  • Ning Tan
  • Jean-Claude Martin
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6758)


In 3D virtual environments and social scenes, virtual characters are not always visible from a front view. Nonverbal expressions of emotions should be designed so as to be properly perceived when viewed from multiple angles. Our platform enables the design of facial expressions including 3D expressive wrinkles and 3D postures. It was used to generate stimuli for studying the impact of front versus side views on the perception of blends of facial and postural expressions of emotions. Results observed in previous studies for front views of incongruent images are confirmed for side views. Furthermore, subjects were less confident in their ratings for side views images than for front view images.


facial expressions posture emotions evaluation study 3D rendering 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Ekman, P., Friesen, W.V.: Unmasking the face. A guide to recognizing emotions from facial clues. Prentice-Hall Inc. Englewood Cliffs (1975)Google Scholar
  2. 2.
    Russell, J.A., Mehrabian, A.: Evidence for a three-factor theory of emotions. Journal of Research on Personality 11(3), 273–294 (1977)CrossRefGoogle Scholar
  3. 3.
    Scherer, K.R.: Analyzing emotion blends. In: 10th Conf. of the Int. Society for Research on Emotions, pp. 142–148 (1998)Google Scholar
  4. 4.
    Scherer, K.R.: Emotion and emotional competence: conceptual and theoretical issues for modelling agents. In: Scherer, K.R., Bänziger, T., Roesch, E. (eds.) Blueprint for Affective Computing, pp. 3–20. Oxford University Press, Oxford (2010)Google Scholar
  5. 5.
    Golan, O., Baron-Cohen, S., Hill, J.: The Cambridge Mindreading (CAM) face-voice battery: testing complex emotion recognition in adults with and without asperger syndrome. Journal of Autism and Developmental Disorders 36(2), 169–183 (2006)CrossRefGoogle Scholar
  6. 6.
    Ekman, P., Friesen, W.V.: Nonverbal behavior and psychopathology. In: Friedman, R.J., Datz, M.M. (eds.) The Psychology of Depression: Contemporary Theory and Research, pp. 203–232. Winston & Sons, Washington (1974)Google Scholar
  7. 7.
    Wallbott, H.G.: Bodily expression of emotion. European Journal of Social Psychology 28, 879–896 (1998)CrossRefGoogle Scholar
  8. 8.
    Kipp, M., Martin, J.C.: Gesture and emotion: can basic gestural form features discriminate emotions? In: Int. Conf. on Affective Computing and Intelligent Interaction. IEEE Press, New York (2009)Google Scholar
  9. 9.
    Coulson, M.: Attributing emotion to static body postures: recognition accuracy, confusions, and viewpoint dependence. Journal of Nonverbal Behavior 28(2), 117–139 (2004)CrossRefMathSciNetGoogle Scholar
  10. 10.
    Kleinsmith, A., de Silva, R., Bianchi-Berthouze, N.: Cross-cultural differences in recognizing affect from body posture. Interacting with Computers 18(6), 1371–1389 (2006)CrossRefGoogle Scholar
  11. 11.
    Bänziger, T., Scherer, K.R.: Using actor portrayals to systematically study multimodal emotion expression: The GEMEP corpus. In: Paiva, A.C.R., Prada, R., Picard, R.W. (eds.) ACII 2007. LNCS, vol. 4738, pp. 476–487. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  12. 12.
    Pantic, M., Patras, I.: Dynamics of facial expression: recognition of facial actions and their temporal segments from face profile image sequences. IEEE Transactions on Systems, Man and Cybernetics 36(2), 433–449 (2006)CrossRefGoogle Scholar
  13. 13.
    Maalej, A., Amor, B.B., Daoudi, M., Srivastava, A., Berretti, S.: 3D face analysis for facial expression recognition. In: Int. Conf. on Pattern Recognition (2010)Google Scholar
  14. 14.
    Courgeon, M., Buisine, S., Martin, J.C.: Impact of expressive wrinkles on perception of a virtual character’s facial expressions of emotions. In: Ruttkay, Z., Kipp, M., Nijholt, A., Vilhjálmsson, H.H. (eds.) IVA 2009. LNCS, vol. 5773, pp. 201–214. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  15. 15.
    de Melo, C., Kenny, P., Gratch, J.: Influence of autonomic signals on perception of emotions in embodied agents. Applied Artificial Intelligence 24(6), 494–509 (2010)CrossRefGoogle Scholar
  16. 16.
    Huang, Y., Fan, H., Li, Y.: Robust symbolic dual-view facial expression recognition with skin wrinkles: local versus global approach. IEEE Transactions on Multimedia 12(6), 536–543 (2010)CrossRefGoogle Scholar
  17. 17.
    Zhang, Y., Sim, T.: Realistic and efficient wrinkle simulation using an anatomy-based face model with adaptive refinement. In: Proc. Computer Graphics International 2005, pp. 3–10 (2005)Google Scholar
  18. 18.
    Bickel, B., Botsch, M., Angst, R., Matusik, W., Otaduy, M., Pfister, H., Gross, M.: Multi-Scale Capture of Facial Geometry and Motion. ACM Transactions on Graphics 26(3) (2007)Google Scholar
  19. 19.
    Hyniewska, S., Niewiadomski, R., Mancini, M., Pelachaud, C.: Expression of affects in embodied conversational agents. In: Scherer, K.R., Bänziger, T., Roesch, E. (eds.) Blueprint for Affective Computing, pp. 213–221. Oxford University Press, Oxford (2010)Google Scholar
  20. 20.
    Gunes, H., Piccardi, M.: Fusing face and body display for bi-modal emotion recognition: Single frame analysis and multi-frame post integration. In: Tao, J., Tan, T., Picard, R.W. (eds.) ACII 2005. LNCS, vol. 3784, pp. 102–111. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  21. 21.
    Hietanen, J.K., Leppänen, J.M.: Judgment of other people’s facial expressions of emotions is influenced by their concurrent affective hand movements. Scandinavian Journal of Psychology 49, 221–230 (2008)CrossRefGoogle Scholar
  22. 22.
    Meeren, H.K.M., van Heijnsbergen, C., de Gelder, B.: Rapid perceptual integration of facial expression and emotional body language. In: Proc. of the National Academy of Sciences of the USA, vol. 102(45), pp. 16518–16523 (2005)Google Scholar
  23. 23.
    de Gelder, B., Van den Stock, J.: Real faces, real emotions: perceiving facial expressions in naturalistic contexts of voices, bodies and scenes. In: Calder, A.J., Rhodes, G., Haxby, J.V., Johnson, M.H. (eds.) The Handbook of Face Perception. Oxford University Press, Oxford (in press)Google Scholar
  24. 24.
    Clavel, C., Plessier, J., Martin, J.C., Ach, L., Morel, B.: Combining facial and postural expressions of emotions in a virtual character. In: Ruttkay, Z., Kipp, M., Nijholt, A., Vilhjálmsson, H.H. (eds.) IVA 2009. LNCS, vol. 5773, pp. 287–300. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  25. 25.
    Courgeon, M., Martin, J.C., Jacquemin, C.: MARC: a Multimodal Affective and Reactive Character. In: Proceedings of the ICMI 2008 Workshop on Affective Interaction in Natural Environments (2008)Google Scholar
  26. 26.
    Pandzic, I., Forchheimer, R.: MPEG-4 facial animation: the standard, implementation and applications. John Wiley and Sons Inc., West Sussex (2003)Google Scholar
  27. 27.
    Ekman, P.: Emotions revealed. Weidenfeld & Nicolson (2003)Google Scholar
  28. 28.
    Fratarcangeli, M., Schaerf, M.: Realistic modeling of animatable faces in MPEG-4. Computer Animation and Social Agents. Computer Graphics Society (CGS) (2004)Google Scholar
  29. 29.
    Courgeon, M., Clavel, C., Martin, J.C.: Appraising emotional events during a real-time interactive game. In: ICMI 2009: Int. Workshop on Affective-Aware Virtual Agents and Social Robots (2009)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Matthieu Courgeon
    • 1
    • 2
  • Céline Clavel
    • 1
  • Ning Tan
    • 1
    • 2
  • Jean-Claude Martin
    • 1
    • 2
  1. 1.LIMSI-CNRSOrsayFrance
  2. 2.University Paris South XIFrance

Personalised recommendations