Skip to main content
SpringerLink
Log in
Book cover

Modeling Communication with Robots and Virtual Humans pp 181–197Cite as

True Emotion vs. Social Intentions in Nonverbal Communication: Towards a Synthesis for Embodied Conversational Agents

  • Conference paper

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 4930))

Abstract

Does a facial expression convey privileged information about a person’s mental state or is it a communicative act, divorced from “true” beliefs, desires and intentions? This question is often cast as a dichotomy between competing theoretical perspectives. Theorists like Ekman argue for the primacy of emotion as a determinant of nonverbal behavior: emotions “leak” and only indirectly serve social ends. In contrast, theorists such as Fridlund argue for the primacy of social ends in determining nonverbal displays. This dichotomy has worked to divide virtual character research. Whereas there have been advances in modeling emotion, this work is often seen as irrelevant to the generation of communicative behavior. In this chapter, I review current findings on the interpersonal function of emotion. I’ll discuss recent developments in Social Appraisal theory as a way to bridge this dichotomy and our attempts to model these functions within the context of embodied conversational agents.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Smith, C.A., Lazarus, R.: Emotion and Adaptation, in Handbook of Personality: theory & research. In: Pervin, L.A. (ed.), pp. 609–637. Guilford Press, New York (1990)

    Google Scholar 

  2. Ekman, P.: Universals and cultural differences in facial expressions of emotions. In: Cole, J. (ed.) Nebraska Symposium on Motivation, pp. 207–283. University of Nebraska Press, Lincoln, Nebraska (1972)

    Google Scholar 

  3. Frank, R.H.: Introducing Moral Emotions into Models of Rational Choice. In: Manstead, A., Frijda, N., Fischer, A. (eds.) Feelings and Emotions, pp. 422–440. Cambridge University Press, Cambridge (2004)

    Google Scholar 

  4. Fridlund, A.: Human facial expression: An evolutionary view. Academic Press, San Diego (1994)

    Google Scholar 

  5. Chovil, N.: Social determinants of facial displays. Journal of Nonverbal Behavior 15, 163–167 (1991)

    Article  Google Scholar 

  6. Krämer, N.C., Iurgel, I., Bente, G.: Emotion and motivation in embodied conversational agents. In: Proceedings of the Symposium “Agents that Want and Like”, Artificial Intelligence and the Simulation of Behavior (AISB). Hatfield, SSAISB (2005)

    Google Scholar 

  7. Cassell, J., et al.: Human conversation as a system framework: Designing embodied conversational agents. In: Cassell, J., et al. (eds.) Embodied Conversational Agents, pp. 29–63. MIT Press, Boston (2000)

    Google Scholar 

  8. Heylen, D.: Challenges Ahead. Head Movements and other social acts in conversation. In: AISB. 2005, Hertfordshire, UK (2005)

    Google Scholar 

  9. Poggi, I., Pelachaud, C.: Emotional Meaning and Expression in Performative Faces. In: Paiva, A. (ed.) Affective Interactions: Towards a New Generation of Computer Interfaces, pp. 182–195. Springer, Berlin (2000)

    Google Scholar 

  10. Lester, J.C., Towns, S.G., FitzGerald, P.J.: Achieving Affective Impact: Visual Emotive Communication in Lifelike Pedagogical Agents. International Journal of Artificial Intelligence in Education 10(3-4), 278–291 (1999)

    Google Scholar 

  11. Elliott, C., Rickel, J., Lester, J.: Lifelike Pedagogical Agents and Affective Computing: An Exploratory Synthesis. In: Wooldridge, M., Veloso, M. (eds.) Artificial Intelligence Today: Recent Trends and Developments, pp. 195–212. Springer, Berlin/Heidelberg (1999)

    Google Scholar 

  12. Lisetti, C., Gmytrasiewicz, P.: Can a rational agent afford to be affectless? A formal approach. Applied Artificial Intelligence 16, 577–609 (2002)

    Article  Google Scholar 

  13. Scheutz, M., Sloman, A.: Affect and agent control: Experiments with simple affective states. In: IAT. 2001, World Scientific Publisher, Singapore (2001)

    Google Scholar 

  14. Blanchard, A., Cañamero, L.: Developing Affect-modulated behaviors: stability, exploration, exploitation, or imitation? in 6th International Workshop on Epigenetic Robotics, Paris (2006)

    Google Scholar 

  15. Marsella, S., Gratch, J.: Modeling coping behaviors in virtual humans: Don’t worry, be happy. In: Second International Joint Conference on Autonomous Agents and Multi-agent Systems, Melbourne, Australia (2003)

    Google Scholar 

  16. Neal Reilly, W.S.: Believable Social and Emotional Agents, Carnegie Mellon University: Pittsburgh, PA (1996)

    Google Scholar 

  17. Smith, C.A., David, B., Kirby, L.D.: Emotion-Eliciting Appraisals of Social Situations. In: Forgas, J. (ed.) Affective Influences on Social Behavior, Psychology Press, New York (2006)

    Google Scholar 

  18. Parkinson, B.: Putting appraisal in context. In: Scherer, K., Schorr, A., Johnstone, T. (eds.) Appraisal processes in emotion: Theory, methods, research, pp. 173–186. Oxford University Press, London (2001)

    Google Scholar 

  19. Keltner, D., Haidt, J.: Social Functions of Emotions at Four Levels of Anysis. Cognition and Emotion 13(5), 505–521 (1999)

    Article  Google Scholar 

  20. Barrett, K.C.: A functionalist approach to shame and guilt. In: Tangney, J.P., Fischer, K.W. (eds.) Self-conscious emotions: The psychology of shame, guilt, embarrassment, and pride, pp. 25–63. Guilford Publications, New York (1995)

    Google Scholar 

  21. Spoor, J.R., Kelly, J.R.: The evolutionary significance of affect in groups: communication and group bonding. Group Processes and Intergroup Prelations 7(4), 398–412 (2004)

    Article  Google Scholar 

  22. de Waal, F.: Primates and Philosophers: How Morality Evolved. In: Macedo, S., Ober, J. (eds.), Princeton University Press, Princeton (2006)

    Google Scholar 

  23. Frank, R.: Passions with reason: the strategic role of the emotions. W. W. Norton, New York (1988)

    Google Scholar 

  24. Lutz, C., White, G.M.: The anthropology of emotions. Annual Review of Anthropology 15, 405–436 (1986)

    Article  Google Scholar 

  25. Clore, G.L., Storbeck, J.: Affect as Information about Liking, Efficacy, and Importance. In: Forgas, J. (ed.) Affective Influences on Social Behavior, Psychology Press, New York (2006)

    Google Scholar 

  26. Tiedens, L.Z.: The Effect of Anger on the Hostile Inferences of Aggressive and Nonaggressive People: Specific Emotions. Cognitive Processing, and Chronic Accessibility Motivation and Emotion 25(3) (2001)

    Google Scholar 

  27. Zak, P.J.: Neuroeconomics. In: Philosophical Transactions of the Royal Society Biology, vol. 359, pp. 1737–1748 (2004)

    Google Scholar 

  28. Izard, C.: Human emotion. Plenum, New York (1977)

    Google Scholar 

  29. Klauer, K.C., Musch, J.: Affective priming: Findings and theories. In: Klauer, K.C., Musch, J., Mahwah, N.J. (eds.) The Psychology of Evaluation: Affective Processes in Cognition and Emotion, pp. 7–49. Lawrence Erlbaum, Mahwah (2003)

    Google Scholar 

  30. Dimberg, U., Öhman, A.: Beyond the wrath: Psychophysiological responses to facial stimuli. Motivation and Emotion 20, 149–182 (1996)

    Article  Google Scholar 

  31. Emde, R.N., Gaensbauer, T.J., Harmon, R.J.: Emotional expression in infancy: A biobehavioral study. Psychological Issues 10 (1976) (1, Monograph No. 37)

    Google Scholar 

  32. Eisenberg, N., et al.: Sympathy and personal distress: Development, gender differences, and interrelations of indexes. In: Eisenberg, N. (ed.) Empathy and related emotional responses, pp. 107–126. Jossey-Bass, San Franciso (1989)

    Google Scholar 

  33. Haviland, J.M., Lelwica, M.: The induced affect response: 10-week-old infants’ responses to three emotion expressions. Developmental Psychology 23, 97–104 (1987)

    Article  Google Scholar 

  34. Brosig, J.: Identifying cooperative behavior: some experimental results in a prisoner’s dilemma game. Journal of Economic Behavior and Organization 47, 275–290 (2002)

    Article  Google Scholar 

  35. Frank, R.H., Gilovich, T., Regan, D.T.: The evolution of one-shot cooperation: an experiment. Ethology and Sociobiology 14, 247–256 (1993)

    Article  Google Scholar 

  36. Dunn, J.: Children as psychologists: The later correlates of individual differences in understanding of emotions and other minds. Cognition and Emotion 9, 187–201 (1995)

    Article  MathSciNet  Google Scholar 

  37. Harris, P.L.: Children and Emotion. Blackwell, Oxford (1989)

    Google Scholar 

  38. Eder, R.A.: Uncovering young children’s psychological selves: Individual and developmenal differences. Child Development 61, 849–863 (1990)

    Article  Google Scholar 

  39. Phillips, M.L.: Facial processing deficits and social dysfunction: how are they related? Brain 127(8), 1691–1692 (2004)

    Article  Google Scholar 

  40. Parkinson, B.: Do facial movements express emotion or communicate motives? Personality and Social Psychology Review 9(4), 278–311 (2005)

    Article  Google Scholar 

  41. Manstead, A., Fischer, A.H., Jakobs, E.B.: The Social and Emotional Functions of Facial Displays. In: Philippot, P., Feldman, R.S., Coats, E.J. (eds.) The Social Context of Nonverbal Behavior (Studies in Emotion and Social Interaction), pp. 287–316. Cambridge Univ Press, Cambridge (1999)

    Google Scholar 

  42. Fernández Dols, J.M., Ruiz-Belda, M.-A.: Spontaneous facial behavior during intense emotional episodes: artistic truth and optical truth. In: Russell, J.A., Fernández Dols, J.M. (eds.) The Psychology of Facial Expression (1997)

    Google Scholar 

  43. Scherer, K.R., Schorr, A., Johnstone, T.: Appraisal Processes in Emotion. In: Davidson, R.J., Ekman, P., Scherer, K.R. (eds.) Affective Science, Oxford University Press, Oxford (2001)

    Google Scholar 

  44. Parkinson, B., Fischer, A.H., Manstead, A.S.R.: Emotion in social relations: Cultural, group, and interpersonal processes. Psychology Press, New York (2005)

    Google Scholar 

  45. Tangney, J.P.: Assessing individual differences in proneness to shame and guilt: development of the self-conscious affect and attribution inventory. Journal of Personality and Social Psychology 59, 102–111 (1990)

    Article  Google Scholar 

  46. Leary, M.R., Landel, J.L., Patton, K.M.: The motivated expression of embarrassment following a self-presentational predicament. Journal of Personality and Social Psychology 64, 619–636 (1996)

    Google Scholar 

  47. Gratch, J., Marsella, S.: Tears and Fears: Modeling Emotions and Emotional Behaviors in Synthetic Agents. In: Fifth International Conference on Autonomous Agents, ACM Press, Montreal, Canada (2001)

    Google Scholar 

  48. Gratch, J., Marsella, S.: A domain independent framework for modeling emotion. Journal of Cognitive Systems Research 5(4), 269–306 (2004)

    Article  Google Scholar 

  49. Marsella, S., Gratch, J., Rickel, J.: Expressive Behaviors for Virtual Worlds. In: Prendinger, H., Ishizuka, M. (eds.) Life-like Characters Tools, Affective Functions and Applications, pp. 317–360. Springer, Berlin (2003)

    Google Scholar 

  50. Swartout, W., et al.: Toward Virtual Humans. AI Magazine 27(1) (2006)

    Google Scholar 

  51. Rickel, J., et al.: Toward a New Generation of Virtual Humans for Interactive Experiences. In: IEEE Intelligent Systems, pp. 32–38 (2002)

    Google Scholar 

  52. Weiner, B.: Responsibility for Social Transgressions: An Attributional Analysis. In: Malle, B.F., Moses, L.J., Baldwin, D.A. (eds.) Intentions and Intentionality: Foundations of Social Cognition, The MIT Press, Cambridge (2001)

    Google Scholar 

  53. Shaver, K.G.: The attribution of blame: Causality, responsibility, and blameworthiness. Springer, Heidelberg (1985)

    Google Scholar 

  54. Mao, W., Gratch, J.: Evaluating a computational model of social causality and responsibility. In: 5th International Joint Conference on Autonomous Agents and Multiagent Systems, Hakodate, Japan (2006)

    Google Scholar 

  55. Mao, W., Gratch, J.: Social Judgment in Multiagent Interactions. In: Third International Joint Conference on Autonomous Agents and Multiagent Systems, New York (2004)

    Google Scholar 

  56. Austin, J.: How to Do Things with Words. Harvard University Press (1962)

    Google Scholar 

  57. Brown, P., Levenson, S.C.: Politeness: Some Universals in Language Usage. Cambridge University Press, New York (1987)

    Google Scholar 

  58. Martinovski, B., et al.: Mitigation Theory: An integrated approach. In: Cognitive Science (2005)

    Google Scholar 

  59. Shannon, C.E., Weaver, W.: A mathematical theory of communication. University of Illinois Press, Urbana (1948)

    Google Scholar 

  60. Fogel, A.: Developing through relationships: Origins of communication, self and culture. Harvester Wheatsheaf, New York (1993)

    Google Scholar 

  61. Agre, P., Chapman, D.: Pengi: An implementation of a theory of activity. In: National Conference on Artificial Intelligence (1987)

    Google Scholar 

  62. Suchman, L.A.: Plans and situated actions: The problem of human-machine communication. Cambridge University Press, New York (1987)

    Google Scholar 

  63. Simon, H.: The sciences of the Artificial. MIT Press, Cambridge (1969)

    Google Scholar 

  64. Jondottir, G.R., et al.: Fluid Semantic Back-Channel Feedback in Dialogue: Challenges and Progress. In: Intelligent Virtual Agents, Paris, France, Springer, Heidelberg (2007)

    Google Scholar 

  65. Ward, N., Tsukahara, W.: Prosodic features which cue back-channel responses in English and Japanese. Journal of Pragmatics 23, 1177–1207 (2000)

    Article  Google Scholar 

  66. Bartlett, M.S., et al.: Fully automatic facial action recognition in spontaneous behavior. In: 7th International Conference on Automatic Face and Gesture Recognition, Southampton, UK (2006)

    Google Scholar 

  67. Morency, L.-P., et al.: Contextual Recognition of Head Gestures. In: 7th International Conference on Multimodal Interactions, Torento, Italy (2005)

    Google Scholar 

  68. Gratch, J., et al.: Virtual Rapport. In: Gratch, J., et al. (eds.) IVA 2006. LNCS (LNAI), vol. 4133, Springer, Heidelberg (2006)

    Google Scholar 

  69. Breazeal, C., Aryananda, L.: Recognition of Affective Communicative Intent in Robot-Directed Speech. Autonomous Robots 12, 83–104 (2002)

    Article  MATH  Google Scholar 

  70. Gratch, J., et al.: Creating Rapport with Virtual Agents. In: 7th International Conference on Intelligent Virtual Agents, Paris, France (2007)

    Google Scholar 

  71. Burleson, W.: Affective Learning Companions: Strategies for Empathetic Agents with Real-Time Multimodal Affective Sensing to Foster Meta-Cognitive and Meta-Affective Approaches to Learning, Motivation, and Perseverance, Unpublished PhD Thesis, MIT Media Lab: Boston (2006)

    Google Scholar 

  72. Wang, N., et al.: Experimental evaluation of polite interaction tactics for pedagogical agents. In: Intelligent User Interfaces (2005)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Southern California,  

    Jonathan Gratch

Authors
  1. Jonathan Gratch
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Ipke Wachsmuth Günther Knoblich

Rights and permissions

Reprints and permissions

Copyright information

© 2008 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Gratch, J. (2008). True Emotion vs. Social Intentions in Nonverbal Communication: Towards a Synthesis for Embodied Conversational Agents. In: Wachsmuth, I., Knoblich, G. (eds) Modeling Communication with Robots and Virtual Humans. Lecture Notes in Computer Science(), vol 4930. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-79037-2_10

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-79037-2_10

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-79036-5

  • Online ISBN: 978-3-540-79037-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation