Advertisement

Expressive Behaviors for Virtual Worlds

  • Stacy Marsella
  • Jonathan Gratch
  • Jeff Rickel
Part of the Cognitive Technologies book series (COGTECH)

Summary

A person’s behavior provides significant information about their emotional state, attitudes, and attention. Our goal is to create virtual humans that convey such information to people while interacting with them in virtual worlds. The virtual humans must respond dynamically to the events surrounding them, which are fundamentally influenced by users’ actions, while providing an illusion of human-like behavior. A user must be able to interpret the dynamic cognitive and emotional state of the virtual humans using the same non-verbal cues that people use to understand one another. Toward these goals, we are integrating and extending components from three prior systems: a virtual human architecture with a wide range of cognitive and motor capabilities, a model of task-oriented emotional appraisal and socially situated planning, and a model of how emotions and coping impact physical behavior. We describe the key research issues and approach in each of these prior systems, as well as our integration and its initial implementation in a leadership training system.

Keywords

Facial Expression Emotional State Virtual World Autonomous Agent Virtual Human 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Ambros-Ingerson, J.A., Steel, S.: Integrating planning, execution and monitoring. In: Proceedings of the Seventh National Conference on Artificial Intelligence (AAAI-88) ( Morgan Kaufmann, San Mateo, 1988 ) pp 83–88Google Scholar
  2. 2.
    Berkowitz, L.: Causes and Consequences of Feelings (Cambridge University Press 2000)CrossRefGoogle Scholar
  3. 3.
    Bindiganavale, R., Schuler, W., Allbeck, J.M., Badler, N.L, Joshi, A.K., Palmer, M.: Dynamically altering agent behaviors using natural language instructions. In: Proceedings of the Fourth International Conference on Autonomous Agents ( ACM Press, New York 2000 ) pp 293–300CrossRefGoogle Scholar
  4. 4.
    Cassell, J., Bickmore, T., Campbell, L., Vilhjálmsson, H., Yan, H.: Conversation as a system framework: Designing embodied conversational agents. Embodied Conversational Agents, ed Cassell, J., Sullivan, J., Prevost, S., Churchill, E. ( MIT Press, Cambridge MA 2000 )Google Scholar
  5. 5.
    Cassell, J., Pelachaud, C., Badler, N., Steedman, M., Achorn, B., Becket, T., Douville, B., Prevost, S., Stone, M.: Animated conversation: Rule-based generation of facial expression, gesture and spoken intonation for multiple conversational agents. In: Proceedings of ACM SIGGRAPH ’94 ( Addison-Wesley, Reading, MA 1994 ) pp 413–420CrossRefGoogle Scholar
  6. 6.
    Cassell, J., Sullivan, J., Prevost, S., Churchill, E. (eds): Embodied Conversational Agents ( MIT Press, Cambridge, MA 2000 )Google Scholar
  7. 7.
    Cassell, J., Thórisson, K.R.: The power of a nod and a glance: Envelope vs. emotional feedback in animated conversational agents. Applied Artificial Intelligence 13: 519–538 (1999)CrossRefGoogle Scholar
  8. 8.
    Cassell, J., Vilhjálmsson, H., Bickmore, T.: BEAT: The behavior expression animation toolkit. In: Proceedings of ACM SIGGRAPH (ACM Press, New York 2001) pp 477–486 (Reprint in this volume)Google Scholar
  9. 9.
    Chopra-Khullar, S., Badler, N.I.: Where to look? Automating attending behaviors of virtual human characters. Autonomous Agents and Multi-Agent Systems 4 (1–2): 9–23 (2001)CrossRefGoogle Scholar
  10. 10.
    Cohen, P.R., Levesque, H.J.: Teamwork. Nous 25 (4): 487–512 (1991)CrossRefGoogle Scholar
  11. 11.
    des Jardins, M., Wolverton, M.J.: Coordinating a distributed planning system. AI Magazine 20(4):45–53 (Winter 1999)Google Scholar
  12. 12.
    Duncan Jr., S.: Some signals and rules for taking speaking turns in conversations. In: Nonverbal Communication, ed Weitz, S. (Oxford University Press 1974 ) pp 298–311Google Scholar
  13. 13.
    Elliott, C.: The Affective Reasoner: A Process Model of Emotions in a Multiagent System. PhD thesis (Northwestern University 1992 )Google Scholar
  14. 14.
    Fleischman, M., Hovy, E.: Emotional variation in speech-based natural language generation. In: Proceedings of the International Natural Language Generation Conference ( Arden House, New York 2002 )Google Scholar
  15. 15.
    Freedman, N.: The analysis of movement behavior during the clinical interview. In: Studies in Dyadic Communication, ed Siegman, A.W., Pope, B. ( Pergamon Press, New York 1972 ) pp 177–210Google Scholar
  16. 16.
    Gratch, J.: Emile: Marshalling passions in training and education. In: Proceedings of the Fourth International Conference on Autonomous Agents ( ACM Press, New York 2000 ) pp 325–332CrossRefGoogle Scholar
  17. 17.
    Gratch, J.: Socially situated planning. In: Socially Intelligent Agents: Creating Relationships with Computers and Robots, ed Cañamero, L., Dautenhahn, K., Bond, A.H., Edmonds, B. ( Kluwer Academic, Dordrecht 2002 ) pp 181–188CrossRefGoogle Scholar
  18. 18.
    Gratch, J., Marsella, S.: Tears and fears: Modeling emotions and emotional behaviors in synthetic agents. In: Proceedings of the Fifth International Conference on Autonomous Agents ( ACM Press, New York 2001 ) pp 278–285CrossRefGoogle Scholar
  19. 19.
    Grosz, B.J., Kraus, S.: Collaborative plans for complex group action. Artificial Intelligence 86 (2): 269–357 (1996)MathSciNetCrossRefGoogle Scholar
  20. 20.
    Grosz, B.J., Sidner, C.L.: Attention, intentions, and the structure of discourse. Computational Linguistics 12 (3): 175–204 (1986)Google Scholar
  21. 21.
    Johnson, W.L., Rickel, J.W., Lester, J.C.: Animated pedagogical agents: Face-to-face interaction in interactive learning environments. International Journal of Artificial Intelligence in Education 11: 47–78 (2000)Google Scholar
  22. 22.
    Johnson, W.L., Narayanan, S., Whitney, R., Das, R., Bulut, M., LaBore, C.: Limited domain synthesis of expressive military speech for animated characters. In: Proceedings of the IEEE Workshop on Speech Synthesis, Santa Monica, CA (2002)Google Scholar
  23. 23.
    Kambhampati, S., Knoblock, C.A., Yang, Q.: Planning as refinement search: A unified framework for evaluating design tradeoffs in partial-order planning. Artificial Intelligence 76: 167–238 (1995)CrossRefGoogle Scholar
  24. 24.
    Laird, J.E., Newell, A., Rosenbloom, P.S.: Soar: An architecture for general intelligence. Artificial Intelligence 33 (1): 1–64 (1987)MathSciNetCrossRefGoogle Scholar
  25. 25.
    Lazarus, R.S.: Emotion and Adaptation (Oxford University Press 1991)Google Scholar
  26. 26.
    Lester, J.C., Towns, S.G., Callaway, C.B., Voerman, J.L., FitzGerald, P.J.: Deictic and emotive communication in animation pedagogical agents. In: Embodied Conversational Agents, ed Cassell, J., Sullivan, J., Prevost, S., Churchill, E. ( MIT Press, Cambridge, MA 2000 )Google Scholar
  27. 27.
    Lester, J.C., Voerman, J.L., Towns, S.G., Callaway, C.B.: Deictic believability: Coordinating gesture, locomotion, and speech in lifelike pedagogical agents. Applied Artificial Intelligence 13: 383–414 (1999)CrossRefGoogle Scholar
  28. 28.
    Lester, J.C., Zettlemoyer, L.S., Gregoire, J., Bares, W.H.: Explanatory lifelike avatars: Performing user-designed tasks in 3d learning environments. In: Proceedings of the Third International Conference on Autonomous Agents ( ACM Press, New York 1999 )Google Scholar
  29. 29.
    Marsella, S., Gratch, J.: Modeling coping behavior in virtual humans: Don’t worry, be happy. In: Proceedings of the Second International Joint Conference on Autonomous Agents and Multi-Agent Systems ( ACM Press, New York 2003 )Google Scholar
  30. 30.
    Marsella, S., Johnson, W.L., LaBore, C.: Interactive pedagogical drama for health interventions. In: Proceedings of the 11th International Conference on Artificial Intelligence in Education (IOS Press 2003 )Google Scholar
  31. 31.
    Marsella, S.C., Johnson, W.L., LaBore, C.: Interactive pedagogical drama. In: Proceedings of the Fourth International Conference on Autonomous Agents ( ACM Press, New York 2000 ) pp 301–308CrossRefGoogle Scholar
  32. 32.
    Matheson, C., Poesio, M., Traum, D.: Modelling grounding and discourse obligations using update rules. In: Proceedings of the First Conference of the North American Chapter of the Association for Computational Linguistics (2000)Google Scholar
  33. 33.
    Neal Reilly, W.S.: Believable Social and Emotional Agents. PhD thesis (School of Computer Science, Carnegie Mellon University 1996), Technical Report CMUCS-96-138Google Scholar
  34. 34.
    Newell, A.: Unified Theories of Cognition (Harvard University Press 1990)Google Scholar
  35. 35.
    Ortony, A., Clore, G.L., Collins, A.: The Cognitive Structure of Emotions (Cambridge University Press 1988 )CrossRefGoogle Scholar
  36. 36.
    Pelachaud, C., Badler, N.I., Steedman, M.: Generating facial expressions for speech. Cognitive Science 20 (1): 1–46 (1996)CrossRefGoogle Scholar
  37. 37.
    Pelachaud, C., Carofiglio, V., De Carolis, B., de Rosis, F., Poggi, I.: Embodied contextual agent in information delivering application. In: Proceedings of the First International Joint Conference on Autonomous Agents and Multi-Agent Systems ( ACM Press, New York 2001 ) pp 758–765Google Scholar
  38. 38.
    Penley, J., Tomaka, J.: Associations among the big five, emotional responses, and coping with acute stress. Personality and Individual Differences 32: 1215–1228 (2002)CrossRefGoogle Scholar
  39. 39.
    Poggi, I., Pelachaud, C.: Emotional meaning and expression in performative faces. In: International Workshop on Affect in Interactions: Towards a New Generation of Interfaces, Siena, Italy (1999)Google Scholar
  40. 40.
    Rickel, J., Johnson, W.L.: Animated agents for procedural training in virtual reality: Perception, cognition, and motor control. Applied Artificial Intelligence 13: 343–382 (1999)CrossRefGoogle Scholar
  41. 41.
    Rickel, J., Johnson, W.L.: Extending virtual humans to support team training in virtual reality. In: Exploring Artificial Intelligence in the New Millenium, ed Lakemayer, G., Nebel, B. ( Morgan Kaufmann, San Francisco 2002 ) pp 217–238Google Scholar
  42. 42.
    Rickel, J., Johnson, W.L.: Task-oriented collaboration with embodied agents in virtual worlds. In: Embodied Conversational Agents, ed Cassell, J., Sullivan, J., Prevost, S., Churchill, E. ( MIT Press, Cambridge, MA 2000 )Google Scholar
  43. 43.
    Rickel, J., Johnson, W.L.: Virtual humans for team training in virtual reality. In: Proceedings of the Ninth International Conference on Artificial Intelligence in Education ( IOS Press, Amsterdam 1999 ) pp 578–585Google Scholar
  44. 44.
    Rickel, J., Marsella, S., Gratch, J., Hill, R., Traum, D., Swartout, W.: Toward a new generation of virtual humans for interactive experiences. IEEE Intelligent Systems 17 (4): 32–38 (2002)CrossRefGoogle Scholar
  45. 45.
    Scherer, K.R.: Appraisal considered as a process of multilevel sequential checking. In: Appraisal Processes in Emotion: Theory, Methods, and Research, ed Scherer, K.R., Schorr, A., Johnstone, T. (Oxford University Press 2001 ) pp 92–120Google Scholar
  46. 46.
    Swartout, W., Hill, R., Gratch, J., Johnson, W.L., Kyriakakis, C., LaBore, C., Lindheim, R., Marsella, S., Miraglia, D., Moore, B., Morie, J., Rickel, J., Thiébaux, M., Tuch, L., Whitney, R., Douglas, J.: Toward the Holodeck: Integrating graphics, sound, character and story. In: Proceedings of the Fifth International Conference on Autonomous Agents (ACM Press, New York 2001) pp 409–416Google Scholar
  47. 47.
    Traum, D., Rickel, J.: Embodied agents for multi-party dialogue in immersive virtual worlds. In: Proceedings of the First International Joint Conference on Autonomous Agents and Multi-Agent Systems ( ACM Press, New York 2002 ) pp 766–773CrossRefGoogle Scholar
  48. 48.
    Traum, D., Rickel, J., Gratch, J., Marsella, S.: Negotiation over tasks in hybrid human-agent teams for simulation-based training. In: Proceedings of the Second International Joint Conference on Autonomous Agents and Multi-Agent Systems ( ACM Press, New York 2003 )Google Scholar
  49. 49.
    Traum, D.R.: A Computational Theory of Grounding in Natural Language Conversation. PhD thesis (Department of Computer Science, University of Rochester, 1994 )Google Scholar
  50. 50.
    Traum, D.R., Hinkelman, E.A.: Conversation acts in task-oriented dialogue. Computational Intelligence 8 (3): 575–599 (1992)CrossRefGoogle Scholar
  51. 51.
    Weld, D.S.: An introduction to least commitment planning. AI Magazine 15 (4): 27–61 (1994)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2004

Authors and Affiliations

  • Stacy Marsella
    • 1
  • Jonathan Gratch
    • 2
  • Jeff Rickel
    • 1
  1. 1.USC Information Sciences InstituteMarina del ReyUSA
  2. 2.USC Institute for Creative TechnologiesMarina del ReyUSA

Personalised recommendations