Skip to main content
SpringerLink
Log in

Affect Detection and an Automated Improvisational AI Actor in E-Drama

  • Conference paper
Artifical Intelligence for Human Computing

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 4451))

Abstract

Enabling machines to understand emotions and feelings of the human users in their natural language textual input during interaction is a challenging issue in Human Computing. Our work presented here has tried to make our contribution toward such machine automation. We report work on adding affect-detection to an existing e-drama program, a text-based software system for dramatic improvisation in simple virtual scenarios, for use primarily in learning contexts. The system allows a human director to monitor improvisations and make interventions, for instance in reaction to excessive, insufficient or inappropriate emotions in the characters’ speeches. Within an endeavour to partially automate directors’ functions, and to allow for automated affective bit-part characters, we have developed an affect-detection module. It is aimed at detecting affective aspects (concerning emotions, moods, value judgments, etc.) of human-controlled characters’ textual “speeches”. The work also accompanies basic research into how affect is conveyed linguistically. A distinctive feature of the project is a focus on the metaphorical ways in which affect is conveyed. Moreover, we have also introduced how the detected affective states activate the animation engine to produce gestures for human-controlled characters. The description of our approach in this paper is taken in part from our previous publications [1, 2] with new contributions mainly on metaphorical language processing (practically and theoretically), 3D emotional animation generation and user testing evaluation. Finally, Our work on affect detection in open-ended improvisational text contributes to the development of automatic understanding of human language and emotion. The generation of emotional believable animations based on detected affective states and the production of appropriate responses for the automated affective bit-part character based on the detection of affect contribute greatly to the ease and innovative user interface in e-drama, which leads to high-level user engagement and enjoyment.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Zhang, L., et al.: Exploitation in Affect Detection in Open-ended Improvisational Text. In: Proceedings of Workshop on Sentiment and Subjectivity at COLING-ACL 2006, Sydney, July 2006 (2006a)

    Google Scholar 

  2. Zhang, L., et al.: Developments in Affect Detection in E-drama. In: Proceedings of EACL 2006, 11th Conference of the European Chapter of the Association for Computational Linguistics, 2006, Trento, Italy, pp. 203–206 (2006b)

    Google Scholar 

  3. Picard, R.W.: Affective Computing. MIT Press, Cambridge (2000)

    Google Scholar 

  4. Ortony, A., Clore, G.L., Collins, A.: The Cognitive Structure of Emotions. Cambridge Univ. Press, Cambridge (1988)

    Google Scholar 

  5. Prendinger, H., Ishizuka, M.: Simulating Affective Communication with Animated Agents. In: Proceedings of Eighth IFIP TC.13 Conference on Human-Computer Interaction, Tokyo, Japan, pp. 182–189 (2001)

    Google Scholar 

  6. Wiltschko, W.R.: Emotion Dialogue Simulator. eDrama learning, Inc. eDrama Front Desk

    Google Scholar 

  7. Mehdi, E.J., et al.: Modeling Character Emotion in an Interactive Virtual Environment. In: Proceedings of AISB 2004 Symposium: Motion, Emotion and Cognition, Leeds, UK (2004)

    Google Scholar 

  8. McCrae, R.R., John, O.P.: An Introduction to the Five Factor Model and Its Application. Journal of Personality 60, 175–215 (1992)

    Article  Google Scholar 

  9. Gratch, J., Marsella, S.: A Domain-Independent Framework for Modeling Emotion. Journal of Cognitive Systems Research 5(4), 269–306 (2004)

    Article  Google Scholar 

  10. Egges, A., Kshirsagar, S., Magnenat-Thalmann, N.: A Model for Personality and Emotion Simulation. In: Palade, V., Howlett, R.J., Jain, L. (eds.) KES 2003. LNCS, vol. 2773, Springer, Heidelberg (2003)

    Google Scholar 

  11. Elliott, C., Rickel, J., Lester, J.: Integrating Affective Computing into Animated Tutoring Agents. In: Proceedings of IJCAI’97 Workshop on Intelligent Interface Agents (1997)

    Google Scholar 

  12. Aylett, R.S., Dias, J., Paiva, A.: An affectively-driven planner for synthetic characters. In: Proceedings of ICAPS (2006)

    Google Scholar 

  13. Mateas, M.: Interactive Drama, Art and Artificial Intelligence. Ph.D. Thesis, School of Computer Science, Carnegie Mellon University (2002)

    Google Scholar 

  14. Zhe, X., Boucouvalas, A.C.: Text-to-Emotion Engine for Real Time Internet Communication. In: Proceedings of International Symposium on Communication Systems, Networks and DSPs, Staffordshire University, UK, pp. 164–168 (2002)

    Google Scholar 

  15. Boucouvalas, A.C.: Real Time Text-to-Emotion Engine for Expressive Internet Communications. In: Riva, G., Davide, F., IJsselsteijn, W. (eds.) Being There: Concepts, Effects and Measurement of User Presence in Synthetic Environments, pp. 305–318 (2002)

    Google Scholar 

  16. Craggs, R., Wood, M.: A Two Dimensional Annotation Scheme for Emotion in Dialogue. In: Proceedings of AAAI Spring Symposium: Exploring Attitude and Affect in Text (2004)

    Google Scholar 

  17. Fussell, S., Moss, M.: Figurative Language in Descriptions of Emotional States. In: Fussell, S.R., Kreuz, R.J. (eds.) Social and cognitive approaches to interpersonal communication, Lawrence Erlbaum, Mahwah (1998)

    Google Scholar 

  18. Kövecses, Z.: Are There Any Emotion-Specific Metaphors? In: Athanasiadou, A., Tabakowska, E. (eds.) Speaking of Emotions: Conceptualization and Expression, pp. 127–151. Mouton de Gruyter, Berlin (1998)

    Google Scholar 

  19. Watson, D., Tellegen, A.: Toward a Consensual Structure of Mood. Psychological Bulletin 98, 219–235 (1985)

    Article  Google Scholar 

  20. Ekman, P.: An Argument for Basic Emotions. Cognition and Emotion 6, 169–200 (1992)

    Article  Google Scholar 

  21. Werry, C.: Linguistic and Interactional Features of Internet Relay Chat. In: Computer-Mediated Communication: Linguistic: Social and Cross-Cultural Perspectives. Pragmatics and Beyond New Series, vol. 39, pp. 47–64. John Benjamins, Amsterdam (1996)

    Google Scholar 

  22. Briscoe, E., Carroll, J.: Robust Accurate Statistical Annotation of General Text. In: Proceedings of the 3rd International Conference on Language Resources and Evaluation, Las Palmas, Gran Canaria, pp. 1499–1504 (2002)

    Google Scholar 

  23. Heise, D.R.: Semantic Differential Profiles for 1,000 Most Frequent English Words. Psychological Monographs 70(8) (Whole 601) (1965)

    Google Scholar 

  24. Barnden, J., et al.: Varieties and Directions of Inter-Domain Influence in Metaphor. Metaphor and Symbol 19(1), 1–30 (2004)

    Article  Google Scholar 

  25. Barnden, J.A.: Forthcoming. Metaphor, Semantic Preferences and Context-sensitivity. Invited chapter for a Festschrifft volume. Kluwer

    Google Scholar 

  26. Esuli, A., Sebastiani, F.: Determining Term Subjectivity and Term Orientation for Opinion Mining. In: Proceedings of EACL-06, 11th Conference of the European Chapter of the Association for Computational Linguistics, Trento, IT, pp. 193–200 (2006)

    Google Scholar 

  27. Sharoff, S.: How to Handle Lexical Semantics in SFL: a Corpus Study of Purposes for Using Size Adjectives. In: Systemic Linguistics and Corpus, Continuum, London (2005)

    Google Scholar 

  28. Garau, M., et al.: The impact of eye gaze on communication using humanoid avatars. In: Proceedings of the SIG-CHI conference on Human factors in computing systems, Seattle, WA, USA, March 31 - April 5, 2001, pp. 309–316 (2001)

    Google Scholar 

  29. Vilhjálmsson, H., Cassell, J.: BodyChat: Autonomous Communicative Behaviors in Avatars. In: Proceedings of ACM Second International Conference on Autonomous Agents, Minneapolis, Minnesota, May 9-13 (1998)

    Google Scholar 

  30. Gillies, M., Crabtree, I.B., Ballin, D.: Individuality and Contextual Variation of Character Behaviour for Interactive Narrative. In: Proceedings of the AISB Workshop on Narrative AI and Games (2006)

    Google Scholar 

  31. Carletta, J.: Assessing Agreement on Classification Tasks: The Kappa statistic. Computational Linguistics 22(2), 249–254 (1996)

    Google Scholar 

  32. Pantic, M., et al.: Human Computing and Machine Understanding of Human Behavior: A Survey. In: Proc. Int’l Conf. Multimodal Interfaces, pp. 239–248 (2006)

    Google Scholar 

  33. Cohn, J.F.: Foundations of human-centered computing: Facial expression and emotion. In: Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI’07), Hyderabad, India (2007)

    Google Scholar 

  34. Nogueiras, A., et al.: Speech emotion recognition using hidden Markov models. In: Proceedings of Eurospeech 2001, Denmark (2001)

    Google Scholar 

  35. Weizenbaum, J.: ELIZA - A Computer Program For the Study of Natural Language Communication Between Man and Machine. Communications of the ACM 9(1), 36–45 (1966)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Computing and Technology, University of East London, Dockland Campus, 4-6 University Way, London, E16 4LZ,  

    Li Zhang

  2. Department of Computer Science, University College London, London, WC1E 6BT,  

    Marco Gillies

  3. School of Computer Science, University of Birmingham, Birmingham, B15 2TT,  

    John A. Barnden, Robert J. Hendley, Mark G. Lee & Alan M. Wallington

Authors
  1. Li Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Marco Gillies
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. John A. Barnden
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Robert J. Hendley
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Mark G. Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Alan M. Wallington
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Thomas S. Huang Anton Nijholt Maja Pantic Alex Pentland

Rights and permissions

Reprints and permissions

Copyright information

© 2007 Springer Berlin Heidelberg

About this paper

Cite this paper

Zhang, L., Gillies, M., Barnden, J.A., Hendley, R.J., Lee, M.G., Wallington, A.M. (2007). Affect Detection and an Automated Improvisational AI Actor in E-Drama. In: Huang, T.S., Nijholt, A., Pantic, M., Pentland, A. (eds) Artifical Intelligence for Human Computing. Lecture Notes in Computer Science(), vol 4451. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-72348-6_17

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-72348-6_17

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-72346-2

  • Online ISBN: 978-3-540-72348-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation