Skip to main content
SpringerLink
Log in
Book cover

International Workshop on Intelligent Virtual Agents

IVA 2006: Intelligent Virtual Agents pp 270–281Cite as

A Story About Gesticulation Expression

  • Conference paper

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 4133))

Abstract

Gesticulation is essential for the storytelling experience thus, virtual storytellers should be endowed with gesticulation expression. This work proposes a gesticulation expression model based on psycholinguistics. The model supports: (a) real-time gesticulation animation described as sequences of constraints on static (Portuguese Sign Language hand shapes, orientations and positions) and dynamic (motion profiles) features; (b) multimodal synchronization between gesticulation and speech; (c) automatic reproduction of annotated gesticulation according to GestuRA, a gesture transcription algorithm. To evaluate the model two studies, involving 147 subjects, were conducted. In both cases, the idea consisted of comparing the narration of the Portuguese traditional story “The White Rabbit” by a human storyteller with a version by a virtual storyteller. Results indicate that synthetic gestures fared well when compared to real gestures however, subjects preferred the human storyteller.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. McNeill, D.: Hand and Mind: What gestures reveal about thought. University of Chicago Press, Chicago (1992)

    Google Scholar 

  2. McNeill, D.: Gesture and Thought. University of Chicago Press, Chicago (2005)

    Google Scholar 

  3. Kendon, A.: Sign languages of Aboriginal Australia: Cultural, semiotic and communicative perspectives. Cambridge University Press, Cambridge (1988)

    Google Scholar 

  4. Kita, S.: The temporal relationship between gesture and speech: A study of Japanese-English bilingual. MhD, Department of Psychology, University of Chicago (1990)

    Google Scholar 

  5. Nobe, S.: Where do most spontaneous representational gestures actually occur with respect to speech? In: McNeill, D. (ed.) Language and Gesture, pp. 186–198. Cambridge University Press, Cambridge (2000)

    Chapter  Google Scholar 

  6. de Ruiter, J.: The production of gesture and speech. In: McNeill, D. (ed.) Language and gesture, pp. 284–311. Cambridge University Press, Cambridge (2000)

    Chapter  Google Scholar 

  7. Krauss, M., Chen, Y., Gottesman, R.: Lexical gestures and lexical access: A process model. In: McNeill, D. (ed.) Language and gesture, pp. 261–283. Cambridge University Press, Cambridge (2000)

    Chapter  Google Scholar 

  8. Kita, S., Özyürek, A.: What does cross-linguistic variation in semantic coordination of speech and gesture reveal? Evidence for an interface representation of spatial thinking and speaking. Journal of Memory and Language 48, 16–32 (2003)

    Google Scholar 

  9. Cassell, J., Pelachaud, C., Badler, N., Steedman, M., Achorn, B., Becket, T., Douville, B., Prevost, S., Stone, M.: Animated Conversation: Rule-based Generation of Facial Expression, Gesture & Spoken Intonation for Multiple Conversational Agent. In: Proc. of SIGGRAPH 1994, pp. 413–420 (1994)

    Google Scholar 

  10. Cassell, J., Bickmore, T., Billinghurst, M., Campbell, L., Chang, K., Vilhjálmsson, H., Yan, H.: Embodiment in Conversational Interfaces: Rea. In: Proc. of the CHI 1999 Conference, Pittsburgh, PA, pp. 520–527 (1999)

    Google Scholar 

  11. Cassell, J., Stone, M.: Living Hand to Mouth: Psychological Theories about Speech and Gesture in Interactive Dialogue Systems. In: Proc. of the AAAI 1999 Fall Symposium on Psychological Models of Communication in Collaborative Systems, North Falmouth, MA, pp. 34–42 (1999)

    Google Scholar 

  12. Cassell, J., Vilhjálmsson, H., Bickmore, T.: BEAT: the Behavior Expression Animation Toolkit. In: Proc. of SIGGRAPH 2001, pp. 477–486 (2001)

    Google Scholar 

  13. Kopp, S., Wachsmuth, I.: A knowledge-based approach for lifelike gesture animation. In: Proc. of the 14th European Conf. on Artificial Intelligence. IOS Press, Amsterdam (2000)

    Google Scholar 

  14. Wachsmuth, I., Kopp, S.: Lifelike Gesture Synthesis and Timing for Conversational Agents. In: Wachsmuth, I., Sowa, T. (eds.) GW 2001. LNCS (LNAI), vol. 2298, pp. 120–133. Springer, Heidelberg (2002)

    Chapter  Google Scholar 

  15. SABLE: A Synthesis Markup Language (v. 1.0), http://www.bell-labs.com/project/tts/sable.html

  16. Kopp, S., Tepper, P., Cassell, J.: Towards Integrated Microplanning of Language and Iconic Gesture for Multimodal Output. In: Proc. of the International Conference on Multi-modal Interfaces (ICMI 2004), pp. 97–104. ACM Press, New York (2004)

    Chapter  Google Scholar 

  17. Arafa, Y., Kamyab, K., Mamdani, E.: Character Animation Scripting Languages: A Comparison. In: Proc. of the 2nd Intl. Conference of Autonomous Agents and Multiagent Systems, pp. 920–921 (2003)

    Google Scholar 

  18. VHML: VHML – Virtual Human Markup Language, http://www.vhml.org/

  19. SMIL: SMIL - Synchronized Multimedia, http://www.w3.org/AudioVideo/

  20. Kranstedt, A., Kopp, S., Wachsmuth, I.: MURML: A Multimodal Utterance Representation Markup Language for Conversational Agents. In: AAMAS 2002 Workshop Embodied conversational agents- let’s specify and evaluate them!, Bologna, Italy (2002)

    Google Scholar 

  21. Ruttkay, Z., Noot, H.: Variations in Gesturing and Speech by GESTYLE. International Journal of Human-Computer Studies, Special Issue on Subtle Expressivity for Characters and Robots 62(2), 211–229 (2005)

    Google Scholar 

  22. de Carolis, B., Pelachaud, C., Poggi, I., Steedman, M.: APML, a Mark-up Language for Believable Behavior Generation. In: Prendinger, H. (ed.) Life-like Characters. Tools, Affective Functions and Applications. Springer, Heidelberg (2004)

    Google Scholar 

  23. Pavlovic, V., Sharma, R., Huang, T.: Visual Interpretation of hand gestures for human computer interaction: A review. IEEE Trans. Pattern Analysis Machine Intelligence 19, 677–695 (1997)

    Article  Google Scholar 

  24. Gavrila, D.: The visual analysis of human movement: A survey in Computer Vision and Image Understanding  73, 82–98 (1999)

    Google Scholar 

  25. de Melo, C., Paiva, A.: Multimodal Expression in Virtual Humans. Accepted for Computer Animation & Social Agents 2006 (CASA 2006) and Journal of Computer Animation and Virtual Worlds (2006)

    Google Scholar 

  26. Blumberg, B., Galyean, T.: Multi-Level Direction of Autonomous Creatures for Real-Time Virtual Environments. In: Proc. of SIGGRAPH 1995, vol. 30(3), pp. 47–54 (1995)

    Google Scholar 

  27. Perlin, K., Goldberg, A.: Improv: A System for Scripting Interactive Actors in Virtual Worlds. In: Proc. of SIGGRAPH 1996, pp. 205–216 (1996)

    Google Scholar 

  28. NASA Man-Systems Integration Manual (NASA-STD-3000)

    Google Scholar 

  29. Black, A., Taylor, P., Caley, R., Clark, R.: Festival, http://www.cstr.ed.ac.uk/projects/festival/

  30. Gut, U., Looks, K., Thies, A., Trippel, T., Gibbon, D.: CoGesT – Conversational Gesture Transcription System. Technical Report, University of Bielefeld (1993)

    Google Scholar 

  31. Kipp, M.: ANVIL – A Generic Annotation Tool for Multimodal Dialogue. In: Proc. of the 7th European Conference on Speech Comm. and Technology, Aalborg, pp. 1367–1370 (2001)

    Google Scholar 

  32. Secretariado Nacional para a Reabilitação e Integração das Pessoas com Deficiência. Gestuário – Língua Gestual Portuguesa – 5th edition

    Google Scholar 

  33. SMIL. SMIL: Synchronized Multimedia, http://www.w3.org/AudioVideo/

  34. Thompson, D., Buford, W., Myers, L., Giurintano, D., Brewer III, J.: A Hand Biomechanics Workstation. Computer Graphics 22(4), 335–343 (1988)

    Article  Google Scholar 

  35. Albrecht, I., Haber, J., Siedel, H.: Construction and Animation of Anatomically Based Human Hand Models. In: SIGGRAPH 2003, pp. 98–109 (2003)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. IST, Technical University of Lisbon and INESC-ID, Avenida Prof. Cavaco Silva – Taguspark, 2780-990, Porto Salvo, Portugal

    Celso de Melo & Ana Paiva

Authors
  1. Celso de Melo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ana Paiva
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Institute for Creative Technologies, University of Southern California, 13274 Fiji Way, 90292, Marina Del Rey, CA, USA

    Jonathan Gratch

  2. Music Department, Goldsmiths, University of London, New Cross, SE14 6NW, London, UK

    Michael Young

  3. School of Mathematical and Computer Sciences, Heriot Watt University, EH14 4AS, Edinburgh, Scotland

    Ruth Aylett

  4. BT plc, Adastral Park, IP5 3RE, Ipswich, UK

    Daniel Ballin

  5. School of Computing Science, Culture Lab, Newcastle University, NE1 7RU, Newcastle upon Tyne, UK

    Patrick Olivier

Rights and permissions

Reprints and permissions

Copyright information

© 2006 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

de Melo, C., Paiva, A. (2006). A Story About Gesticulation Expression. In: Gratch, J., Young, M., Aylett, R., Ballin, D., Olivier, P. (eds) Intelligent Virtual Agents. IVA 2006. Lecture Notes in Computer Science(), vol 4133. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11821830_22

Download citation

  • DOI: https://doi.org/10.1007/11821830_22

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-37593-7

  • Online ISBN: 978-3-540-37594-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation