Skip to main content
SpringerLink
Log in

Individualized Gesture Production in Embodied Conversational Agents

  • Chapter
Human-Computer Interaction: The Agency Perspective

Part of the book series: Studies in Computational Intelligence ((SCI,volume 396))

Abstract

Gesturing behavior is subject to great variations across situations, individuals, or cultures. These variations make gestures hard for systematic studies and modeling attempts. However, gesture research on real humans and modeling approaches with virtual agents have made significant progress in the last years. In this chapter we discuss the state of research and present results from an extensive empirical study on human iconic gestures in direction giving dialogues. It is described how machine learning methods can be employed to extract different speakers’ gesturing style and to generate individualized language and gestures in ECAs. Evaluations show that human observers rate virtual agents better in terms of competence, human-likeness, or likability when a consistent individual gesture style is produced.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Ball, G., Breese, J.: Emotion and personality in a conversational agent. In: Cassell, J., Sullivan, J., Prevost, S., Churchill, E. (eds.) Embodied Conversational Agents, pp. 189–219. MIT Press, Cambridge (2000)

    Google Scholar 

  2. Bavelas, J., Gerwing, J., Sutton, C., Prevost, D.: Gesturing on the telephone: Independent effects of dialogue and visibility. Journal of Memory and Language 58, 495–520 (2008)

    Article  Google Scholar 

  3. Bavelas, J., Kenwood, C., Johnson, T., Philips, B.: An experimental study of when and how speakers use gestures to communicate. Gesture 2(1), 1–17 (2002)

    Article  Google Scholar 

  4. Bente, G., Haug Leuschner, A.I., Blascovich, J.: The others: Universals and cultural specificities in the perception of status and dominance from nonverbal behavior. Consciousness and Cognition 19(3), 762–777 (2010)

    Article  Google Scholar 

  5. Bergmann, K., Kopp, S.: GNetIc – using bayesian decision networks for iconic gesture generation. In: Ruttkay, Z., Kipp, M., Nijholt, A., Vilhjálmsson, H.H. (eds.) IVA 2009. LNCS, vol. 5773, pp. 76–89. Springer, Heidelberg (2009)

    Chapter  Google Scholar 

  6. Bergmann, K., Kopp, S.: Increasing expressiveness for virtual agents–Autonomous generation of speech and gesture in spatail description tasks. In: Decker, K., Sichman, J., Sierra, C., Castelfranchi, C. (eds.) Proceedings of the 8th International Conference on Autonomous Agents and Multiagent Systems, IFAAMAS, Budapest, Hungary, pp. 361–368 (2009)

    Google Scholar 

  7. Bergmann, K., Kopp, S.: Modelling the production of co-verbal iconic gestures by learning bayesian decision networks. Applied Artificial Intelligence 24(6), 530–551 (2010)

    Article  Google Scholar 

  8. Bergmann, K., Kopp, S., Eyssel, F.: Individualized gesturing outperforms average gesturing – Evaluating gesture production in virtual humans. In: Allbeck, J., Badler, N., Bickmore, T., Pelachaud, C., Safonova, A. (eds.) Proceedings of the 10th Conference on Intelligent Virtual Agents, pp. 104–117. Springer, Berlin (2010)

    Chapter  Google Scholar 

  9. Cassell, J., Stone, M., Douville, B., Prevost, S., Achorn, B., Steedman, M., Badler, N., Catherine, P.: Modeling the interaction between speech and gesture. In: Ram, A., Eiselt, K. (eds.) Proceedings of the Sixteenth Annual Conference of the Cognitive Science Society, pp. 153–158. Lawrence Erlbaum Associates (1994)

    Google Scholar 

  10. Chi, D., Costa, M., Zhao, L., Badler, N.: The EMOTE model for effort and shape. In: Akeley, K. (ed.) Proceedings of SIGGRAPH 2000, pp. 173–182. Addison-Wesley Longman (2000)

    Google Scholar 

  11. Foster, M., Oberlander, J.: Corpus-based generation of head and eyebrow motion for an embodied conversational agent. Language Resources and Evaluation 41, 305–323 (2007)

    Article  Google Scholar 

  12. Hartmann, B., Mancini, M., Pelachaud, C.: Implementing expressive gesture synthesis for embodied conversational agents. In: Gibet, S., Courty, N., Kamp, J.-F. (eds.) GW 2005. LNCS (LNAI), vol. 3881, pp. 188–199. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  13. Hostetter, A., Alibali, M.: Raise your hand if you’re spatial–Relations between verbal and spatial skills and gesture production. Gesture 7(1), 73–95 (2007)

    Article  Google Scholar 

  14. Howard, R., Matheson, J.: Influence diagrams. Decision Analysis 2(3), 127–143 (2005)

    Article  Google Scholar 

  15. Kendon, A.: Gesture–Visible Action as Utterance. Cambridge University Press (2004)

    Google Scholar 

  16. Kimbara, I.: On gestural mimicry. Gesture 6(1), 39–61 (2006)

    Article  Google Scholar 

  17. Kita, S.: How representational gestures helps speaking. In: McNeill, D. (ed.) Language and gesture, pp. 162–185. Cambridge University Press, Cambridge (2000)

    Chapter  Google Scholar 

  18. Kopp, S.: Social resonance and embodied coordination in face-to-face conversation with artificial interlocutors. Speech Communication 52, 587–597 (2010)

    Article  Google Scholar 

  19. Kopp, S., Bergmann, K., Wachsmuth, I.: Multimodal communication from multimodal thinking–Towards an integrated model of speech and gesture production. International Journal of Semantic Computing 2(1), 115–136 (2008)

    Article  Google Scholar 

  20. Kopp, S., Tepper, P., Ferriman, K., Striegnitz, K., Cassell, J.: Trading spaces: How humans and humanoids use speech and gesture to give directions. In: Nishida, T. (ed.) Conversational Informatics, pp. 133–160. John Wiley, New York (2007)

    Chapter  Google Scholar 

  21. Kopp, S., Wachsmuth, I.: Synthesizing multimodal utterances for conversational agents. Computer Animation and Virtual Worlds 15(1), 39–52 (2004)

    Article  Google Scholar 

  22. Lauritzen, S.L.: The EM algorithm for graphical association models with missing data. Computational Statistics and Data Analysis 19, 191–201 (1995)

    Article  MATH  Google Scholar 

  23. Madsen, A., Jensen, F., Kjærulff, U., Lang, M.: HUGIN–The tool for bayesian networks and influence diagrams. International Journal of Artificial Intelligence Tools 14(3), 507–543 (2005)

    Article  Google Scholar 

  24. McNeill, D.: Gesture and Thought. Univ. of Chicago Press, Chicago (2005)

    Book  Google Scholar 

  25. Melinger, A., Levelt, W.: Gesture and the communicative intention of the speaker. Gesture 4(2), 119–141 (2004)

    Article  Google Scholar 

  26. Müller, C.: Redebegleitende Gesten: Kulturgeschichte–Theorie–Sprachvergleich. Berlin Verlag, Berlin (1998)

    Google Scholar 

  27. Nass, C., Isbister, K., Lee, E.-J.: Truth is beauty: Researching embodied conversational agents. In: Cassell, J., Sullivan, J., Prevost, S., Churchill, E. (eds.) Embodied Conversational Agents, pp. 374–402. MIT Press, Cambridge (2000)

    Google Scholar 

  28. Neff, M., Kipp, M., Albrecht, I., Seidel, H.-P.: Gesture modeling and animation based on a probabilistic re-creation of speaker style. ACM Transactions on Graphics 27(1), 1–24 (2008)

    Article  Google Scholar 

  29. Ruttkay, Z.: Presenting in style by virtual humans. In: Esposito, A., Faundez-Zanuy, M., Keller, E., Marinaro, M. (eds.) COST Action 2102. LNCS (LNAI), vol. 4775, pp. 23–36. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

  30. Steck, H., Tresp, V.: Bayesian belief networks for data mining. In: Proceedings of the 2nd Workshop on Data Mining and Data Warehousing (1999)

    Google Scholar 

  31. Stone, M., DeCarlo, D., Oh, I., Rodriguez, C., Stere, A., Lees, A., Bregler, C.: Speaking with hands: Creating animated conversational characters from recordings of human performance. In: Proceedings of SIGGRAPH 2004, pp. 506–513 (2004)

    Google Scholar 

  32. Stone, M., Doran, C., Webber, B., Bleam, T., Palmer, M.: Microplanning with Communicative Intentions: The SPUD System. Comput. Intelligence 19(4), 311–381 (2003)

    Article  MathSciNet  Google Scholar 

  33. Streeck, J.: Depicting by gesture. Gesture 8(3), 285–301 (2008)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Sociable Agents Group, CITEC, Faculty of Technology, Bielefeld University, Bielefeld, Germany

    Stefan Kopp

  2. SFB 673 Alignment in Communication, Bielefeld University, Bielefeld, Germany

    Kirsten Bergmann

Authors
  1. Stefan Kopp
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kirsten Bergmann
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. UAlg Informatics Lab, University of Algarve , Campus de Gambelas, 8000-062, Faro, Portugal

    Marielba Zacarias  & José Valente de Oliveira  & 

Rights and permissions

Reprints and permissions

Copyright information

© 2012 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Kopp, S., Bergmann, K. (2012). Individualized Gesture Production in Embodied Conversational Agents. In: Zacarias, M., de Oliveira, J.V. (eds) Human-Computer Interaction: The Agency Perspective. Studies in Computational Intelligence, vol 396. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-25691-2_12

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-25691-2_12

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-25690-5

  • Online ISBN: 978-3-642-25691-2

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation