Skip to main content
SpringerLink
Log in

Systematicity and Idiosyncrasy in Iconic Gesture Use: Empirical Analysis and Computational Modeling

  • Conference paper
Gesture in Embodied Communication and Human-Computer Interaction (GW 2009)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 5934))

Included in the following conference series:

Abstract

Why an iconic gesture takes its particular form is a largely open question, given the variations one finds across both situations and speakers. We present results of an empirical study that analyzes correlations between contextual factors (referent features, discourse) and gesture features, and tests whether they are systematic (shared among speakers) or idiosyncratic (inter-individually different). Based on this, a computational model of gesture formation is presented that combines data-based, probabilistic and model-based decision making.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Alibali, M., Kita, S., Young, A.: Gesture and the process of speech production: We think, therefore we gesture. Language and cognitive processes 15, 593–613 (2000)

    Article  Google Scholar 

  2. Bavelas, J., Gerwing, J., Sutton, C., Prevost, D.: Gesturing on the telephone: Independent effects of dialogue and visibility. Journal of Memory and Language 58, 495–520 (2008)

    Article  Google Scholar 

  3. Bavelas, J., Kenwood, C., Johnson, T., Philips, B.: An experimental study of when and how speakers use gestures to communicate. Gesture 1, 1–17 (2002)

    Article  Google Scholar 

  4. Bergmann, K., Kopp, S.: Increasing expressiveness for virtual agents–Autonomous generation of speech and gesture for spatial description tasks. In: Decker, K., Sichman, J., Sierra, C., Castelfranchi, C. (eds.) Proceedings of the 8th International Conference on Autonomous Agents and Multiagent Systems, pp. 361–368 (2009)

    Google Scholar 

  5. Bergmann, K., Kopp, S.: GNetIc – using bayesian decision networks for iconic gesture generation. In: Ruttkay, Z., Kipp, M., Nijholt, A., Vilhjálmsson, H.H. (eds.) IVA 2009. LNCS, vol. 5773, pp. 76–89. Springer, Heidelberg (2009)

    Chapter  Google Scholar 

  6. Cassell, J., Stone, M., Yan, H.: Coordination and context-dependence in the generation of embodied conversation. In: Proceedings of the First International Conference on Natural Language Generation (2000)

    Google Scholar 

  7. de Ruiter, J.: The production of gesture and speech. In: McNeill, D. (ed.) Language and gesture. Cambridge University Press, Cambridge (2000)

    Google Scholar 

  8. Denis, M.: The description of routes: A cognitive approach to the production of spatial discourse. Current Psychology of Cognition 16, 409–458 (1997)

    Google Scholar 

  9. Gullberg, M., Hendriks, H., Hickmann, M.: Learning to talk and gesture about motion in french. First Language 28(2), 200–236 (2008)

    Article  Google Scholar 

  10. Gwet, K.: Handbook of Inter-Rater Reliability. STATAXIS Publishing Company (2001)

    Google Scholar 

  11. Halliday, M.: Notes on transitivity and theme in english (part 2). Journal of Linguistics 3, 199–247 (1967)

    Article  Google Scholar 

  12. Hostetter, A., Alibali, M.: Raise your hand if you’re spatial–Relations between verbal and spatial skills and gesture production. Gesture 7(1), 73–95 (2007)

    Article  Google Scholar 

  13. Hostetter, A., Alibali, M., Kita, S.: I see it in my hands’ eye: Representational gestures reflect conceptual demands. Language and Cogn. Processes 22, 313–336 (2007)

    Article  Google Scholar 

  14. Howard, R., Matheson, J.: Influence diagrams. Decision Analysis 2(3), 127–143 (2005)

    Article  Google Scholar 

  15. Kendon, A.: Gesture–Visible Action as Utterance. Cambridge University Press, Cambridge (2004)

    Google Scholar 

  16. Kita, S., Özyürek, A.: What does cross-linguistic variation in semantic coordination of speech and gesture reveal?: Evidence for an interface representation of spatial thinking and speaking. Journal of Memory and Language 48, 16–32 (2003)

    Article  Google Scholar 

  17. Kopp, S., Bergmann, K., Wachsmuth, I.: Multimodal communication from multimodal thinking–Towards an integrated model of speech and gesture production. International Journal of Semantic Computing 2(1), 115–136 (2008)

    Article  Google Scholar 

  18. Kopp, S., Tepper, P., Cassell, J.: Towards integrated microplanning of language and iconic gesture for multimodal output. In: Proc. Intern. Conf. on Multimodal Interfaces, pp. 97–104. ACM Press, New York (2004)

    Chapter  Google Scholar 

  19. Kopp, S., Tepper, P., Ferriman, K., Striegnitz, K., Cassell, J.: Trading spaces: How humans and humanoids use speech and gesture to give directions. In: Nishida, T. (ed.) Conversational Informatics, pp. 133–160. John Wiley, New York (2007)

    Chapter  Google Scholar 

  20. Kopp, S., Wachsmuth, I.: Synthesizing multimodal utterances for conversational agents. Computer Animation and Virtual Worlds 15(1), 39–52 (2004)

    Article  Google Scholar 

  21. Kruijff-Korbayova, I., Steedman, M.: Discourse and information structure. Journal of Logic, Language and Information 12, 249–259 (2003)

    Article  MATH  Google Scholar 

  22. Lauritzen, S.L.: The EM algorithm for graphical association models with missing data. Computational Statistics and Data Analysis 19, 191–201 (1995)

    Article  MATH  Google Scholar 

  23. Madsen, A., Jensen, F., Kjærulff, U., Lang, M.: HUGIN–The tool for bayesian networks and influence diagrams. International Journal of Artificial Intelligence Tools 14(3), 507–543 (2005)

    Article  Google Scholar 

  24. McNeill, D.: Hand and Mind - What Gestures Reveal about Thought. University of Chicago Press, Chicago (1992)

    Google Scholar 

  25. McNeill, D., Duncan, S.: Growth points in thinking-for-speaking. In: Language and gesture. Cambridge Univ. Press, Cambridge (2000)

    Chapter  Google Scholar 

  26. Müller, C.: Redebegleitende Gesten: Kulturgeschichte–Theorie–Sprachvergleich. Berlin Verlag, Berlin (1998)

    Google Scholar 

  27. Neff, M., Kipp, M., Albrecht, I., Seidel, H.-P.: Gesture modeling and animation based on a probabilistic re-creation of speaker style. ACM Transactions on Graphics 27(1), 1–24 (2008)

    Article  Google Scholar 

  28. Ritz, J., Dipper, S., Götze, M.: Annotation of information structure: An evaluation across different types of texts. In: Proceedings of the 6th LREC conference, pp. 2137–2142 (2008)

    Google Scholar 

  29. Sowa, T., Wachsmuth, I.: A model for the representation and processing of shape in coverbal iconic gestures. In: Proc. KogWis 2005, pp. 183–188 (2005)

    Google Scholar 

  30. Steck, H., Tresp, V.: Bayesian belief networks for data mining. In: Proceedings of the 2nd Workshop on Data Mining and Data Warehousing (1999)

    Google Scholar 

  31. Stone, M., Doran, C., Webber, B., Bleam, T., Palmer, M.: Microplanning with Communicative Intentions: The SPUD System. Comput. Intelligence 19(4), 311–381 (2003)

    Article  MathSciNet  Google Scholar 

  32. Streeck, J.: Depicting by gesture. Gesture 8(3), 285–301 (2008)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. SFB 673 Alignment in Communication, Bielefeld University, P.O. Box 100 131, D-33615, Bielefeld, Germany

    Kirsten Bergmann

  2. Sociable Agents Group, CITEC, Bielefeld University, P.O. Box 100 131, D-33615, Bielefeld, Germany

    Stefan Kopp

Authors
  1. Kirsten Bergmann
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Stefan Kopp
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. CITEC, Bielefeld University, P.O. Box 100131, 33501, Bielefeld, Germany

    Stefan Kopp

  2. Artificial Intelligence Group, Faculty of Technology, Bielefeld University, 33594, Bielefeld, Germany

    Ipke Wachsmuth

Rights and permissions

Reprints and permissions

Copyright information

© 2010 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Bergmann, K., Kopp, S. (2010). Systematicity and Idiosyncrasy in Iconic Gesture Use: Empirical Analysis and Computational Modeling. In: Kopp, S., Wachsmuth, I. (eds) Gesture in Embodied Communication and Human-Computer Interaction. GW 2009. Lecture Notes in Computer Science(), vol 5934. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-12553-9_16

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-12553-9_16

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-12552-2

  • Online ISBN: 978-3-642-12553-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation