On Factoring Out a Gesture Typology from the Bielefeld Speech-and-Gesture-Alignment Corpus (SAGA)

  • Hannes Rieser
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5934)


The paper is based on the Bielefeld Speech-And-Gesture-Alignment corpus (SAGA). From this corpus one video film is taken to establish a typological grid for iconic and referring gesture types, i.e. a multiple inheritance hierarchy of types proceeding from single gestural features like hand shape to sequences of entities filling up the whole gesture space. Types are mapped onto a partial ontology specifying their respective meaning. Multi-modal meaning is generated via linking verbal meaning and gestural meaning. How verbal and gestural meaning interface is shown with an example using a quantified NP. It is argued that gestural meaning extends the restriction of the original quantified NP. On the other hand it is shown that gestural meaning is not strong enough to resolve the underspecification of the lexical information.


SAGA corpus iconic gesture gesture typology partial ontology speech-gesture interface 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Abeillé, A., Rambow, O.: Tree Adjoining Grammars. CSLI Publications, Stanford (2000)zbMATHGoogle Scholar
  2. Bergmann, K., Kopp, S.: Systematicity and idiosyncrasy in iconic gestures – A probabilistic model of gesture use. In: Kopp, S., Wachsmuth, I. (eds.) GW 2009. LNCS (LNAI), vol. 5934, pp. 179–191. Springer, Heidelberg (2010)Google Scholar
  3. Bergmann, K., Fröhlich, C., Hahn, F., Kopp, S., Lücking, A., Rieser, H.: Wegbeschreibungsexperiment: Grobannotationsschema. MS., Univ. Bielefeld (June 2007)Google Scholar
  4. Bergmann, K., Damm, O., Fröhlich, C., Hahn, F., Kopp, S., L”ucking, A., Rieser, H., Thomas, N.: Annotationsmanual zur Gestenmorphologie. MS., Univ. Bielefeld (June 2008)Google Scholar
  5. Carpenter, B.: The Logic of Typed Feature Structures. CUP, Cambridge (1992)zbMATHCrossRefGoogle Scholar
  6. Ekman, P., Friesen, W.: The repertoire of nonverbal behavior: categories, origins, usage and coding. Semiotica 1, 49–98 (1969)Google Scholar
  7. Essler, K.: Wissenschaftstheorie I. Definition und Reduktion. Freiburg, Alber (1970)Google Scholar
  8. Goldin-Meadow, S.: When gesture and words speak differently. Current Directions in Psychological Science 6, 138–143 (1997)CrossRefGoogle Scholar
  9. Grice, H.P.: Logic and Conversation. The William James Lectures. Harvard University, Cambridge (1989) Published as Part 1 of Grice; Grice, H. P.: Studies in the Way of Words. Harvard University Press, Cambridge (1967)Google Scholar
  10. Karttunen, L., Peters, S.: Conventional Implicature. In: Syntax and Semantics 11: Presuppositions, pp. 1–56. Academic Press, London (1979)Google Scholar
  11. Kendon, A.: Gesture: Visible Action as Utterance. CUP, Cambridge (2004)Google Scholar
  12. Kita, S. (ed.): Pointing: Where Language, Culture and Cognition Meet. Lawr. Erlb., Mahwah (2003)Google Scholar
  13. McNeill, D.: Hand and Mind. What Gestures Reveal About Thought. Univ. Chic. Press, Chicago (1992)Google Scholar
  14. Paul, G.: Approaches to abductive reasoning: an overview. Artificial Intelligence Review 7, 109–152 (1993)CrossRefGoogle Scholar
  15. Peirce, C.S.: Syllabus. In: Hartshorne, C., Weiss, P. (eds.) Collected papers of Charles Sanders Peirce, vol. 2, p. 274. Belknap Press of Harvard Univ. Press (1965)Google Scholar
  16. Peirce, C.S.: On the Algebra of Logic. In: Houser, N., Kloesel, C. (eds.) The Essential Peirce, vol. 1, p. 226. Indiana Univ. Pr., Bloomington (1992)Google Scholar
  17. Poggi, I.: From a Typology of Gestures to a Procedure for Gesture Production. In: Wachsmuth, I., Sowa, T. (eds.) GW 2001. LNCS (LNAI), vol. 2298, pp. 158–168. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  18. Reiter, R.: A logic for default reasoning. Artificial Intelligence 13, 81–132 (1980)zbMATHCrossRefMathSciNetGoogle Scholar
  19. Rieser, H., Kopp, S., Wachsmuth, I.: Speech-Gesture Alignment. In: ISGS Abstracts, Integrating Gestures, Northwestern University, Evanston, Chicago, pp. 25–27 (2007)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Hannes Rieser
    • 1
  1. 1.Bielefeld University 

Personalised recommendations