The Role of Iconic Gestures in Production and Comprehension of Language: Evidence from Brain and Behavior
Speakers in all cultures and ages use gestures as they speak (i.e., cospeech gestures). There have been different views in the literature with regard to whether and how a specific type of gestures speakers use, i.e., iconic gestures, interacts with language processing. Here I review evidence showing that iconic gestures are not produced merely from the spatial and/or motoric imagery but from an in interface representation of imagistic and linguistic representation during online speaking Similarly, for comprehension, neuroimaging and behavioral studies indicate that speech and gesture influences semantic processing of each other during online comprehension. These findings show overall that processing of information in both modalities interacts during both comprehension and production of language arguing against models that propose independent processing of each modality. They also have implications for AI models that aim to simulate cospeech gesture use in conversational agents.
Keywordsiconic cospeech gesture interface production comprehension brain behavior
Unable to display preview. Download preview PDF.
- 2.McNeill, D.: Hand and mind. University of Chicago Press, Chicago (1992)Google Scholar
- 4.Özyürek, A.: Gesture in sign and spoken language. In: Pfau, R., Steinbach, M., Woll, B. (eds.) Sign language: An international handbook. Mouton, Berlin (in press)Google Scholar
- 5.Kendon, A.: Gesture. Cambridge University Press, Cambridge (2004)Google Scholar
- 9.Hostetter, A.B., Alibali, M.W.: Visible embodiment: Gestures as simulated action. Psychonomic Bulletin and Review (2008)Google Scholar
- 11.Feyreisen, P., Lanoy, J.D.: Gestures and speech: Psychological investigations. Cambridge University Press, Cambridge (1991)Google Scholar
- 16.Kelly, S., Özyürek, A., Maris, E.: Two sides of the same coin: Speech and gesture interact to enhance comprehension. Psychological Science (in press)Google Scholar
- 17.Özyürek, A., Kita, S.: Expressing manner and path in English and Turkish: Differences in speech, gesture, and conceptualization. In: Hahn, M., Stoness, S.C. (eds.) Proceedings of the twenty first annual conference of the Cognitive Science Society, pp. 507–512. Lawrence Erlbaum, Mahwah (1999)Google Scholar
- 18.Kita, S.: How representational gestures help speaking. In: McNeill, D. (ed.) Language and Gesture, pp. 261–284. Cambridge University Press, Cambridge (2000)Google Scholar
- 19.Talmy, L.: Semantics and syntax of motion. In: Shopen, T. (ed.) Language typology and syntactic description. Grammatical categories and the lexicon, vol. 3, pp. 57–149. Cambridge University Press, Cambridge (1985)Google Scholar
- 21.Levelt, P.: Speaking. MIT Press, Cambridge (1989)Google Scholar