Abstract
A taxonomy of hand gestures and a digital tool (CodGest) are proposed in order to describe different types of gesture used by speaker during speech in different social contexts. It is an exhaustive and mutually exclusive categories system to be shared within the scientific community to study multimodal signals and their contribute to the interaction. Classical taxonomies from gesture literature were integrated within a comprehensive taxonomy, which was tested in five different social contexts and its reliability was measured across them through inter-observer agreement indexes. A multi-media tool was realized as digital support for coding gestures in observational research.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Bakeman, R., Gottman, J.M.: Observing interaction. An introduction to sequential analysis, II edn. Cambridge University Press, New York (1997)
Bavelas, J.B., Chovil, N., Lawrie, D.A., Wade, A.: Interactive gestures. Discourse Processes 15, 469–489 (1992)
Ekman, P., Friesen, W.V.: The repertoire of nonverbal behavior. Semiotica 1, 49–98 (1969)
Kendon, A.: Gesture and speech: How they interact. In: Wiemann, J.M., Harrison, R.P. (eds.) Nonverbal Interaction, pp. 13–45. Sage Publications, Beverly Hills (1983)
McNeill, D.: Hand and Mind. The University of Chicago Press, Chicago (1992)
Kendon, A.: Gesture: Visible Action as Utterance. Cambridge University Press, Cambridge (2004)
Krauss, R.M., Chen, Y., Chawla, P.: Nonverbal behavior and nonverbal communication: What do conversational hand gestures tell us? Adv. Exp. Soc. Psychol. 28, 389–450 (1996)
Poggi, I.: Iconicity in different types of gestures. Gesture 8, 45–61 (2008)
McNeill, D.: So you think gesture are nonverbal? Psychol. Rev. 92, 350–371 (1985)
Kipp, M., Neff, M., Albrecht, I.: An Annotation Scheme for Conversational Gestures: How to economically capture timing and form. Lang. Resour. Eval. 41(3-4), 325–339 (2007)
Allwood, J., Cerrato, L., Jokinen, K., Navarretta, C., Paggio, P.: The MUMIN coding scheme for the annotation of feedback, turn management and sequencing phenomena. Lang. Resour. Eval. 41(3-4), 273–287 (2007)
Bakeman, R., Gnisci, A.: Sequential observational methods. In: Eid, M., Dieneer, E. (eds.) Handbook of Multimethod Measurement in Psychology, pp. 127–140. American Psychological Association, Washington (2006)
Cohen, J.A.: Coefficient of agreement for nominal scales. Educ. Psychol. Meas. 20, 37–46 (1960)
Krippendorff, K.: Estimating the reliability, systematic error, and random error of interval data. Educ. Psychol. Meas. 30, 61–70 (1970)
Kendon, A.: Gestures as illocutionary and discourse structure markers in Southern Italian conversation. J. Pragmatics 23, 247–279 (1995)
McNeill, D., Levy, E.T.: Cohesion and gesture. Discourse Processes 16, 363–386 (1993)
Rosenfeld, H.M.: Instrumental affiliative functions of facial and gestural expressions. J. Pers. Soc. Psychol. 4, 65–72 (1966)
Edelmann, R.J., Hampson, S.: Embarrassment in dyadic interaction. Soc. Behav. Personal. 9, 171–178 (1981)
Duncan, S.: Coding manual (2004), Technical Report availale from www.mcneilllab.uchicago.edu
Maricchiolo, F., Livi, S., Bonaiuto, M., Gnisci, A.: Hand gestures and perceived influence in small group interaction. Span. J. Psychol. 14, 755–764 (2011)
Caso, L., Maricchiolo, F., Bonaiuto, M., Vrij, A., Mann, S.: The impact of deception and suspicion on different hand movements. J. Nonverbal Behav. 30, 1–19 (2006)
Jefferson, G.: On the interactional unpackaging of a “Gloss”. Lang. Soc. 14, 435–466 (1985)
Robinson, B.F., Bakeman, R.: ComKappa: A Windows 95 Program for Calculating Kappa and Related Statistics. Behav. Res. Meth. Ins. C. 30, 731–732 (1998)
Fleiss, J.L.: Measuring nominal scale agreement among many raters. Psychol. Bull. 76, 378–382 (1971)
Kita, S.: Cross-cultural variation of speech-accompanying gesture: A review. Lang. Cognitive Proc. 24, 145–167 (2009)
Bonaiuto, M., Gnisci, A., Maricchiolo, F.: Struttura e funzioni dei gesti delle mani durante la conversazione: nodi concettuali e possibili sviluppi futuri. Presentation to “Gesture in the Mediterranean: Recent Research in Southern Europe”, Procida, October 21-23 (2005)
Bonaiuto, M., Maricchiolo, F., Orlacchio, T.: Cultura e gestualità delle mani durante la conversazione: un confronto tra donne native dell’Italia e del Burkina Faso. Presentation to Workshop: “Intersoggettività. Identità e Cultura”, Urbino, October 14-15 (2005)
Beattie, G., Shovelton, H.: Iconic hand gestures and predistability of words in context in spontaneous speech. Brit. J. Psychol. 91, 473–492 (2000)
Beattie, G., Shovelton, H.: An experimental investigation of some properties of individual iconic gestures that mediate their communicative power. Brit J. Psychol. 93, 179–192 (2002)
Contento, S., Stame, S.: Déixis verbale et non verbale dans la construction de l’espace interpersonnel. Dialogue Analisis 5, 427–433 (1997)
Feyereisen, P., Havard, I.: Mental imagery and production of hand gestures while speaking in younger and older adults. J. Nonverbal Behav. 23, 153–171 (1999)
Cronbach, L.J., Gleser, G.C., Nanda, H., Rajaratnam, N.: The dependability of behavioural measurement: Theory of generazability for scores and profiles. Wiley, New York (1972)
Maricchiolo, F., Bonaiuto, M., Gnisci, A.: Hand gestures in speech: studies on their roles in social interaction. In: Mondada, L. (ed.) Proceedings of the 2nd ISGS Conference, Interacting Bodies, Lyon-France, June 15-18. Ecole Normale Supérieure Lettres et Sciences humaines, Lyon (2007)
Calbris, G.: Elements of Meaning in Gesture. John Benjamins Publishing, Amsterdam (2011)
Wiener, M., Devoe, S., Rubinow, S., Geller, J.: Nonverbal behavior and nonverbal communication. Psychol. Rev. 79, 185–214 (1972)
Barakat, R.: Arabic gestures. J. Pop. Cult. 6, 749–792 (1973)
De Jorio, A.: La mimica degli antichi investigata nel gestire napoletano (Ancient’s mimicry investigated through gesturing Neapolitan). Fibreno, Napoli (1832)
Ricci Bitti, P.E., Poggi, I.A.: Symbolic nonverbal behavior: Talking through gestures. In: Feldman, R.S., Rimé, B. (eds.) Fundamentals of Nonverbal Behavior, pp. 433–457. Cambridge University Press, New York (1991)
Morris, D.: Manwatching. A field guide to human behavior. Andromeda Oxford Limited and Jonathan cape Limited, London (1977)
Haviland, J.B.: Pointing, gesture spaces, and mental maps.. In: McNeill, D. (ed.) Language and Gesture, pp. 13–46. Cambridge University Press, Cambridge (2002)
Rosenfeld, H.M.: Instrumental affiliative functions of facial and gestural expressions. J. Personal. Soc. Psycho. 4, 65–72 (1966)
Freedman, N., Hoffman, S.P.: Kinetic behavior in altered clinical states: approach to objective analysis of motor behavior during clinical interviews. Percept. Motor Skill. 24, 527–539 (1967)
Kimura, D.: Manual activity during speaking. Neuropsychologia 11, 45–55 (1973)
Bull, P., Connelly, G.: Body movement and emphasis in speech. J. Nonverbal Behav. 9, 169–187 (1985)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Maricchiolo, F., Gnisci, A., Bonaiuto, M. (2012). Coding Hand Gestures: A Reliable Taxonomy and a Multi-media Support. In: Esposito, A., Esposito, A.M., Vinciarelli, A., Hoffmann, R., Müller, V.C. (eds) Cognitive Behavioural Systems. Lecture Notes in Computer Science, vol 7403. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-34584-5_36
Download citation
DOI: https://doi.org/10.1007/978-3-642-34584-5_36
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-34583-8
Online ISBN: 978-3-642-34584-5
eBook Packages: Computer ScienceComputer Science (R0)