Abstract
This paper deals with the results of a machine learning experiment conducted on annotated gesture data from two case studies (Danish and Estonian). The data concern mainly facial displays, that are annotated with attributes relating to shape and dynamics, as well as communicative function. The results of the experiments show that the granularity of the attributes used seems appropriate for the task of distinguishing the desired communicative functions. This is a promising result in view of a future automation of the annotation task.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Allwood, J.: Dialog Coding – Function and Grammar. Gothenburg Papers. In: Theoretical Linguistics. Department of Linguistics, vol. 85, Gothenburg University (2001a)
Allwood, J.: The Structure of Dialog. In: Taylor, M., Bouwhuis, D., Nel, F. (eds.) The Structure of Multimodal Dialogue II, pp. 3–24. Amsterdam, Benjamins (2001b)
Allwood, J., Cerrato, L.: A study of gestural feedback expressions. In: Paggio, P., et al. (eds.) Proceedings of the First Nordic Symposium on Multimodal Communication, pp. 7–22 (2003)
Allwood, J., Nivre, J., Ahlsén, E.: On the Semantics and Pragmatics of Linguistic Feedback. Journal of Semantics 9, 1–26 (1992)
Allwood, J., Cerrato, L., Jokinen, K., Navarretta, C., Paggio, P.: The MUMIN coding scheme for the annotation of feedback, turn management and sequencing phenomena. In: Martin, J.C., et al. (eds.) Multimodal Corpora for Modelling Human Multimodal Behaviour. Special issue of the International Journal of Language Resources and Evaluation, vol. 41(3–4), pp. 273–287. Springer, Heidelberg (2007)
Campbell, N.: Tools and Resources for Visualising Conversational-Speech Interaction. In: LREC 2008, Marrakesh, Morocco (2008)
Carletta, J.: Assessing agreement on classification tasks: the kappa statistics. Computational Linguistics 22(2), 249–254 (1996)
Carletta, J.: Unleashing the killer corpus: experiences in creating the multi-everything AMI Meeting Corpus. In: Language Resources and Evaluation, vol. 41(2), pp. 181–190. Springer, Heidelberg (2007)
Cerrato, L.: Investigating Communicative Feedback Phenomena across Languages and Modalities. PhD Thesis in Speech and Music Communication, Stockholm, KTH (2007)
Clark, H.H., Schaefer, E.F.: Contributing to Discourse. Cognitive Science 13, 259–294 (1989)
Douxchamps, D., Campbell, N.: Robust real-time tracking for the analysis of human behaviour. In: Popescu-Belis, A., Renals, S., Bourlard, H. (eds.) MLMI 2007. LNCS, vol. 4892. Springer, Heidelberg (2008)
Duncan Jr., S., Fiske, D.W.: Face-to-Face Interaction: Research, Methods and Theory. John Wiley & Sons/Lawrence Erlbaum, Mahwah (1977)
Jokinen, K., Ragni, A.: Clustering experiments on the communicative properties of gaze and gestures. In: Proceeding of the 3rd. Baltic Conference on Human Language Technologies, Kaunas (2007)
Kendon, A.: Gesture, Cambridge (2004)
Kipp, M.: Anvil – A Generic Annotation Tool for Multimodal Dialogue. In: Proceedings of Eurospeech 2001, pp. 1367–1370 (2001)
Martin, J.C., Paggio, P., Kuenlein, P., Stiefelhagen, R., Pianesi, F. (eds.): Multimodal Corpora for Modelling Human Multimodal Behaviour. Special issue of the International Journal of Language Resources and Evaluation, vol. 41(3–4). Springer, Heidelberg (2007)
McNeill, D.: Hand and Mind: What Gestures Reveal About Thought. University of Chicago Press, Chicago (1992)
Mostefa, D., Moreau, N., Choukri, K., Potamianos, G., Chu, S.M., Tyagi, A., Casas, J.R., Turmo, J., Cristoforetti, L., Tobia, F., Pnevmatikakis, A., Mylonakis, V., Talantzis, F., Burger, S., Stiefelhagen, R., Bernardin, K., Rochet, C.: The CHIL audiovisual corpus for lecture and meeting analysis inside smart rooms. In: Language Resources and Evaluation, vol. 41(3-4), pp. 389–407. Springer, Heidelberg (2007)
Peirce, C.S.: Elements of Logic. In: Hartshorne, C., Weiss, P. (eds.) Collected Papers of Charles Sanders Peirce, vol. 2, Harvard University Press, Cambridge (1931)
Witten, I.H., Frank, E.: Data Mining: Practical machine learning tools and techniques, 2nd edn. Morgan Kaufmann, San Francisco (2005)
Platt, J.: Sequential minimal optimization: A fast algorithm for training support vector machines. Technical Report MSR-TR-98-14, Microsoft Research (1998)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Jokinen, K., Navarretta, C., Paggio, P. (2008). Distinguishing the Communicative Functions of Gestures. In: Popescu-Belis, A., Stiefelhagen, R. (eds) Machine Learning for Multimodal Interaction. MLMI 2008. Lecture Notes in Computer Science, vol 5237. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-85853-9_4
Download citation
DOI: https://doi.org/10.1007/978-3-540-85853-9_4
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-85852-2
Online ISBN: 978-3-540-85853-9
eBook Packages: Computer ScienceComputer Science (R0)