Abstract
Synchronization of synthetic gestures with speech output is one of the goals for embodied conversational agents which have become a new paradigm for the study of gesture and for human-computer interface. In this context, this contribution presents an operational model that enables lifelike gesture animations of an articulated figure to be rendered in real-time from representations of spatiotemporal gesture knowledge. Based on various findings on the production of human gesture, the model provides means for motion representation, planning, and control to drive the kinematic skeleton of a figure which comprises 43 degrees of freedom in 29j oints for the main body and 20 DOF for each hand. The model is conceived to enable cross-modal synchrony with respect to the coordination of gestures with the signal generated by a text-to-speech system.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
P. Becheiraz and D. Thalmann. A behavioral animation system for autonomous actors personified by emotions. In Proc. First Workshop on Embodied Conversational Characters (WECC’ 98), Lake Tahoe, CA, 1998.
J. Beskow and S. McGlashan. Olga-a conversational agent with gestures. In E. André, editor, Proc. of IJCAI-97 Workshop on Animated Interface Agents: Making Them Intelligent, 1997.
J. Cassell, J. Sullivan, S. Prevost, and E. Churchill, editors. Embodied Conversatinal Agents. The MIT Press, Cambridge (MA), 2000.
J. deRuiter. The production of gesture and speech. In D. McNeill, editor, Language and gesture, chapter 14, pages 284–311. Cambridge University Press, 2000.
T. Dutoit. An Introdutcion to Text-To-Speech Synthesis. Kluwer Academic Publishers, Dordrecht, 1997.
S. Gibet, T. Lebourque, and P. Marteau. High-level specification and animation of communicative gestures. Journal of Visual Languages and Computing, 12(6):657–687, 2001.
S. Kopp and I. Wachsmuth. A knowledge-based approach for lifelike gesture animation. In W. Horn, editor, ECAI 2000 Proceedings of the 14th European Conference on Artificial Intelligence, pages 661–667, Amsterdam, 2000. IOS Press.
M. Latoschik, M. Fröhlich, B. Jung, and I. Wachsmuth. Utilize speech and gestures to realize natural interaction in a virtual environment. In IECON’98-Proceedings of the 24th Annual Conference of the IEEE Industrial Electronics Society, volume 4, pages 2028–2033. IEEE, 1998.
D. McNeill. Hand and Mind: What Gestures Reveal about Thought. University of Chicago Press, Chicago, 1992.
C. Pelachaud and S. Prevost. Talking heads: Physical, linguistic and cognitive issues in facial animation. Course Notes, Computer Graphics International, June 1995.
K. Perlin and A. Goldberg. Improv: A system for scripting interactive actors in virtual worlds. In SIGGRAPH 96, Proceedings of the 23rd annual conference on Computer Graphics, pages 205–216, 1996.
T. Portele, F. Höfer, and W. Hess. A mixed inventory structure for german concatenative synthesis. In Proceedings of the 2nd ESCA/IEEE Workshop on Speech Synthesis, pages 115–118, 1994.
S. Prillwitz, R. Leven, H. Zienert, T. Hamke, and J. Henning. HamNoSys Version 2.0: Hamburg Notation System for Sign Languages: An Introductory Guide, volume 5 of International Studies on Sign Language and Communication of the Deaf. Signum Press, Hamburg, Germany, 1989.
J. Rickel and W. Johnson. Animated agents for procedural training in virtual reality: Perception, cognition, and motor control. Applied Artificial Intelligence, 13:343–382, 1999.
Sable Consortium. Sable: A synthesis markup language (version 10.). 〈http://www.bell-labs.com/project/tts/sable.html〉, 2000. Bell Laboratories, 25.2.2001.
T. Sowa and I. Wachsmuth. Interpretation of shape-related iconic gestures in virtual environments. This volume.
I. Wachsmuth. Communicative rhythm in gesture and speech. In A. Braffort et al., editor, Gesture-Based Communication in Human-Computer Interaction-Proceedings International Gesture Workshop, pages 277–289, Berlin, March 1999. Springer (LNAI 1739).
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2002 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Wachsmuth, I., Kopp, S. (2002). Lifelike Gesture Synthesis and Timing for Conversational Agents. In: Wachsmuth, I., Sowa, T. (eds) Gesture and Sign Language in Human-Computer Interaction. GW 2001. Lecture Notes in Computer Science(), vol 2298. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-47873-6_13
Download citation
DOI: https://doi.org/10.1007/3-540-47873-6_13
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-43678-2
Online ISBN: 978-3-540-47873-7
eBook Packages: Springer Book Archive