Real-Time Synthesis of Body Movements Based on Learned Primitives

  • Martin A. Giese
  • Albert Mukovskiy
  • Aee-Ni Park
  • Lars Omlor
  • Jean-Jacques E. Slotine
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5604)


The synthesis of realistic complex body movements in real-time is a difficult problem in computer graphics and in robotics. High realism requires the accurate modeling of the details of the trajectories for a large number of degrees of freedom. At the same time, real-time animation necessitates flexible systems that can react in an online fashion, adapting to external constraints. Such online systems are suitable for the self-organization of complex behavior by the dynamic interaction between multiple autonomous characters in the scene. In this paper we present a novel approach for the online synthesis of realistic human body movements. The proposed model is inspired by concepts from motor control. It approximates movements by superposition of movement primitives (synergies) that are learned from motion capture data applying a new blind source separation algorithm. The learned generative model can synthesize periodic and non-periodic movements, achieving high degrees of realism with a very small number of synergies. For obtaining a system that is suitable for real-time synthesis, the primitives are approximated by the solutions of low-dimensional nonlinear dynamical systems (dynamic primitives). The application of a new type of stability analysis (contraction theory) permits the design of complex networks of such dynamic primitives, resulting in a stable overall system architecture. We discuss a number of applications of this framework and demonstrate that it is suitable for the self-organization of complex behaviors, such as navigation, synchronized crowd behavior and dancing.


Support Vector Regression Central Pattern Generator Movement Primitive Motion Capture Data Motor Primitive 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Rose, C., Cohen, M., Bodenheimer, B.: Verbs and adverbs: Multidimensional motion interpolation. IEEE Computer Graphics and Applications 18(5), 32–40 (1998)CrossRefGoogle Scholar
  2. 2.
    Witkin, A., Popovi\(\acute{c}\), Z.: Motion warping. In: Proc. ACM SIGGRAPH 1995, vol. 29, pp. 105–108 (1995)Google Scholar
  3. 3.
    Gleicher, M.: Retargetting motion to new characters. In: Proc. ACM SIGGRAPH 1998, pp. 33–42 (1998)Google Scholar
  4. 4.
    Gleicher, M., Shin, H.J., Kovar, L., Jepsen, A.: Snap-together motion: Assembling run-time animation. ACM Trans. on Graphics, SIGGRAPH 2003 22(3), 702 (2003)Google Scholar
  5. 5.
    Arikan, O., Forsyth, D.A., O’Brien, J.F.: Motion synthesis from annotations. ACM Trans. on Graphics, SIGGRAPH 2003 22(3), 402–408 (2003)zbMATHCrossRefGoogle Scholar
  6. 6.
    Grzeszczuk, R., Terzopoulos, D., Hinton, G.: Neuroanimator: Fast neural network emulation and control of physics based models. In: Int. Conf. on Comp. Graph. and Interactive Techniques, Proc. ACM SIGGRAPH 1998, pp. 9–20 (1998)Google Scholar
  7. 7.
    Shao, W., Terzopoulos, D.: Artificial intelligence for animation: Autonomous pedestrians. Proc. ACM SIGGRAPH 2005 69(5-6), 19–28 (2005)Google Scholar
  8. 8.
    Hsu, E., Pulli, K., Popovi\(\acute{c}\), J.: Style translation for human motion. ACM Trans. on Graphics, SIGGRAPH 2005 24(3), 1082–1089 (2005)Google Scholar
  9. 9.
    Chai, J., Hodgins, J.: Performance animation from low-dimensional control signals. ACM Trans. on Graphics, SIGGRAPH 2005 24(3), 686–696 (2005)CrossRefGoogle Scholar
  10. 10.
    Bernstein, N.A.: The coordination and regulation of movements. Pergamon Press, Oxford (1967)Google Scholar
  11. 11.
    Flash, T., Hochner, B.: Motor primitives in vertebrates and invertebrates. Curr. Opin. Neurobiol. 15(6), 660–666 (2005)CrossRefGoogle Scholar
  12. 12.
    d’Avella, A., Bizzi, E.: Shared and specific muscle synergies in neural motor behaviours. Proc. Natl. Acad. Sci. 102(8), 3076–3081 (2005)CrossRefGoogle Scholar
  13. 13.
    Ivanenko, Y., Poppele, R., Lacquaniti, F.: Five basic muscle activation patterns account for muscle activity during human locomotion. Journal of Physiology 556, 267–282 (2004)CrossRefGoogle Scholar
  14. 14.
    Santello, M., Flanders, M., Soechting, J.: Postural hand synergies for tool use. Journal of Neuroscience 18(23), 10105–10115 (1998)Google Scholar
  15. 15.
    Safanova, A., Hodgins, J.K., Pollard, N.S.: Synthesizing physically realistic human motion in low-dimensional, behavior-specific spaces. ACM Trans. on Graphics, Proc. SIGGRAPH 2004 23(3), 514–521 (2004)CrossRefGoogle Scholar
  16. 16.
    Park, A., Mukovskiy, A., Omlor, L., Giese, M.A.: Synthesis of character behaviour by dynamic interaction of synergies learned from motion capture data. In: The 16-th Int. Conf. in Central Europe on Comp. Graphics, Visualization and Computer Vision 2008, WSCG 2008, pp. 9–16 (2008)Google Scholar
  17. 17.
    Park, A., Mukovskiy, A., Omlor, L., Giese, M.A.: Self organized character animation based on learned synergies from full-body motion capture data. In: International Conference on Cognitive Systems, CogSys 2008 (2008)Google Scholar
  18. 18.
    Unuma, M., Anjyo, K., Takeuchi, R.: Fourier principles for emotion-based human figure animation. Proc. ACM SIGGRAPH 1995 22, 91–96 (1995)Google Scholar
  19. 19.
    Schaal, S., Ijspeert, A., Billard, A.: Computational approaches to motor learning by imitation. Philos. Trans. R. Soc. of London, Biol. Sci. 358(1431), 537–547 (2003)CrossRefGoogle Scholar
  20. 20.
    Ijspeert, A., Nakanishi, J., Schaal, S.: Learning attractor landscapes for learning motor primitives. Adv. Neural Inf. Process Systems 15, 1547–1554 (2002)Google Scholar
  21. 21.
    Omlor, L., Giese, M.A.: Extraction of spatio-temporal primitives of emotional body expressions. Neurocomputing 70, 10–12 (2007)CrossRefGoogle Scholar
  22. 22.
    Omlor, L., Giese, M.A.: Blind source separation for over-determined delayed mixtures. Adv. in Neural Inf. Process Systems 19, 1049–1056 (2006)Google Scholar
  23. 23.
    Schöner, G., Dose, M., Engels, C.: Dynamics of behavior: Theory and applications for autonomous robot architectures. Robotics and Autonomous Systems 16(2-4), 213–245 (1995)CrossRefGoogle Scholar
  24. 24.
    Buchli, J., Righetti, L., Ijspeert, A.J.: Engineering entrainment and adaptation in limit cycle systems - from biological inspiration to applications in robotics. Biological Cybernetics 95(6), 645–664 (2006)MathSciNetzbMATHCrossRefGoogle Scholar
  25. 25.
    Fod, A., Matari\(\acute{c}\), M.J., Jenkins, O.C.: Motor primitives in vertebrates and invertebrates. Autonomous Robots 12(1), 39–54 (2002)Google Scholar
  26. 26.
    Soechting, J., Lacquaniti, F.: Invariant characteristics of a pointing movement in man. Journal of Neuroscience 1, 710–720 (1981)Google Scholar
  27. 27.
    Morasso, P.: Spatial control of arm movements. Exp. Brain Res. 42(2) (1981)Google Scholar
  28. 28.
    Vapnik, N.V.: Statistical Learning Theory. Wiley Interscience, Hoboken (1998)Google Scholar
  29. 29.
    Chang, C.C., Lin, C.J.: LIBSVM: a library for support vector machines (2001),
  30. 30.
    Wang, W., Slotine, J.J.E.: On partial contraction analysis for coupled nonlinear oscillators. Biological Cybernetics 92(1), 38–53 (2005)MathSciNetzbMATHCrossRefGoogle Scholar
  31. 31.
    Warren, W.: The dynamics of perception and action. Psychological Review 113(2), 358–389 (2006)MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Martin A. Giese
    • 1
  • Albert Mukovskiy
    • 1
  • Aee-Ni Park
    • 1
  • Lars Omlor
    • 1
  • Jean-Jacques E. Slotine
    • 2
  1. 1.Section Computational Sensomotorics, Hertie Institute for Clinical Brain Research & Center for Integrative NeuroscienceUniversity of TübingenGermany
  2. 2.Nonlinear Systems LaboratoryMassachusetts Institute of TechnologyUSA

Personalised recommendations