Recognition and Synthesis of Human Movements by Parametric HMMs

  • Dennis Herzog
  • Volker Krüger
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5604)


The representation of human movements for recognition and synthesis is important in many application fields such as: surveillance, human-computer interaction, motion capture, and humanoid robots. Hidden Markov models (HMMs) are a common statistical framework in this context, since they are generative and are able to deal with the intrinsic dynamic variation of movements performed by humans. In this work we argue that many human movements are parametric, i.e., a parametric variation of the movements in dependence of, e.g., a position a person is pointing at. The parameter is part of the semantic of a movement. And while classic HMMs treat them as noise, we will use parametric HMMs (PHMMs) [6,19] to model the parametric variability of human movements explicitly. In this work, we discuss both types of PHMMs, as introduced in [6] and [19], and we will focus our considerations on the recognition and synthesis of human arm movements. Furthermore, we will show in various experiments the use of PHMMs for the control of a humanoid robot by synthesizing movements for relocating objects at arbitrary positions. In vision-based interaction experiments, PHMM are used for the recognition of pointing movements, where the recognized parameterization conveys to a robot the important information which object to relocate and where to put it. Finally, we evaluate the accuracy of recognition and synthesis for pointing and grasping arm movements and discuss that the precision of the synthesis is within the natural uncertainty of human movements.


Hide Markov Model Motion Capture Humanoid Robot Dynamic Time Warping Hide Markov Model Model 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Asfour, T., Welke, K., Ude, A., Azad, P., Hoeft, J., Dillmann, R.: Perceiving objects and movements to generate actions on a humanoid robot. In: Proc. Workshop: From features to actions – Unifying perspectives in computational and robot vision, ICRA, Rome, Italy (April 2007)Google Scholar
  2. 2.
    Dariush, B.: Human Motion Analysis for Biomechanics and Biomedicine. Machine Vision and Applications 14, 202–205 (2003)CrossRefGoogle Scholar
  3. 3.
    Grest, D., Woetzel, J., Koch, R.: Nonlinear Body Pose Estimation from Depth Images. In: Kropatsch, W.G., Sablatnig, R., Hanbury, A. (eds.) DAGM 2005. LNCS, vol. 3663, pp. 285–292. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  4. 4.
    Gunter, S., Bunke, H.: Optimizing the number of states, training iterations and Gaussians in an HMM-based handwritten word recognizer. In: Proc. Seventh International Conference on Document Analysis and Recognition, vol. 01, pp. 472–476 (2003)Google Scholar
  5. 5.
    Herzog, D., Grest, D., Krueger, V.: An Online Recognition Demo,
  6. 6.
    Herzog, D., Krüger, V., Grest, D.: Parametric Hidden Markov Models for Recognition and Synthesis of Movements. In: Proceedings of the British Machine Vision Conference (BMVC), Leeds, UK, September 2008, vol. 1, pp. 163–172 (2008)Google Scholar
  7. 7.
    Herzog, D., Ude, A., Krueger, V.: HOAP-3 Online Demo,
  8. 8.
    Herzog, D., Ude, A., Krueger, V.: Motion Imitation and Recognition using Parametric Hidden Markov Models. In: Proc. 8th IEEE-RAS International Conference on Humanoid Robots Humanoids 2008, December 2008, pp. 339–346 (2008)Google Scholar
  9. 9.
    Huang, X., Ariki, Y., Jack, M.: Hidden Markov Models for Speech Recognition. Edinburgh University Press (1990)Google Scholar
  10. 10.
    Jacobs, R., Jordan, M., Nowlan, S., Hinton, G.: Adaptive mixtures of local experts. Neural Computation 3, 79–87 (1991)CrossRefGoogle Scholar
  11. 11.
    Keogh, E., Pazzani, M.: Derivative dynamic time warping. In: Proceedings of the 1st SIAM International Conference on Data Mining (SDM 2001), Chicago, USA (2001)Google Scholar
  12. 12.
    Lu, C., Ferrier, N.: Repetitive Motion Analysis: Segmentation and Event Classification. IEEE Transactions on Pattern Analysis and Machine Intelligence 26(2), 258–263 (2004)CrossRefGoogle Scholar
  13. 13.
    Moeslund, T., Hilton, A., Krueger, V.: A survey of advances in vision-based human motion capture and analysis. Computer Vision and Image Understanding 104(2-3), 90–127 (2006)CrossRefGoogle Scholar
  14. 14.
    Murphy, K.P.: Hidden semi-Markov models (HSMMs). In: unpublished notes (2002)Google Scholar
  15. 15.
    Rabiner, L.R., Juang, B.H.: An introduction to hidden Markov models. In: IEEE ASSP Magazine, January 1986, pp. 4–15 (1986)Google Scholar
  16. 16.
    Ramana, P.K.R., Grest, D., Krüger, V.: Human Action Recognition in Table-top Scenarios: An HMM-based Analysis to Optimize the Performance. In: Proceedings of Computer Analysis of Images and Patterns, Vienna, pp. 101–108 (2007)Google Scholar
  17. 17.
    Schaal, S.: Is Imitation Learning the Route to Humanoid Robots? Trends in Cognitive Sciences 3(6), 233–242 (1999)Google Scholar
  18. 18.
    Vecchio, D., Murray, R., Perona, P.: Decomposition of Human Motion into Dynamics-based Primitives with Application to Drawing Tasks. Automatica 39(12), 2085–2098 (2003)MathSciNetzbMATHCrossRefGoogle Scholar
  19. 19.
    Wilson, A.D., Bobick, A.F.: Parametric hidden Markov models for gesture recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence 21(9), 884–900 (1999)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Dennis Herzog
    • 1
  • Volker Krüger
    • 1
  1. 1.Computer Vision and Machine Intelligence LabAalborg University CopenhagenBallerupDenmark

Personalised recommendations