Human Motion Tracking with Monocular Video by Introducing a Graph Structure into Gaussian Process Dynamical Models

  • Jianfeng Xu
  • Koichi Takagi
  • Shigeyuki Sakazawa
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7087)


This paper presents a novel approach to tracking articulated human motion with monocular video. In a conventional tracking system based on particle filters, it is very challenging to track a complex human pose with many degrees of freedom. A typical solution to this problem is to track the pose in a low dimensional latent space by manifold learning techniques, e.g., the Gaussian process dynamical model (GPDM model). In this paper, we extend the GPDM model into a graph structure (called GPDM graph) to better express the diverse dynamics of human motion, where multiple latent spaces are constructed and dynamically connected to each other appropriately by an unsupervised learning method. Basically, the proposed model has both intra-transitions (in each latent space) and inter-transitions (among latent spaces). Moreover, the probability of inter-transition is dynamic, depending on the current latent state. Using the proposed GPDM graph model, we can track human motion with monocular video, where the average tracking errors are improved from the state-of-the-art methods in our experiments.


motion tracking monocular video manifold learning Gaussian process dynamical model motion graph 


  1. 1.
    Agarwal, A., Triggs, B.: Recovering 3d human pose from monocular images. IEEE Trans. on PAMI 28(1), 44–58 (2006)CrossRefGoogle Scholar
  2. 2.
    Arikan, O.: Compression of motion capture databases. ACM Trans. on Graphics 25(3), 890–897 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  3. 3.
    Arikan, O., Forsyth, D.A.: Interactive motion generation from examples. ACM Trans. on Graphics 21(3), 483–490 (2002)CrossRefzbMATHGoogle Scholar
  4. 4.
    Arulampalam, M., Maskell, S., Gordon, N., Clapp, T.: A tutorial on particle filters for online nonlinear/non-gaussian bayesian tracking. IEEE Trans. on Signal Processing 50(2), 174–188 (2002)CrossRefGoogle Scholar
  5. 5.
    Balan, A., Black, M.J.: An adaptive appearancemodel approach formodel-based articulated object tracking. In: IEEE CVPR, vol. 1, pp. 758–765 (2006)Google Scholar
  6. 6.
    Moeslund, T.B., Hilton, A., Kruger, V.: A survey of advances in vision based human motion capture and analysis. CVIU 104(2), 90–126 (2006)Google Scholar
  7. 7.
    Chen, J., Kim, M., Wang, Y., Ji, Q.: Switching gaussian process dynamic models for simultaneous composite motion tracking and recognition. In: IEEE CVPR, pp. 2655–2662 (2009)Google Scholar
  8. 8.
    Kovar, L., Gleicherl, M., Pighin, F.: Motion graphs. ACM Trans. on Graphics 21(3), 473–482 (2002)CrossRefGoogle Scholar
  9. 9.
    Lawrence, N.: Gaussian process latent variable models for visualization. In: Proc. Adv. Neural Inf. Process. pp. 329–336 (2003)Google Scholar
  10. 10.
    Lee, J., Chai, J., Reitsma, P.S.A., Hodgins, J.K., Pollard, N.S.: Interactive control of avatars animated with human motion data. ACM Trans. on Graphics 21(3), 491–500 (2002)Google Scholar
  11. 11.
    Poppe, R.: Vision-based human motion analysis: an overview. CVIU 108(1/2), 4–18 (2007)Google Scholar
  12. 12.
    Quirion, S., Duchesne, C., Laurendeau, D., Marchand, M.: Comparing gplvm approaches for dimensionality reduction in character animation. Journal of WSCG 16(1-3), 41–48 (2008)Google Scholar
  13. 13.
    Sigal, L., Balan, A., Black, M.J.: Humaneva: Synchronized video and motion capture dataset and baseline algorithm for evaluation of articulated human motion. International Journal of Computer Vision 87(1), 4–27 (2010)CrossRefGoogle Scholar
  14. 14.
    Taylor, G.W., Sigal, L., Fleet, D.J., Hinton, G.E.: Dynamical binary latent variable models for 3d human pose tracking. In: IEEE CVPR, pp. 631–638 (2010)Google Scholar
  15. 15.
    Urtasun, R., Darrell, T.: Sparse probabilistic regression for activity independent human pose inference. In: IEEE CVPR, pp. 1–8 (2008)Google Scholar
  16. 16.
    Urtasun, R., Fleet, D., Fua, P.: 3d people tracking with gaussian process dynamical models. In: IEEE CVPR, pp. 238–245 (2006)Google Scholar
  17. 17.
    Wang, J.M., Fleet, D.J., Hertzmann, A.: Gaussian process dynamical models for human motion. IEEE Trans. on PAMI 30(2), 283–298 (2008)CrossRefGoogle Scholar
  18. 18.
    Wang, J., Bodenheimer, B.: Synthesis and evaluation of linear motion transitions. ACM Trans. on Graphics 27(1), 1:1–1:15 (2008)Google Scholar
  19. 19.
    Xu, J., Takagi, K., Yoneyama, A.: Beat induction from motion capture data using short-term principal component analysis. The Journal of The Institute of Image Information and Television Engineers 64(4), 577–583 (2010)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Jianfeng Xu
    • 1
  • Koichi Takagi
    • 1
  • Shigeyuki Sakazawa
    • 1
  1. 1.KDDI R&D Laboratories Inc.Japan

Personalised recommendations