Modeling Human Locomotion with Topologically Constrained Latent Variable Models

  • Raquel Urtasun
  • David J. Fleet
  • Neil D. Lawrence
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4814)


Learned, activity-specific motion models are useful for human pose and motion estimation. Nevertheless, while the use of activity-specific models simplifies monocular tracking, it leaves open the larger issues of how one learns models for multiple activities or stylistic variations, and how such models can be combined with natural transitions between activities. This paper extends the Gaussian process latent variable model (GP-LVM) to address some of these issues. We introduce a new approach to constraining the latent space that we refer to as the locally-linear Gaussian process latent variable model (LL-GPLVM). The LL-GPLVM allows for an explicit prior over the latent configurations that aims to preserve local topological structure in the training data. We reduce the computational complexity of the GPLVM by adapting sparse Gaussian process regression methods to the GP-LVM. By incorporating sparsification, dynamics and back-constraints within the LL-GPLVM we develop a general framework for learning smooth latent models of different activities within a shared latent space, allowing the learning of specific topologies and transitions between different activities.


Radial Basis Function Latent Space Gaussian Process Human Motion Kernel Matrix 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Arikan, O., Forsyth, D.: Interactive motion generation from examples. In: SIGGRAPH, pp. 483–490 (2002)Google Scholar
  2. 2.
    Bissacco, A.: Modeling and Learning Contact Dynamics in Human Motion. In: CVPR, pp. 421–428 (2005)Google Scholar
  3. 3.
    Brand, M., Hertzmann, A.: Style Machines. In: SIGGRAPH, pp. 183–192 (2000)Google Scholar
  4. 4.
    Elgammal, A., Lee, C.: Separating Style and Content on a Nonlinear Manifold. In: CVPR, vol. 1, pp. 478–485 (2004)Google Scholar
  5. 5.
    Grochow, K., Martin, S., Hertzmann, A., Popovic, Z.: Style-based inverse kinematics. In: SIGGRAPH, pp. 522–531 (2004)Google Scholar
  6. 6.
    Vasilescu, M.A.: Human Motion Signatures: Analysis, Synthesis. In: ICPR, pp. 456–460 (2002)Google Scholar
  7. 7.
    Kovar, L., Gleicher, M., Pighin, F.: Motion Graphs. In: SIGGRAPH, pp. 473–482 (2002)Google Scholar
  8. 8.
    Lawrence, N., Quinonero-Candela, J.: Local distance preservation in the GP-LVM through back constraints. In: ICML, pp. 513–520 (2006)Google Scholar
  9. 9.
    Lawrence, N.D., Seeger, M., Herbrich, R.: Fast sparse Gaussian process methods: The informative vector machine. In: NIPS, pp. 609–616 (2003)Google Scholar
  10. 10.
    Lawrence, N.D.: Gaussian Process Models for Visualisation of High Dimensional Data. In: NIPS (2004)Google Scholar
  11. 11.
    Lawrence, N.: Learning for larger datasets with the Gaussian process latent variable model. In: AISTATS (2007)Google Scholar
  12. 12.
    Li, R., Yang, M.H., Sclaroff, S., Tian, T.: Monocular Tracking of 3D Human Motion with a Coordinated Mixture of Factor Analyzers. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3952, pp. 137–150. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  13. 13.
    Li, Y., Wang, T., Shum, H.: Motion Texture: A Two-Level Statistical Model for Character Motion Synthesis. In: SIGGRAPH, pp. 465–472 (2002)Google Scholar
  14. 14.
    Moon, K., Pavlovic, V.: Impact of Dynamics on Subspace Embedding and Tracking of Sequences. In: CVPR, pp. 198–205 (2006)Google Scholar
  15. 15.
    Pavlovic, J.M., Rehg, J., MacCormick, J.: Learning switching linear models of human motion. In: NIPS, pp. 981–987 (2000)Google Scholar
  16. 16.
    Quiñonero-Candela, J., Rasmussen, C.E.: A Unifying View of Sparse Approximate Gaussian Process Regression. In: JMLR, vol. 6, pp. 1939–1959 (2006)Google Scholar
  17. 17.
    Rasmussen, C.E., Williams, C.K.: Gaussian Process for Machine Learning. MIT Press, Cambridge (2006)Google Scholar
  18. 18.
    Roweis, S., Saul, L.: Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500), 2323–2326 (2000)CrossRefGoogle Scholar
  19. 19.
    Sidenbladh, H., Black, M.J., Sigal, L.: Implicit Probabilistic Models of Human Motion for Synthesis and Tracking. In: Tistarelli, M., Bigun, J., Jain, A.K. (eds.) ECCV 2002. LNCS, pp. 784–800. Springer, Heidelberg (2002)Google Scholar
  20. 20.
    Sminchisescu, C., Jepson, A.: Generative Modeling for Continuous Non-Linearly Embedded Visual Inference. In: ICML, pp. 96–103 (2004)Google Scholar
  21. 21.
    Snelson, E., Ghahramani, Z.: Sparse Gaussian processes using pseudo-inputs. In: NIPS, pp. 1257–1264 (2006)Google Scholar
  22. 22.
    Taylor, G.W., Hinton, G.E., Roweis, S.: Modeling human motion using binary latent variables. In: NIPS, pp. 1345–1352 (2007)Google Scholar
  23. 23.
    Tenenbaum, J., de Silva, V., Langford, J.: A global geometric framework for nonlinear dimensionality reduction. Science 290(5500), 2319–2323 (2000)CrossRefGoogle Scholar
  24. 24.
    Tian, T., Li, R., Sclaroff, S.: Articulated Pose Estimation in a Learned Smooth Space of Feasible Solutions. In: CVPR Learning Workshop (2005)Google Scholar
  25. 25.
    Urtasun, R.: Motion Models for Robust 3D Human Body Tracking. PhD thesis, EPFL (2006)Google Scholar
  26. 26.
    Urtasun, R., Fleet, D.J., Fua, P.: 3d people tracking with gaussian process dynamical models. In: CVPR, vol. 1, pp. 238–245 (2006)Google Scholar
  27. 27.
    Urtasun, R., Fleet, D.J., Hertzman, A., Fua, P.: Priors for people tracking from small training sets. In: ICCV, pp. 403–410 (2005)Google Scholar
  28. 28.
    Wang, J., Fleet, D.J., Hertzman, A.: Gaussian Process dynamical models. In: NIPS, pp. 1441–1448 (2005)Google Scholar
  29. 29.
    Wang, J., Fleet, D.J., Hertzman, A.: Multifactor Gaussian Process Models for Style-Content Separation. In: ICML, pp. 975–982 (2007)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2007

Authors and Affiliations

  • Raquel Urtasun
    • 1
  • David J. Fleet
    • 2
  • Neil D. Lawrence
    • 3
  1. 1.Massachusetts Institute of Technology, Cambridge, MA 02139USA
  2. 2.University of Toronto, M5S 3H5Canada
  3. 3.School of Computer Science, University of Manchester, M13 9PLU.K.

Personalised recommendations