Advertisement

Motion Models for People Tracking

  • David J. Fleet

Abstract

This chapter provides an introduction to models of human pose and motion for use in 3D human pose tracking. We concentrate on probabilistic latent variable models of kinematics, most of which are learned from motion capture data, and on recent physics-based models. We briefly discuss important open problems and future research challenges.

Keywords

Gaussian Process Latent Variable Model Locally Linear Embedding Linear Dynamic System Restrict Boltzmann Machine 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Notes

Acknowledgements

Many thanks to research colleagues with whom I have worked on modeling human motion: Michael Black, Marcus Brubaker, Aaron Hertzmann, Geoff Hinton, Hedvig Kjellström. Neil Lawrence, Roland Memisevic, Leonid Sigal, Graham Taylor, Niko Troje, Raquel Urtasun, and Jack Wang. We gratefully acknowledge generous financial support from NSERC Canada and the Canadian Institute for Advanced Research (CIfAR).

References

  1. 1.
    Andriluka, M., Roth, S., Schiele, B.: Monocular 3d pose estimation and tracking by detection. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2010) Google Scholar
  2. 2.
    Brubaker, M.A., Fleet, D.J.: The kneed walker for human pose tracking. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2008) Google Scholar
  3. 3.
    Brubaker, M.A., Fleet, D.J., Hertzmann, A.: Physics-based person tracking using the anthropomorphic walker. Int. J. Comput. Vis. 87(1–2), 140–155 (2010) CrossRefGoogle Scholar
  4. 4.
    Brubaker, M.A., Sigal, L., Fleet, D.J.: Estimating contact dynamics. In: IEEE International Conference on Computer Vision, pp. 2389–2396 (2009) CrossRefGoogle Scholar
  5. 5.
    Brubaker, M.A., Sigal, L., Fleet, D.J.: Physics-based human motion modeling for people tracking: A short tutotial. In: Notes from IEEE ICCV Tutorial (2009). (Available from http://www.cs.toronto.edu/~ls/iccv2009tutorial/) Google Scholar
  6. 6.
    Choo, K., Fleet, D.J.: People tracking using hybrid Monte Carlo filtering. In: IEEE International Conference on Computer Vision, vol. II, pp. 321–328 (2001) Google Scholar
  7. 7.
    Collins, S.H., Ruina, A.: A bipedal walking robot with efficient and human-like gait. In: International Conference on Robotics and Automation (2005) Google Scholar
  8. 8.
    Corazza, S., Muendermann, L., Chaudhari, A., Demattio, T., Cobelli, C., Andriacchi, T.: A markerless motion capture system to study musculoskeletal biomechanics: Visual hull and simulated annealing approach. Ann. Biomed. Eng. 34(6), 1019–1029 (2006) CrossRefGoogle Scholar
  9. 9.
    Darby, J., Li, B., Costens, N., Fleet, D.J., Lawrence, N.D.: Backing off: Hierarchical decomposition of activity for 3d novel pose recovery. In: British Machine Vision Conference (2009) Google Scholar
  10. 10.
    de La Gorce, M., Fleet, D.J., Paragios, N.: Model-based 3d hand pose estimation from monocular video. IEEE Trans. Pattern Anal. Mach. Intell. 33(9), 1793–1805 (2011) Google Scholar
  11. 11.
    Deutscher, J., Reid, I.: Articulated body motion capture by stochastic search. Int. J. Comput. Vis. 61(2), 185–205 (2005) CrossRefGoogle Scholar
  12. 12.
    Doretto, G., Chiuso, A., Wu, Y.N., Soatto, S.: Dynamic textures. Int. J. Comput. Vis. 51(2), 91–109 (2003) MATHCrossRefGoogle Scholar
  13. 13.
    Doucet, A., Godsill, S., Andrieu, C.: On sequential Monte Carlo sampling methods for Bayesian filtering. Stat. Comput. 10(3), 197–208 (2000) CrossRefGoogle Scholar
  14. 14.
    Elgammal, A.M., Lee, C.-S.: Inferring 3d body pose from silhouettes using activity manifold learning. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 681–688 (2004) Google Scholar
  15. 15.
    Elgammal, A.M., Lee, C.-S.: Separating style and content on a nonlinear manifold. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 478–485 (2004) Google Scholar
  16. 16.
    Fox, E.B., Sudderth, E.B., Jordan, M.I., Willsky, A.S.: Nonparametric Bayesian learning of switching linear dynamical systems. In: Advances in Neural Information Processing Systems, pp. 457–464 (2008) Google Scholar
  17. 17.
    Full, R.J., Koditschek, D.E.: Templates and anchors: Neuromechanical hypotheses of legged locomotion on land. J. Exp. Biol. 202, 3325–3332 (1999) Google Scholar
  18. 18.
    Gall, J., Rosenhahn, B., Brox, T., Seidel, H.-P.: Optimization and filtering for human motion capture. Int. J. Comput. Vis. 87(1–2), 75–92 (2010) CrossRefGoogle Scholar
  19. 19.
    Gordon, N.J., Salmond, D.J., Smith, A.F.M.: Novel approach to nonlinear/non-Gaussian Bayesian state estimation. IEE Proc., F, Radar Signal Process. 140, 107–113 (1993) CrossRefGoogle Scholar
  20. 20.
    Grochow, K., Martin, S.L., Hertzmann, A., Popovic, Z.: Style-based inverse kinematics. ACM Trans. Graph. 23(3), 522–531 (2004) CrossRefGoogle Scholar
  21. 21.
    Hauberg, S., Sommer, S., Pedersen, K.S.: Gaussian-like spatial priors for articulated tracking. In: European Conference on Computer Vision, vol. 1, pp. 425–437 (2010) Google Scholar
  22. 22.
    Herda, L., Urtasun, R., Fua, P.: Hierarchical implicit surface joint limits for human body tracking. Comput. Vis. Image Underst. 99(2), 189–209 (2005) CrossRefGoogle Scholar
  23. 23.
    Hinton, G.E.: Training products of experts by minimizing contrastive divergence. Neural Comput. 14(8), 1771–1800 (2002) MATHCrossRefGoogle Scholar
  24. 24.
    Hinton, G.E.: A practical guide to training restricted Boltzmann machines. Technical Report UTML TR 2010-003, Department of Computer Science, University of Toronto (2010) Google Scholar
  25. 25.
    Hinton, G.E., Osindero, S., Teh, Y.W.: A fast learning algorithm for deep belief nets. Neural Comput. 18(7), 1527–1554 (2006) MathSciNetMATHCrossRefGoogle Scholar
  26. 26.
    Hou, S., Galata, A., Caillette, F., Thacker, N.A., Bromiley, P.A.: Real-time body tracking using a Gaussian process latent variable model. In: IEEE International Conference on Computer Vision (2007) Google Scholar
  27. 27.
    Howe, N.R., Leventon, M.E., Freeman, W.T.: Bayesian reconstruction of 3d human motion from single-camera video. In: Advances in Neural Information Processing Systems, pp. 820–826 (1999) Google Scholar
  28. 28.
    Hyndman, M., Jepson, A.D., Fleet, D.J.: Higher-order autoregressive models for dynamic textures. In: British Machine Vision Conference (2007) Google Scholar
  29. 29.
    Isard, M., Blake, A.: CONDENSATION – Conditional density propagation for visual tracking. Int. J. Comput. Vis. 29(1), 5–28 (1998) CrossRefGoogle Scholar
  30. 30.
    Kakadiaris, L., Metaxas, D.: Model-based estimation of 3D human motion. IEEE Trans. Pattern Anal. Mach. Intell. 22(12), 1453–1459 (2000) CrossRefGoogle Scholar
  31. 31.
    Kong, A., Liu, J.S., Wong, W.H.: Sequential imputations and Bayesian missing data problems. J. Am. Stat. Assoc. 89(425), 278–288 (1994) MATHCrossRefGoogle Scholar
  32. 32.
    Kuo, A.D.: A simple model of bipedal walking predicts the preferred speed–step length relationship. J. Biomech. Eng. 123(3), 264–269 (2001) CrossRefGoogle Scholar
  33. 33.
    Lawrence, N.D.: Probabilistic non-linear principal component analysis with Gaussian process latent variable models. J. Mach. Learn. Res. 6, 1783–1816 (2005) MathSciNetGoogle Scholar
  34. 34.
    Lawrence, N.D., Moore, A.J.: Hierarchical Gaussian process latent variable models. In: International Conference on Machine Learning, pp. 481–488 (2007) Google Scholar
  35. 35.
    Lawrence, N.D., Quiñonero-Candela, J.: Local distance preservation in the gp-lvm through back constraints. In: International Conference on Machine Learning, pp. 513–520 (2006) Google Scholar
  36. 36.
    Li, R.: Simultaneous learning of non-linear manifold and dynamical models for high-dimensional time series. PhD thesis, Boston University (2009) Google Scholar
  37. 37.
    Li, R., Tian, T.-P., Sclaroff, S., Yang, M.-H.: 3d human motion tracking with a coordinated mixture of factor analyzers. Int. J. Comput. Vis. 87(1–2), 170–190 (2010) CrossRefGoogle Scholar
  38. 38.
    Li, R., Yang, M.-H., Sclaroff, S., Tian, T.-P.: Monocular tracking of 3d human motion with a coordinated mixture of factor analyzers. In: European Conference on Computer Vision, vol. 2, pp. 137–150 (2006) Google Scholar
  39. 39.
    Liu, C.K., Hertzmann, A., Popović, Z.: Learning physics-based motion style with nonlinear inverse optimization. ACM Trans. Graph. 24(3), 1071–1081 (2005) CrossRefGoogle Scholar
  40. 40.
    Lu, Z., Carreira-Perpiñán, M.Á., Sminchisescu, C.: People tracking with the Laplacian eigenmaps latent variable model. In: Advances in Neural Information Processing Systems, pp. 1705–1712 (2007) Google Scholar
  41. 41.
    MacKay, D.J.C.: Information Theory, Inference, and Learning Algorithms. Cambridge University Press, Cambridge (2003) MATHGoogle Scholar
  42. 42.
    McGeer, T.: Dynamics and control of bipedal locomotion. J. Theor. Biol. 163, 277–314 (1993) CrossRefGoogle Scholar
  43. 43.
    Metaxas, D., Terzopoulos, D.: Shape and nonrigid motion estimation through physics-based synthesis. IEEE Trans. Pattern Anal. Mach. Intell. 15(6), 580–591 (1993) CrossRefGoogle Scholar
  44. 44.
    North, B., Blake, A., Isard, M., Rittscher, J.: Learning and classification of complex dynamics. IEEE Trans. Pattern Anal. Mach. Intell. 22(9), 1016–1034 (2000) CrossRefGoogle Scholar
  45. 45.
    Oh, S.M., Rehg, J.M., Balch, T.R., Dellaert, F.: Learning and inferring motion patterns using parametric segmental switching linear dynamic systems. Int. J. Comput. Vis. 77(1–3), 103–124 (2008) CrossRefGoogle Scholar
  46. 46.
    Pan, W., Torresani, L.: Unsupervised hierarchical modeling of locomotion styles. In: International Conference on Machine Learning, p. 99 (2009) Google Scholar
  47. 47.
    Pavlovic, V., Rehg, J.M., MacCormick, J.: Learning switching linear models of human motion. In: Advances in Neural Information Processing Systems, pp. 981–987 (2000) Google Scholar
  48. 48.
    Poon, E., Fleet, D.J.: Hybrid Monte Carlo filtering: Edge-based people tracking. In: Workshop on Motion and Video Computing, pp. 151–158 (2002) CrossRefGoogle Scholar
  49. 49.
    Quiñonero-Candela, J., Rasmussen, C.E.: A unifying view of sparse approximate Gaussian process regression. J. Mach. Learn. Res. 6, 1939–1959 (2005) MathSciNetGoogle Scholar
  50. 50.
    Raskin, L.M., Rivlin, E., Rudzsky, M.: Using Gaussian process annealing particle filter for 3d human tracking. EURASIP J. Adv. Sig. Proc. (2008) Google Scholar
  51. 51.
    Rasmussen, C.E., Williams, C.K.I.: Gaussian Processes for Machine Learning. MIT Press, Cambridge (2006) MATHGoogle Scholar
  52. 52.
    Robertson, D.G.E., Caldwell, G.E., Hamill, J., Kamen, G., Whittlesey, S.N.: Research Methods in Biomechanics. Human Kinetics, Champaign (2004) Google Scholar
  53. 53.
    Rose, C., Cohen, M.F., Bodenheimer, B.: Verbs and adverbs: Multidimensional motion interpolation. IEEE Comput. Graph. Appl. 18(5), 32–40 (1998) CrossRefGoogle Scholar
  54. 54.
    Roweis, S.T., Saul, L.K.: Nonlinear dimensionality reduction by locally linear embedding. Science 290(550), 2323–2326 (2000) CrossRefGoogle Scholar
  55. 55.
    Roweis, S.T., Saul, L.K., Hinton, G.E.: Global coordination of local linear models. In: Advances in Neural Information Processing Systems, pp. 889–896 (2001) Google Scholar
  56. 56.
    Shoemake, K.: Animating rotation with quaternion curves. In: ACM Transactions on Graphics, pp. 245–254 (1985) Google Scholar
  57. 57.
    Sidenbladh, H., Black, M.J., Fleet, D.J.: Stochastic tracking of 3d human figures using 2d image motion. In: European Conference on Computer Vision, vol. 2, pp. 702–718 (2000) Google Scholar
  58. 58.
    Sigal, L., Balan, A.O., Black, M.J.: Humaneva: Synchronized video and motion capture dataset and baseline algorithm for evaluation of articulated human motion. Int. J. Comput. Vis. 87(1–2), 4–27 (2010) CrossRefGoogle Scholar
  59. 59.
    Sigal, L., Fleet, D.J., Troje, N., Livne, M.: Human attributes from 3d pose tracking. In: European Conference on Computer Vision (2010) Google Scholar
  60. 60.
    Sminchisescu, C., Jepson, A.: Generative modeling for continuous non-linearly embedded visual inference. In: International Conference on Machine Learning, pp. 759–766 (2004) Google Scholar
  61. 61.
    Sminchisescu, C., Triggs, B.: Kinematic jump processes for monocular 3D human tracking. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 69–76, Madison (2003) Google Scholar
  62. 62.
    Taylor, G.W., Hinton, G.E.: Factored conditional restricted Boltzmann machines for modeling motion style. In: International Conference on Machine Learning (2009) Google Scholar
  63. 63.
    Taylor, G.W., Hinton, G.E., Roweis, S.T.: Modeling human motion using binary latent variables. In: Advances in Neural Information Processing Systems, pp. 1345–1352 (2006) Google Scholar
  64. 64.
    Taylor, G.W., Sigal, L., Fleet, D.J., Hinton, G.E.: Dynamical binary latent variable models for 3d human pose tracking. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 631–638 (2010) CrossRefGoogle Scholar
  65. 65.
    Tenenbaum, J.B., Freeman, W.T.: Separating style and content with bilinear models. Neural Comput. 12(6), 1247–1283 (2000) CrossRefGoogle Scholar
  66. 66.
    Troje, N.: Decomposing biological motion: A framework for analysis and synthesis of human gait patterns. J. Vis. 2(5), 371–387 (2002) CrossRefGoogle Scholar
  67. 67.
    Urtasun, R., Fleet, D.J., Fua, P.: 3D people tracking with Gaussian process dynamical models. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 238–245 (2006) Google Scholar
  68. 68.
    Urtasun, R., Fleet, D.J., Hertzmann, A., Fua, P.: Priors for people tracking from small training sets. In: IEEE International Conference on Computer Vision, vol. 1, pp. 403–410 (2005) Google Scholar
  69. 69.
    Urtasun, R., Fleet, D.J., Fua, P.: Motion models for 3D people tracking. Comput. Vis. Image Underst. 104(2–3), 157–177 (2006) CrossRefGoogle Scholar
  70. 70.
    Urtasun, R., Fleet, D.J., Geiger, A., Popovic, J., Darrell, T., Lawrence, N.D.: Topologically-constrained latent variable models. In: International Conference on Machine Learning, pp. 1080–1087 (2008) Google Scholar
  71. 71.
    Van Overschee, P., De Moor, B.: N4SID: Subspace algorithms for the identification of combined deterministic-stochastic systems. Automatica 30(1), 75–93 (1994) MathSciNetMATHCrossRefGoogle Scholar
  72. 72.
    Vasilescu, M.A.O.: Human motion signatures: Analysis, synthesis, recognition. In: International Conference on Pattern Recognition, vol. III, pp. 456–460 (2002) Google Scholar
  73. 73.
    Verbeek, J.J.: Learning nonlinear image manifolds by global alignment of local linear models. IEEE Trans. Pattern Anal. Mach. Intell. 28(8), 1236–1250 (2006) CrossRefGoogle Scholar
  74. 74.
    Wachter, S., Nagel, H.H.: Tracking persons in monocular image sequences. Comput. Vis. Image Underst. 74(3), 174–192 (1999) CrossRefGoogle Scholar
  75. 75.
    Wang, J.M.: Locomotion synthesis methods for humanoid characters. PhD thesis, University of Toronto (2010) Google Scholar
  76. 76.
    Wang, J.M., Fleet, D.J., Hertzmann, A.: Multifactor Gaussian process models for style-content separation. In: International Conference on Machine Learning, pp. 975–982 (2007) Google Scholar
  77. 77.
    Wang, J.M., Fleet, D.J., Hertzmann, A.: Gaussian process dynamical models for human motion. IEEE Trans. Pattern Anal. Mach. Intell. 30(2), 283–298 (2008) CrossRefGoogle Scholar
  78. 78.
    Wang, J.M., Fleet, D.J., Hertzmann, A.: Optimizing walking controllers. ACM Trans. Graph. 28(5) (2009) Google Scholar
  79. 79.
    Wang, J.M., Fleet, D.J., Hertzmann, A.: Optimizing walking controllers for uncertain inputs and environments. ACM Trans. Graph. 29(4) (2010) Google Scholar
  80. 80.
    Wren, C.R., Pentland, A.: Dynamic models of human motion. In: International Conference on Automatic Face and Gesture Recognition, pp. 22–27 (1998) Google Scholar
  81. 81.
    Yin, K., Loken, K., van de Panne, M.: Simbicon: Simple biped locomotion control. ACM Trans. Graph. 26(3) (2007) Google Scholar

Copyright information

© Springer-Verlag London Limited 2011

Authors and Affiliations

  1. 1.Department of Computer ScienceUniversity of TorontoTorontoCanada

Personalised recommendations