Human Pose Tracking Using Multi-level Structured Models

  • Mun Wai Lee
  • Ram Nevatia
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3953)


Tracking body poses of multiple persons in monocular video is a challenging problem due to the high dimensionality of the state space and issues such as inter-occlusion of the persons’ bodies. We proposed a three-stage approach with a multi-level state representation that enables a hierarchical estimation of 3D body poses. At the first stage, humans are tracked as blobs. In the second stage, parts such as face, shoulders and limbs are estimated and estimates are combined by grid-based belief propagation to infer 2D joint positions. The derived belief maps are used as proposal functions in the third stage to infer the 3D pose using data-driven Markov chain Monte Carlo. Experimental results on realistic indoor video sequences show that the method is able to track multiple persons during complex movement such as turning movement with inter-occlusion.


Belief Propagation State Candidate Observation Function Motion Capture Data Body Joint 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Agarwal, A., Triggs, B.: 3D human pose from silhouettes by relevance vector regression. In: CVPR 2004 (2004)Google Scholar
  2. 2.
    Choo, K., Fleet, D.J.: People tracking with hybrid Monte Carlo. In: ICCV (2001)Google Scholar
  3. 3.
    Deutscher, J., Davison, A., Reid, I.: Automatic partitioning of high dimensional search spaces associated with articulated body motion capture. In: CVPR 2001 (2001)Google Scholar
  4. 4.
    Garofolo, J.S., Laprun, C.D., Michel, M., Stanford, V.M., Tabassi, E.: The NIST Meeting Room Pilot Corpus. In: Proc. 4th International Conference on Language Resources and Evaluation (LREC 2004), Lisbon, Portugal, May 26-28 (2004)Google Scholar
  5. 5.
    Gilks, W., Richardson, S., Spiegelhalter, D.: Markov Chain Monte Carlo in Practice. Chapman and Hall, Boca Raton (1996)zbMATHGoogle Scholar
  6. 6.
    Hua, G., Yang, M., Wu, Y.: Learning to estimate human pose with data driven belief propagation. In: CVPR 2005 (2005)Google Scholar
  7. 7.
    Lee, M., Cohen, I.: Proposal Maps driven MCMC for Estimating Human Body Pose in Static Images. In: CVPR 2004 (2004)Google Scholar
  8. 8.
    Lee, M., Nevatia, R.: Dynamic Human Pose Estimation using Markov chain Monte Carlo Approach. In: Motion 2005 (2005)Google Scholar
  9. 9.
    Mori, G., Ren, X., Efros, A., Malik, J.: Recovering Human Body Configurations: Combining Segmentation and Recognition. In: CVPR 2004 (2004)Google Scholar
  10. 10.
    Isard, M.: PAMPAS: Real-valued graphical models for computer vision. In: CVPR 2003 (2003)Google Scholar
  11. 11.
    Ramanan, D., Forsyth, D.A.: Finding and tracking people from the bottom up. In: CVPR 2003 (2003)Google Scholar
  12. 12.
    Roberts, T.J., McKenna, S.J., Ricketts, I.W.: Human Pose Estimation Using Learnt Probabilistic Region Similarities and Partial Configurations. In: Pajdla, T., Matas, J(G.) (eds.) ECCV 2004. LNCS, vol. 3024, pp. 291–303. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  13. 13.
    Shakhnarovich, G., Viola, P., Darrell, T.: Face pose estimation with parameter sensitive hashing. In: ICCV 2003 (2003)Google Scholar
  14. 14.
    Sigal, L., Bhatia, S., Roth, S., Black, M.J., Isard, M.: Tracking Loose-limbed People. In: CVPR 2004 (2004)Google Scholar
  15. 15.
    Sminchisescu, C., Triggs, B.: Kinematic Jump Processes for Monocular Human Tracking. In: CVPR 2003 (2003)Google Scholar
  16. 16.
    Stauffer, C., Grimson, W.: Adaptive background mixture models for real-time tracking. In: CVPR 1999 (1999)Google Scholar
  17. 17.
    Sudderth, E.B., Ihler, A.T., Freeman, W.T., Willsky, A.S.: Nonparametric belief propagation. In: CVPR 2003 (2003)Google Scholar
  18. 18.
    Sudderth, E.B., Mandel, M.I., Freeman, W.T., Willsky, A.S.: Distributed occlusion reasoning for tracking with nonparametric belief propagation. In: NIPS 2004 (2004)Google Scholar
  19. 19.
    Tu, Z.W., Zhu, S.C.: Image Segmentation by Data-Driven Markov Chain Monte Carlo. PAMI 24(5), 657–672 (2002)Google Scholar
  20. 20.
    Wu, Y., Hua, G., Yu, T.: Tracking articulated body by dynamic Markov network. In: CVPR 2003 (2003)Google Scholar
  21. 21.
    Zhu, S., Zhang, R., Tu, Z.: Integrating bottom-up/top-down for object recognition by data driven Markov chain Monte Carlo. In: CVPR 2000 (2000)Google Scholar
  22. 22.
    Zhang, J., Collins, R., Liu, Y.: Representation and Matching of Articulated Shapes. In: CVPR 2004 (2004)Google Scholar
  23. 23.
    Zhao, T., Nevatia, R.: Tracking Multiple Humans in Crowded Environment. In: CVPR 2004 (2004)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Mun Wai Lee
    • 1
  • Ram Nevatia
    • 1
  1. 1.Institute for Robotics and Intelligent SystemUniversity of Southern CaliforniaLos AngelesUSA

Personalised recommendations