Real-Time Human Pose Tracking from Range Data

  • Varun Ganapathi
  • Christian Plagemann
  • Daphne Koller
  • Sebastian Thrun
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7577)


Tracking human pose in real-time is a difficult problem with many interesting applications. Existing solutions suffer from a variety of problems, especially when confronted with unusual human poses. In this paper, we derive an algorithm for tracking human pose in real-time from depth sequences based on MAP inference in a probabilistic temporal model. The key idea is to extend the iterative closest points (ICP) objective by modeling the constraint that the observed subject cannot enter free space, the area of space in front of the true range measurements. Our primary contribution is an extension to the articulated ICP algorithm that can efficiently enforce this constraint. The resulting filter runs at 125 frames per second using a single desktop CPU core. We provide extensive experimental results on challenging real-world data, which show that the algorithm outperforms the previous state-of-the-art trackers both in computational efficiency and accuracy.


Motion Capture Depth Image Body Model Iterative Close Point Tracking Accuracy 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Ganapathi, V., Plagemann, C.: Project website and data sets (March 2010),
  2. 2.
    Pons-Moll, G., Rosenhahn, B.: Model-based pose estimation. In: Visual Analysis of Humans, pp. 139–170 (2011)Google Scholar
  3. 3.
    Stoll, C., Hasler, N., Gall, J., Seidel, H.P., Theobalt, C.: Fast articulated motion tracking using a sums of gaussians body model. In: IEEE International Conference on Computer Vision, ICCV (2011)Google Scholar
  4. 4.
    de Aguiar, E., Theobalt, C., Stoll, C., Seidel, H.P.: Marker-less deformable mesh tracking for human shape and motion capture. In: CVPR, pp. 1–8 (2007)Google Scholar
  5. 5.
    Corazza, S., Mundermann, L., Chaudhari, A., Demattio, T., Cobelli, C., Andriacchi, T.: A markerless motion capture system to study musculoskeletal biomechanics: Visual hull and simulated annealing approach. Annals of Bio. Eng. (2006)Google Scholar
  6. 6.
    Van den Bergh, M., Koller-Meier, E., Van Gool, L.: Real-time body pose recognition using 2D or 3D haarlets. Int. Journal of Computer Vision 83, 72–84 (2009)CrossRefGoogle Scholar
  7. 7.
    Agarwal, A., Triggs, B.: 3D human pose from silhouettes by relevance vector regression. In: Computer Vision and Pattern Recognition (CVPR) (2004)Google Scholar
  8. 8.
    Sun, Y., Bray, M., Thayananthan, A., Yuan, B., Torr, P.H.S.: Regression-based human motion capture from voxel data. In: British Machine Vision Conf. (2006)Google Scholar
  9. 9.
    Plagemann, C., Ganapathi, V., Koller, D., Thrun, S.: Realtime identification and localization of body parts from depth images. In: IEEE Int. Conference on Robotics and Automation (ICRA), Anchorage, Alaska, USA (2010)Google Scholar
  10. 10.
    Shotton, J., Fitzgibbon, A.W., Cook, M., Sharp, T., Finocchio, M., Moore, R., Kipman, A., Blake, A.: Real-time human pose recognition in parts from single depth images. In: CVPR (2011)Google Scholar
  11. 11.
    Girshick, R., Shotton, J., Kohli, P., Criminisi, A., Fitzgibbon, A.: Efficient regression of general-activity human poses from depth images. In: ICCV (2011)Google Scholar
  12. 12.
    Grest, D., Woetzel, J., Koch, R.: Nonlinear Body Pose Estimation from Depth Images. In: Kropatsch, W.G., Sablatnig, R., Hanbury, A. (eds.) DAGM 2005. LNCS, vol. 3663, pp. 285–292. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  13. 13.
    Plankers, R., Fua, P.: Articulated soft objects for multiview shape and motion capture. Pattern Analysis and Machine Intelligence 25(9), 1182–1187 (2003)CrossRefGoogle Scholar
  14. 14.
    Demirdjian, D., Ko, T., Darrell, T.: Constraining Human Body Tracking. In: IEEE International Conference on Computer Vision, vol. 2 (2003)Google Scholar
  15. 15.
    Hähnel, D., Thrun, S., Burgard, W.: An extension of the ICP algorithm for modeling nonrigid objects with mobile robots (2003)Google Scholar
  16. 16.
    Knoop, S., Vacek, S., Dillmann, R.: Sensor fusion for 3D human body tracking with an articulated 3D body model. In: ICRA (2006)Google Scholar
  17. 17.
    Balan, A., Sigal, L., Black, M., Davis, J., Haussecker, H.: Detailed human shape and pose from images. In: Computer Vision and Pattern Recognition (CVPR), pp. 1–8. IEEE (2007)Google Scholar
  18. 18.
    Zhu, Y., Fujimura, K.: Bayesian 3D Human Body Pose Tracking from Depth Image Sequences. In: Zha, H., Taniguchi, R.-I., Maybank, S. (eds.) ACCV 2009, Part II. LNCS, vol. 5995, pp. 267–278. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  19. 19.
    Ganapathi, V., Plagemann, C., Thrun, S., Koller, D.: Real time motion capture using a single time-of-flight camera. In: Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), San Francisco, CA, USA (June 2010)Google Scholar
  20. 20.
    Siddiqui, M., Medioni, G.: Human pose estimation from a single view point, real-time range sensor. In: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 1–8 (June 2010)Google Scholar
  21. 21.
    Baak, A., Müller, M., Bharaj, G., Seidel, H.P., Theobalt, C.: A data-driven approach for real-time full body pose reconstruction from a depth camera. In: IEEE 13th International Conference on Computer Vision (ICCV), pp. 1092–1099. IEEE (November 2011)Google Scholar
  22. 22.
    Bregman, L.M.: The relaxation method of finding the common point of convex sets and its application to the solution of problems in convex programming. USSR Computational Mathematics and Mathematical Physics 7(3), 200–217 (1967)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Varun Ganapathi
    • 1
    • 2
  • Christian Plagemann
    • 1
    • 2
  • Daphne Koller
    • 1
  • Sebastian Thrun
    • 1
    • 2
  1. 1.Computer Science DepartmentStanford UniversityStanfordUSA
  2. 2.Google Inc.Mountain ViewUSA

Personalised recommendations