Human Context: Modeling Human-Human Interactions for Monocular 3D Pose Estimation

  • Mykhaylo Andriluka
  • Leonid Sigal
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7378)


Automatic recovery of 3d pose of multiple interacting subjects from unconstrained monocular image sequence is a challenging and largely unaddressed problem. We observe, however, that by tacking the interactions explicitly into account, treating individual subjects as mutual “context” for one another, performance on this challenging problem can be improved. Building on this observation, in this paper we develop an approach that first jointly estimates 2d poses of people using multi-person extension of the pictorial structures model and then lifts them to 3d. We illustrate effectiveness of our method on a new dataset of dancing couples and challenging videos from dance competitions.


Pictorial Structure Interaction Aspect Projection Parameter Human Context People Detection 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Andriluka, M., Roth, S., Schiele, B.: Monocular 3d pose estimation and tracking by detection. In: CVPR (2010)Google Scholar
  2. 2.
    Pellegrini, S., Edd, A., Schindler, K., van Gool, L.: You’ll never walk alone: Modeling social behaviour for multi-target tracking. In: ICCV (2009)Google Scholar
  3. 3.
    Yao, B., Fei-Fei, L.: Modeling mutual context of object and human pose in human-object interaction activities. In: CVPR (2010)Google Scholar
  4. 4.
    Kjellström, H., Kragic, D., Black, M.J.: Tracking people interacting with objects. In: CVPR (2010)Google Scholar
  5. 5.
    Ionescu, C., Bo, L., Sminchisescu, C.: Structured svm for visual localization and continuous state estimation. In: ICCV (2009)Google Scholar
  6. 6.
    Eichner, M., Ferrari, V.: We Are Family: Joint Pose Estimation of Multiple Persons. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6311, pp. 228–242. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  7. 7.
    Wang, J.M., Fleet, D.J., Hertzmann, A.: Gaussian process dynamical models for human motion. PAMI 30 (2008)Google Scholar
  8. 8.
    Felzenszwalb, P., Girshick, R., McAllester, D., Ramanan, D.: Object detection with discriminatively trained part based models. PAMI 32 (2010)Google Scholar
  9. 9.
    Andriluka, M., Roth, S., Schiele, B.: Pictorial structures revisited: People detection and articulated pose estimation. In: CVPR (2009)Google Scholar
  10. 10.
    Eichner, M., Ferrari, V.: Better appearance models for pictorial structures. In: BMVC (2009)Google Scholar
  11. 11.
    Ramanan, D.: Learning to parse images of articulated objects. In: NIPS (2006)Google Scholar
  12. 12.
    Felzenszwalb, P., Huttenlocher, D.: Pictorial structures for object recognition. International Journal of Computer Vision (2005)Google Scholar
  13. 13.
    Johnson, S., Everingham, M.: Clustered pose and nonlinear appearance models for human pose estimation. In: BMVC (2010)Google Scholar
  14. 14.
    Sapp, B., Weiss, D., Taskar, B.: Parsing human motion with stretchable models. In: CVPR (2011)Google Scholar
  15. 15.
    Yang, Y., Ramanan, D.: Articulated pose estimation with flexible mixtures-of-parts. In: CVPR (2011)Google Scholar
  16. 16.
    Ferrari, V., Marin-Jimenez, M., Zisserman, A.: Pose search: retrieving people using their pose. In: CVPR (2009)Google Scholar
  17. 17.
    Sigal, L., Black, M.J.: Measure locally, reason globally: Occlusion-sensitive articulated pose estimation. In: CVPR (2006)Google Scholar
  18. 18.
    Ferrari, V., Marin-Jimenez, M., Zisserman, A.: Progressive search space reduction for human pose estimation. In: CVPR (2008)Google Scholar
  19. 19.
    Tian, T.P., Sclaroff, S.: Fast globally optimal 2d human derection with loopy graph models. In: CVPR (2010)Google Scholar
  20. 20.
    Urtasun, R., Fleet, D., Fua, P.: 3d people tracking with gaussian process dynamical models. In: CVPR (2006)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Mykhaylo Andriluka
    • 1
  • Leonid Sigal
    • 2
  1. 1.Max Planck Institute for InformaticsSaarbrückenGermany
  2. 2.Disney ResearchPittsburghUSA

Personalised recommendations