Advertisement

Feature-Based Pose Estimation

  • Cristian Sminchisescu
  • Liefeng Bo
  • Catalin Ionescu
  • Atul Kanaujia

Abstract

In this chapter we review challenges and methodology for feature-based predictive three-dimensional human pose reconstruction, based on image and video data. We argue that reliable 3D human pose prediction can be achieved through an alliance between image descriptors that encode multiple levels of selectivity and invariance and models that are capable to represent multiple structured solutions. For monocular systems, key to reliability is the capacity to leverage prior knowledge in order to bias solutions not only to kinematically feasible sets, but also toward typical configurations that humans are likely to assume in everyday surroundings. In this context, we discuss several predictive methods including large-scale mixture of experts, supervised spectral latent variable models and structural support vector machines, asses the impact of the various choices of image descriptors, review open problems, and give pointers to datasets and code available online.

Keywords

Image Encodings Latent Variable Model Image Descriptor Motion Capture System Expert Model 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Notes

Acknowledgements

This work has been supported in part by the European Commission under MCEXT-025481 and by CNCSIS-UEFISCU under project PN II-RU-RC-2/2009. This chapter reviews research presented in [9, 11, 25, 28].

References

  1. 1.
    CMU Human Motion Capture Database. Available online at http://mocap.cs.cmu.edu/search.html (2003)
  2. 2.
    Agarwal, A., Triggs, B.: Hyperfeatures – multilevel local coding for visual recognition. In: European Conference on Computer Vision (2006) Google Scholar
  3. 3.
    Agarwal, A., Triggs, B.: A local basis representation for estimating human pose from cluttered images. In: Asian Conference on Computer Vision (2006) Google Scholar
  4. 4.
    Andriluka, M., Roth, S., Schiele, B.: People tracking-by-detection and people-detection-by-tracking. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2008) Google Scholar
  5. 5.
    Balan, A., Black, M., Haussecker, H., Sigal, L.: Shining a light on human pose: On shadows, shading and the estimation of pose and shape. In: IEEE International Conference on Computer Vision (2007) Google Scholar
  6. 6.
    Battu, B., Krappers, A., Koenderink, J.: Ambiguity in pictorial depth. Perception 36, 1290–1304 (2007) CrossRefGoogle Scholar
  7. 7.
    Bishop, C., Svensen, M.: Bayesian mixtures of experts. In: Conference on Uncertainty in Artificial Intelligence (2003) Google Scholar
  8. 8.
    Blaschko, M.B., Lampert, C.H.: Learning to localize objects with structured output regression. In: European Conference on Computer Vision, pp. 2–15 (2008) Google Scholar
  9. 9.
    Bo, L., Sminchisescu, C.: Supervised spectral latent variable models. In: International Conference on Artificial Intelligence and Statistics (2009) Google Scholar
  10. 10.
    Bo, L., Sminchisescu, C.: Twin Gaussian processes for structured prediction. Int. J. Comput. Vis. 87 (2010) Google Scholar
  11. 11.
    Bo, L., Sminchisescu, C., Kanaujia, A., Metaxas, D.: Fast algorithms for large scale conditional 3D prediction. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2008) Google Scholar
  12. 12.
    Böhning, D.: Multinomial logistic regression algorithm. Ann. Inst. Stat. Math. 44, 197–200 (2001) CrossRefGoogle Scholar
  13. 13.
    Bourdev, L., Malik, J.: Poselets: Body part detectors trained using 3d human pose annotations. In: IEEE International Conference on Computer Vision (2009) Google Scholar
  14. 14.
    Brubaker, M., Fleet, D.: The Kneed Walker for human pose tracking. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2008) Google Scholar
  15. 15.
    Choo, K., Fleet, D.: People tracking using hybrid Monte Carlo filtering. In: IEEE International Conference on Computer Vision (2001) Google Scholar
  16. 16.
    Cook, R.D.: Regression Graphics. Wiley-Interscience, New York (1988) Google Scholar
  17. 17.
    Cortes, C., Mohri, M., Weston, J.: A general regression technique for learning transductions. In: International Conference on Machine Learning, pp. 153–160 (2005) Google Scholar
  18. 18.
    Deutscher, J., Blake, A., Reid, I.: Articulated body motion capture by annealed particle filtering. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2000) Google Scholar
  19. 19.
    Deutscher, J., Davidson, A., Reid, I.: Articulated partitioning of high dimensional search spaces associated with articulated body motion capture. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2001) Google Scholar
  20. 20.
    Edwards, D., Lauritzen, S.: The TM algorithm for maximising a conditional likelihood function. Biometrika 88(4), 961–972 (2001) MathSciNetMATHCrossRefGoogle Scholar
  21. 21.
    Eichner, M., Ferrari, V.: We are family: Joint pose estimation of multiple persons. In: European Conference on Computer Vision (2010) Google Scholar
  22. 22.
    Felzenszwalb, P., Girshick, R., McAllester, D., Ramanan, D.: Object detection with discriminatively trained part-based models. IEEE Trans. Pattern Anal. Mach. Intell. 32, 1627–1645 (2010) CrossRefGoogle Scholar
  23. 23.
    Felzenszwalb, P., Huttenlocher, D.: Pictorial structures for object recognition. Int. J. Comput. Vis. 61(1) (2005) Google Scholar
  24. 24.
    Friedman, J.: Greedy function approximation: A gradient boosting machine. Ann. Stat. 29(5), 1189–1232 (2001) MATHCrossRefGoogle Scholar
  25. 25.
    Ionescu, C., Bo, L., Sminchisescu, C.: Structural SVM for visual localization and continuous state estimation. In: IEEE International Conference on Computer Vision (2009) Google Scholar
  26. 26.
    Joachims, T., Finley, T., Yu, C.: Cutting-plane training of structural SVMs. Mach. Learn. 77, 27–59 (2009) CrossRefGoogle Scholar
  27. 27.
    Jordan, M., Jacobs, R.: Hierarchical mixtures of experts and the EM algorithm. Neural Comput. 6, 181–214 (1994) CrossRefGoogle Scholar
  28. 28.
    Kanaujia, A., Sminchisescu, C., Metaxas, D.: Semi-supervised hierarchical models for 3D human pose reconstruction. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2006) Google Scholar
  29. 29.
    Kanaujia, A., Sminchisescu, C., Metaxas, D.: Spectral latent variable models for perceptual inference. In: IEEE International Conference on Computer Vision, vol. 1 (2007) Google Scholar
  30. 30.
    Kehl, R., Bray, M., Gool, L.V.: Full body tracking from multiple views using stochastic sampling. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2005) Google Scholar
  31. 31.
    Kim, M., Pavlovic, V.: Dimensionality reduction using covariance operator inverse regression. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2008) Google Scholar
  32. 32.
    Koenderink, J.: Pictorial relief. Philos. Trans. R. Soc., Math. Phys. Eng. Sci. 356(1740), 1071–1086 (1998) MathSciNetMATHCrossRefGoogle Scholar
  33. 33.
    Koenderink, J., van Doorn, A.: The internal representation of solid shape with respect to vision. Biol. Cybern. 32(3), 211–216 (1979) MATHCrossRefGoogle Scholar
  34. 34.
    Krishnapuram, B., Carin, L., Figueiredo, M.T., Hartemink, A.J.: Sparse multinomial logistic regression: Fast algorithms and generalization bounds. IEEE Trans. Pattern Anal. Mach. Intell. 27(6), 957–968 (2005) CrossRefGoogle Scholar
  35. 35.
    Lange, K., Hunter, D., Yang, I.: Optimization transfer using surrogate objective functions. J. Comput. Graph. Stat. 9, 1–59 (2001) MathSciNetGoogle Scholar
  36. 36.
    Lawrence, N.: Probabilistic non-linear component analysis with Gaussian process latent variable models. J. Mach. Learn. Res. 6, 1783–1816 (2005) MathSciNetGoogle Scholar
  37. 37.
    Lazebnik, S., Schmid, C., Ponce, J.: Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2006) Google Scholar
  38. 38.
    LeCun, Y., Chopra, S., Hadsell, R., Ranzato, R.M., Huang, F.: A tutorial on energy-based learning. In: Predicting Structured Data. MIT Press, Cambridge (2006) Google Scholar
  39. 39.
    Lee, H.J., Chen, Z.: Determination of 3D human body postures from a single view. Comput. Vis. Graph. Image Process. 30, 148–168 (1985) MathSciNetCrossRefGoogle Scholar
  40. 40.
    Lu, Z., Perpinan, M.C., Sminchisescu, C.: People tracking with the Laplacian eigenmaps latent variable model. In: Advances in Neural Information Processing Systems (2007) Google Scholar
  41. 41.
    Morris, D., Rehg, J.: Singularity analysis for articulated object tracking. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 289–296 (1998) Google Scholar
  42. 42.
    Nistér, D., Stévenius, H.: Scalable recognition with a vocabulary tree. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2006) Google Scholar
  43. 43.
    Poppe, R.: Evaluating example-based human pose estimation: Experiments on HumanEva sets. In: HumanEva Workshop CVPR (2007) Google Scholar
  44. 44.
    Ramanan, D., Sminchisescu, C.: Training deformable models for localization. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2006) Google Scholar
  45. 45.
    Rosales, R., Sclaroff, S.: Learning body pose via specialized maps. In: Advances in Neural Information Processing Systems (2002) Google Scholar
  46. 46.
    Sapp, B., Toshev, A., Taskar, B.: Cascaded models for articulated pose estimation. In: European Conference on Computer Vision (2010) Google Scholar
  47. 47.
    Serre, T., Wolf, L., Poggio, T.: Object recognition with features inspired by visual cortex. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 994–1000 (2005) Google Scholar
  48. 48.
    Shakhnarovich, G., Viola, P., Darrell, T.: Fast pose estimation with parameter sensitive hashing. In: IEEE International Conference on Computer Vision (2003) Google Scholar
  49. 49.
    Sidenbladh, H., Black, M.: Learning image statistics for Bayesian tracking. In: IEEE International Conference on Computer Vision (2001) Google Scholar
  50. 50.
    Sidenbladh, H., Black, M., Fleet, D.: Stochastic tracking of 3D human figures using 2D image motion. In: European Conference on Computer Vision (2000) Google Scholar
  51. 51.
    Sidenbladh, H., Black, M., Sigal, L.: Implicit probabilistic models of human motion for synthesis and tracking. In: European Conference on Computer Vision (2002) Google Scholar
  52. 52.
    Sigal, L., Balan, A., Black, M.: HumanEva: Synchronized video and motion capture dataset and baseline algorithm for evaluation of articulated human motion. Int. J. Comput. Vis. 87, 4–27 (2006) CrossRefGoogle Scholar
  53. 53.
    Sigal, L., Balan, A., Black, M.J.: Combined discriminative and generative articulated pose and non-rigid shape estimation. In: Advances in Neural Information Processing Systems (2007) Google Scholar
  54. 54.
    Sigal, L., Bhatia, S., Roth, S., Black, M., Isard, M.: Tracking loose-limbed people. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2004) Google Scholar
  55. 55.
    Sigal, L., Black, M.: Predicting 3d people from 2d pictures. In: International Workshop on Articulated Motion and Deformable Objects (2006) Google Scholar
  56. 56.
    Sigal, L., Memisevic, R., Fleet, D.: Shared kernel information embedding for discriminative inference. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2009) Google Scholar
  57. 57.
    Sminchisescu, C., Jepson, A.: Generative modeling for continuous non-linearly embedded visual inference. In: International Conference on Machine Learning, pp. 759–766 (2004) Google Scholar
  58. 58.
    Sminchisescu, C., Jepson, A.: Variational mixture smoothing for non-linear dynamical systems. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 608–615 (2004) Google Scholar
  59. 59.
    Sminchisescu, C., Kanaujia, A., Li, Z., Metaxas, D.: Discriminative density propagation for 3D human motion estimation. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 390–397 (2005) Google Scholar
  60. 60.
    Sminchisescu, C., Kanaujia, A., Metaxas, D.: Learning joint top–down and bottom–up processes for 3D visual inference. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2006) Google Scholar
  61. 61.
    Sminchisescu, C., Kanaujia, A., Metaxas, D.: BM 3 E: Discriminative density propagation for visual tracking. IEEE Trans. Pattern Anal. Mach. Intell. (2007) Google Scholar
  62. 62.
    Sminchisescu, C., Triggs, B.: Building roadmaps of local minima of visual models. In: European Conference on Computer Vision, vol. 1, pp. 566–582 (2002) Google Scholar
  63. 63.
    Sminchisescu, C., Triggs, B.: Hyperdynamics importance sampling. In: European Conference on Computer Vision, vol. 1, pp. 769–783 (2002) Google Scholar
  64. 64.
    Sminchisescu, C., Triggs, B.: Kinematic jump processes for monocular 3D human tracking. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 69–76 (2003) Google Scholar
  65. 65.
    Sminchisescu, C., Triggs, B.: Mapping minima and transitions in visual models. Int. J. Comput. Vis. 61(1) (2005) Google Scholar
  66. 66.
    Sminchisescu, C., Welling, M.: Generalized darting Monte-Carlo. In: International Conference on Artificial Intelligence and Statistics, vol. 1 (2007) Google Scholar
  67. 67.
    Tipping, M., Faul, A.: Fast marginal likelihood maximisation for sparse Bayesian models. In: International Conference on Artificial Intelligence and Statistics (2003) Google Scholar
  68. 68.
    Tsochantaridis, I., Joachims, T., Hofmann, T., Altun, Y.: Large margin methods for structured and interdependent output variables. J. Mach. Learn. Res. 6, 1453–1484 (2005) MathSciNetGoogle Scholar
  69. 69.
    Urtasun, R., Fleet, D., Hertzmann, A., Fua, P.: Priors for people tracking in small training sets. In: IEEE International Conference on Computer Vision (2005) Google Scholar
  70. 70.
    Urtasun, R., Fleet, D., Geiger, A., Popovic, J., Darrell, T., Lawrence, N.: Topologically-constrained latent variable models. In: International Conference on Machine Learning (2008) Google Scholar
  71. 71.
    Vincent, P., Bengio, Y.: Kernel matching pursuit. Mach. Learn. 48, 165–187 (2002) MATHCrossRefGoogle Scholar
  72. 72.
    Vlasic, D., Adelsberger, R., Vannucci, G., Barwell, J., Gross, M., Matusik, W., Popovic, J.: Practical motion capture in everyday surroundings. ACM Trans. Graph. (2007) Google Scholar
  73. 73.
    Vondrak, M., Sigal, L., Jenkins, O.C.: Physical simulation for probabilistic motion tracking. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2008) Google Scholar
  74. 74.
    Wang, J., Fleet, D.J., Hertzmann, A.: Gaussian process dynamical models. In: IEEE Trans. Pattern Anal. Mach. Intell. (2008) Google Scholar
  75. 75.
    Weston, J., Schölkopf, B., Bousquet, O., Mann, T., Noble, W.: Joint kernel maps. In: International Work-Conference on Artificial Neural Networks, pp. 176–191 (2005) Google Scholar

Copyright information

© Springer-Verlag London Limited 2011

Authors and Affiliations

  • Cristian Sminchisescu
    • 1
    • 2
  • Liefeng Bo
    • 3
  • Catalin Ionescu
    • 4
  • Atul Kanaujia
    • 5
  1. 1.Institute for Numerical Simulation (INS), Faculty of Mathematics and Natural ScienceUniversity of BonnBonnGermany
  2. 2.Institute for Mathematics of the Romanian Academy (IMAR)BucharestRomania
  3. 3.University of WashingtonSeattleUSA
  4. 4.INSUniversity of BonnBonnGermany
  5. 5.ObjectVideoRestonUSA

Personalised recommendations