Abstract
In this chapter we review challenges and methodology for feature-based predictive three-dimensional human pose reconstruction, based on image and video data. We argue that reliable 3D human pose prediction can be achieved through an alliance between image descriptors that encode multiple levels of selectivity and invariance and models that are capable to represent multiple structured solutions. For monocular systems, key to reliability is the capacity to leverage prior knowledge in order to bias solutions not only to kinematically feasible sets, but also toward typical configurations that humans are likely to assume in everyday surroundings. In this context, we discuss several predictive methods including large-scale mixture of experts, supervised spectral latent variable models and structural support vector machines, asses the impact of the various choices of image descriptors, review open problems, and give pointers to datasets and code available online.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsNotes
- 1.
Standard kernel regression models can be viewed as a special case of a conditional mixture with a single expert (M=1). We will thus not include separate derivations for kernel regression, the restriction to M=1 in (12.1) being straightforward.
- 2.
Sampled configurations have parenthesized superscripts; subscripts index training datapoints.
References
CMU Human Motion Capture Database. Available online at http://mocap.cs.cmu.edu/search.html (2003)
Agarwal, A., Triggs, B.: Hyperfeatures – multilevel local coding for visual recognition. In: European Conference on Computer Vision (2006)
Agarwal, A., Triggs, B.: A local basis representation for estimating human pose from cluttered images. In: Asian Conference on Computer Vision (2006)
Andriluka, M., Roth, S., Schiele, B.: People tracking-by-detection and people-detection-by-tracking. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2008)
Balan, A., Black, M., Haussecker, H., Sigal, L.: Shining a light on human pose: On shadows, shading and the estimation of pose and shape. In: IEEE International Conference on Computer Vision (2007)
Battu, B., Krappers, A., Koenderink, J.: Ambiguity in pictorial depth. Perception 36, 1290–1304 (2007)
Bishop, C., Svensen, M.: Bayesian mixtures of experts. In: Conference on Uncertainty in Artificial Intelligence (2003)
Blaschko, M.B., Lampert, C.H.: Learning to localize objects with structured output regression. In: European Conference on Computer Vision, pp. 2–15 (2008)
Bo, L., Sminchisescu, C.: Supervised spectral latent variable models. In: International Conference on Artificial Intelligence and Statistics (2009)
Bo, L., Sminchisescu, C.: Twin Gaussian processes for structured prediction. Int. J. Comput. Vis. 87 (2010)
Bo, L., Sminchisescu, C., Kanaujia, A., Metaxas, D.: Fast algorithms for large scale conditional 3D prediction. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2008)
Böhning, D.: Multinomial logistic regression algorithm. Ann. Inst. Stat. Math. 44, 197–200 (2001)
Bourdev, L., Malik, J.: Poselets: Body part detectors trained using 3d human pose annotations. In: IEEE International Conference on Computer Vision (2009)
Brubaker, M., Fleet, D.: The Kneed Walker for human pose tracking. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2008)
Choo, K., Fleet, D.: People tracking using hybrid Monte Carlo filtering. In: IEEE International Conference on Computer Vision (2001)
Cook, R.D.: Regression Graphics. Wiley-Interscience, New York (1988)
Cortes, C., Mohri, M., Weston, J.: A general regression technique for learning transductions. In: International Conference on Machine Learning, pp. 153–160 (2005)
Deutscher, J., Blake, A., Reid, I.: Articulated body motion capture by annealed particle filtering. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2000)
Deutscher, J., Davidson, A., Reid, I.: Articulated partitioning of high dimensional search spaces associated with articulated body motion capture. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2001)
Edwards, D., Lauritzen, S.: The TM algorithm for maximising a conditional likelihood function. Biometrika 88(4), 961–972 (2001)
Eichner, M., Ferrari, V.: We are family: Joint pose estimation of multiple persons. In: European Conference on Computer Vision (2010)
Felzenszwalb, P., Girshick, R., McAllester, D., Ramanan, D.: Object detection with discriminatively trained part-based models. IEEE Trans. Pattern Anal. Mach. Intell. 32, 1627–1645 (2010)
Felzenszwalb, P., Huttenlocher, D.: Pictorial structures for object recognition. Int. J. Comput. Vis. 61(1) (2005)
Friedman, J.: Greedy function approximation: A gradient boosting machine. Ann. Stat. 29(5), 1189–1232 (2001)
Ionescu, C., Bo, L., Sminchisescu, C.: Structural SVM for visual localization and continuous state estimation. In: IEEE International Conference on Computer Vision (2009)
Joachims, T., Finley, T., Yu, C.: Cutting-plane training of structural SVMs. Mach. Learn. 77, 27–59 (2009)
Jordan, M., Jacobs, R.: Hierarchical mixtures of experts and the EM algorithm. Neural Comput. 6, 181–214 (1994)
Kanaujia, A., Sminchisescu, C., Metaxas, D.: Semi-supervised hierarchical models for 3D human pose reconstruction. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2006)
Kanaujia, A., Sminchisescu, C., Metaxas, D.: Spectral latent variable models for perceptual inference. In: IEEE International Conference on Computer Vision, vol. 1 (2007)
Kehl, R., Bray, M., Gool, L.V.: Full body tracking from multiple views using stochastic sampling. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2005)
Kim, M., Pavlovic, V.: Dimensionality reduction using covariance operator inverse regression. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2008)
Koenderink, J.: Pictorial relief. Philos. Trans. R. Soc., Math. Phys. Eng. Sci. 356(1740), 1071–1086 (1998)
Koenderink, J., van Doorn, A.: The internal representation of solid shape with respect to vision. Biol. Cybern. 32(3), 211–216 (1979)
Krishnapuram, B., Carin, L., Figueiredo, M.T., Hartemink, A.J.: Sparse multinomial logistic regression: Fast algorithms and generalization bounds. IEEE Trans. Pattern Anal. Mach. Intell. 27(6), 957–968 (2005)
Lange, K., Hunter, D., Yang, I.: Optimization transfer using surrogate objective functions. J. Comput. Graph. Stat. 9, 1–59 (2001)
Lawrence, N.: Probabilistic non-linear component analysis with Gaussian process latent variable models. J. Mach. Learn. Res. 6, 1783–1816 (2005)
Lazebnik, S., Schmid, C., Ponce, J.: Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2006)
LeCun, Y., Chopra, S., Hadsell, R., Ranzato, R.M., Huang, F.: A tutorial on energy-based learning. In: Predicting Structured Data. MIT Press, Cambridge (2006)
Lee, H.J., Chen, Z.: Determination of 3D human body postures from a single view. Comput. Vis. Graph. Image Process. 30, 148–168 (1985)
Lu, Z., Perpinan, M.C., Sminchisescu, C.: People tracking with the Laplacian eigenmaps latent variable model. In: Advances in Neural Information Processing Systems (2007)
Morris, D., Rehg, J.: Singularity analysis for articulated object tracking. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 289–296 (1998)
Nistér, D., Stévenius, H.: Scalable recognition with a vocabulary tree. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2006)
Poppe, R.: Evaluating example-based human pose estimation: Experiments on HumanEva sets. In: HumanEva Workshop CVPR (2007)
Ramanan, D., Sminchisescu, C.: Training deformable models for localization. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2006)
Rosales, R., Sclaroff, S.: Learning body pose via specialized maps. In: Advances in Neural Information Processing Systems (2002)
Sapp, B., Toshev, A., Taskar, B.: Cascaded models for articulated pose estimation. In: European Conference on Computer Vision (2010)
Serre, T., Wolf, L., Poggio, T.: Object recognition with features inspired by visual cortex. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 994–1000 (2005)
Shakhnarovich, G., Viola, P., Darrell, T.: Fast pose estimation with parameter sensitive hashing. In: IEEE International Conference on Computer Vision (2003)
Sidenbladh, H., Black, M.: Learning image statistics for Bayesian tracking. In: IEEE International Conference on Computer Vision (2001)
Sidenbladh, H., Black, M., Fleet, D.: Stochastic tracking of 3D human figures using 2D image motion. In: European Conference on Computer Vision (2000)
Sidenbladh, H., Black, M., Sigal, L.: Implicit probabilistic models of human motion for synthesis and tracking. In: European Conference on Computer Vision (2002)
Sigal, L., Balan, A., Black, M.: HumanEva: Synchronized video and motion capture dataset and baseline algorithm for evaluation of articulated human motion. Int. J. Comput. Vis. 87, 4–27 (2006)
Sigal, L., Balan, A., Black, M.J.: Combined discriminative and generative articulated pose and non-rigid shape estimation. In: Advances in Neural Information Processing Systems (2007)
Sigal, L., Bhatia, S., Roth, S., Black, M., Isard, M.: Tracking loose-limbed people. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2004)
Sigal, L., Black, M.: Predicting 3d people from 2d pictures. In: International Workshop on Articulated Motion and Deformable Objects (2006)
Sigal, L., Memisevic, R., Fleet, D.: Shared kernel information embedding for discriminative inference. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2009)
Sminchisescu, C., Jepson, A.: Generative modeling for continuous non-linearly embedded visual inference. In: International Conference on Machine Learning, pp. 759–766 (2004)
Sminchisescu, C., Jepson, A.: Variational mixture smoothing for non-linear dynamical systems. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 608–615 (2004)
Sminchisescu, C., Kanaujia, A., Li, Z., Metaxas, D.: Discriminative density propagation for 3D human motion estimation. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 390–397 (2005)
Sminchisescu, C., Kanaujia, A., Metaxas, D.: Learning joint top–down and bottom–up processes for 3D visual inference. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2006)
Sminchisescu, C., Kanaujia, A., Metaxas, D.: BM 3 E: Discriminative density propagation for visual tracking. IEEE Trans. Pattern Anal. Mach. Intell. (2007)
Sminchisescu, C., Triggs, B.: Building roadmaps of local minima of visual models. In: European Conference on Computer Vision, vol. 1, pp. 566–582 (2002)
Sminchisescu, C., Triggs, B.: Hyperdynamics importance sampling. In: European Conference on Computer Vision, vol. 1, pp. 769–783 (2002)
Sminchisescu, C., Triggs, B.: Kinematic jump processes for monocular 3D human tracking. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 69–76 (2003)
Sminchisescu, C., Triggs, B.: Mapping minima and transitions in visual models. Int. J. Comput. Vis. 61(1) (2005)
Sminchisescu, C., Welling, M.: Generalized darting Monte-Carlo. In: International Conference on Artificial Intelligence and Statistics, vol. 1 (2007)
Tipping, M., Faul, A.: Fast marginal likelihood maximisation for sparse Bayesian models. In: International Conference on Artificial Intelligence and Statistics (2003)
Tsochantaridis, I., Joachims, T., Hofmann, T., Altun, Y.: Large margin methods for structured and interdependent output variables. J. Mach. Learn. Res. 6, 1453–1484 (2005)
Urtasun, R., Fleet, D., Hertzmann, A., Fua, P.: Priors for people tracking in small training sets. In: IEEE International Conference on Computer Vision (2005)
Urtasun, R., Fleet, D., Geiger, A., Popovic, J., Darrell, T., Lawrence, N.: Topologically-constrained latent variable models. In: International Conference on Machine Learning (2008)
Vincent, P., Bengio, Y.: Kernel matching pursuit. Mach. Learn. 48, 165–187 (2002)
Vlasic, D., Adelsberger, R., Vannucci, G., Barwell, J., Gross, M., Matusik, W., Popovic, J.: Practical motion capture in everyday surroundings. ACM Trans. Graph. (2007)
Vondrak, M., Sigal, L., Jenkins, O.C.: Physical simulation for probabilistic motion tracking. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2008)
Wang, J., Fleet, D.J., Hertzmann, A.: Gaussian process dynamical models. In: IEEE Trans. Pattern Anal. Mach. Intell. (2008)
Weston, J., Schölkopf, B., Bousquet, O., Mann, T., Noble, W.: Joint kernel maps. In: International Work-Conference on Artificial Neural Networks, pp. 176–191 (2005)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag London Limited
About this chapter
Cite this chapter
Sminchisescu, C., Bo, L., Ionescu, C., Kanaujia, A. (2011). Feature-Based Pose Estimation. In: Moeslund, T., Hilton, A., Krüger, V., Sigal, L. (eds) Visual Analysis of Humans. Springer, London. https://doi.org/10.1007/978-0-85729-997-0_12
Download citation
DOI: https://doi.org/10.1007/978-0-85729-997-0_12
Publisher Name: Springer, London
Print ISBN: 978-0-85729-996-3
Online ISBN: 978-0-85729-997-0
eBook Packages: Computer ScienceComputer Science (R0)