Skip to main content
SpringerLink
Log in

Contours, Optic Flow, and Prior Knowledge: Cues for Capturing 3D Human Motion in Videos

  • Chapter
Book cover Human Motion

Part of the book series: Computational Imaging and Vision ((CIVI,volume 36))

Human 3D motion tracking from video is an emerging research field with many applications demanding highly detailed results. This chapter surveys a high quality generative method, which employs the person’s silhouette extracted from one or multiple camera views for fitting an a priori given 3D body surface model. A coupling between pose estimation and contour extraction allows for reliable tracking in cluttered scenes without the need of a static background. The optic flow computed between two successive frames is used for pose prediction. It improves the quality of tracking in case of fast motion and/or low frame rates. In order to cope with unreliable or insufficient data, the framework is further extended by the use of prior knowledge on static joint angle configurations.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 54.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Agarwal A. and Triggs B. Recovering 3D human pose from monocular images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(1):44-58, Jan. 2006.

    Article  Google Scholar 

  2. Alvarez L., Weickert J., and Sánchez J. Reliable estimation of dense optical flow fields with large displacements. International Journal of Computer Vision, 39 (1):41-56, Aug. 2000.

    Article  MATH  Google Scholar 

  3. Anandan P. A computational framework and an algorithm for the measurement of visual motion. International Journal of Computer Vision, 2:283-310, 1989.

    Article  Google Scholar 

  4. Besl P. and McKay N. A method for registration of 3D shapes. IEEE Transac-tions on Pattern Analysis and Machine Intelligence, 12:239-256, 1992.

    Article  Google Scholar 

  5. Black M.J. and Anandan P. The robust estimation of multiple motions: para-metric and piecewise smooth flow fields. Computer Vision and Image Under-standing, 63(1):75-104, Jan. 1996.

    Article  Google Scholar 

  6. Blake A. and Zisserman A. Visual Reconstruction. MIT Press, Cambridge, MA, 1987.

    Google Scholar 

  7. Bregler C. and Malik J. Tracking people with twists and exponential maps. In Proc. IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 8-15, Santa Barbara, California, 1998.

    Google Scholar 

  8. Bregler C., Malik J. and Pullen K. Twist based acquisition and tracking of animal and human kinematics. International Journal of Computer Vision, 56(3):179-194, 2004.

    Article  Google Scholar 

  9. Brox T., Bruhn A., Papenberg N., and Weickert J. High accuracy optical flow estimation based on a theory for warping. In T. Pajdla and J. Matas, editors, Proc.8th European Conference on Computer Vision, volume 3024 of LNCS, pp. 25-36. Springer, May 2004.

    Google Scholar 

  10. Brox T., Rosenhahn B., Cremers D., and Seidel H.-P. High accuracy optical flow serves 3-D pose tracking: exploiting contour and flow based constraints. In A. Leonardis, H. Bischofand, A. Prinz, editors, Proc.European Conference on Computer Vision, volume 3952 of LNCS, pp. 98-111, Graz, Austria, Springer, May 2006.

    Google Scholar 

  11. Brox T., Rosenhahn B., Kersting U., and Cremers D. Nonparametric density estimation for human pose tracking. In K.F. et al., editor, Pattern Recognition, volume 4174 of LNCS, pp. 546-555, Berlin, Germany, Sept. 2006. Springer.

    Chapter  Google Scholar 

  12. Brox T. and Weickert J. A TV flow based local scale estimate and its appli-cation to texture discrimination. Journal of Visual Communication and Image Representation, 17(5):1053-1073, Oct. 2006.

    Article  Google Scholar 

  13. Brox T. and Cremers D. On the statistical interpretation of the piecewise smooth Mumford-Shah functional. In Scale Space and Variational Methods in Computer Vision, volume 4485 of LNCS, pp. 203-213 Springer, 2007.

    Google Scholar 

  14. Bruhn A. and Weickert J. Towards ultimate motion estimation: Combining highest accuracy with real-time performance. In Proc.10th International Confer-ence on Computer Vision, pp. 749-755. IEEE Computer Society Press, Beijing, China, Oct. 2005.

    Google Scholar 

  15. Chan T. and Vese L. Active contours without edges. IEEE Transactions on Image Processing, 10(2):266-277, Feb. 2001.

    Article  MATH  Google Scholar 

  16. Chetverikov D. A simple and efficient algorithm for detection of high curvature points. In N. Petkov and M. Westenberg, editors, Computer Analysis of Images and Patterns, volume 2756 of LNCS, pp. 746-753, Groningen, Springer, 2003.

    Google Scholar 

  17. Cremers D. Dynamical statistical shape priors for level set based tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(8):1262-1273, Aug. 2006.

    Article  Google Scholar 

  18. Cremers D., Kohlberger T. and Schnörr C. Shape statistics in kernel space for variational image segmentation. Pattern Recognition, 36(9):1929-1943, Sept. 2003.

    Article  MATH  Google Scholar 

  19. Cremers D., Osher S., and Soatto S. Kernel density estimation and intrinsic alignment for shape priors in level set segmentation. International Journal of Computer Vision, 69(3):335-351, 2006.

    Article  Google Scholar 

  20. Cremers D., Rousson M., and Deriche R. A review of statistical approaches to level set segmentation: integrating color, texture, motion and shape. Interna- tional Journal of Computer Vision, 72(2):195-215, 2007.

    Article  Google Scholar 

  21. DeCarlo D. and Metaxas D. Optical flow constraints on deformable models with applications to face tracking. International Journal of Computer Vision, 38(2):99-127, July 2000.

    Article  MATH  Google Scholar 

  22. Dempster A., Laird N., and Rubin D. Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society series B, 39:1-38, 1977.

    MATH  MathSciNet  Google Scholar 

  23. Dervieux A. and Thomasset F. A finite element method for the simulation of Rayleigh-Taylor instability. In R. Rautman, editor, Approximation Methods for Navier-Stokes Problems, volume 771 of Lecture Notes in Mathematics, pp. 145-158. Berlin, Springer, 1979.

    Chapter  Google Scholar 

  24. Dunn D., Higgins W.E. and Wakeley J. Texture segmentation using 2-D Gabor elementary functions. IEEE Transactions on Pattern Analysis and Machine Intelligence, 16(2):130-149, Feb. 1994.

    Article  Google Scholar 

  25. Elgammal A. and Lee C. Inferring 3D body pose from silhouettes using activity manifold learning. In Proc.International Conference on Computer Vision and Pattern Recognition, pp. 681-688, Washington DC, 2004.

    Google Scholar 

  26. Gavrila D. and Davis L.3D model based tracking of humans in action: a multiview approach. In ARPA Image Understanding Workshop, pp. 73-80, Palm Springs, 1996.

    Google Scholar 

  27. Grochow K., Martin S.L., Hertzmann A., and Popović Z. Style-based inverse kinematics. In ACM Transactions on Graphics (Proc.SIGGRAPH), volume 23, pp. 522-531, 2004.

    Article  Google Scholar 

  28. Heiler M. and Schnörr C. Natural image statistics for natural image segmenta- tion. International Journal of Computer Vision, 63(1):5-19, 2005.

    Article  Google Scholar 

  29. Horn B. and Schunck B. Determining optical flow. Artificial Intelligence, 17:185-203,1981.

    Article  Google Scholar 

  30. Horprasert T., Harwood D., and Davis L. A statistical approach for real-time robust background subtraction and shadow detection. In International Confer-ence on Computer Vision, FRAME-RATE Workshop, Kerkyra, Greece, 1999. Available at www.vast.uccs.edu/∼tboult/FRAME.

  31. Kadir T. and Brady M. Unsupervised non-parametric region segmentation using level sets. In Proc.Ninth IEEE International Conference on Computer Vision, volume 2, pp. 1267-1274, 2003.

    Article  Google Scholar 

  32. Kim J., Fisher J., Yezzi A., Cetin M., and Willsky A. A nonparametric statistical method for image segmentation using information theory and curve evolution. IEEE Transactions on Image Processing, 14(10):1486-1502, 2005.

    Article  MathSciNet  Google Scholar 

  33. Klette R. and Rosenfeld A. Digital Geometry-Geometric Methods for Digital Picture Analysis. Morgan Kaufmann, San Francisco, 2004.

    MATH  Google Scholar 

  34. Klette R., Schlüns K., and Koschan A. Computer Vision. Three-Dimensional Data from Images. Singapore, Springer, 1998.

    MATH  Google Scholar 

  35. Lawrence N.D. Gaussian process latent variable models for visualisation of high dimensional data. In Neural Information Processing Systems 16.

    Google Scholar 

  36. Leventon M.E., Grimson W.E.L., and Faugeras O. Statistical shape influence in geodesic active contours. In Proc.2000 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), volume 1, pp. 316-323, Hilton Head, SC, June 2000.

    Google Scholar 

  37. Lowe D. Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 60(2):91-110, 2004.

    Article  Google Scholar 

  38. Marchand E., Bouthemy P., and Chaumette F. A 2D-3D model-based approach to real-time visual tracking. Image and Vision Computing, 19(13):941-955, Nov. 2001.

    Article  Google Scholar 

  39. McLachlan G. and Krishnan T. The EM Algorithm and Extensions. Wiley series in probability and statistics. Wiley, 1997.

    Google Scholar 

  40. Mémin E. and Pérez P. Dense estimation and object-based segmentation of the optical flow with robust techniques. IEEE Transactions on Image Processing, 7(5):703-719, May 1998.

    Article  Google Scholar 

  41. Mumford D. and Shah J. Optimal approximations by piecewise smooth func- tions and associated variational problems. Communications on Pure and Applied Mathematics, 42:577-685, 1989.

    Article  MATH  MathSciNet  Google Scholar 

  42. Murray R., Li Z., and Sastry S. Mathematical Introduction to Robotic Manipu- lation. CRC Press, Baton Rouge, 1994.

    Google Scholar 

  43. Nagel H.-H. and Enkelmann W. An investigation of smoothness constraints for the estimation of displacement vector fields from image sequences. IEEE Transactions on Pattern Analysis and Machine Intelligence, 8:565-593, 1986.

    Article  Google Scholar 

  44. Osher S. and Sethian J.A. Fronts propagating with curvature-dependent speed: Algorithms based on Hamilton-Jacobi formulations. Journal of Computational Physics, 79:12-49, 1988.

    Article  MATH  MathSciNet  Google Scholar 

  45. Ö zuysal M., Lepetit V., Fleuret F., and Fua P. Feature harvesting for tracking-by-detection. In Proc.European Conference on Computer Vision, volume 3953 of LNCS, pp. 592-605. Graz, Austria, Springer, 2006.

    Google Scholar 

  46. Paragios N. and Deriche R. Geodesic active regions: A new paradigm to deal with frame partition problems in computer vision. Journal of Visual Communication and Image Representation, 13(1/2):249-268, 2002.

    Article  Google Scholar 

  47. Parzen E. On the estimation of a probability density function and the mode. Annals of Mathematical Statistics, 33:1065-1076, 1962.

    Article  MATH  MathSciNet  Google Scholar 

  48. Rasmussen C.E. and Williams C.K.I. Gaussian Processes for Machine Learning. MIT Press, Cambridge, MA, 2006.

    MATH  Google Scholar 

  49. Rosales R. and Sclaroff S. Learning body pose via specialized maps. In Proc. Neural Information Processing Systems, Dec. 2001.

    Google Scholar 

  50. Rosenblatt F. Remarks on some nonparametric estimates of a density function. Annals of Mathematical Statistics, 27:832-837, 1956.

    Article  MATH  MathSciNet  Google Scholar 

  51. Rosenhahn B., Brox T., Cremers D., and Seidel H.-P. A comparison of shape matching methods for contour based pose estimation. In R. Reulke, U. Eckhardt, B. Flach, U. Knauer and K. Polthier, editors, Proc.International Workshop on Combinatorial Image Analysis, volume 4040 of LNCS, pp. 263-276, Berlin, Germany, Springer, June 2006.

    Chapter  Google Scholar 

  52. Rosenhahn B., Brox T., Kersting U., Smith A., Gurney J., and Klette R. A system for marker-less motion capture. Künstliche Intelligenz, (1):45-51, 2006.

    Google Scholar 

  53. Rosenhahn B., Brox T., and Weickert J.. Three-dimensional shape knowledge for joint image segmentation and pose tracking. International Journal of Computer Vision, 73(3):243-262, July 2007.

    Article  Google Scholar 

  54. Rousson M., Brox T., and Deriche R. Active unsupervised texture segmentation on a diffusion based feature space. In Proc.International Conference on Com-puter Vision and Pattern Recognition, pp. 699-704, Madison, WI, June 2003.

    Google Scholar 

  55. Shevlin F. Analysis of orientation problems using Plücker lines. In International Conference on Pattern Recognition (ICPR), volume 1, pp. 685-689, Brisbane, 1998.

    Google Scholar 

  56. Shi J. and Tomasi C. Good features to track. In Proc.International Conference on Computer Vision and Pattern Recognition, pp. 593-600, 2004.

    Google Scholar 

  57. Sidenbladh H., Black M., and Sigal L. Implicit probabilistic models of human motion for synthesis and tracking. In A. Heyden, G. Sparr, M. Nielsen and P. Johansen, editors, Proc.European Conference on Computer Vision, volume 2353 of LNCS, pp. 784-800. Springer, 2002.

    Google Scholar 

  58. Silverman B.W. Density Estimation for Statistics and Data Analysis. Chapman & Hall, New York, 1986.

    MATH  Google Scholar 

  59. Sminchisescu C. and Jepson A. Generative modelling for continuous non-linearly embedded visual inference. In Proc.International Conference on Machine Learn-ing, 2004.

    Google Scholar 

  60. Sminchisescu C., Kanaujia A., Li Z., and Metaxas D. Discriminative density propagation for 3D human motion estimation. In Proc.International Conference on Computer Vision and Pattern Recognition, pp. 390-397, 2005.

    Google Scholar 

  61. Sminchisescu C., Kanaujia A., and Metaxas D. Learning joint top-down and bottom-up processes for 3D visual inference. In Proc.International Conference on Computer Vision and Pattern Recognition, pp. 1743-1752, 2006.

    Google Scholar 

  62. Sminchisescu C. and Triggs B. Estimating articulated human motion with co-variance scaled sampling. International Journal of Robotics Research, 22(6):371-391,2003.

    Article  Google Scholar 

  63. Sommer G., editor. Geometric Computing with Clifford Algebra: Theoreti-cal Foundations and Applications in Computer Vision and Robotics. Berlin, Springer, 2001.

    Google Scholar 

  64. Tsai A., Yezzi A., and Willsky A. Curve evolution implementation of the Mumford-Shah functional for image segmentation, denoising, interpolationand magnification. IEEE Transactions on Image Processing, 10(8):1169-1186, 2001.

    Article  MATH  Google Scholar 

  65. Urtasun R., Fleet D.J., and Fua P. 3D people tracking with Gaussian process dynamical models. In Proc.International Conference on Computer Vision and Pattern Recognition, pp. 238-245. IEEE Computer Society Press, 2006.

    Google Scholar 

  66. Zhu S.-C. and Yuille A. Region competition: unifying snakes, region growing, and Bayes/MDL for multiband image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 18(9):884-900, Sept. 1996.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. CVPR Group, University of Bonn, Römerstr. 164, 53117, Bonn, Germany

    Thomas Brox & Daniel Cremers

  2. Max-Planck Institute for Computer Science, Stuhlsatzhausenweg 85, D-66123, Saarbrücken, Germany

    Bodo Rosenhahn

Authors
  1. Thomas Brox
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bodo Rosenhahn
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Daniel Cremers
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Max-Planck Institute for Computer Science, Stuhlsatzhausenweg 85, D-66123, Saarbrücken, Germany

    Bodo Rosenhahn

  2. The University of Auckland, New Zealand

    Reinhard Klette

  3. Rutgers University, Piscataway, NJ, USA

    Dimitris Metaxas

Rights and permissions

Reprints and permissions

Copyright information

© 2008 Springer

About this chapter

Cite this chapter

Brox, T., Rosenhahn, B., Cremers, D. (2008). Contours, Optic Flow, and Prior Knowledge: Cues for Capturing 3D Human Motion in Videos. In: Rosenhahn, B., Klette, R., Metaxas, D. (eds) Human Motion. Computational Imaging and Vision, vol 36. Springer, Dordrecht. https://doi.org/10.1007/978-1-4020-6693-1_11

Download citation

  • DOI: https://doi.org/10.1007/978-1-4020-6693-1_11

  • Publisher Name: Springer, Dordrecht

  • Print ISBN: 978-1-4020-6692-4

  • Online ISBN: 978-1-4020-6693-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation