Skip to main content
SpringerLink
Log in

Marker-Less 3D Feature Tracking for Mesh-Based Human Motion Capture

  • Conference paper
Human Motion – Understanding, Modeling, Capture and Animation (HuMo 2007)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 4814))

Included in the following conference series:

Abstract

We present a novel algorithm that robustly tracks 3D trajectories of features on a moving human who has been recorded with multiple video cameras. Our method does so without special markers in the scene and can be used to track subjects wearing everyday apparel. By using the paths of the 3D points as constraints in a fast mesh deformation approach, we can directly animate a static human body scan such that it performs the same motion as the captured subject. Our method can therefore be used to directly animate high quality geometry models from unaltered video data which opens the door to new applications in motion capture, 3D Video and computer animation. Since our method does not require a kinematic skeleton and only employs a handful of feature trajectories to generate lifelike animations with realistic surface deformations, it can also be used to track subjects wearing wide apparel, and even animals. We demonstrate the performance of our approach using several captured real-world sequences, and also validate its accuracy.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Moeslund, T.B., Granum, E.: A survey of computer vision-based human motion capture. CVIU 81(3), 231–268 (2001)

    MATH  Google Scholar 

  2. Silaghi, M.C., Plänkers, R., Boulic, R., Fua, P., Thalmann, D.: Local and global skeleton fitting techniques for optical motion capture. In: Magnenat-Thalmann, N., Thalmann, D. (eds.) CAPTECH 1998. LNCS (LNAI), vol. 1537, pp. 26–40. Springer, Heidelberg (1998)

    Google Scholar 

  3. Lewis, J.P., Cordner, M., Fong, N.: Pose space deformation: a unified approach to shape interpolation and skeleton-driven deformation. ACM Trans. Graph, 165–172 (2000)

    Google Scholar 

  4. Allen, B., Curless, B., Popović, Z.: Articulated body deformation from range scan data. ACM Trans. Graph (SIGGRAPH 2002), 612–619 (2002)

    Google Scholar 

  5. Park, S.I., Hodgins, J.K.: Capturing and animating skin deformation in human motion. ACM Trans. Graph (SIGGRAPH 2006) 25(3) (2006)

    Google Scholar 

  6. Sand, P., McMillan, L., Popovic, J.: Continuous capture of skin deformation. ACM Trans. Graph. 22(3), 578–586 (2003)

    Article  Google Scholar 

  7. Goldluecke, B., Magnor, M.: Space-time isosurface evolution for temporally coherent 3d reconstruction. In: CVPR 2004, vol. 1, pp. 350–355 (2004)

    Google Scholar 

  8. Furukawa, Y., Ponce, J.: Carved visual hulls for image-based modeling. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3951, pp. 564–577. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  9. Metaxas, D., Terzopoulos, D.: Shape and nonrigid motion estimation through physics-based synthesis. IEEE Trans. Pattern Anal. Mach. Intell. 15(6), 580–591 (1993)

    Article  Google Scholar 

  10. Lepetit, V., Fua, P.: Monocular model-based 3d tracking of rigid objects. Found. Trends. Comput. Graph. Vis. 1(1), 1–89 (2006)

    Article  Google Scholar 

  11. Yilmaz, A., Javed, O., Shah, M.: Object tracking: A survey. ACM Comput. Surv. 38(4), 13 (2006)

    Article  Google Scholar 

  12. Anguelov, D., Srinivasan, P., Koller, D., Thrun, S., Rodgers, J., Davis, J.: Scape: shape completion and animation of people. ACM Trans. Graph. 24(3), 408–416 (2005)

    Article  Google Scholar 

  13. Rosenhahn, B., Kersting, U., Powel, K., Seidel, H.P.: Cloth x-ray: Mocap of people wearing textiles. In: DAGM, pp. 495–504 (2006)

    Google Scholar 

  14. Zitnick, C.L., Kang, S.B., Uyttendaele, M., Winder, S., Szeliski, R.: High-quality video view interpolation using a layered representation. ACM Trans. Graph. 23(3), 600–608 (2004)

    Article  Google Scholar 

  15. Decarlo, D., Metaxas, D.: Optical flow constraints on deformable models with applications to face tracking. Int. J. Comput. Vision 38(2), 99–127 (2000)

    Article  MATH  Google Scholar 

  16. Pritchard, D., Heidrich, W.: Cloth motion capture. Eurographics, 263–271 (September 2003)

    Google Scholar 

  17. de Aguiar, E., Theobalt, C., Magnor, M., Seidel, H.P.: Reconstructing human shape and motion from multi-view video. In: CVMP 2005, pp. 42–49 (2005)

    Google Scholar 

  18. Plänkers, R., Fua, P.: Articulated soft objects for multiview shape and motion capture. IEEE Trans. Pattern Anal. Mach. Intell. 25(9), 1182–1187 (2003)

    Article  Google Scholar 

  19. Hasler, N., Asbach, M., Rosenhahn, B., Ohm, J.R., Seidel, H.P.: Physically based tracking of cloth. In: Proc of VMV 2006, Aachen, Germany, pp. 49–56 (2006)

    Google Scholar 

  20. Salzmann, M., Ilic, S., Fua, P.: Physically valid shape parameterization for monocular 3-d deformable surface tracking. In: British Machine Vision Conference (2005)

    Google Scholar 

  21. Sorkine, O.: Differential representations for mesh processing. Computer Graphics Forum 25(4) (2006)

    Google Scholar 

  22. Balan, A.O., Black, M.J.: An adaptive appearance model approach for model-based articulated object tracking. In: Proc. of CVPR 2006, pp. 758–765. IEEE Computer Society Press, Los Alamitos (2006)

    Google Scholar 

  23. Brox, T., Rosenhahn, B., Cremers, D., Seidel, H.P.: High accuracy optical flow serves 3-d pose tracking: Exploiting contour and flow based constraints. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3952, pp. 98–111. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  24. Kehl, R., Gool, L.V.: Markerless tracking of complex human motions from multiple views. Comput. Vis. Image Underst. 104(2), 190–209 (2006)

    Article  Google Scholar 

  25. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision 20, 91–110 (2004)

    Article  Google Scholar 

  26. Lowe, D.G.: Object recognition from local scale-invariant features. In: Proc. of ICCV, pp. 1150–1157 (1999)

    Google Scholar 

  27. Mikolajczyk, K., Schmid, C.: A performance evaluation of local descriptors (2003)

    Google Scholar 

  28. Brox, T., Bruhn, A., Papenberg, N., Weickert, J.: High accuracy optical flow estimation based on a theory for warping. In: Pajdla, T., Matas, J. (eds.) ECCV 2004. LNCS, vol. 3024, pp. 25–36. Springer, Heidelberg (2004)

    Google Scholar 

  29. de Aguiar, E., Theobalt, C., Stoll, C., Seidel, H.P.: Rapid animation of laser-scanned humans. In: IEEE Virtual Reality 2007, pp. 223–226. IEEE Computer Society Press, Los Alamitos (2007)

    Chapter  Google Scholar 

  30. Lipman, Y., Sorkine, O., Cohen-Or, D., Levin, D., Rössl, C., Seidel, H.P.: Differential coordinates for interactive mesh editing. In: SMI 2004, pp. 181–190 (2004)

    Google Scholar 

  31. Stoll, C., Karni, Z., Rössl, C., Yamauchi, H., Seidel, H.P.: Template deformation for point cloud fitting. In: Symposium on Point-Based Graphics, pp. 27–35 (2006)

    Google Scholar 

  32. Kruskal, J.B.: On the shortest spanning subtree of a graph and the traveling salesman problem. Proc. of the American Mathematical Society 7, 48–50 (1956)

    Article  MathSciNet  Google Scholar 

  33. Zayer, R., Rössl, C., Karni, Z., Seidel, H.P.: Harmonic guidance for surface deformation. In: Proc. of Eurographics 2005, vol. 24, pp. 601–609 (2005)

    Google Scholar 

  34. Pebay, P.P., Baker, T.J.: A comparison of triangle quality measures. In: Proc. of the 10th International Meshing Roundtable, pp. 327–340 (2001)

    Google Scholar 

  35. Huang, J., Shi, X., Liu, X., Zhou, K., Wei, L.Y., Teng, S.H., Bao, H., Guo, B., Shum, H.Y.: Subspace gradient domain mesh deformation. ACM Trans. Graph. 25(3), 1126–1134 (2006)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. MPI Informatik, Germany

    Edilson de Aguiar, Carsten Stoll & Hans-Peter Seidel

  2. Stanford University, USA

    Christian Theobalt

Authors
  1. Edilson de Aguiar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Christian Theobalt
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Carsten Stoll
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hans-Peter Seidel
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Ahmed Elgammal Bodo Rosenhahn Reinhard Klette

Rights and permissions

Reprints and permissions

Copyright information

© 2007 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

de Aguiar, E., Theobalt, C., Stoll, C., Seidel, HP. (2007). Marker-Less 3D Feature Tracking for Mesh-Based Human Motion Capture. In: Elgammal, A., Rosenhahn, B., Klette, R. (eds) Human Motion – Understanding, Modeling, Capture and Animation. HuMo 2007. Lecture Notes in Computer Science, vol 4814. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-75703-0_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-75703-0_1

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-75702-3

  • Online ISBN: 978-3-540-75703-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation