Advertisement

Markerless Motion Capture through Visual Hull, Articulated ICP and Subject Specific Model Generation

Abstract

An approach for accurately measuring human motion through Markerless Motion Capture (MMC) is presented. The method uses multiple color cameras and combines an accurate and anatomically consistent tracking algorithm with a method for automatically generating subject specific models. The tracking approach employed a Levenberg-Marquardt minimization scheme over an iterative closest point algorithm with six degrees of freedom for each body segment. Anatomical consistency was maintained by enforcing rotational and translational joint range of motion constraints for each specific joint. A subject specific model of the subjects was obtained through an automatic model generation algorithm (Corazza et al. in IEEE Trans. Biomed. Eng., 2009) which combines a space of human shapes (Anguelov et al. in Proceedings SIGGRAPH, 2005) with biomechanically consistent kinematic models and a pose-shape matching algorithm. There were 15 anatomical body segments and 14 joints, each with six degrees of freedom (13 and 12, respectively for the HumanEva II dataset). The overall method is an improvement over (Mündermann et al. in Proceedings of CVPR, 2007) in terms of both accuracy and robustness. Since the method was originally developed for ≥8 cameras, the method performance was tested both (i) on the HumanEva II dataset (Sigal and Black, Technical Report CS-06-08, 2006) in a 4 camera configuration, (ii) on a series of motions including walking trials, a very challenging gymnastic motion and a dataset with motions similar to HumanEva II but with variable number of cameras.

This is a preview of subscription content, log in to check access.

Access options

Buy single article

Instant unlimited access to the full article PDF.

US$ 39.95

Price includes VAT for USA

Subscribe to journal

Immediate online access to all issues from 2019. Subscription will auto renew annually.

US$ 199

This is the net price. Taxes to be calculated in checkout.

References

  1. Aggarwal, J., & Cai, Q. (1999). Human motion analysis: a review. Computer Vision and Image Understanding, 73(3), 295–304.

  2. Andriacchi, T. P., Alexander, E. J., Toney, M. K., Dyrby, C. O., & Sum, J. A. (1998). A point cluster method for in vivo motion analysis: applied to a study of knee kinematics. Journal of Biomechanical Engineering, 120, 743–749.

  3. Anguelov, D., Koller, D., Pang, H., Srinivasan, P., & Thrun, S. (2004). Recovering articulated object models from 3D range data. In Proceedings UAI.

  4. Anguelov, D., Srinivasan, P., Koller, D., Thrun, S., Rodgers, J., & Davis, J. (2005). SCAPE: shape completion and animation of people. In Proceedings SIGGRAPH.

  5. Balan, A. O., Sigal, L., Black, M. J., Davis, J. E., & Haussecker, H. W. (2007). Detailed human shape and pose from images. In Proceedings CVPR.

  6. Baran, I., & Popovic, J. (2007). Automatic rigging and animation of 3D characters. In Proceedings of SIGGRAPH.

  7. Besl, P., & McKay, N. (1992). A method for registration of 3D shapes. Transactions on Pattern Analysis and Machine Intelligence, 14(2), 239–256.

  8. Bharatkumar, A. G., Daigle, K. E., Pandy, M. G., Cai, Q., & Aggarwal, J. K. (1994). Lower limb kinematics of human walking with the medial axis transformation. In IEEE Workshop on Non-Rigid Motion, Austin, USA (pp. 70–76).

  9. Bottino, A., & Laurentini, A. (2001). A silhouette based technique for the reconstruction of human movement. Computer Vision and Image Understanding, 83, 79.

  10. Bregler, C., & Malik, J. (1997). Tracking people with twists and exponential maps. In Proceedings CVPR.

  11. Cedras, C., & Shah, M. (1995). Motion-based recognition: a survey. Image and Vision Computing, 13(2), 129–155.

  12. Cheung, K., Baker, S., & Kanade, T. (2005). Shape-from-silhouette across time part I: Theory and algorithm. International Journal of Computer Vision, 62, 221–247.

  13. Corazza, S., Mündermann, L., Chaudhari, A. M., Demattio, T., Cobelli, C., & Andriacchi, T. P. (2006). A markerless motion capture system to study musculoskeletal biomechanics: visual hull and simulated annealing approach. Annals Biomedical Engineering, 34(6), 1019–1029.

  14. Corazza, S., Gambaretto, E., Mündermann, L., & Andriacchi, T. (2009). Automatic generation of a subject specific model for accurate markerless motion capture and biomechanical applications. IEEE Transactions on Biomedical Engineering, in press.

  15. Delamarre, Q., & Faugeras, O. (1999). 3D articulated models and multiview tracking with silhouettes. In Proceedings ICCV.

  16. Demirdjian, D. (2004). Combining geometric- and view-based approaches for articulated pose. In Proceedings ECCV04 (Vol. III, pp. 183–194).

  17. Deutscher, J., Blake, A., & Reid, I. (2000). Articulated body motion capture by annealed particle filtering. In Proceedings CVPR (pp. 2126–2133).

  18. Gavrila, D. (1999). The visual analysis of human movement: a survey. Computer Vision and Image Understanding, 73(3), 82–98.

  19. Gavrila, D., & Davis, L. (1996). 3-D model based tracking of humans in action:a multiview approach. In Proceedings CVPR (pp. 73–80).

  20. Hogg, D. (1983). Model-based vision: a program to see a walking person. Image and Vision Computing, 1, 5.

  21. Isard, M., & Blake, A. (1996). Estimating 3D hand pose using hierarchical multi-label classification. In Proceedings of 4th European Conference on Computer Vision, Cambridge, UK.

  22. Kakadiaris, I. A., & Metaxas, D. (1998). Three-dimensional human body model acquisition from multiple views. International Journal of Computer Vision, 30, 191.

  23. Kanade, T., Saito, H., & Vedula, S. (1998). The 3D Room: Digitizing time-varying 3D events by synchronized multiple video streams (Tech. report CMU-RI-TR-98-34). Robotics Institute, Carnegie Mellon University.

  24. Knossow, D., Ronfard, R., & Horaud, R. P. (2008). Human motion tracking with a kinematic parameterization of extremal contours. International Journal of Computer Vision, 79(2), 247–269.

  25. Kohli, P., Rihan, J., Bray, M., & Torr, P. H. S. (2008). Simultaneous segmentation and pose estimation of humans using dynamic graph cuts. International Journal of Computer Vision, 79(3), 285–298.

  26. Laurentini, A. (1994). The Visual Hull concept for silhouette base image understanding. IEEE Transactions on Pattern Analysis and Machine Intelligence, 16, 150–162.

  27. Leardini, A., Chiari, L., Della Croce, U., & Cappozzo, A. (2005). Human movement analysis using stereophotogrammetry. Part 3: Soft tissue artifact assessment and compensation. Gait and Posture, 21, 221–225.

  28. Lee, H. J., & Chen, Z. (1985). Determination of 3D human body posture from a single view. Computer Vision, Graphics, and Image Processing, 30, 148–168.

  29. Lee, W., Gu, J., & Magnenat-Thalmann, N. (2000). Generating animatable 3D virtual humans from photographs. In Proceedings Computer Graphics Forum—Eurographics (pp. 1–10).

  30. Legrand, L., Marzani, F., & Dusserre, L. (1998). A marker-free system for the analysis of. movement disabilities. Medinfo, 9, 1066–1070.

  31. Liu, Q., & Prakash, E. C. (2003). The parametrization of joint rotation with the unit quaternion. In Proceedings of 7° Digital Image Computing.

  32. Marzani, F., Calais, E., & Legrand, L. (2001). A 3-D marker-free system for the analysis of movement disabilities—an application to the legs. IEEE Transactions on Information Technology in Biomedicine, 5(1), 18–26.

  33. Mikic, I., Trivedi, M., Hunter, E., & Cosman, P. (2003). Human body model acquisition and tracking using voxel data. International Journal of Computer Vision, 53, 199–223.

  34. Moeslund, T. B., Hilton, A., & Krüger, V. (2006). A survey of advances in vision-based human motion capture and analysis. Computer Vision and Image Understanding, 104(2), 90–126.

  35. Moon, H., Chellappa, R., & Rosenfeld, A. (2001). 3D object tracking using shape-encoded particle propagation. In Proceedings ICCV.

  36. Mündermann, L., Corazza, S., Chaudhari, A. M., Alexander, E. J., & Andriacchi, T. P. (2005). Most favorable camera configuration for a shape-from-silhouette markerless motion capture system for biomechanical analysis. Proceedings of SPIE-IS&T Electronic Imaging, 5665, 278–287.

  37. Mündermann, L., Corazza, S., & Andriacchi, T.P. (2006). The evolution of methods for the capture of human movement leading to markerless motion capture for biomechanical applications. Journal of Neuroengineering and Rehabilitation, 3(1).

  38. Mündermann, L., Corazza, S., & Andriacchi, T. (2007). Accurately measuring human movement using articulated ICP with soft-joint constraints and a repository of articulated models. In Proceedings of CVPR.

  39. Narayanan, P. J., Rander, P., & Kanade, T. (1995). Synchronous capture of image sequences from multiple cameras (Technical Report CMU-RI-TR-95-25). Robotics Institute, Carnegie Mellon University.

  40. Nielsen, H. B. (1999). Damping parameter in Marquardt’s method (Technical Report IMM-REP-1999-05). Technical University of Denmark.

  41. Niskanen, M., Boyer, E., & Horaud, R. (2005). Articulated motion capture from 3-D points and normals. In Proceedings of BMVC’05.

  42. O’Rourke, J., & Badler, N. I. (1980). Model-based image analysis of human motion using constraint propagation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2, 522–536.

  43. Plankers, R., & Fua, P. (2003). Articulated soft objects for multiview shape and motion capture. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25, 1182–1187.

  44. Rosenhahn, B., & Klette, R. (2005). Automatic human model generation. Computer Analysis of Images and Patterns, 230–237.

  45. Rosenhahn, B., Brox, T., Kersting, U. G., Smith, A. W., Gurney, J. K., & Klette, R. (2006). A system for marker-less motion capture. Künstliche Intelligenz (KI), 1, 45–51.

  46. Sigal, L., & Black, M. J. (2006). HumanEva: synchronized video and motion capture dataset for evaluation of articulated human motion (Technical Report CS-06-08). Brown University.

  47. Wagg, D. K., & Nixon, M. S. (2004). Automated markerless extraction of walking people using deformable contour models. Computer Animation and Virtual Worlds, 15, 399–406.

  48. Wren, C. R., Azarbayejani, A., Darrell, T., & Pentland, A. P. (1997). Pfinder—real-time tracking of the human body. Transactions on Pattern Analysis and Machine Intelligence, 19, 780–785.

  49. Yamamoto, M., & Koshikawa, K. (1991). Human motion analysis based on a robot arm model. In Proceedings Computer Vision and Pattern Recognition.

Download references

Author information

Correspondence to Stefano Corazza.

Electronic Supplementary Material

Below is the link to the electronic supplementary material. (WMV 1,841 kB)

Below is the link to the electronic supplementary material. (WMV 1,961 kB)

Below is the link to the electronic supplementary material. (MOV 2,275 kB)

Below is the link to the electronic supplementary material. (WMV 1,841 kB)

Below is the link to the electronic supplementary material. (WMV 1,961 kB)

Below is the link to the electronic supplementary material. (MOV 2,275 kB)

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Corazza, S., Mündermann, L., Gambaretto, E. et al. Markerless Motion Capture through Visual Hull, Articulated ICP and Subject Specific Model Generation. Int J Comput Vis 87, 156 (2010) doi:10.1007/s11263-009-0284-3

Download citation

Keywords

  • Markerless motion capture
  • Tracking
  • 3D reconstruction
  • Human body model
  • Shape from silhouette