Skip to main content
SpringerLink
Log in

Reconstructing Human Shape, Motion and Appearance from Multi-view Video

  • Chapter
Book cover Three-Dimensional Television

Part of the book series: Signals and Communication Technology ((SCT))

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 219.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. B. Bodenheimer, C. Rose, S. Rosenthal, and J. Pella. The process of motion capture: Dealing with the data. In Proc. of Eurographics Computer Animation and Simulation, 1997.

    Google Scholar 

  2. G. Johannson. Visual perception of biological motion and a model for its analysis. In Perception and Psychophysics, 14(2):201–211, 1973.

    Google Scholar 

  3. M. Gleicher. Animation from observation: Motion capture and motion editing. In Computer Graphics, 4(33):51–55, November 1999.

    Google Scholar 

  4. L. Herda, P. Fua, R. Plaenkers, R. Boulic, and D. Thalmann. Skeleton-based motion capture for robust reconstruction of human motion. In Proc. of Computer Animation 2000, IEEE CS Press 2000.

    Google Scholar 

  5. M. Ringer and J. Lasenby. Multiple-hypothesis tracking for automatic human motion capture. In Proc. of European Conference on Computer Vision, 1: 524–536, 2002.

    Google Scholar 

  6. www.vicon.com.

    Google Scholar 

  7. T.B. Moeslund and E. Granum. A survey of computer vision-based human motion capture. In CVIU, 81(3):231–268, 2001.

    MATH  Google Scholar 

  8. K.M. Cheung, T. Kanade, J.-Y. Bouguet, and M. Holler. A real time system for robust 3D voxel reconstruction of human motions. In Proc. of CVPR, 2: 714–720, June 2000.

    Google Scholar 

  9. I. Mikić, M. Triverdi, E. Hunter, and P. Cosman. Articulated body posture estimation from multicamera voxel data. In Proc. of CVPR, 1:455ff, 2001.

    Google Scholar 

  10. C. Sminchisescu and B. Triggs. Kinematic jump processes for monocular 3d human tracking. In Proc. of IEEE International Conference on Computer Vision and Pattern Recognition, I 69–76, 2003.

    Google Scholar 

  11. D.M. Gavrila and L.S. Davis. 3D model-based tracking of humans in action: A multi-view approach. In CVPR 96, 73–80, 1996.

    Google Scholar 

  12. I.A. Kakadiaris and D. Metaxas. Model-based estimation of 3D human motion with occlusion based on active multi-viewpoint selection. In Proc. CVPR, 81–87, Los Alamitos, California, USA, 1996. IEEE Computer Society.

    Google Scholar 

  13. H. Sidenbladh, M.J. Black, and J.D. Fleet. Stochastic tracking of 3D human figures using 2D image motion. In Proc. of ECCV, 2:702–718, 2000.

    Google Scholar 

  14. L. Goncalves, E. DiBernardo, E. Ursella, and P. Perona. Monocular tracking of the human arm in 3D. In Proc. of CVPR, 764–770, 1995.

    Google Scholar 

  15. R. Plaenkers and P. Fua. Articulated soft objects for multi-view shape and motion capture. In IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(10), 2003.

    Google Scholar 

  16. A. Mittal, L. Zhao, and L.S. Davis. Human body pose estimation using silhouette shape analysis. In Proc. of Conference on Advanced Video and Signal-based Surveillance (AVSS), 263ff, 2003.

    Google Scholar 

  17. J. O’Rourke and N.I. Badler. Model-based image analysis of human motion using constraint propagation. In PAMI, 2(6), 1980.

    Google Scholar 

  18. Z. Chen and H. Lee. Knowledge-guided visual perception of 3d human gait from a single image sequence. In IEEE Transactions on Systems, Man and Cybernetics, 22(2):336–342, 1992.

    Article  Google Scholar 

  19. N. Grammalidis, G. Goussis, G. Troufakos, and M.G. Strintzis. Estimating body animation parameters from depth images using analysis by synthesis. In Proc. of Second International Workshop on Digital and Computational Video (DCV’01), 93ff, 2001.

    Google Scholar 

  20. R. Koch. Dynamic 3D scene analysis through synthesis feedback control. In PAMI, 15(6):556–568, 1993.

    Google Scholar 

  21. G. Martinez. 3D motion estimation of articulated objects for object-based analysis-synthesis coding (OBASC). In VLBV 95, 1995.

    Google Scholar 

  22. I.A. Kakadiaris and D. Metaxas. 3D human body model acquisition from multiple views. In Proc. of ICCV’ 95, 618–623, 1995.

    Google Scholar 

  23. Q. Delamarre and O. Faugeras. 3D articulated models and multi-view tracking with silhouettes. In ICCV99, 716–721, 1999.

    Google Scholar 

  24. S. Yonemoto, D. Arita, and R. Taniguchi. Real-time human motion analysis and ik-based human figure control. In Proc. of IEEE Workshop on Human Motion, 149–154, 2000.

    Google Scholar 

  25. C. Bregler and J. Malik. Tracking people with twists and exponential maps. In Proc. of CVPR 98, 8–15, 1998.

    Google Scholar 

  26. M.M. Covelle, A. Rahimi, M. Harville, and T.J. Darrell. Articulated pose estimation using brighness and depth constancy constraints. In Proc. of IEEE International Conference on Computer Vision and Pattern Recognition, 2: 438–445, 2000.

    Google Scholar 

  27. B. Rosenhahn, T. Brox, and J. Weickert. Three-dimensional shape knowledge for joint image segmentation and pose tracking. In To appear in International Journal of Computer Vision, 2006.

    Google Scholar 

  28. B. Deutscher, A. Blake, and I. Reid. Articulated body motion capture by annealed particle filtering. In Proc. of CVPR’00, 2: 2126ff, 2000.

    Google Scholar 

  29. T. Drummond and R. Cipolla. Real-time tracking of highly articulated structures in the presence of noisy measurements. In Proc. of ICCV, 2: 315–320, 2001.

    Google Scholar 

  30. J. MacCormick and M. Isard. Partitioned sampling, articulated objects, and interface-quality hand tracking. In Proc. of European Conference on Computer Vision, 2:3–19, 2000.

    Google Scholar 

  31. H. Sidenbladh, M. Black, and R. Sigal. Implicit probabilistic models of human motion for synthesis and tracking. In Proc. of ECCV, 1:784–800, 2002.

    Google Scholar 

  32. C. Theobalt, M. Magnor, P. Schüler, and H.-P. Seidel. Combining 2d feature tracking and volume reconstruction for online video-based human motion capture. In Proc. of the 10th Pacific Conference on Computer Graphics and Applications (Pacific Graphics 2002), pages 96–103, Beijing, China, 2002. IEEE.

    Google Scholar 

  33. A. Bottino and A. Laurentini. A silhouette based technique for the reconstruction of human movement. In CVIU, 83:79–95, 2001.

    MATH  Google Scholar 

  34. G. Cheung, S. Baker, and T. Kanada. Shape-from-silhouette of articulated objects and its use for human body kinematics estimation and motion capture. In Proc. of CVPR, 2003.

    Google Scholar 

  35. J. Carranza, C. Theobalt, M.A. Magnor, and H.-P. Seidel. Free-viewpoint video of human actors. In Proc. of SIGGRAPH’ 03, 569–577, 2003.

    Google Scholar 

  36. E. de Aguiar, C. Theobalt, M. Magnor, and H.-P. Seidel. Reconstructing human shape and motion from multi-view video. In 2nd European Conference on Visual Media Production (CVMP), pages 42–49, London, UK, December 2005. The IEE.

    Google Scholar 

  37. E. de Aguiar, R. Zayer, C. Theobalt, M. Magnor, and H.-P. Seidel. A framework for natural animation of digitized models. Research Report MPI-I-2006-4-003, Saarbruecken, Germany, July 2006. Max-Planck-Institut fuer Informatik.

    Google Scholar 

  38. C. Theobalt, E. de Aguiar, M. Magnor, H. Theisel, and H.-P. Seidel. Marker-free kinematic skeleton estimation from sequences of volume data. In ACM Symposium on Virtual Reality Software and Technology (VRST 2004), 57–64 Hong Kong, China, November 2004. ACM.

    Google Scholar 

  39. E. de Aguiar, C. Theobalt, M. Magnor, H. Theisel, and H.-P. Seidel. M3: Marker-free model reconstruction and motion tracking from 3d voxel data. In 12th Pacific Conference on Computer Graphics and Applications, PG 2004, pages 101–110, Seoul, Korea, October 2004. IEEE.

    Google Scholar 

  40. E. de Aguiar, C. Theobalt, and H.-P. Seidel. Automatic learning of articulated skeletons from 3d marker trajectories. In Proc. of ISVC’06, 2006.

    Google Scholar 

  41. B. Allen, B. Curless, and Z. Popovic. Articulated body deformations from range scan data. In Proc. of ACM SIGGRAPH 02, 612–619, 2002.

    Google Scholar 

  42. P. Sand, L. McMillan, and J. Popovic. Continuous capture of skin deformation. In ACM Transactions on. Graphics, 22(3):578–586, 2003.

    Google Scholar 

  43. D. Anguelov, P. Srinivasan, D. Koller, S. Thrun, J. Rogers, and J. Davis. SCAPE - shape completion and animation of people. In ACM Transactions on Graphics (Proc. of SIGGRAPH’05), 24(3): 408–416, 2005.

    Google Scholar 

  44. W. Matusik, C. Buehler, R. Raskar, S.J. Gortler, and L. McMillan. Image-based visual hulls. In Proc. of ACM SIGGRAPH 00, 369–374, 2000.

    Google Scholar 

  45. S. Würmlin, E. Lamboray, O.G. Staadt, and M.H. Gross. 3d video recorder. In Proc. of IEEE Pacific Graphics, 325–334, 2002.

    Google Scholar 

  46. T. Matsuyama and T. Takai. Generation, visualization, and editing of 3D video. In Proc. of 1st International Symposium on 3D Data Processing Visualization and Transmission (3DPVT’02), 234ff, 2002.

    Google Scholar 

  47. M.H. Gross, S. Würmlin, M. Näf, E. Lamboray, C.P. Spagno, A.M. Kunz, E. Koller-Meier, T. Svoboda, L.J. Van Gool, S. Lang, K. Strehlke, A. Vande Moere, and O.G. Staadt. blue-c: a spatially immersive display and 3d video portal for telepresence. In ACM Transactions on Graphics (Proc. of SIGGRAPH’03), 22(3):819–827, 2003.

    Article  Google Scholar 

  48. M. Li, H. Schirmacher, M. Magnor, and H.-P. Seidel. Combining stereo and visual hull information for on-line reconstruction and rendering of dynamic scenes. In Proc. of IEEE Multimedia and Signal Processing, 9–12, 2002.

    Google Scholar 

  49. C. Lawrence Zitnick, S. Bing Kang, M. Uyttendaele, S. Winder, and R. Szeliski. High-quality video view interpolation using a layered representation. In ACM TOC (Proc. SIGGRAPH’04), 23(3):600–608, 2004.

    Google Scholar 

  50. T. Kanade, P. Rander, and P.J. Narayanan. Virtualized reality: Constructing virtual worlds from real scenes. In IEEE MultiMedia, 4(1):34–47, 1997.

    Google Scholar 

  51. M. Waschbüsch, S. Würmlin, D. Cotting, F. Sadlo, and M. Gross. Scalable 3D video of dynamic scenes. In Proc. of Pacific Graphics, 629–638, 2005.

    Google Scholar 

  52. M. Levoy and P. Hanrahan. Light field rendering. In Proc. of ACM SIGGRAPH’ 96, 31–42, 1996.

    Google Scholar 

  53. W. Matusik and H. Pfister. 3d tv: A scalable system for real-time acquisition, transmission, and autostereoscopic display of dynamic scenes. In ACM Transactions on Graphics (Proc. of SIGGRAPH’04), 23(3):814–824, 2004.

    Article  Google Scholar 

  54. C. Theobalt, J. Carranza, M. Magnor, and H.-P. Seidel. A parallel framework for silhouette-based human motion capture. In Vision, Modeling and Visualization 2003 (VMV-03): Proc., pages 207–214, Munich, Germany, November 2003.

    Google Scholar 

  55. C. Theobalt, J. Carranza, M. Magnor, and H.-P. Seidel. Enhancing silhouette-based human motion capture with 3d motion fields. In Jon Rokne, Reinhard Klein, and Wenping Wang, editors, 11th Pacific Conference on Computer Graphics and Applications (PG-03), pages 185–193, Canmore, Canada, October 2003. IEEE.

    Google Scholar 

  56. C. Theobalt, J. Carranza, M. Magnor, and H.-P. Seidel. Combining 3d flow fields with silhouette-based human motion capture for immersive video. In Graphical Models, 66:333–351, September 2004.

    Google Scholar 

  57. C. Theobalt, J. Carranza, M. Magnor, and H.-P. Seidel. 3d video – being part of the movie. In ACM SIGGRAPH Computer Graphics, 38(3):18–20, August 2004.

    Article  Google Scholar 

  58. N. Ahmed, E. de Aguiar, C. Theobalt, M. Magnor, and H.-P. Seidel. Automatic generation of personalized human avatars from multi-view video. In VRST ’05: Proc. of the ACM Symposium on Virtual Reality Software and Technology, pages 257–260, Monterey, USA, December 2005. ACM.

    Google Scholar 

  59. M. Alexa, M.-P. Cani, and K. Singh. Interactive shape modeling. In Eurographics Course Notes. 2005.

    Google Scholar 

  60. O. Sorkine. Differential representations for mesh processing. In Computer Graphics Forum, 25(4), 2006.

    Google Scholar 

  61. R. Zayer, C. Rössl, Z. Karni, and H.-P. Seidel. Harmonic guidance for surface deformation. In Marc Alexa and Joe Marks, editors, Proc. of Eurographics 2005, 24:601–609, 2005.

    Google Scholar 

  62. R.W. Sumner and J. Popovic. Deformation transfer for triangle meshes. In ACM Transactions on Graphics, 23(3):399–405, 2004.

    Article  Google Scholar 

  63. R.W. Sumner, M. Zwicker, C. Gotsman, and J. Popovic. Mesh-based inverse kinematics. In ACM Transactions on Graphics, 24(3):488–495, 2005.

    Article  Google Scholar 

  64. K.G. Der, R.W. Sumner, and J. Popovic. Inverse kinematics for reduced deformable models. In ACM Transactions on Graphics, 25(3):1174–1179, 2006.

    Article  Google Scholar 

  65. L. Shi, Y. Yu, N. Bell, and W.-W. Feng. A fast multigrid algorithm for mesh deformation. In ACM Transactions on Graphics, 25(3):1108–1117, 2006.

    Article  Google Scholar 

  66. J. Huang, X. Shi, X. Liu, K. Zhou, L.-Y. Wei, S.-H. Teng, H. Bao, B. Guo, and H.-Y. Shum. Subspace gradient domain mesh deformation. In ACM Transactions on Graphics, 25(3):1126–1134, 2006.

    Article  Google Scholar 

  67. H.P.A. Lensch, W. Heidrich, and H.-P. Seidel. A silhouette-based algorithm for texture registration and stitching. In Graphic Models, 63(4): 245–262, 2001.

    Article  MATH  Google Scholar 

  68. W.H. Press, S.A. Teukolsky, W.T. Vetterling, and B.P. Flannery. In Numerical Recipes in C++. 2002, Cambridge University Press.

    Google Scholar 

  69. P. Fua and Y.G. Leclerc. Object-centered surface reconstruction: Combining multi-image stereo and shading. In International Journal of Computerised Vision, 16(1):35–55, 1995.

    Article  Google Scholar 

  70. M.N. Kolountzakis and K.N. Kutulakos. Fast computation of the euclidian distance maps for binary images. In Information Processing Letters, 43(4):181–184, 1992.

    Article  MATH  MathSciNet  Google Scholar 

  71. P.P. Pebay and T.J. Baker. A comparison of triangle quality measures. In Proc. to the 10th International Meshing Roundtable, 327–340, 2001.

    Google Scholar 

  72. R. Byrd, P. Lu, J. Nocedal, and C. Zhu. A limited memory algorithm for bound constrained optimization. In SIAM Journal of Science Comparative, 16(5): 1190–1208, 1995.

    Article  MATH  MathSciNet  Google Scholar 

  73. G. Farin. Curves and Surfaces for CAGD: A Practical Guide, 1999. Morgan Kaufmann.

    Google Scholar 

  74. Y. Lipman, O. Sorkine, D. Cohen-Or, D. Levin, C. Rössl, and H.-P. Seidel. Differential coordinates for interactive mesh editing. In Franca Giannini and Alexander Pasko, editors, Shape Modeling International 2004 (SMI 2004), pages 181–190, Genova, Italy, 2004. IEEE.

    Google Scholar 

  75. C. Theobalt, N. Ahmed, E. de Aguiar, G. Ziegler, H.P.A. Lensch, M. Magnor, and H.-P. Seidel. Joint motion and reflectance capture for creating relightable 3d videos. Research Report MPI-I-2005-4-004, Saarbruecken, Germany, April 2005, Max-Planck-Institut fuer Informatik.

    Google Scholar 

  76. C. Theobalt, N. Ahmed, H.P.A. Lensch, M. Magnor, and H.-P. Seidel. Enhanced dynamic reflectometry for relightable free-viewpoint video, Research Report MPI-I-2006-4-006, Saarbrücken, Germany, 2006, Max-Planck-Institut fuer Informatik.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. MPI Informatik, Saarbrücken, Germany

    Christian Theobalt, Edilson de Aguiar & Hans-Peter Seidel

  2. TU Braunschweig, Braunschweig, Germany

    Marcus A. Magnor

Authors
  1. Christian Theobalt
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Edilson de Aguiar
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Marcus A. Magnor
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hans-Peter Seidel
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Dept. of Electrical Engineering, Bilkent University, TR-06800 Bilkent, Ankara, Turkey

    Haldun M. Ozaktas (Prof.) & Levent Onural (Prof.) (Prof.) &  (Prof.)

Rights and permissions

Reprints and permissions

Copyright information

© 2008 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Theobalt, C., Aguiar, E.d., Magnor, M.A., Seidel, HP. (2008). Reconstructing Human Shape, Motion and Appearance from Multi-view Video. In: Ozaktas, H.M., Onural, L. (eds) Three-Dimensional Television. Signals and Communication Technology. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-72532-9_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-72532-9_3

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-72531-2

  • Online ISBN: 978-3-540-72532-9

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation