Abstract
In this paper, a dynamic surface is represented by a triangle mesh with dense vertices whose 3D positions change over time. These time-varying positions are reconstructed by finding their corresponding projections in the images captured by two calibrated and synchronized video cameras. To achieve accurate dense correspondences across views and frames, we first match sparse feature points and rely on them to provide good initialization and strong constraints in optimizing dense correspondence. Spatio-temporal consistency is utilized in matching both features and image points. Three synergistic constraints, image similarity, epipolar geometry and motion clue, are jointly used to optimize stereo and temporal correspondences simultaneously. Tracking failure due to self-occlusion or large appearance change are automatically handled. Experimental results show that complex shape and motion of dynamic surfaces like fabrics and skin can be successfully reconstructed with the proposed method.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
de Aguiar, E., Theobalt, C., Stoll, C., Seidel, H.P.: Marker-less deformable mesh tracking for human shape and motion capture. In: IEEE Conference on Computer Vision and Pattern Recognition (2007)
Ahmed, N., Theobalt, C., Rossl, C., Thrun, S., Seidel, H.P.: Dense correspondence finding for parametrization-free animation reconstruction from video. In: IEEE Conference on Computer Vision and Pattern Recognition (2008)
Brox, T., Bruhn, A., Papenberg, N., Weickert, J.: High accuracy optical flow estimation based on a theory for warping. In: Pajdla, T., Matas, J(G.) (eds.) ECCV 2004. LNCS, vol. 3024, pp. 25–36. Springer, Heidelberg (2004)
Carceroni, R.L., Kutulakos, K.N.: Multi-view scene capture by surfel sampling: from video streams to non-rigid 3D motion, shape and reflectance. The International Journal of Computer Vision 49, 175–214 (2002)
Chivers, K., Clocksin, W.: Inspection of surface strain in materials using optical flow. In: British Machine Vision Conference, pp. 392–401 (2000)
Du, H., Zou, D., Chen, Y.Q.: Relative epipolar motion of tracked features for correspondence in binocular stereo. In: IEEE International Conference on Computer Vision (2007)
Furukawa, Y., Ponce, J.: Accurate, dense, and robust multi-view stereopsis. In: IEEE Conference on Computer Vision and Pattern Recognition (2007)
Furukawa, Y., Ponce, J.: Dense 3D motion capture from synchronized video streams. In: IEEE Conference on Computer Vision and Pattern Recognition (2008)
Huguet, F., Devernay, F.: A variational method for scene flow estimation from stereo sequences. In: IEEE International Conference on Computer Vision (2007)
Lagarias, J.C., Reeds, J.A., Wright, M.H., Wright, P.E.: Convergence properties of the Nelder-Mead simplex method in low dimensions. SIAM Journal of Optimization 9, 112–147 (1998)
Li, R., Sclaroff, S.: Multi-scale 3D scene flow from binocular stereo sequences. Computer Vision and Image Understanding 110, 75–90 (2008)
Lowe, D.G.: Distinctive image features from scale-invariant keypoints. The International Journal of Computer Vision 60, 91–110 (2004)
Neumann, J., Aloimonos, Y.: Spatio-temporal stereo using multi-resolution subdivision surfaces. The International Journal of Computer Vision 47, 181–193 (2002)
Pons, J.P., Keriven, R., Faugeras, O.: Multi-view stereo reconstruction and scene flow estimation with a global image-based matching score. The International Journal of Computer Vision 72, 179–193 (2007)
Susskind, J.M., Lee, D.H., Cusi, A., Feiman, R., Grabski, W., Anderson, A.K.: Expressing fear enhances sensory acquisition. Nature Neuroscience 11, 843–850 (2008)
Vedula, S., Baker, S., Rander, P., Collins, R., Kanade, T.: Three-dimensional scene flow. IEEE Transactions on Pattern Analysis and Machine Intelligence 27, 475–480 (2005)
Wedel, A., Rabe, C., Vaudrey, T., Brox, T., Franke, U., Cremers, D.: Efficient dense scene flow from sparse or dense stereo data. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part I. LNCS, vol. 5302, pp. 739–751. Springer, Heidelberg (2008)
White, R., Crane, K., Forsyth, D.A.: Capturing and animating occluded cloth. In: SIGGRAPH (2007)
Zhang, Z.: A flexible new technique for camera calibration. IEEE Transactions on Pattern Analysis and Machine Intelligence 22, 1330–1334 (2000)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Zhou, Y., Chen, Y.Q. (2011). Feature-Assisted Dense Spatio-temporal Reconstruction from Binocular Sequences. In: Kimmel, R., Klette, R., Sugimoto, A. (eds) Computer Vision – ACCV 2010. ACCV 2010. Lecture Notes in Computer Science, vol 6495. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-19282-1_35
Download citation
DOI: https://doi.org/10.1007/978-3-642-19282-1_35
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-19281-4
Online ISBN: 978-3-642-19282-1
eBook Packages: Computer ScienceComputer Science (R0)