Abstract
Dense and markerless elastic 3D motion estimation based on stereo sequences is a challenge in computer vision. Solutions based on scene flow and 3D registration are mostly restricted to simple non-rigid motions, and suffer from the error accumulation. To address this problem, this paper proposes a globally optimal approach to non-rigid motion estimation which simultaneously recovers the 3D surface as well as its non-rigid motion over time. The instantaneous surface of the object is represented as a set of points which is reconstructed from the matched stereo images, meanwhile its deformation is captured by registering the points over time under spatio-temporal constraints. A global energy is defined on the constraints of stereo, spatial smoothness and temporal continuity, which is optimized via an iterative algorithm to approximate the minimum. Our extensive experiments on real video sequences including different facial expressions, cloth flapping, flag waves, etc. proved the robustness of our method and showed the method effectively handles complex nonrigid motions.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
White, R., Crane, K., Forsyth, D.: Capturing and animating occluded cloth. ACM Transactions on Graphics (2007)
Di, H., Tao, L., Xu, G.: A Mixture of Transformed Hidden Markov Models for Elastic Motion Estimation. IEEE Trans. PAMI 31(10), 1817–1830 (2009)
Park, S.I., Hodgins, J.K.: Capturing and animating skin deformation in human motion. ACM ToG 25(3) (2006)
Koterba, S.C., Baker, S., Matthews, I., Hu, C., Xiao, J., Cohn, J., Kanade, T.: Multi-view AAM fitting and camera calibration. In: Proc. ICCV (2005)
Jianke, Z., Steven, C.H., Zenglin, X., Lyu, M.R.: An Effective Approach to 3D Deformable Surface Tracking. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part III. LNCS, vol. 5304, pp. 766–779. Springer, Heidelberg (2008)
Carceroni, R.L., Kutulakos, K.N.: Multi-view scene capture by surfel sampling: From video streams to non-rigid 3D motion, shape and reflectance. IJCV (2002)
Huguet, F., Devernay, F.: A variational method for scene flow estimation from stereo sequences. In: Proc. ICCV (2007)
Wedel, A., Rabe, C., Vaudrey, T., Brox, T., Franke, U., Cremers, D.: Efficient Dense Scene Flow from Sparse or Dense Stereo Data. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part I. LNCS, vol. 5302, pp. 739–751. Springer, Heidelberg (2008)
Vedula, S., Baker, S., Kanade, T.: Image-based spatiotemporal modeling and view interpolation of dynamic events. ACM ToG (2005)
Vedula, S., Baker, S.: Three-dimensional scene flow. IEEE Trans. PAMI (2005)
Li, R., Sclaroff, S.: Multi-scale 3D scene flow from binocular stereo sequences. In: WACV/MOTION (2005)
Pons, J.-P., Keriven, R., Faugeras, O.: Modelling dynamic scenes by registering multi-view image sequences. In: Proc. CVPR (2005)
Neumann, J., Aloimonos, Y.: Spatio-temporal stereo using multi-resolution subdivision surfaces. In: IJCV (2002)
Bradley, D., Popa, T., Sheffer, A., Heidrich, W., Boubekeur, T.: Markerless Garment Capture. ACM Trans. on SIGGRAPH (2008)
Furukawa, Y., Ponce, J.: Dense 3D Motion Capture from Synchronized Video Streams. In: Proc. CVPR (2008)
Canny, J.: A computational approach to edge detection. PAMI (1986)
Di, H., Iqbal, R.N., Xu, G., Tao, L.: Groupwise shape registration on raw edge sequence via a spatio-temporal generative model. In: CVPR (2007)
Felzenszwalb, P.F., Huttenlocher, D.P.: Efficient belief propagation for early vision. IJCV (2006)
Chui, H., Rangarajan, A., Zhang, J., Leonard, C.: Unsupervised learning of an atlas from unlabeled point-sets. IEEE Trans. PAMI (2004)
Bookstein, F.L.: Principal warps: Thin-plate splines and the decomposition of deformations. IEEE Trans. PAMI (1989)
Forsyth, D., Ponce, J.: Chapter tracking with linear dynamic models, computer vision: A modern approach. Prentice Hall, Inc., Englewood Cliffs (2003)
Chui, H., Rangarajan, A.: A new point matching algorithm for non-rigid registration. CVIU (2003)
Scharstein, D., Szeliski, R.: A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. IJCV (2002)
Jian, S., Heung-Yeung, S., Nanning, Z.: Stereo Matching Using Belief Propagation. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2351, pp. 510–524. Springer, Heidelberg (2002)
Wang, J., Xu, K., Zhou, K., Lin, S., Hu, S., Guo, B.: Spherical Harmonics Scaling. In: Pacific Conference on Computer Graphics and Applications (2006)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
1 Electronic Supplementary Material
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Wang, Q., Tao, L., Di, H. (2010). A Globally Optimal Approach for 3D Elastic Motion Estimation from Stereo Sequences. In: Daniilidis, K., Maragos, P., Paragios, N. (eds) Computer Vision – ECCV 2010. ECCV 2010. Lecture Notes in Computer Science, vol 6314. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-15561-1_38
Download citation
DOI: https://doi.org/10.1007/978-3-642-15561-1_38
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-15560-4
Online ISBN: 978-3-642-15561-1
eBook Packages: Computer ScienceComputer Science (R0)