Abstract
This paper deals with the problem of computing optical flow between each of the images in a sequence and a reference frame when the camera is viewing a non-rigid object. We exploit the high correlation between 2D trajectories of different points on the same non-rigid surface by assuming that the displacement sequence of any point can be expressed in a compact way as a linear combination of a low-rank motion basis. This subspace constraint effectively acts as a long term regularization leading to temporally consistent optical flow. We formulate it as a robust soft constraint within a variational framework by penalizing flow fields that lie outside the low-rank manifold. The resulting energy functional includes a quadratic relaxation term that allows to decouple the optimization of the brightness constancy and spatial regularization terms, leading to an efficient optimization scheme. We provide a new benchmark dataset, based on motion capture data of a flag waving in the wind, with dense ground truth optical flow for evaluation of multi-view optical flow of non-rigid surfaces. Our experiments, show that our proposed approach provides comparable or superior results to state of the art optical flow and dense non-rigid registration algorithms.
This work is supported by the European Research Council under ERC Starting Grant agreement 204871-HUMANIS. We are grateful to D. Pizarro for providing results of their method [1] and tracks for the synthetic sequence.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Pizarro, D., Bartoli, A.: Feature-based deformable surface detection with self-occlusion reasoning. In: International Symposium on 3D Data Processing, Visualization and Transmission, 3DPVT 2010 (2010)
Weickert, J., Schnörr, C.: Variational optic flow computation with a spatio-temporal smoothness constraint. JMIV 14, 245–255 (2001)
Irani, M.: Multi-frame correspondence estimation using subspace constraints. IJCV (2002)
Torresani, L., Yang, D., Alexander, E., Bregler, C.: Tracking and modeling non-rigid objects with rank constraints. In: CVPR (2001)
Torresani, L., Bregler, C.: Space-time tracking. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2350, pp. 801–812. Springer, Heidelberg (2002)
Garg, R., Pizarro, L., Rueckert, D., Agapito, L.: Dense multi-frame optic flow for non-rigid objects using subspace constraints. In: Kimmel, R., Klette, R., Sugimoto, A. (eds.) ACCV 2010, Part IV. LNCS, vol. 6495, pp. 460–473. Springer, Heidelberg (2011)
Nir, T., Bruckstein, A.M., Kimmel, R.: Over-parameterized variational optical flow. IJCV 76, 205–216 (2008)
Chambolle, A.: An algorithm for total variation minimization and applications. JMIV 20, 89–97 (2004)
Chambolle, A., Pock, T.: A first-order primal-dual algorithm for convex problems with applications to imaging. JMIV (2011)
Horn, B., Schunck, B.: Determining optical flow. Artificial Intelligence 17, 185–203 (1981)
Brox, T., Malik, J.: Large displacement optical flow: Descriptor matching in variational motion estimation. TPAMI (2010)
Brox, T., Bruhn, A., Papenberg, N., Weickert, J.: High accuracy optical flow estimation based on a theory for warping (chapter). In: Pajdla, T., Matas, J(G.) (eds.) ECCV 2004. LNCS, vol. 3024, pp. 25–36. Springer, Heidelberg (2004)
Zach, C., Pock, T., Bischof, H.: A duality based approach for realtime TV- L1 optical flow. In: Hamprecht, F.A., Schnörr, C., Jähne, B. (eds.) DAGM 2007. LNCS, vol. 4713, pp. 214–223. Springer, Heidelberg (2007)
Wedel, A., Pock, T., Braun, J., Franke, U., Cremers, D.: Duality TV-L1 flow with fundamental matrix prior. In: Image and Vision Computing, New Zealand (2008)
Wedel, A., Cremers, D., Pock, T., Bischof, H.: Structure- and motion-adaptive regularization for high accuracy optic flow. In: ICCV (2009)
Alvarez, L., Weickert, J., Sánchez, J.: Reliable estimation of dense optical flow fields with large displacements. IJCV 39, 41–56 (2000)
Steinbruecker, F., Pock, T., Cremers, D.: Large displacement optical flow computation without warping. In: ICCV (2009)
Pock, T., Cremers, D., Bischof, H., Chambolle, A.: Global solutions of variational models with convex regularization. SIAM Journal on Imaging Sciences (2010)
Stühmer, J., Gumhold, S., Cremers, D.: Real-time dense geometry from a handheld camera. In: Goesele, M., Roth, S., Kuijper, A., Schiele, B., Schindler, K. (eds.) Pattern Recognition. LNCS, vol. 6376, pp. 11–20. Springer, Heidelberg (2010)
Tian, Y., Narasimhan, S.: A globally optimal data-driven approach for image distortion estimation. In: CVPR (2010)
Baker, S., Scharstein, D., Lewis, J., Roth, S., Black, M., Szeliski, R.: A database and evaluation methodology for optical flow. IJCV 92, 1–31 (2011)
Bregler, C., Hertzmann, A., Biermann, H.: Recovering non-rigid 3D shape from image streams. In: CVPR (2000)
Torresani, L., Hertzmann, A., Bregler., C.: Non-rigid structure-from-motion: Estimating shape and motion with hierarchical priors. PAMI 30 (2008)
Shi, J., Tomasi, C.: Good features to track. CVPR (1994)
Rudin, L., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Physica D 60, 259–268 (1992)
White, R., Crane, K., Forsyth, D.: Capturing and animating occluded cloth. ACM Trans. on Graphics (2007)
Wedel, A., Pock, T., Zach, C., Bischof, H., Cremers, D.: An improved algorithm for TV-L1 optical flow. In: Cremers, D., Rosenhahn, B., Yuille, A.L., Schmidt, F.R. (eds.) Statistical and Geometrical Approaches to Visual Motion Analysis. LNCS, vol. 5604, pp. 23–45. Springer, Heidelberg (2009)
Lucas, B.D., Kanade, T.: An iterative image registration technique with an application to stereo vision. In: IJCAI 1981 (1981)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Garg, R., Roussos, A., Agapito, L. (2011). Robust Trajectory-Space TV-L1 Optical Flow for Non-rigid Sequences. In: Boykov, Y., Kahl, F., Lempitsky, V., Schmidt, F.R. (eds) Energy Minimization Methods in Computer Vision and Pattern Recognition. EMMCVPR 2011. Lecture Notes in Computer Science, vol 6819. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-23094-3_22
Download citation
DOI: https://doi.org/10.1007/978-3-642-23094-3_22
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-23093-6
Online ISBN: 978-3-642-23094-3
eBook Packages: Computer ScienceComputer Science (R0)