Abstract
Scene flow is defined as the motion field in 3D space, and can be computed from a single view when using an RGBD sensor. We propose a new scene flow approach that exploits the local and piecewise rigidity of real world scenes. By modeling the motion as a field of twists, our method encourages piecewise smooth solutions of rigid body motions. We give a general formulation to solve for local and global rigid motions by jointly using intensity and depth data. In order to deal efficiently with a moving camera, we model the motion as a rigid component plus a non-rigid residual and propose an alternating solver. The evaluation demonstrates that the proposed method achieves the best results in the most commonly used scene flow benchmark. Through additional experiments we indicate the general applicability of our approach in a variety of different scenarios.
This work was supported by a collaborative research program between Inria Grenoble and University of Freiburg. We gratefully acknowledge partial funding by CMIRA 2013 (Region Rhône-Alpes), GDR ISIS (CNRS), and COLCIENCIAS (Colombia).
Chapter PDF
Similar content being viewed by others
Keywords
References
Basha, T., Moses, Y., Kiryati, N.: Multi-view scene flow estimation: A view centered variational approach. In: Conference on Computer Vision and Pattern Recognition, pp. 1506–1513 (2010)
Brox, T., Malik, J.: Large displacement optical flow: Descriptor matching in variational motion estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence 33(3), 500–513 (2011)
Chambolle, A., Pock, T.: A first-order primal-dual algorithm for convex problems with applications to imaging. Journal of Mathematical Imaging and Vision 40(1), 120–145 (2011)
Goldluecke, B., Strekalovskiy, E., Cremers, D.: The natural vectorial total variation which arises from geometric measure theory. SIAM Journal on Imaging Sciences 5(2), 537–563 (2012)
Hadfield, S., Bowden, R.: Scene particles: Unregularized particle-based scene flow estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence 36(3), 564–576 (2014)
Herbst, E., Ren, X., Fox, D.: RGB-D flow: Dense 3-D motion estimation using color and depth. In: International Conference on Robotics and Automation (ICRA), pp. 2276–2282 (2013)
Horn, B.K., Schunck, B.G.: Determining optical flow. Artificial Intelligence 17, 185–203 (1981)
Huguet, F., Devernay, F.: A variational method for scene flow estimation from stereo sequences. In: International Conference on Computer Vision, pp. 1–7 (2007)
Kerl, C., Sturm, J., Cremers, D.: Robust odometry estimation for RGB-D cameras. In: International Conference on Robotics and Automation (ICRA), pp. 3748–3754 (2013)
Letouzey, A., Petit, B., Boyer, E.: Scene flow from depth and color images. In: British Machine Vision Conference, BMVC 2011 (2011)
Murray, R.M., Sastry, S.S., Zexiang, L.: A Mathematical Introduction to Robotic Manipulation, 1st edn. CRC Press, Inc., Boca Raton (1994)
Quiroga, J., Devernay, F., Crowley, J.: Scene flow by tracking in intensity and depth data. In: Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 50–57 (2012)
Quiroga, J., Devernay, F., Crowley, J.: Local/global scene flow estimation. In: International Conference on Image Processing (ICIP), pp. 3850–3854 (2013)
Rosman, G., Bronstein, A.M., Bronstein, M.M., Tai, X.-C., Kimmel, R.: Group-valued regularization for analysis of articulated motion. In: Fusiello, A., Murino, V., Cucchiara, R. (eds.) ECCV 2012 Ws/Demos, Part I. LNCS, vol. 7583, pp. 52–62. Springer, Heidelberg (2012)
Scharstein, D., Szeliski, R.: High-accuracy stereo depth maps using structured light. In: Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 195–202 (2003)
Spies, H., Jahne, B., Barron, J.: Dense range flow from depth and intensity data. In: International Conference on Pattern Recognition, vol. 1, pp. 131–134 (2000)
Sturm, J., Engelhard, N., Endres, F., Burgard, W., Cremers, D.: A benchmark for the evaluation of RGB-D slam systems. In: International Conference on Intelligent Robot Systems (IROS), pp. 573–580 (2012)
Vedula, S., Baker, S., Rander, P., Collins, R.: Three-dimensional scene flow. In: International Conference on Computer Vision, vol. 2, pp. 722–729 (1999)
Vogel, C., Schindler, K., Roth, S.: 3D scene flow estimation with a rigid motion prior. In: International Conference on Computer Vision, pp. 1291–1298 (2011)
Wang, Y., Yang, J., Yin, W., Zhang, Y.: A new alternating minimization algorithm for total variation image reconstruction. SIAM J. Img. Sci. 1(3), 248–272 (2008)
Wedel, A., Rabe, C., Vaudrey, T., Brox, T., Franke, U., Cremers, D.: Efficient dense scene flow from sparse or dense stereo data. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part I. LNCS, vol. 5302, pp. 739–751. Springer, Heidelberg (2008)
Wedel, A., Pock, T., Zach, C., Bischof, H., Cremers, D.: An improved algorithm for TV-l 1 optical flow. In: Cremers, D., Rosenhahn, B., Yuille, A.L., Schmidt, F.R. (eds.) Visual Motion Analysis. LNCS, vol. 5604, pp. 23–45. Springer, Heidelberg (2009)
Werlberger, M., Trobin, W., Pock, T., Wedel, A., Cremers, D., Bischof, H.: Anisotropic Huber-L1 Optical Flow. In: Proceedings of the British Machine Vision Conference (BMVC) (2009)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Quiroga, J., Brox, T., Devernay, F., Crowley, J. (2014). Dense Semi-rigid Scene Flow Estimation from RGBD Images. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds) Computer Vision – ECCV 2014. ECCV 2014. Lecture Notes in Computer Science, vol 8695. Springer, Cham. https://doi.org/10.1007/978-3-319-10584-0_37
Download citation
DOI: https://doi.org/10.1007/978-3-319-10584-0_37
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-10583-3
Online ISBN: 978-3-319-10584-0
eBook Packages: Computer ScienceComputer Science (R0)