Abstract
We propose a method that uses a monocular sequence for joint direct estimation of dense scene flow and relative depth, a problem that has been generally tackled in the literature with binocular or stereo image sequences. The problem is posed as the optimization of a functional containing two terms: a data term, which relates three-dimensional (3D) velocity to depth in terms of the spatiotemporal visual pattern and an L 2 regularization term. Based on expressing the optical flow gradient constraint in terms of scene flow velocity and depth, our formulation is analogous to the classical Horn and Schunck optical flow estimation although it involves 3D motion and depth rather than 2D image motion. The discretized Euler-Lagrange equations yield a large scale sparse system of linear equations, which we order so that the corresponding matrix is symmetric positive definite. This implies that Gauss-Seidel iterations converge, point-wise or block-wise, and afford highly efficient means of solving the equations. Examples are given to verify the scheme and its implementation.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Vogel, C., Schindler, K., Roth, S.: Piecewise rigid scene flow. In: IEEE International Conference on Computer Vision, ICCV (2013)
Baker, S., Scharstein, D., Lewis, J.P., Roth, S., Black, M.J., Szeliski, R.: A database and evaluation methodology for optical flow. International Journal of Computer Vision 92(1), 1–31 (2011)
Sun, D., Roth, S., Black, M.J.: Secrets of optical flow estimation and their principles. In: CVPR, pp. 2432–2439 (2010)
Basha, T., Moses, Y., Kiryati, N.: Multi-view scene flow estimation: A view centered variational approach. International Journal of Computer Vision 101(1), 6–21 (2013)
Wedel, A., Brox, T., Vaudrey, T., Rabe, C., Franke, U., Cremers, D.: Stereoscopic scene flow computation for 3d motion understanding. International Journal of Computer Vision 95(1), 29–51 (2011)
Rabe, C., Müller, T., Wedel, A., Franke, U.: Dense, robust, and accurate motion field estimation from stereo image sequences in real-time. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part IV. LNCS, vol. 6314, pp. 582–595. Springer, Heidelberg (2010)
Huguet, F., Devernay, F.: A variational method for scene flow estimation from stereo sequences. In: IEEE International Conference on Computer Vision (ICCV), pp. 1–7 (2007)
Pons, J.P., Keriven, R., Faugeras, O.D.: Multi-view stereo reconstruction and scene flow estimation with a global image-based matching score. International Journal of Computer Vision 72(2), 179–193 (2007)
Vedula, S., Baker, S., Rander, P., Collins, R., Kanade, T.: Three-dimensional scene flow. IEEE Transactions on Pattern Analysis and Machine Intelligence 27, 475–480 (2005)
Mitiche, A., Ben Ayed, I.: Variational and level set methods in image segmentation. Springer (2010)
Mitiche, A., Sekkati, H.: Optical flow 3D segmentation and interpretation: A variational method with active curve evolution and level sets. IEEE Transactions on Pattern Analysis and Machine Intelligence 28(11), 1818–1829 (2006)
Sekkati, H., Mitiche, A.: Concurrent 3D motion segmentation and 3D interpretation of temporal sequences of monocular images. IEEE Transactions on Image Processing 15(3), 641–653 (2006)
Zhang, Y., Kambhamettu, C.: Integrated 3d scene flow and structure recovery from multiview image sequences. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), vol. 2, pp. 674–681 (2000)
Mitiche, A., Letang, J.M.: Stereokinematic analysis of visual data in active, convergent stereoscopy. Journal of Robotics and Autonomous Systems 705, 43–71 (1998)
Brox, T., Bruhn, A., Papenberg, N., Weickert, J.: High accuracy optical flow estimation based on a theory for warping. In: Pajdla, T., Matas, J(G.) (eds.) ECCV 2004. LNCS, vol. 3024, pp. 25–36. Springer, Heidelberg (2004)
Vogel, C., Schindler, K., Roth, S.: 3d scene flow estimation with a rigid motion prior. In: IEEE International Conference on Computer Vision (ICCV), pp. 1291–1298 (2011)
Longuet-Higgins, H., Prazdny, K.: The interpretation of a moving retinal image. Proceedings of the Royal Society of London, B 208, 385–397 (1981)
Pons, J.P., Keriven, R., Faugeras, O., Hermosillo, G.: Variational stereovision and 3d scene flow estimation with statistical similarity measures. In: IEEE International Conference on Computer Vision (ICCV), pp. 597–602 (2003)
Horn, B., Schunk, B.: Determining optical flow. Artificial Intelligence 17(17), 185–203 (1981)
Mitiche, A., Mansouri, A.: On convergence of the horn and schunck optical flow estimation method. IEEE Transactions on Image Processing 13(11), 1473–1490 (2004)
Ciarlet, P.: Introduction a l’analyse numerique matricielle et a l’optimisation, 5th edn., Masson (1994)
Stoer, J., Bulirsch, R.: Introduction to Numerical Analysis, 3rd edn. Texts in Applied Mathematics, vol. 12. Springer, New York (2002)
Forsythe, G., Malcolm, M., Moler, C.: Computer methods for mathematical computations. Prentice-Hall (1977)
Sekkati, H., Mitiche, A.: A variational method for the recovery of dense 3D structure from motion. Journal of Robotics and Autonomous Systems 55, 597–607 (2007)
Debrunner, C., Ahuja, N.: Segmentation and factorization-based motion and structure estimation for long image sequences. IEEE Transactions on Pattern Analysis and Machine Intelligence 20, 206–211 (1998)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Mathlouthi, Y., Mitiche, A., Ben Ayed, I. (2014). Direct Estimation of Dense Scene Flow and Depth from a Monocular Sequence. In: Bebis, G., et al. Advances in Visual Computing. ISVC 2014. Lecture Notes in Computer Science, vol 8887. Springer, Cham. https://doi.org/10.1007/978-3-319-14249-4_11
Download citation
DOI: https://doi.org/10.1007/978-3-319-14249-4_11
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-14248-7
Online ISBN: 978-3-319-14249-4
eBook Packages: Computer ScienceComputer Science (R0)