A Prediction-Correction Approach for Real-Time Optical Flow Computation Using Stereo

  • Maxime DeromeEmail author
  • Aurelien Plyer
  • Martial Sanfourche
  • Guy Le Besnerais
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9796)


Estimating the optical flow robustly in real-time is still a challenging issue as revealed by current KITTI benchmarks. We propose an original two-step method for fast and performant optical flow estimation from stereo vision. The first step is the prediction of the flow due to the ego-motion, efficiently conducted by stereo-matching and visual odometry. The correction step estimates the motion of mobile objects. Algorithmic choices are justified by empirical studies on real datasets. Our method achieves framerate processing on images of realistic size, and provides results comparable or better than methods having computation times one or two orders of magnitude higher.


Optical Flow Camera Motion Visual Odometry Feature Point Extraction Scene Flow 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



This work was sponsored by the Direction Générale de l’Armement (DGA) of the French Ministry of Defense.


  1. 1.
    Brox, T., Bruhn, A., Papenberg, N., Weickert, J.: High accuracy optical flow estimation based on a theory for warping. In: Pajdla, T., Matas, J.G. (eds.) ECCV 2004. LNCS, vol. 3024, pp. 25–36. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  2. 2.
    Brox, T., Malik, J.: Large displacement optical flow: descriptor matching in variational motion estimation. PAMI 33(3), 500–513 (2001)CrossRefGoogle Scholar
  3. 3.
    Chakrabarti, A., Xiong, Y., Gortler, S.J., Zickler, T.: Low-level vision by consensus in a spatial hierarchy of regions. In: CVPR (2015)Google Scholar
  4. 4.
    Derome, M., Plyer, A., Sanfourche, M., Le Besnerais, G.: Real-time mobile object detection using stereo. In: ICARCV (2014)Google Scholar
  5. 5.
    Derome, M., Plyer, A., Sanfourche, M., Le Besnerais, G.: Moving object detection in real-time using stereo from a mobile platform. Unmanned Syst. 3(4), 253–266 (2015)CrossRefGoogle Scholar
  6. 6.
    Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. ACM 24(6), 381–395 (1981)MathSciNetCrossRefGoogle Scholar
  7. 7.
    Gehrig, S.K., Eberli, F., Meyer, T.: A real-time low-power stereo vision engine using semi-global matching. In: Fritz, M., Schiele, B., Piater, J.H. (eds.) ICVS 2009. LNCS, vol. 5815, pp. 134–143. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  8. 8.
    Geiger, A., Lenz, P., Stiller, C., Urtasun, R.: Vision meets robotics: the KITTI dataset. Int. J. Robot. Res. 32(11), 1231–1237 (2013)CrossRefGoogle Scholar
  9. 9.
    Geiger, A., Roser, M., Urtasun, R.: Efficient large-scale stereo matching. In: Kimmel, R., Klette, R., Sugimoto, A. (eds.) ACCV 2010, Part I. LNCS, vol. 6492, pp. 25–38. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  10. 10.
    Hirschmller, H.: Stereo processing by semiglobal matching and mutual information. Pattern Anal. Mach. Intell. 30(2), 328–341 (2008)CrossRefGoogle Scholar
  11. 11.
    Horn, B.K., Schunck, B.G.: Determining optical flow. Artif. Intell. 17, 185–203 (1981)CrossRefGoogle Scholar
  12. 12.
    Le Besnerais, G., Champagnat, F.: Dense optical flow by iterative local window registration. In: ICIP (2005)Google Scholar
  13. 13.
    Lucas, B.D., Kanade, T.: An iterative image registration technique with an application to stereo vision. In: IJCAI, pp. 674–679 (1981)Google Scholar
  14. 14.
    Menze, M., Geiger, A.: Object scene flow for autonomous vehicles. In: CVPR (2015)Google Scholar
  15. 15.
    Müller, T., Rannacher, J., Rabe, C., Franke, U.: Feature- and depth-supported modified total variation optical flow for 3D motion field estimation in real scenes. In: CVPR (2011)Google Scholar
  16. 16.
    Plyer, A., Le Besnerais, G., Champagnat, F.: Massively parallel Lucas Kanade optical flow for real-time video processing applications. J. Real-Time Image Proc. 11(4), 713–730 (2016)CrossRefGoogle Scholar
  17. 17.
    Revaud, J., Weinzaepfel, P., Harchaoui, Z., Schmid, C.: EpicFlow: edge-preserving interpolation of correspondences for optical flow. In: CVPR (2015)Google Scholar
  18. 18.
    Sanfourche, M., Vittori, V., Le Besnerais, G.: eVO: a realtime embedded stereo odometry for MAV applications. In: IROS (2013)Google Scholar
  19. 19.
    Sizintsev, M., Kuthirummal, S., Samarasekera, S., Kumar, R., Sawhney, H.S., Chaudhry, A.: GPU accelerated realtime stereo for augmented reality. In: 3DPVT (2010)Google Scholar
  20. 20.
    Vogel, C., Schindler, K., Roth, S.: 3D scene flow estimation with a piecewise rigid scene model. Int. J. Comput. Vis. 115, 1 (2015)MathSciNetCrossRefGoogle Scholar
  21. 21.
    Wedel, A., Brox, T., Vaudrey, T., Rabe, C., Franke, U., Cremers, D.: Stereoscopic scene flow computation for 3D motion understanding. Int. J. Comput. Vis. 95(1), 29–51 (2011)CrossRefzbMATHGoogle Scholar
  22. 22.
    Wedel, A., Pock, T., Zach, C., Bischof, H., Cremers, D.: An improved algorithm for TV-L \(^1\) optical flow. In: Cremers, D., Rosenhahn, B., Yuille, A.L., Schmidt, F.R. (eds.) Statistical and Geometrical Approaches to Visual Motion Analysis. LNCS, vol. 5604, pp. 23–45. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  23. 23.
    Weinzaepfel, P., Revaud, J., Harchaoui, Z., Schmid, C.: Deepflow: large displacement optical flow with deep matching. In: ICCV (2013)Google Scholar
  24. 24.
    Yamaguchi, K., McAllester, D., Urtasun, R.: Efficient joint segmentation, occlusion labeling, stereo and flow estimation. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014, Part V. LNCS, vol. 8693, pp. 756–771. Springer, Heidelberg (2014)Google Scholar
  25. 25.
    Zach, C., Pock, T., Bischof, H.: A duality based approach for realtime TV-L \(^{1}\) optical flow. In: Hamprecht, F.A., Schnörr, C., Jähne, B. (eds.) DAGM 2007. LNCS, vol. 4713, pp. 214–223. Springer, Heidelberg (2007)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  • Maxime Derome
    • 1
    Email author
  • Aurelien Plyer
    • 1
  • Martial Sanfourche
    • 1
  • Guy Le Besnerais
    • 1
  1. 1.ONERA - The French Aerospace LabPalaiseauFrance

Personalised recommendations