“What Is Optical Flow For?”: Workshop Results and Summary

  • Fatma Güney
  • Laura Sevilla-Lara
  • Deqing Sun
  • Jonas WulffEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11134)


Traditionally, computer vision problems have been classified into three levels: low (image to image), middle (image to features), and high (features to analysis) [11]. Some typical low-level vision problems include optical flow [7], stereo [10] and intrinsic image decomposition [1]. The solution to these problems would then be combined to solve higher level problems, such as action recognition and visual question answering.


  1. 1.
    Barrow, H., Tenenbaum, J., Hanson, A., Riseman, E.: Recovering intrinsic scene characteristics. Comput. Vis. Syst. 2, 3–26 (1978)Google Scholar
  2. 2.
    Black, M.J.: Robust incremental optical flow. Ph.D. thesis. Yale university (1992)Google Scholar
  3. 3.
    Dosovitskiy, A., et al.: FlowNet: learning optical flow with convolutional networks (2015)Google Scholar
  4. 4.
    Gao, R., Xiong, B., Grauman, K.: Im2Flow: motion hallucination from static images for action recognition. In: CVPR (2018)Google Scholar
  5. 5.
    Gibson, J.J.: The Perception of the Visual World. Houghton Mifflin, Boston (1950)Google Scholar
  6. 6.
    Gu, C., et al.: AVA: a video dataset of spatio-temporally localized atomic visual actions. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR (2018)Google Scholar
  7. 7.
    Horn, B., Schunck, B.: Determining optical flow. Artif. Intell. 17, 185–203 (1981)CrossRefGoogle Scholar
  8. 8.
    Ilg, E., Mayer, N., Saikia, T., Keuper, M., Dosovitskiy, A., Brox, T.: FlowNet 2.0: evolution of optical flow estimation with deep networks (2017)Google Scholar
  9. 9.
    Jain, S., Xiong, B., Grauman, K.: Pixel objectness. arXiv preprint arXiv:1701.05349 (2017)
  10. 10.
    Lucas, B., Kanade, T.: An iterative image registration technique with an application to stereo vision, pp. 674–679 (1981)Google Scholar
  11. 11.
    Marr, D.: Vision: A Computational Investigation into the Human Representation and Processing of Visual Information. W.H. Freeman, New York (1982)Google Scholar
  12. 12.
    Nakayama, K.: Biological image motion processing: a review. Vis. Res. 25(5), 625–660 (1985)CrossRefGoogle Scholar
  13. 13.
    Oh, T.-H., et al.: Learning-based video motion magnification. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11208, pp. 663–679. Springer, Cham (2018). Scholar
  14. 14.
    Ranjan, A., Black, M.J.: Optical flow estimation using a spatial pyramid network (2017)Google Scholar
  15. 15.
    Sevilla-Lara, L., Liao, Y., Guney, F., Jampani, V., Geiger, A., Black, M.J.: On the integration of optical flow and action recognition. arXiv preprint arXiv:1712.08416 (2017)
  16. 16.
    Su, Y.C., Grauman, K.: Learning compressible 360\(^{\circ }\) video isomers. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, June 2018Google Scholar
  17. 17.
    Sun, C., Shrivastava, A., Vondrick, C., Murphy, K., Sukthankar, R., Schmid, C.: Actor-centric relation network. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11215, pp. 335–351. Springer, Cham (2018). Scholar
  18. 18.
    Sun, D., Yang, X., Liu, M.Y., Kautz, J.: Models matter, so does training: an empirical study of CNNs for optical flow estimation. arXiv preprint arXiv:1809.05571 (2018)
  19. 19.
    Sun, D., Yang, X., Liu, M.Y., Kautz, J.: PWC-Net: CNNs for optical flow using pyramid, warping, and cost volume (2018)Google Scholar
  20. 20.
    Ummenhofer, B., et al.: DeMoN: depth and motion network for learning monocular stereo. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR (2017).
  21. 21.
    Wachowski, L., Wachowski, L.: The Matrix Reloaded (2003)Google Scholar
  22. 22.
    Wadhwa, N., Rubinstein, M., Durand, F., Freeman, W.T.: Phase-based video motion processing. ACM Trans. Graph. 32(4), 80 (2013). (Proceedings SIGGRAPH 2013)CrossRefGoogle Scholar
  23. 23.
    Ward, V.: What Dreams May Come (1998)Google Scholar
  24. 24.
    Xue, T., Rubinstein, M., Liu, C., Freeman, W.T.: A computational approach for obstruction-free photography. ACM Trans. Graph. 34(4), 79 (2015). (Proc. SIGGRAPH)CrossRefGoogle Scholar
  25. 25.
    Zhou, H., Ummenhofer, B., Brox, T.: DeepTAM: deep tracking and mapping. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11220, pp. 851–868. Springer, Cham (2018). Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Fatma Güney
    • 1
  • Laura Sevilla-Lara
    • 2
  • Deqing Sun
    • 3
  • Jonas Wulff
    • 4
    Email author
  1. 1.Oxford UniversityOxfordUK
  2. 2.Facebook ResearchMenlo ParkUSA
  3. 3.NVIDIASanta ClaraUSA
  4. 4.Massachusetts Institute of TechnologyCambridgeUSA

Personalised recommendations