Lessons and Insights from Creating a Synthetic Optical Flow Benchmark

  • Jonas Wulff
  • Daniel J. Butler
  • Garrett B. Stanley
  • Michael J. Black
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7584)


With the MPI-Sintel Flow dataset, we introduce a naturalistic dataset for optical flow evaluation derived from the open source CGI movie Sintel. In contrast to the well-known Middlebury dataset, the MPI-Sintel Flow dataset contains longer and more varied sequences with image degradations such as motion blur, defocus blur, and atmospheric effects. Animators use a variety of techniques that produce pleasing images but make the raw animation data inappropriate for computer vision applications if used “out of the box”. Several changes to the rendering software and animation files were necessary in order to produce data for flow evaluation and similar changes are likely for future efforts to construct a scientific dataset from an animated film. Here we distill our experience with Sintel into a set of best practices for using computer animation to generate scientific data for vision research.


  1. 1.
    Baker, S., Scharstein, D., Lewis, J.P., Roth, S., Black, M.J., Szeliski, R.: A Database and Evaluation Methodology for Optical Flow. Int. J. Comput. Vision 92, 1–31 (2011)CrossRefGoogle Scholar
  2. 2.
    Butler, D.J., Wulff, J., Stanley, G.B., Black, M.J.: A Naturalistic Open Source Movie for Optical Flow Evaluation. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part VI. LNCS, vol. 7577, pp. 611–625. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  3. 3.
    Roth, S., Black, M.: On the spatial statistics of optical flow. In: ICCV 2005, vol. 1, pp. 42–49 (2005)Google Scholar
  4. 4.
    Geiger, A., Lenz, P., Urtasun, R.: Are we ready for Autonomous Driving? The KITTI Vision Benchmark Suite. In: CVPR 2012 (2012)Google Scholar
  5. 5.
    Barron, J.L., Fleet, D.J., Beauchemin, S.S.: Performance of optical flow techniques. Int. J. Comput. Vision 12, 43–77 (1994)CrossRefGoogle Scholar
  6. 6.
    McCane, B., Novins, K., Crannitch, D., Galvin, B.: On benchmarking optical flow. Comput. Vis. Image Underst. 84, 126–143 (2001)zbMATHCrossRefGoogle Scholar
  7. 7.
    Mac Aodha, O., Humayun, A., Pollefeys, M., Brostow, G.J.: Learning a Confidence Measure for Optical Flow. IEEE Trans. Pattern Anal. Mach. Intell. 34 (to appear, 2012)Google Scholar
  8. 8.
    Vaudrey, T., Rabe, C., Klette, R., Milburn, J.: Differences between stereo and motion behavior on synthetic and real-world stereo sequences. In: 23rd International Conference of Image and Vision Computing New Zealand (IVCNZ 2008), pp. 1–6 (2008)Google Scholar
  9. 9.
    Meister, S., Kondermann, D.: Real versus realistically rendered scenes for optical flow evaluation. In: CEMT 2011, pp. 1–6. IEEE (2011)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Jonas Wulff
    • 1
  • Daniel J. Butler
    • 2
  • Garrett B. Stanley
    • 3
  • Michael J. Black
    • 1
  1. 1.Max-Planck Institute for Intelligent SystemsTübingenGermany
  2. 2.University of WashingtonSeattleUSA
  3. 3.Georgia Institute of TechnologyAtlantaUSA

Personalised recommendations