Skip to main content
SpringerLink
Log in

Motion Compensated Frame Interpolation with a Symmetric Optical Flow Constraint

  • Conference paper
Advances in Visual Computing (ISVC 2012)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 7431))

Included in the following conference series:

Abstract

We consider the problem of interpolating frames in an image sequence. For this purpose accurate motion estimation can be very helpful. We propose to move the motion estimation from the surrounding frames directly to the unknown frame by parametrizing the optical flow objective function such that the interpolation assumption is directly modeled. This reparametrization is a powerful trick that results in a number of appealing properties, in particular the motion estimation becomes more robust to noise and large displacements, and the computational workload is more than halved compared to usual bidirectional methods. The proposed reparametrization is generic and can be applied to almost every existing algorithm. In this paper we illustrate its advantages by considering the classic TV-L 1 optical flow algorithm as a prototype. We demonstrate that this widely used method can produce results that are competitive with current state-of-the-art methods. Finally we show that the scheme can be implemented on graphics hardware such that it becomes possible to double the frame rate of 640×480 video footage at 30 fps, i.e. to perform frame doubling in realtime.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Christensen, G.E., Johnson, H.J.: Consistent image registration. IEEE Transactions on Medical Imaging 20, 568–582 (2001)

    Article  Google Scholar 

  2. Alvarez, L., Deriche, R., Papadopoulo, T., Sánchez, J.: Symmetrical dense optical flow estimation with occlusions detection. International Journal of Computer Vision 75, 371–385 (2007)

    Article  Google Scholar 

  3. Alvarez, L., Castaño, C.A., García, M., Krissian, K., Mazorra, L., Salgado, A., Sánchez, J.: Symmetric Optical Flow. In: Moreno Díaz, R., Pichler, F., Quesada Arencibia, A. (eds.) EUROCAST 2007. LNCS, vol. 4739, pp. 676–683. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

  4. Chen, W.: Surface velocity estimation from satellite imagery using displaced frame central difference equation. To appear in IEEE Transactions on Geoscience and Remote Sensing (2012)

    Google Scholar 

  5. Bresson, X., Chan, T.: Fast dual minimization of the vectorial total variation norm and application to color image processing. Inverse Problems and Imaging 2, 455–484 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  6. Goldluecke, B., Strekalovskiy, E., Cremers, D.: The natural total variation which arises from geometric measure theory. SIAM Journal on Imaging Sciences 5, 537–563 (2012)

    Article  Google Scholar 

  7. Zach, C., Pock, T., Bischof, H.: A Duality Based Approach for Realtime TV-L 1 Optical Flow. In: Hamprecht, F.A., Schnörr, C., Jähne, B. (eds.) DAGM 2007. LNCS, vol. 4713, pp. 214–223. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

  8. Rakêt, L.L., Roholm, L., Nielsen, M., Lauze, F.: TV-L 1 Optical Flow for Vector Valued Images. In: Boykov, Y., Kahl, F., Lempitsky, V., Schmidt, F.R. (eds.) EMMCVPR 2011. LNCS, vol. 6819, pp. 329–343. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  9. Baker, S., Scharstein, D., Lewis, J.P., Roth, S., Black, M.J., Szeliski, R.: A database and evaluation methodology for optical flow. International Journal of Computer Vision 31, 1–31 (2011)

    Article  Google Scholar 

  10. Herbst, E., Seitz, S., Baker, S.: Occlusion reasoning for temporal interpolation using optical flow. Technical Report UW-CSE-09-08-01, Department of Computer Science and Engineering, University of Washington (2009)

    Google Scholar 

  11. Werlberger, M., Pock, T., Unger, M., Bischof, H.: Optical Flow Guided TV-L1 Video Interpolation and Restoration. In: Boykov, Y., Kahl, F., Lempitsky, V., Schmidt, F.R. (eds.) EMMCVPR 2011. LNCS, vol. 6819, pp. 273–286. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  12. Stich, T., Linz, C., Albuquerque, G., Magnor, M.: View and time interpolation in image space. Computer Graphics Forum 27, 1781–1787 (2008)

    Article  Google Scholar 

  13. Huang, X., Rakêt, L.L., Luong, H.V., Nielsen, M., Lauze, F., Forchhammer, S.: Multi-hypothesis transform domain Wyner-Ziv video coding including optical flow. In: Multimedia Signal Processing (2011)

    Google Scholar 

  14. Keller, S., Lauze, F., Nielsen, M.: Temporal super resolution using variational methods. In: Mrak, M., Grgic, M., Kunt, M. (eds.) High-Quality Visual Experience: Creation, Processing and Interactivity of High-Resolution and High-Dimensional Video Signals. Springer (2010)

    Google Scholar 

  15. Wedel, A., Pock, T., Zach, C., Bischof, H., Cremers, D.: An Improved Algorithm for TV-1 Optical Flow. In: Cremers, D., Rosenhahn, B., Yuille, A.L., Schmidt, F.R. (eds.) Statistical and Geometrical Approaches to Visual Motion Analysis. LNCS, vol. 5604, pp. 23–45. Springer, Heidelberg (2009)

    Chapter  Google Scholar 

  16. Chen, K., Lorenz, D.: Image sequence interpolation using optimal control. Journal of Mathematical Imaging and Vision 41, 222–238 (2011)

    Article  MathSciNet  Google Scholar 

  17. Bruhn, A., Weickert, J., Schnörr, C.: Lucas/Kanade meets Horn/Schunck: Combining local and global optic flow methods. International Journal of Computer Vision 61, 211–231 (2005)

    Article  Google Scholar 

  18. Weickert, J.: Theoretical foundations of anisotropic diffusion in image processing. In: Kropatsch, W.G., Klette, R., Solina, F. (eds.) Theoretical Foundations of Computer Vision, Computing Supplement, vol. 11, pp. 221–236. Springer (1994)

    Google Scholar 

  19. Galić, I., Weickert, J., Welk, M., Bruhn, A., Belyaev, A., Seidel, H.P.: Image compression with anisotropic diffusion. Journal of Mathematical Imaging and Vision 31, 255–269 (2008)

    Article  MathSciNet  Google Scholar 

  20. Ghodstinat, M., Bruhn, A., Weickert, J.: Deinterlacing with Motion-Compensated Anisotropic Diffusion. In: Cremers, D., Rosenhahn, B., Yuille, A.L., Schmidt, F.R. (eds.) Statistical and Geometrical Approaches to Visual Motion Analysis. LNCS, vol. 5604, pp. 91–106. Springer, Heidelberg (2009)

    Chapter  Google Scholar 

  21. Volz, S., Bruhn, A., Valgaerts, L., Zimmer, H.: Modeling temporal coherence for optical flow. In: Metaxas, D.N., Quan, L., Sanfeliu, A., Gool, L.J.V. (eds.) IEEE International Conference on Computer Vision (ICCV), pp. 1116–1123 (2011)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, University of Copenhagen, Universitetsparken 5, 2100, Copenhagen, Denmark

    Lars Lau Rakêt & Lars Roholm

  2. Institute for Visualization and Interactive Systems, University of Stuttgart, Universitätsstraße 38, 70569, Stuttgart, Germany

    Andrés Bruhn

  3. Mathematical Image Analysis Group, Saarland University, Campus E 1.7, 66123, Saarbrücken, Germany

    Joachim Weickert

Authors
  1. Lars Lau Rakêt
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lars Roholm
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Andrés Bruhn
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Joachim Weickert
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Computer Science, University of Nevada, 89557, Reno, NV, USA

    George Bebis

  2. NASA Ames Research Center, 94035, Moffett Field, CA, USA

    Richard Boyle

  3. Lawrence Berkeley National Laboratory, 94720, Berkeley, CA, USA

    Bahram Parvin

  4. Desert Research Institute, 89512, Reno, NV, USA

    Darko Koracin

  5. Department of Computer Science, University of California at Irvine, 92697-3435, Irvine, CA

    Charless Fowlkes

  6. Eastman Kodak Company, 14650-2102, Rochester, NY, USA

    Sen Wang

  7. Department of Computer Science and Engineering, University of Colorado at Denver, 80217, Denver, CO, USA

    Min-Hyung Choi

  8. VRVis Zentrum für Virtual Reality and Visualisierung, 1220, Vienna, Austria

    Stephan Mantler

  9. California Institute for Telecommunications and Information Technology, University of California, 92093, San Diego, La Jolla, CA, USA

    Jürgen Schulze

  10. KAUST Visualizatioin Core Lab., 23955-6900, Thurwal, Saudi Arabia

    Daniel Acevedo

  11. Stony Brook University, 11794-4400, NY, USA

    Klaus Mueller

  12. Argonne National Laboratory, 60439, IL, USA

    Michael Papka

Rights and permissions

Reprints and permissions

Copyright information

© 2012 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Rakêt, L.L., Roholm, L., Bruhn, A., Weickert, J. (2012). Motion Compensated Frame Interpolation with a Symmetric Optical Flow Constraint. In: Bebis, G., et al. Advances in Visual Computing. ISVC 2012. Lecture Notes in Computer Science, vol 7431. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-33179-4_43

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-33179-4_43

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-33178-7

  • Online ISBN: 978-3-642-33179-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation