Skip to main content
SpringerLink
Log in

Temporal Frame Interpolation (TFI)

  • Chapter
  • First Online:
Novel Motion Anchoring Strategies for Wavelet-based Highly Scalable Video Compression

Part of the book series: Springer Theses ((Springer Theses))

  • 318 Accesses

Abstract

As discussed in the last chapter, the temporal scalability of standardized scalable video codecs, e.g., H.264/SVC [15] and SHVC [16], is limited to reducing the framerate. One of the main reasons that the framerate can not be increased is that the target-frame anchored motion is estimated in an opportunistic way, which means that it does not in general describe the “true” motion trajectory of objects in the scene. In contrast, in the motion anchoring strategies explored in this thesis, motion information is anchored at reference frames, and temporal frame interpolation (TFI) is the essential building block that allows us to form predictions of the target frames.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. H. Schwarz, D. Marpe, T. Wiegand, Overview of the scalable video coding extension of the H.264/AVC standard. IEEE Trans. Circuit Syst. Video Technol. 17(9), 1103–1120 (2007)

    Article  Google Scholar 

  2. P. Helle, H. Lakshman, M. Siekmann, J. Stegemann, T. Hinz, H. Schwarz, D. Marpe, T. Wiegand, A scalable video coding extension of HEVC, in Proceedings of the IEEE Data Compression Conference (2013)

    Google Scholar 

  3. S.H. Chan, T.Q. Nguyen, LCD motion blur: modeling, analysis, and algorithm. IEEE Trans. Image Process. 20(8), 2352–2365 (2011)

    Article  MATH  MathSciNet  Google Scholar 

  4. B. Girod, A.M. Aaron, S. Rane, D. Rebollo-Monedero, Distributed video coding. Proc. IEEE 93(1), 71–83 (2005)

    Google Scholar 

  5. G. De Haan, P.W.A.C. Biezen, H. Huijgen, O.A. Ojo, True- motion estimation with 3-D recursive search block matching. IEEE Trans. Circuit Syst. Video Technol. 3(5), 368–379 (1993)

    Article  Google Scholar 

  6. A. Beric, G. De Haan, J. Van Meerbergen, R. Sethuraman, Towards an Efficient High Quality Picture-rate Up-converter, 2003

    Google Scholar 

  7. T. Ha, S. Lee, J. Kim, Motion compensated frame interpolation by new block-based motion estimation algorithm. IEEE Trans. Consum. Electron. 50(2), 752–759 (2004)

    Article  Google Scholar 

  8. Q. Lu, N. Xu, X. Fang, Motion-compensated frame interpolation with multiframe based occlusion handling. IEEE J. Disp. Technol. 11(4), (2015)

    Google Scholar 

  9. B.K. Horn, B.G. Schunck, Determining optical flow. Artif. Intell. 17, 185–203 (1981)

    Article  Google Scholar 

  10. T. Brox, A. Bruhn, N. Papenberg, J. Weickert, High accuracy optical flow estimation based on a theory for warping, in European Conference on Computer Vision (2004), pp. 25–36

    Google Scholar 

  11. D. Sun, S. Roth, J. Lewis, M.J. Black, Learning optical flow, in European Conference on Computer Vision (2008), pp. 83–97

    Google Scholar 

  12. A. Wedel, D. Cremers, T. Pock, H. Bischof, Structure-and motion- adaptive regularization for high accuracy optic flow, in Proceedings of the IEEE International Conference on Computer Vision (2009), pp. 1663–1668

    Google Scholar 

  13. L. Xu, J. Jia, Y. Matsushita, Motion detail preserving optical flow estimation. IEEE Trans. Pattern Anal. Mach. Intell. 34, 1744–1757 (2012)

    Article  Google Scholar 

  14. J. Wulff, M.J. Black, Modeling blurred video with layers, in European Conference on Computer Vision (2014), vol. 8694, pp. 236–252

    Google Scholar 

  15. J. Revaud, P. Weinzaepfel, Z. Harchaoui, C. Schmid, EpicFlow: edge-preserving interpolation of correspondences for optical flow, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2015)

    Google Scholar 

  16. M. Menze, C. Heipke, A. Geiger, Discrete optimization for optical flow, in German Conference on Pattern Recognition (2015)

    Google Scholar 

  17. Q. Chen, V. Koltun, Full flow: optical flow estimation by global optimization over regular grid, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2016), vol. 2016, pp. 4706–4714

    Google Scholar 

  18. S. Dikbas, Y. Altunbasak, Novel true-motion estimation algorithm and its application to motion-compensated temporal frame interpolation. IEEE Trans. Image Process. 22(8), 2931–2945 (2013)

    Article  MATH  MathSciNet  Google Scholar 

  19. B.-D. Choi, J.-W. Han, C.-S. Kim, S.-J. Ko, Motion-compensated frame interpolation using bilateral motion estimation and adaptive overlapped block motion compensation. IEEE Trans. Circuit Syst. Video Technol. 17(4), 407–416 (2007)

    Article  Google Scholar 

  20. C. Wang, L. Zhang, Y. He, Y.-P. Tan, Frame rate up-conversion using trilateral filtering. IEEE Trans. Circuit Syst. Video Technol. 20(6), 886–893 (2010)

    Article  Google Scholar 

  21. A. Veselov, M. Gilmutdinov, Iterative hierarchical true motion estimation for temporal frame interpolation, in IEEE International Workshop on Multimedia Signal Processing (2014)

    Google Scholar 

  22. L.L. Rakêt, L. Roholm, A. Bruhn, J. Weickert, Motion compensated frame interpolation with a symmetric optical flow constraint. Adv. Vis. Comput. 447–457 (2012)

    Google Scholar 

  23. S.-G. Jeong, C. Lee, C.-S. Kim, Motion-compensated frame interpolation based on multihypothesis motion estimation and texture optimization. IEEE Trans. Image Process. 22, 4497–4509 (2013)

    Article  MATH  MathSciNet  Google Scholar 

  24. Y. Chin, C.-J. Tsai, Dense true motion field compensation for video coding, in Proceedings of the IEEE International Conference on Image Processing (2013), pp. 1958–1961

    Google Scholar 

  25. D. Sun, S. Roth, M.J. Black, Secrets of optical flow estimation and their principles, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2010), pp. 2432–2439

    Google Scholar 

  26. D. Kim, H. Lim, H. Park, Iterative true motion estimation for motion-compensated frame interpolation. IEEE Trans. Circuit Syst. Video Technol. 23(3), 445–454 (2013)

    Article  Google Scholar 

  27. Y.-H. Cho, H.-Y. Lee, D.-S. Park, Temporal frame interpolation based on multiframe feature trajectory. IEEE Trans. Circuit Syst. Video Technol. 23(12), 2105–2115 (2013)

    Article  Google Scholar 

  28. D. Kim, H. Park, An efficient motion-compensated frame for high-resolution videos. IEEE J. Disp. Technol. 11(7), 580–588 (2015)

    Article  Google Scholar 

  29. E. Herbst, S. Seitz, S. Baker, Occlusion Reasoning for Temporal Interpolation using Optical Flow, Department of Computer Science and Engineering, University of Washington, Technical report. UW-CSE-09- 08-01, 2009

    Google Scholar 

  30. T. Stich, C. Linz, C. Wallraven, D. Cunningham, M. Magnor, Perception-motivated interpolation of image sequences. ACM Trans. Appl. Percept. 8(2), 1–25 (2011)

    Article  Google Scholar 

  31. W.R. Mark, L. McMillan, G. Bishop, Post-rendering 3D warping, in Proceedings of the Symposium on Interactive 3D Graphics (1997), pp. 7–16

    Google Scholar 

  32. R. Leonardi, A. Iocco, Time-varying motion estimation on a sequence of images. Multimed. Commun. Video Coding, 309–315 (1996)

    Google Scholar 

  33. P. Csillag, L. Boroczky, Estimation of accelerated motion for motion-compensated frame interpolation. Vis. Commun. Image Process. 604–614 (1996)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Electrical Engineering and Telecommunications, UNSW Sydney, Sydney, NSW, Australia

    Dominic Rüfenacht

Authors
  1. Dominic Rüfenacht
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dominic Rüfenacht .

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer Nature Singapore Pte Ltd.

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Rüfenacht, D. (2018). Temporal Frame Interpolation (TFI). In: Novel Motion Anchoring Strategies for Wavelet-based Highly Scalable Video Compression. Springer Theses. Springer, Singapore. https://doi.org/10.1007/978-981-10-8225-2_3

Download citation

  • DOI: https://doi.org/10.1007/978-981-10-8225-2_3

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-10-8224-5

  • Online ISBN: 978-981-10-8225-2

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation