Abstract
We propose a method for making temporal super-resolution video from a single video by exploiting the self-similarity that exists in the spatio-temporal domain of videos. Temporal super-resolution is inherently ill-posed problem because there are an infinite number of high temporal resolution frames that can produce the same low temporal resolution frame. The key idea in this work to solve this ambiguity is exploiting self-similarity, i.e., a self-similar appearance that represents integrated motion of objects during each exposure time of videos with different temporal resolutions. In contrast with other methods that try to generate plausible intermediate frames based on temporal interpolation, our method can increase the temporal resolution of a given video, for instance by resolving one frame to two frames. Based on the quantitative evaluation of experimental results, we demonstrate that our method can generate enhanced videos with increased temporal resolution thereby recovering appearances of dynamic scenes.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Choi, B.T., Lee, S.H., Ko, S.J.: New frame rate up-conversion using bi-directional motion estimation. IEEE Trans. CE 46, 603–609 (2000)
Ha, T., Lee, S., Kim, J.: Motion compensated frame interpolation by new block-based motion estimation algorithm. IEEE Trans. CE 50, 752–759 (2004)
ying Kuo, T., Kim, J., Jay Kuo, C.: Motion-compensated frame interpolation scheme for h.263 codec. In: Proc. ISCAS 1999, pp. 491–494 (1999)
Baker, S., Scharstein, D., Lewis, J., Roth, S., Black, M., Szeliski, R.: A database and evaluation methodology for optical flow. In: Proc. ICCV 2007, pp. 1–8 (2007)
Glasner, D., Bagon, S., Irani, M.: Super-resolution from a single image. In: Proc. ICCV 2009, pp. 349–356 (2009)
Mahajan, D., Huang, F.C., Matusik, W., Ramamoorthi, R., Belhumeur, P.: Moving gradients: a path-based method for plausible image interpolation. In: Proc. SIGGRAPH 2009, pp. 1–11 (2009)
Ben-Ezra, M., Nayar, S.K.: Motion deblurring using hybrid imaging. In: Proc. CVPR 2003, pp. I–657–I–664 (2003)
Tai, Y.W., Du, H., Brown, M.S., Lin, S.: Image/video deblurring using a hybrid camera. In: Proc. CVPR 2008, pp. 1–8 (2008)
Watanabe, K., Iwai, Y., Nagahara, H., Yachida, M., Suzuki, T.: Video synthesis with high spatio-temporal resolution using motion compensation and image fusion in wavelet domain. In: Narayanan, P.J., Nayar, S.K., Shum, H.-Y. (eds.) ACCV 2006. LNCS, vol. 3851, pp. 480–489. Springer, Heidelberg (2006)
Shechtman, E., Caspi, Y., Irani, M.: Space-time super-resolution. IEEE Trans. PAMI 27, 531–545 (2005)
Agrawal, A., Gupta, M., Veeraraghavan, A., Narasimhan, S.: Optimal coded sampling for temporal super-resolution. In: Proc. CVPR 2010, pp. 599–606 (2010)
Baker, S., Kanade, T.: Limits on super-resolution and how to break them. IEEE Trans. PAMI 24, 1167–1183 (2002)
Freeman, W., Jones, T., Pasztor, E.: Example-based super-resolution. IEEE Computer Graphics and Applications 22, 56–65 (2002)
Brox, T., Bruhn, A., Papenberg, N., Weickert, J.: High accuracy optical flow estimation based on a theory for warping. In: Pajdla, T., Matas, J(G.) (eds.) ECCV 2004. LNCS, vol. 3024, pp. 25–36. Springer, Heidelberg (2004)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Shimano, M., Okabe, T., Sato, I., Sato, Y. (2011). Video Temporal Super-Resolution Based on Self-similarity. In: Kimmel, R., Klette, R., Sugimoto, A. (eds) Computer Vision – ACCV 2010. ACCV 2010. Lecture Notes in Computer Science, vol 6492. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-19315-6_8
Download citation
DOI: https://doi.org/10.1007/978-3-642-19315-6_8
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-19314-9
Online ISBN: 978-3-642-19315-6
eBook Packages: Computer ScienceComputer Science (R0)