Abstract
We present the first systematic evaluation of the data terms for multi-frame super-resolution within a variational model. The various data terms are derived by permuting the order of the blur-, downsample-, and warp-operators in the image acquisition model. This yields six different basic models. Our experiments using synthetic images with known ground truth show that two models are preferable: the widely-used warp-blur-downsample model that is physically plausible if atmospheric blur is negligible, and the hardly considered blur-warp-downsample model. We show that the quality of motion estimation plays the decisive role on which of these two models works best: While the classic warp-blur-downsample model requires optimal motion estimation, the rarely-used blur-warp-downsample model should be preferred in practically relevant scenarios when motion estimation is suboptimal. This confirms a widely ignored result by Wang and Qi (2004). Last but not least, we propose a new modification of the blur-warp-downsample model that offers a very significant speed-up without substantial loss in the reconstruction quality.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Babacan, S., Molina, R., Katsaggelos, A.: Variational Bayesian super resolution. IEEE Trans. Image Process. 20(4), 984–999 (2011)
Belekos, S., Galatsanos, N., Katsaggelos, A.: Maximum a posteriori video super-resolution using a new multichannel image prior. IEEE Trans. Image Process. 19(6), 1451–1464 (2010)
Black, M., Anandan, P.: A framework for robust estimation of optical flow. In: Proceedings of the Fourth International Conference on Computer Vision, pp. 231–236. IEEE Computer Society Press, Berlin, Germany, May 1993
Brox, T., Bruhn, A., Papenberg, N., Weickert, J.: High accuracy optical flow estimation based on a theory for warping. In: Pajdla, T., Matas, J. (eds.) ECCV 2004. LNCS, vol. 3024, pp. 25–36. Springer, Heidelberg (2004). doi:10.1007/978-3-540-24673-2_3
Chaudhuri, S.: Super-Resolution Imaging. Springer, New York (2001)
Elad, M., Feuer, A.: Restoration of a single superresolution image from several blurred, noisy, and undersampled measured images. IEEE Trans. Image Process. 6(12), 1646–1658 (1997)
Elad, M., Hel-Or, Y.: A fast super-resolution reconstruction algorithm for pure translational motion and common space-invariant blur. IEEE Trans. Image Process. 10(8), 1187–1193 (2001)
Farsiu, S., Robinson, M.D., Elad, M., Milanfar, P.: Fast and robust multi-frame super resolution. IEEE Trans. Image Process. 13(10), 1327–1364 (2004)
Glasner, D., Shai, B., Irani, M.: Super-resolution from a single image. In: Proceedings of the Twelfth International Conference on Computer Vision, pp. 349–356. IEEE, Kyoto, Japan, September 2009
He, H., Kondi, L.: An image super-resolution algorithm for different error levels per frame. IEEE Trans. Image Process. 15(3), 592–603 (2006)
Li, X., Hu, Y., Gao, X., Tao, D., Ning, B.: A multi-frame image super-resolution method. Sig. Process. 90(2), 405–414 (2010)
Marquina, A., Stanley, J.: Image super-resolution by TV-regularization and Bregman iteration. Springer J. Sci. Comput. 37(3), 367–382 (2008)
Milanfar, P.: Super-Resolution Imaging. CRC Press, Boca Raton (2010)
Mitzel, D., Pock, T., Schoenemann, T., Cremers, D.: Video super resolution using duality based TV-L1 optical flow. In: Denzler, J., Notni, G., Süße, H. (eds.) DAGM 2009. LNCS, vol. 5748, pp. 432–441. Springer, Heidelberg (2009). doi:10.1007/978-3-642-03798-6_44
Ng, M., Shen, H., Lam, E., Zhang, L.: A total variation regularization based super-resolution reconstruction algorithm for digital video. EURASIP J. Adv. Sig. Process. 1, 1–6 (2007)
Nguyen, N., Milanfar, P., Golub, G.: Efficient generalized cross-validation with applications to parametric image restoration and resolution enhancement. IEEE Trans. Process. 10(9), 1299–1308 (2001)
Park, S., Park, M., Kang, M.: Super-resolution image reconstruction: a technical overview. IEEE Sig. Process. Mag. 20(3), 21–36 (2003)
Rochefort, G., Champagnat, F., Besnerais, G.L., Giovannelli, J.: An improved observation model for super-resolution under affine motion. IEEE Trans. Image Process. 15(11), 3325–3337 (2006)
Tian, J., Ma, K.: A survey on super-resolution imaging. Sig. Image Video Process. 5(3), 329–342 (2011)
Tsai, R., Huang, T.: Multiframe image restoration and registration. Adv. Comput. Vis. Image Process. 1(2), 317–339 (1984)
Wang, Z., Qi, F.: On ambiguities in super-resolution modeling. IEEE Sig. Process. Lett. 11(8), 678–681 (2004)
Weickert, J., Grewenig, S., Schroers, C., Bruhn, A.: Cyclic schemes for PDE-based image analysis. Int. J. Comput. Vis. 118(3), 275–299 (2016)
Yang, J., Wright, J., Huang, T., Ma, Y.: Image super-resolution via sparse representation. IEEE Trans. Image Process. 19(11), 2681–2873 (2010)
Zhang, L., Zhang, H., Shen, H., Li, P.: A super-resolution reconstruction algorithm for surveillance images. Sig. Process. 90(3), 848–859 (2010)
Zhang, X., Jiang, J., Peng, S.: Commutability of blur and affine warping in super-resolution with application to joint estimation of triple-coupled variables. IEEE Trans. Image Process. 21(4), 1796–1808 (2012)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Bodduna, K., Weickert, J. (2017). Evaluating Data Terms for Variational Multi-frame Super-Resolution. In: Lauze, F., Dong, Y., Dahl, A. (eds) Scale Space and Variational Methods in Computer Vision. SSVM 2017. Lecture Notes in Computer Science(), vol 10302. Springer, Cham. https://doi.org/10.1007/978-3-319-58771-4_47
Download citation
DOI: https://doi.org/10.1007/978-3-319-58771-4_47
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-58770-7
Online ISBN: 978-3-319-58771-4
eBook Packages: Computer ScienceComputer Science (R0)