Abstract
We propose a novel single-image super-resolution approach based on the geostatistical method of kriging. Kriging is a zero-bias minimum-variance estimator that performs spatial interpolation based on a weighted average of known observations. Rather than solving for the kriging weights via the traditional method of inverting covariance matrices, we propose a supervised form in which we learn a deep network to generate said weights. We combine the kriging weight generation and kriging process into a joint network that can be learned end-to-end. Our network achieves competitive super-resolution results as other state-of-the-art methods. In addition, since the super-resolution process follows a known statistical framework, we are able to estimate bias and variance, something which is rarely possible for other deep networks.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsNotes
- 1.
We refer the reader to our supplementary materials for a more detailed comparison of the two.
- 2.
For concise notation, we use \(\hat{f}_*\) to denote \(\widehat{f}(x^*)\), \(f_i\) to denote \(f(x_i)\) and \(w_i^* =w_i(x^*)\).
- 3.
Of course, we cannot account for the low resolution process that produced \(\tilde{F}\).
References
Anbarjafari, G., Demirel, H.: Image super resolution based on interpolation of wavelet domain high frequency subbands and the spatial domain input image. ETRI J. 32(3), 390–394 (2010)
Bevilacqua, M., Roumy, A., Guillemot, C., Alberi-Morel, M.: Low-complexity single-image super-resolution based on nonnegative neighbor embedding (2012)
De Brabandere, B., Jia, X., Tuytelaars, T., Gool, L.V.: Dynamic filter networks. In: NIPS (2016)
Chang, H., Yeung, D., Xiong, Y.: Super-resolution through neighbor embedding. In: CVPR, vol. 1, p. I. IEEE (2004)
Cressie, N.: Statistics for Spatial Data. Wiley, Hoboken (2015)
Damianou, A., Lawrence, N.: Deep Gaussian processes. In: Artificial Intelligence and Statistics, pp. 207–215 (2013)
Dong, W., Zhang, L., Shi, G., Wu, X.: Image deblurring and super-resolution by adaptive sparse domain selection and adaptive regularization. IEEE Trans. Image Process. (TIP) 20(7), 1838–1857 (2011)
Donoho, D.L.: Compressed sensing. IEEE Trans. Inf. Theory 52(4), 1289–1306 (2006)
Freeman, W., Jones, T., Pasztor, E.: Example-based super-resolution. IEEE Comput. Graph. Appl. 22(2), 56–65 (2002)
He, H., Siu, W.C.: Single image super-resolution using Gaussian process regression. In: CVPR (2011)
Huang, J., Singh, A., Ahuja, N.: Single image super-resolution from transformed self-exemplars. In: CVPR, pp. 5197–5206 (2015)
Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. In: International Conference on Machine Learning, pp. 448–456 (2015)
Keys, R.: Cubic convolution interpolation for digital image processing. IEEE Trans. Acoust. Speech Signal Process. 29(6), 1153–1160 (1981)
Kim, J., Lee, J.K., Lee, K.M.: Accurate image super-resolution using very deep convolutional networks. In: CVPR (2016)
Kim, J., Lee, J.K., Lee, K.M.: Deeply-recursive convolutional network for image super-resolution. In: CVPR (2016)
LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)
Lim, B., Son, S., Kim, H., Nah, S., Lee, K.M.: Enhanced deep residual networks for single image super-resolution. In: CVPR Workshops (2017)
Marquina, A., Osher, S.: Image super-resolution by TV-regularization and Bregman iteration. J. Sci. Comput. 37(3), 367–382 (2008)
Martin, D., Fowlkes, C., Tal, D., Malik, J.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: ICCV, vol. 2, pp. 416–423 (2001)
Matheron, G.: Random Sets and Integral Geometry. Wiley, Hoboken (1975)
Meier, F., Hennig, P., Schaal, S.: Local Gaussian regression. arXiv preprint arXiv:1402.0645 (2014)
Peleg, T., Elad, M.: A statistical prediction model based on sparse representations for single image super-resolution. IEEE Trans. Image Process. (TIP) 23(6), 2569–2582 (2014)
Pronzato, L., Rendas, M.J.: Bayesian local kriging. Technometrics 59, 293–304 (2017)
Sajjadi, M.S.M., Scholkopf, B., Hirsch, M.: EnhanceNet: single image super-resolution through automated texture synthesis. In: ICCV, October 2017
Tai, Y., Yang, J., Liu, X.: Image super-resolution via deep recursive residual network. In: CVPR, June 2017
Tong, T., Li, G., Liu, X., Gao, Q.: Image super-resolution using dense skip connections. In: ICCV, October 2017
Wang, Z., Bovik, A., Sheikh, H., Simoncelli, E.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. (TIP) 13(4), 600–612 (2004)
Wilson, A.G., Hu, Z., Salakhutdinov, R., Xing, E.P.: Deep kernel learning. In: AISTATS (2016)
Xu, L., Jia, J.: Two-phase kernel estimation for robust motion deblurring. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6311, pp. 157–170. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-15549-9_12
Yang, J., Wang, Z., Lin, Z., Cohen, S., Huang, T.: Coupled dictionary training for image super-resolution. IEEE Trans. Image Process. (TIP) 21(8), 3467–3478 (2012)
Yang, J., Wright, J., Huang, T., Ma, Y.: Image super-resolution via sparse representation. IEEE Trans. Image Process. (TIP) 19(11), 2861–2873 (2010)
Zhang, K., Gao, X., Tao, D., Li, X.: Single image super-resolution with non-local means and steering kernel regression. IEEE Trans. Image Process. (TIP) 21(11), 4544–4556 (2012)
Zhao, S., Han, H., Peng, S.: Wavelet-domain HMT-based image super-resolution. In: International Conference on Image Processing (ICIP), vol. 2, pp. II–953. IEEE (2003)
Acknowledgement
This research was funded by the German Research Foundation (DFG) as part of the research training group GRK 1564 Imaging New Modalities.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Franchi, G., Yao, A., Kolb, A. (2019). Supervised Deep Kriging for Single-Image Super-Resolution. In: Brox, T., Bruhn, A., Fritz, M. (eds) Pattern Recognition. GCPR 2018. Lecture Notes in Computer Science(), vol 11269. Springer, Cham. https://doi.org/10.1007/978-3-030-12939-2_44
Download citation
DOI: https://doi.org/10.1007/978-3-030-12939-2_44
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-12938-5
Online ISBN: 978-3-030-12939-2
eBook Packages: Computer ScienceComputer Science (R0)