Advertisement

Supervised Deep Kriging for Single-Image Super-Resolution

  • Gianni FranchiEmail author
  • Angela Yao
  • Andreas Kolb
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11269)

Abstract

We propose a novel single-image super-resolution approach based on the geostatistical method of kriging. Kriging is a zero-bias minimum-variance estimator that performs spatial interpolation based on a weighted average of known observations. Rather than solving for the kriging weights via the traditional method of inverting covariance matrices, we propose a supervised form in which we learn a deep network to generate said weights. We combine the kriging weight generation and kriging process into a joint network that can be learned end-to-end. Our network achieves competitive super-resolution results as other state-of-the-art methods. In addition, since the super-resolution process follows a known statistical framework, we are able to estimate bias and variance, something which is rarely possible for other deep networks.

Notes

Acknowledgement

This research was funded by the German Research Foundation (DFG) as part of the research training group GRK 1564 Imaging New Modalities.

Supplementary material

480455_1_En_44_MOESM1_ESM.pdf (15.9 mb)
Supplementary material 1 (pdf 16310 KB)

References

  1. 1.
    Anbarjafari, G., Demirel, H.: Image super resolution based on interpolation of wavelet domain high frequency subbands and the spatial domain input image. ETRI J. 32(3), 390–394 (2010)CrossRefGoogle Scholar
  2. 2.
    Bevilacqua, M., Roumy, A., Guillemot, C., Alberi-Morel, M.: Low-complexity single-image super-resolution based on nonnegative neighbor embedding (2012)Google Scholar
  3. 3.
    De Brabandere, B., Jia, X., Tuytelaars, T., Gool, L.V.: Dynamic filter networks. In: NIPS (2016)Google Scholar
  4. 4.
    Chang, H., Yeung, D., Xiong, Y.: Super-resolution through neighbor embedding. In: CVPR, vol. 1, p. I. IEEE (2004)Google Scholar
  5. 5.
    Cressie, N.: Statistics for Spatial Data. Wiley, Hoboken (2015)zbMATHGoogle Scholar
  6. 6.
    Damianou, A., Lawrence, N.: Deep Gaussian processes. In: Artificial Intelligence and Statistics, pp. 207–215 (2013)Google Scholar
  7. 7.
    Dong, W., Zhang, L., Shi, G., Wu, X.: Image deblurring and super-resolution by adaptive sparse domain selection and adaptive regularization. IEEE Trans. Image Process. (TIP) 20(7), 1838–1857 (2011)MathSciNetCrossRefGoogle Scholar
  8. 8.
    Donoho, D.L.: Compressed sensing. IEEE Trans. Inf. Theory 52(4), 1289–1306 (2006)MathSciNetCrossRefGoogle Scholar
  9. 9.
    Freeman, W., Jones, T., Pasztor, E.: Example-based super-resolution. IEEE Comput. Graph. Appl. 22(2), 56–65 (2002)CrossRefGoogle Scholar
  10. 10.
    He, H., Siu, W.C.: Single image super-resolution using Gaussian process regression. In: CVPR (2011)Google Scholar
  11. 11.
    Huang, J., Singh, A., Ahuja, N.: Single image super-resolution from transformed self-exemplars. In: CVPR, pp. 5197–5206 (2015)Google Scholar
  12. 12.
    Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. In: International Conference on Machine Learning, pp. 448–456 (2015)Google Scholar
  13. 13.
    Keys, R.: Cubic convolution interpolation for digital image processing. IEEE Trans. Acoust. Speech Signal Process. 29(6), 1153–1160 (1981)MathSciNetCrossRefGoogle Scholar
  14. 14.
    Kim, J., Lee, J.K., Lee, K.M.: Accurate image super-resolution using very deep convolutional networks. In: CVPR (2016)Google Scholar
  15. 15.
    Kim, J., Lee, J.K., Lee, K.M.: Deeply-recursive convolutional network for image super-resolution. In: CVPR (2016)Google Scholar
  16. 16.
    LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)CrossRefGoogle Scholar
  17. 17.
    Lim, B., Son, S., Kim, H., Nah, S., Lee, K.M.: Enhanced deep residual networks for single image super-resolution. In: CVPR Workshops (2017)Google Scholar
  18. 18.
    Marquina, A., Osher, S.: Image super-resolution by TV-regularization and Bregman iteration. J. Sci. Comput. 37(3), 367–382 (2008)MathSciNetCrossRefGoogle Scholar
  19. 19.
    Martin, D., Fowlkes, C., Tal, D., Malik, J.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: ICCV, vol. 2, pp. 416–423 (2001)Google Scholar
  20. 20.
    Matheron, G.: Random Sets and Integral Geometry. Wiley, Hoboken (1975)zbMATHGoogle Scholar
  21. 21.
    Meier, F., Hennig, P., Schaal, S.: Local Gaussian regression. arXiv preprint arXiv:1402.0645 (2014)
  22. 22.
    Peleg, T., Elad, M.: A statistical prediction model based on sparse representations for single image super-resolution. IEEE Trans. Image Process. (TIP) 23(6), 2569–2582 (2014)MathSciNetCrossRefGoogle Scholar
  23. 23.
    Pronzato, L., Rendas, M.J.: Bayesian local kriging. Technometrics 59, 293–304 (2017)MathSciNetCrossRefGoogle Scholar
  24. 24.
    Sajjadi, M.S.M., Scholkopf, B., Hirsch, M.: EnhanceNet: single image super-resolution through automated texture synthesis. In: ICCV, October 2017Google Scholar
  25. 25.
    Tai, Y., Yang, J., Liu, X.: Image super-resolution via deep recursive residual network. In: CVPR, June 2017Google Scholar
  26. 26.
    Tong, T., Li, G., Liu, X., Gao, Q.: Image super-resolution using dense skip connections. In: ICCV, October 2017Google Scholar
  27. 27.
    Wang, Z., Bovik, A., Sheikh, H., Simoncelli, E.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. (TIP) 13(4), 600–612 (2004)CrossRefGoogle Scholar
  28. 28.
    Wilson, A.G., Hu, Z., Salakhutdinov, R., Xing, E.P.: Deep kernel learning. In: AISTATS (2016)Google Scholar
  29. 29.
    Xu, L., Jia, J.: Two-phase kernel estimation for robust motion deblurring. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6311, pp. 157–170. Springer, Heidelberg (2010).  https://doi.org/10.1007/978-3-642-15549-9_12CrossRefGoogle Scholar
  30. 30.
    Yang, J., Wang, Z., Lin, Z., Cohen, S., Huang, T.: Coupled dictionary training for image super-resolution. IEEE Trans. Image Process. (TIP) 21(8), 3467–3478 (2012)MathSciNetCrossRefGoogle Scholar
  31. 31.
    Yang, J., Wright, J., Huang, T., Ma, Y.: Image super-resolution via sparse representation. IEEE Trans. Image Process. (TIP) 19(11), 2861–2873 (2010)MathSciNetCrossRefGoogle Scholar
  32. 32.
    Zhang, K., Gao, X., Tao, D., Li, X.: Single image super-resolution with non-local means and steering kernel regression. IEEE Trans. Image Process. (TIP) 21(11), 4544–4556 (2012)MathSciNetCrossRefGoogle Scholar
  33. 33.
    Zhao, S., Han, H., Peng, S.: Wavelet-domain HMT-based image super-resolution. In: International Conference on Image Processing (ICIP), vol. 2, pp. II–953. IEEE (2003)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Institute for Vision and GraphicsUniversity of SiegenSiegenGermany
  2. 2.University of BonnBonnGermany

Personalised recommendations