Advertisement

Single Image Super-Resolution Reconstruction Based on Edge-Preserving with External and Internal Gradient Prior Knowledge

  • Ruxin Wang
  • Congying HanEmail author
  • Mingqiang Li
  • Tiande Guo
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10116)

Abstract

Single image super-resolution (SISR) reconstruction is currently a very fundamental and significant task in image processing. Instead of upscaling the image in spatial domain, we propose a novel SISR method based on edge preserving integrating the external gradient priors by deep learning method (auto-encoder network) and internal gradient priors using non-local total variation (NLTV). The gradient domain effectively reflects the high frequency details and edge information of nature image to some extent. The joint perspective exploits the complementary advantages of external and internal gradient prior knowledge for reconstructing the HR image. The experimental results demonstrate the effectiveness of our approach over several state-of-art SISR methods.

Keywords

Convolutional Neural Network Gradient Domain Image Processing Problem Internal Gradient Patch Pair 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Notes

Acknowledgement

The authors would like to sincerely thank J. C. Yang, L. He, R. Timofte and C. Dong et al. for sharing the source codes of the ScSR, BP-JDL, NE+NNLS, NE+LLE, ANR and SRCNN methods. This work was funded by the Chinese National Natural Science Foundation (11331012, 71271204, 11571014).

References

  1. 1.
    Li, X., Orchard, M.T.: New edge-directed interpolation. IEEE Trans. Image Process. 10, 1521–1527 (2001)CrossRefGoogle Scholar
  2. 2.
    Zhang, L., Wu, X.: An edge-guided image interpolation algorithm via directional filtering and data fusion. IEEE Trans. Image Process. 15, 2226–2238 (2006)CrossRefGoogle Scholar
  3. 3.
    Marquina, A., Osher, S.J.: Image super-resolution by TV-regularization and bregman iteration. J. Sci. Comput. 37, 367–382 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    Zhang, K., Gao, X., Tao, D., Li, X.: Single image super-resolution with non-local means and steering kernel regression. IEEE Trans. Image Process. 21, 4544–4556 (2012)MathSciNetCrossRefGoogle Scholar
  5. 5.
    Lin, Z., Shum, H.Y.: Fundamental limits of reconstruction-based super resolution algorithms under local translation. IEEE Trans. Pattern Anal. Mach. Intell. 26, 83–97 (2004)CrossRefGoogle Scholar
  6. 6.
    Yang, J., Wright, J., Huang, T.S., Ma, Y.: Image super-resolution via sparse representation. IEEE Trans. Image Process. 19, 2861–2873 (2010)MathSciNetCrossRefGoogle Scholar
  7. 7.
    Dong, C., Loy, C.C., He, K., Tang, X.: Learning a deep convolutional network for image super-resolution. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8692, pp. 184–199. Springer, Cham (2014). doi: 10.1007/978-3-319-10593-2_13 Google Scholar
  8. 8.
    Timofte, R., De Smet, V., Van Gool, L.: A+: adjusted anchored neighborhood regression for fast super-resolution. In: Cremers, D., Reid, I., Saito, H., Yang, M.-H. (eds.) ACCV 2014. LNCS, vol. 9006, pp. 111–126. Springer, Cham (2015). doi: 10.1007/978-3-319-16817-3_8 Google Scholar
  9. 9.
    Osher, S., Burger, M., Goldfarb, D., Xu, J., Yin, W.: An iterative regularization method for total variation-based image restoration. Multiscale Model. Simul. 4, 460–489 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
  10. 10.
    Buades, A., Coll, B., Morel, J.M.: A non-local algorithm for image denoising. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2005, vol. 2, pp. 60–65. IEEE (2005)Google Scholar
  11. 11.
    Tai, Y.W., Liu, S., Brown, M.S., Lin, S.: Super resolution using edge prior and single image detail synthesis. In: 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2400–2407. IEEE (2010)Google Scholar
  12. 12.
    Yang, C.Y., Yang, M.H.: Fast direct super-resolution by simple functions. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 561–568 (2013)Google Scholar
  13. 13.
    Kim, K.I., Kwon, Y.: Single-image super-resolution using sparse regression and natural image prior. IEEE Trans. Pattern Anal. Mach. Intell. 32, 1127–1133 (2010)CrossRefGoogle Scholar
  14. 14.
    Zhang, K., Gao, X., Li, X., Tao, D.: Partially supervised neighbor embedding for example-based image super-resolution. IEEE J. Sel. Top. Sign. Proces. 5, 230–239 (2011)CrossRefGoogle Scholar
  15. 15.
    Zeyde, R., Elad, M., Protter, M.: On single image scale-up using sparse-representations. In: Boissonnat, J.-D., Chenin, P., Cohen, A., Gout, C., Lyche, T., Mazure, M.-L., Schumaker, L. (eds.) Curves and Surfaces 2010. LNCS, vol. 6920, pp. 711–730. Springer, Heidelberg (2012). doi: 10.1007/978-3-642-27413-8_47 CrossRefGoogle Scholar
  16. 16.
    Timofte, R., Smet, V., Gool, L.: Anchored neighborhood regression for fast example-based super-resolution. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1920–1927 (2013)Google Scholar
  17. 17.
    Kim, J., Lee, J.K., Lee, K.M.: Accurate image super-resolution using very deep convolutional networks. arXiv preprint (2015). arXiv:1511.04587
  18. 18.
    Timofte, R., Rothe, R., Van Gool, L.: Seven ways to improve example-based single image super resolution. arXiv preprint (2015). arXiv:1511.02228
  19. 19.
    Sun, J., Xu, Z., Shum, H.Y.: Gradient profile prior and its applications in image super-resolution and enhancement. IEEE Trans. Image Process. 20, 1529–1542 (2011)MathSciNetCrossRefGoogle Scholar
  20. 20.
    Wang, L., Xiang, S., Meng, G., Wu, H., Pan, C.: Edge-directed single-image super-resolution via adaptive gradient magnitude self-interpolation. IEEE Trans. Circuits Syst. Video Technol. 23, 1289–1299 (2013)CrossRefGoogle Scholar
  21. 21.
    Li, Y., Liu, J., Yang, W., Guo, Z.: Neighborhood regression for edge-preserving image super-resolution. In: 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1201–1205. IEEE (2015)Google Scholar
  22. 22.
    Si, D., Hu, Y., Gan, Z., Cui, Z., Liu, F.: Edge directed single image super resolution through the learning based gradient regression estimation. In: Zhang, Y.-J. (ed.) ICIG 2015. LNCS, vol. 9218, pp. 226–239. Springer, Cham (2015). doi: 10.1007/978-3-319-21963-9_21 CrossRefGoogle Scholar
  23. 23.
    Bengio, Y.: Learning deep architectures for AI. Found. Trends Mach. Learn. 2, 1–127 (2009)CrossRefzbMATHGoogle Scholar
  24. 24.
    Zhang, X., Burger, M., Bresson, X., Osher, S.: Bregmanized nonlocal regularization for deconvolution and sparse reconstruction. SIAM J. Imaging Sci. 3, 253–276 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  25. 25.
    Cho, T.S., Zitnick, C.L., Joshi, N., Kang, S.B., Szeliski, R., Freeman, W.T.: Image restoration by matching gradient distributions. IEEE Trans. Pattern Anal. Mach. Intell. 34, 683–694 (2012)CrossRefGoogle Scholar
  26. 26.
    Gilboa, G., Sochen, N.A., Zeevi, Y.Y.: Regularized shock filters and complex diffusion. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2350, pp. 399–413. Springer, Heidelberg (2002). doi: 10.1007/3-540-47969-4_27 CrossRefGoogle Scholar
  27. 27.
    Liu, W., Li, S.: Sparse representation with morphologic regularizations for single image super-resolution. Sig. Process. 98, 410–422 (2014)CrossRefGoogle Scholar
  28. 28.
    Zhou, Q., Chen, S., Liu, J., Tang, X.: Edge-preserving single image super-resolution. In: Proceedings of the 19th ACM International Conference on Multimedia, pp. 1037–1040. ACM (2011)Google Scholar
  29. 29.
    Dong, C., Loy, C., He, K., Tang, X.: Image super-resolution using deep convolutional networks. IEEE Trans. Pattern Anal. Mach. Intell. 38, 295–307 (2016)CrossRefGoogle Scholar
  30. 30.
    Bevilacqua, M., Roumy, A., Guillemot, C., Alberi-Morel, M.L.: Low-complexity single-image super-resolution based on nonnegative neighbor embedding (2012)Google Scholar
  31. 31.
    Chang, H., Yeung, D.Y., Xiong, Y.: Super-resolution through neighbor embedding. In: Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2004, vol. 1, p. 1. IEEE (2004)Google Scholar
  32. 32.
    He, L., Qi, H., Zaretzki, R.: Beta process joint dictionary learning for coupled feature spaces with application to single image super-resolution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 345–352 (2013)Google Scholar
  33. 33.
    Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13, 600–612 (2004)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Ruxin Wang
    • 1
  • Congying Han
    • 1
    Email author
  • Mingqiang Li
    • 1
  • Tiande Guo
    • 1
  1. 1.Key Laboratory of Big Data Mining and Knowledge Management, School of Mathematical ScienceUniversity of Chinese Academy of Sciences (UCAS)BeijingChina

Personalised recommendations