Computational Visual Media

, Volume 3, Issue 1, pp 73–82 | Cite as

Multi-example feature-constrained back-projection method for image super-resolution

Open Access
Research Article

Abstract

Example-based super-resolution algorithms, which predict unknown high-resolution image information using a relationship model learnt from known high- and low-resolution image pairs, have attracted considerable interest in the field of image processing. In this paper, we propose a multi-example feature-constrained back-projection method for image super-resolution. Firstly, we take advantage of a feature-constrained polynomial interpolation method to enlarge the low-resolution image. Next, we consider low-frequency images of different resolutions to provide an example pair. Then, we use adaptive kNN search to find similar patches in the low-resolution image for every image patch in the high-resolution low-frequency image, leading to a regression model between similar patches to be learnt. The learnt model is applied to the low-resolution high-frequency image to produce high-resolution high-frequency information. An iterative back-projection algorithm is used as the final step to determine the final high-resolution image. Experimental results demonstrate that our method improves the visual quality of the high-resolution image.

Keywords

feature constraints back-projection super-resolution (SR) 

Notes

Acknowledgements

The authors would like to thank the anonymous reviewers for giving valuable suggestions that greatly improved the paper. The authors also thank other researchers who provided the code for their algorithms for comparative testing. This project was supported by the National Natural Science Foundation of China (Grant Nos. 61572292, 61332015, 61373078, and 61272430), and the National Research Foundation for the Doctoral Program of Higher Education of China (Grant No. 20110131130004).

References

  1. [1]
    Glasner, D.; Bagon, S.; Irani, M. Super-resolution from a single image. In: Proceedings of the IEEE 12th International Conference on Computer Vision, 349–356, 2009.Google Scholar
  2. [2]
    Park, S. C.; Park, M. K.; Kang, M. G. Super-resolution image reconstruction: A technical overview. IEEE Signal Processing Magazine Vol. 20, No. 3, 21–36, 2003.CrossRefGoogle Scholar
  3. [3]
    Kolte, R.; Arora, A. Image super-resolution. Available at https://pdfs.semanticscholar.org/20de/2880a4196a733314252a717f1a55f5f0ea64.pdf.Google Scholar
  4. [4]
    Hou, H.; Andrews, H. Cubic splines for image interpolation and digital filtering. IEEE Transactions on Acoustics, Speech, and Signal Processing Vol. 26, No. 6, 508–517, 1978.CrossRefMATHGoogle Scholar
  5. [5]
    McKinley, S.; Levine, M. Cubic spline interpolation. College of the Redwoods Vol. 45, No. 1, 1049–1060, 1998.Google Scholar
  6. [6]
    Keys, R. Cubic convolution interpolation for digital image processing. IEEE Transactions on Acoustics, Speech, and Signal Processing Vol. 29, No. 6, 1153–1160, 1981.MathSciNetCrossRefMATHGoogle Scholar
  7. [7]
    Wang, H.; Gao, X.; Zhang, K.; Li, J. Singleimage super-resolution using active-sampling Gaussian process regression. IEEE Transactions on Image Processing Vol. 25, No. 2, 935–948, 2016.MathSciNetCrossRefGoogle Scholar
  8. [8]
    Irani, M.; Peleg, S. Improving resolution by image registration. CVGIP: Graphical Models and Image Processing Vol. 53, No. 3, 231–239, 1991.Google Scholar
  9. [9]
    Dong, W.; Zhang, L.; Shi, G.; Wu, X. Nonlocal back-projection for adaptive image enlargement. In: Proceedings of the 16th IEEE International Conference on Image Processing, 349–352, 2009.Google Scholar
  10. [10]
    Adelson, E. H.; Anderson, C. H.; Bergen, J. R.; Burt, P. J.; Ogden, J. M. Pyramid methods in image processing. RCA Engineer Vol. 29, No. 6, 33–41, 1984.Google Scholar
  11. [11]
    Bevilacqua, M.; Roumy, A.; Guillemot, C.; Alberi-Morel, M. L. Low-complexity single-image superresolution based on nonnegative neighbor embedding. In: Proceedings of British Machine Vision Conference, 135.1–135.10, 2012.Google Scholar
  12. [12]
    Yang, C.-Y.; Huang, J.-B.; Yang, M.-H. Exploiting self-similarities for single frame super-resolution. In: Computer Vision–ACCV 2010. Kimmel, R.; Klette, R.; Sugimoto, A. Eds. Springer Berlin Heidelberg, 497–510, 2010.Google Scholar
  13. [13]
    Yang, J.; Wright, J.; Huang, T. S.; Ma, Y. Image super-resolution via sparse representation. IEEE Transactions on Image Processing Vol. 19, No. 11, 2861–2873, 2010.MathSciNetCrossRefMATHGoogle Scholar
  14. [14]
    Dong, W.; Shi, G.; Zhang, L.; Wu, X. Super-resolution with nonlocal regularized sparse representation. In: Proceedings of SPIE7744, Visual Communications and Image Processing, 77440H, 2010.Google Scholar
  15. [15]
    Yang, J.; Wright, J.; Huang, T.; Ma, Y. Image super-resolution as sparse representation of raw image patches. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1–8, 2008.Google Scholar
  16. [16]
    Zhang, H.; Zhang, Y.; Huang, T. S. Efficient sparse representation based image super resolution via dual dictionary learning. In: Proceedings of the IEEE International Conference on Multimedia and Expo, 1–6, 2011.Google Scholar
  17. [17]
    Zhao, Y.; Yang, J.; Zhang, Q.; Song, L.; Cheng, Y.; Pan, Q. Hyperspectral imagery super-resolution by sparse representation and spectral regularization. EURASIP Journal on Advances in Signal Processing Vol. 2011, 87, 2011.CrossRefGoogle Scholar
  18. [18]
    Chang, H.; Yeung, D.-Y.; Xiong, Y. Super-resolution through neighbor embedding. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, I, 2004.Google Scholar
  19. [19]
    Gao, X.; Zhang, K.; Tao, D.; Li, X. Image superresolution with sparse neighbor embedding. IEEE Transactions on Image Processing Vol. 21, No. 7, 3194–3205, 2012.MathSciNetCrossRefMATHGoogle Scholar
  20. [20]
    Roweis, S. T.; Saul, L. K. Nonlinear dimensionality reduction by locally linear embedding. Science Vol. 290, No. 5500, 2323–2326, 2000.CrossRefGoogle Scholar
  21. [21]
    BenAbdelkader, C.; Cutler, R.; Nanda, H.; Davis, L. EigenGait: Motion-based recognition of people using image self-similarity. In: Audio-and Video-Based Biometric Person Authentication. Bigun, J.; Smeraldi, F. Eds. Springer Berlin Heidelberg, 284–294, 2001.CrossRefGoogle Scholar
  22. [22]
    Church, K. W.; Helfman, J. I. Dotplot: A program for exploring self-similarity in millions of lines of text and code. Journal of Computational and Graphical Statistics Vol. 2, No. 2, 153–174, 1993.Google Scholar
  23. [23]
    Shechtman, E.; Irani, M. Matching local selfsimilarities across images and videos. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1–8, 2007.Google Scholar
  24. [24]
    Caiming, Z.; Xin, Z.; Xuemei, L.; Fuhua, C. Cubic surface fitting to image with edges as constraints. In: Proceedings of the IEEE International Conference on Image Processing, 1046–1050, 2013.Google Scholar
  25. [25]
    Chan, T.-M.; Zhang, J.; Pu, J.; Huang, H. Neighbor embedding based super-resolution algorithm through edge detection and feature selection. Pattern Recognition Letters Vol. 30, No. 5, 494–502, 2009.CrossRefGoogle Scholar
  26. [26]
    Freedman, G.; Fattal, R. Image and video upscaling from local self-examples. ACM Transactions on Graphics Vol. 30, No. 2, Article No. 12, 2011.Google Scholar
  27. [27]
    Dong, W.; Zhang, L.; Lukac, R.; Shi, G. Sparse representation based image interpolation with nonlocal autoregressive modeling. IEEE Transactions on Image Processing Vol. 22, No. 4, 1382–1394, 2013.MathSciNetCrossRefMATHGoogle Scholar
  28. [28]
    Hore, A.; Ziou, D. Image quality metrics: PSNR vs. SSIM. In: Proceedings of the 20th International Conference on Pattern Recognition, 2366–2369, 2010.Google Scholar

Copyright information

© The Author(s) 2016

Open Access The articles published in this journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Other papers from this open access journal are available free of charge from http://www.springer.com/journal/41095. To submit a manuscript, please go to https://www.editorialmanager.com/cvmj.

Authors and Affiliations

  • Junlei Zhang
    • 1
  • Dianguang Gai
    • 2
  • Xin Zhang
    • 1
  • Xuemei Li
    • 1
  1. 1.School of Computer Science and TechnologyShandong UniversityJinanChina
  2. 2.Earthquake Administration of Shandong ProvinceShandongChina

Personalised recommendations