Hybrid Example-Based Single Image Super-Resolution

  • Yang XianEmail author
  • Xiaodong Yang
  • Yingli Tian
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9475)


Image super-resolution aims to recover a visually pleasing high resolution image from one or multiple low resolution images. It plays an essential role in a variety of real-world applications. In this paper, we propose a novel hybrid example-based single image super-resolution approach which integrates learning from both external and internal exemplars. Given an input image, a proxy image with the same resolution as the target high-resolution image is first generated from a set of externally-learnt regression models. We then perform a coarse-to-fine gradient-level self-refinement on the proxy image guided by the input image. Finally, the refined high-resolution gradients are fed into a uniform energy function to recover the final output. Extensive experiments demonstrate that our framework outperforms the recent state-of-the-art single image super-resolution approaches both quantitatively and qualitatively.


Input Image Gaussian Mixture Model Gaussian Component Ground Truth Image External Dataset 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



This work was supported in part by ONR grant N000141310450 and NSF grants EFRI-1137172, IIP-1343402.


  1. 1.
    Boulanger, J., Kervrann, C., Bouthemy, P.: Space-time adaptation for patch-based image sequence restoration. IEEE Trans. Pattern Anal. Mach. Intell. 29, 1096–1102 (2007)CrossRefGoogle Scholar
  2. 2.
    Farsiu, S., Robinson, M.D., Elad, M., Milanfar, P.: Fast and robust multiframe super resolution. IEEE Trans. Image Process. 13, 1327–1344 (2004)CrossRefGoogle Scholar
  3. 3.
    Protter, M., Elad, M., Tekeda, H., Milanfar, P.: Generalizing the non-local-means to super-resolution reconstruction. IEEE Trans. Image Process. 18, 36–51 (2009)CrossRefMathSciNetGoogle Scholar
  4. 4.
    Shi, B., Zhao, H., Ben-Ezra, M., Yeung, S.-K., Fernandez-Cull, C., Shepard, R.H., Barsi, C., Raskar, R.: Sub-pixel layout for super-resolution with images in the octic group. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014, Part I. LNCS, vol. 8689, pp. 250–264. Springer, Heidelberg (2014) Google Scholar
  5. 5.
    Li, X., Orchard, M.T.: New edge-directed interpolation. IEEE Trans. Image Process. 10, 1521–1527 (2001)CrossRefGoogle Scholar
  6. 6.
    Su, D., Willis, P.: Image interpolation by pixel-level data-dependent triangulation. Comput. Graph. Forum 23, 189–201 (2004)CrossRefGoogle Scholar
  7. 7.
    Tai, Y., Liu, S., Brown, M., Lin, S.: Super resolution using edge prior and single image detail synthesis. In: CVPR (2010)Google Scholar
  8. 8.
    Aly, H.A., Dubois, E.: Image up-sampling using total-variation regularization with a new observation model. IEEE Trans. Image Process. 14, 1647–1659 (2005)CrossRefMathSciNetGoogle Scholar
  9. 9.
    Shan, Q., Li, Z., Jia, J., Tang, C.K.: Fast image/video upsampling. In: ACM SIGGRAPH Asia (2008)Google Scholar
  10. 10.
    Fattal, R.: Image upsampling via imposed edge statistics. In: ACM SIGGRAPH (2007)Google Scholar
  11. 11.
    Sun, J., Sun, J., Xu, Z., Shum, H.Y.: Image super-resolution using gradient profile prior. In: CVPR (2008)Google Scholar
  12. 12.
    Huang, J., Mumford, D.: Statistics of natural images and models. In: CVPR (1999)Google Scholar
  13. 13.
    Freeman, W.T., Pasztor, E.C., Carmichael, O.T.: Learning low-level vision. Int. J. Comput. Vis. 40, 25–47 (2000)zbMATHCrossRefGoogle Scholar
  14. 14.
    Freeman, W.T., Jones, T.R., Pasztor, E.C.: Example-based super-resolution. Comput. Graph. Appl. 22, 56–65 (2002)CrossRefGoogle Scholar
  15. 15.
    Yang, J., Wright, J., Huang, T., Ma, Y.: Image super-resolution as sparse representation of raw image patches. In: CVPR (2008)Google Scholar
  16. 16.
    Yang, J., Wright, J., Huang, T.S., Ma, Y.: Image super-resolution via sparse representation. IEEE Trans. Image Process. 19, 2861–2873 (2010)CrossRefMathSciNetGoogle Scholar
  17. 17.
    HaCohen, Y., Fattal, R., Lischinski, D.: Image upsampling via texture hallucination. In: ICCP (2010)Google Scholar
  18. 18.
    Sun, J., Zhu, J., Tappen, M.F.: Context-constrained hallucination for image super-resolution. In: CVPR (2010)Google Scholar
  19. 19.
    Timofte, R., Smet, V.D., Gool, L.V.: Anchored neighborhood regression for fast example-based super-resolution. In: ICCV (2013)Google Scholar
  20. 20.
    Zhu, Y., Zhang, Y., Yuille, A.L.: Single image super-resolution using deformable patches. In: CVPR (2014)Google Scholar
  21. 21.
    Yang, C.Y., Yang, M.H.: Fast direct super-resolution by simple functions. In: ICCV (2013)Google Scholar
  22. 22.
    Glasner, D., Bagon, S., Irani, M.: Super-resolution from a single image. In: ICCV (2009)Google Scholar
  23. 23.
    Freedman, G., Fattal, R.: Image and video upscaling from local self-examples. ACM Trans. Graph. 28, 1–10 (2010)Google Scholar
  24. 24.
    Huang, J.B., Singh, A., Ahuja, N.: Single image super-resolution from transformed self-exemplars. In: CVPR (2015)Google Scholar
  25. 25.
    Yang, J., Lin, Z., Cohen, S.: Fast image super-resolution based on in-place example regression. In: CVPR (2013)Google Scholar
  26. 26.
    Zontak, M., Irani, M.: Internal statistics of a single natural image. In: CVPR (2011)Google Scholar
  27. 27.
    Martin, D., Fowlkes, C., Tal, D., Malik, J.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: ICCV (2001)Google Scholar
  28. 28.
    Bevilacqua, M., Roumy, A., Guillemot, C., Morel, M.A.: Low-complexity single-image super-resolution based on nonnegative neighbor embedding. In: BMVC (2012)Google Scholar
  29. 29.
    Zeyde, R., Elad, M., Protter, M.: On single image scale-up using sparse-representations. Curves Surf. 6920, 711–730 (2010)CrossRefMathSciNetGoogle Scholar
  30. 30.
    Dong, C., Loy, C.C., He, K., Tang, X.: Learning a deep convolutional network for image super-resolution. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014, Part IV. LNCS, vol. 8692, pp. 184–199. Springer, Heidelberg (2014) Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  1. 1.The Graduate CenterThe City University of New YorkNew YorkUSA
  2. 2.NVIDIA ResearchSanta ClaraUSA
  3. 3.The City CollegeThe City University of New YorkNew YorkUSA

Personalised recommendations