Advertisement

Image Quality Constrained GAN for Super-Resolution

  • Jingwen SuEmail author
  • Yao Peng
  • Hujun Yin
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11871)

Abstract

As one of the most important research topics in image processing, super-resolution aims to estimate high resolution images from single or multiple low resolution images taken from the same scene. With the advent of deep learning techniques, generative adversarial networks are widely adopted for solving various image processing problems including super resolution. We investigate the effect of introducing image quality constraints into the training objective function of a generative adversarial network for super resolution. Experiment results demonstrate that network performance has great potential to be improved with such constraints.

Keywords

Image quality constraints Super-resolution Generative adversarial networks 

References

  1. 1.
    Bergen, J.R., Anandan, P., Hanna, K.J., Hingorani, R.: Hierarchical model-based motion estimation. In: Sandini, G. (ed.) ECCV 1992. LNCS, vol. 588, pp. 237–252. Springer, Heidelberg (1992).  https://doi.org/10.1007/3-540-55426-2_27CrossRefGoogle Scholar
  2. 2.
    Blau, Y., Michaeli, T.: The perception-distortion tradeoff. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6228–6237 (2018)Google Scholar
  3. 3.
    Bulat, A., Yang, J., Tzimiropoulos, G.: To learn image super-resolution, use a GAN to learn how to do image degradation first. In: Proceedings of the European Conference on Computer Vision, pp. 185–200 (2018)CrossRefGoogle Scholar
  4. 4.
    Freeman, W.T., Pasztor, E.C.: Learning to estimate scenes from images. In: Proceedings of the International Conference on Advances in Neural Information Processing Systems, pp. 775–781 (1999)Google Scholar
  5. 5.
    Gerchberg, R.: Super-resolution through error energy reduction. Opt. Acta: Int. J. Opt. 21(9), 709–720 (1974)CrossRefGoogle Scholar
  6. 6.
    Goodfellow, I., et al.: Generative adversarial nets. In: Proceedings of the International Conference on Advances in Neural Information Processing Systems, pp. 2672–2680 (2014)Google Scholar
  7. 7.
    Haris, M., Shakhnarovich, G., Ukita, N.: Deep back-projection networks for super-resolution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1664–1673 (2018)Google Scholar
  8. 8.
    Irani, M., Peleg, S.: Super resolution from image sequences. In: Proceedings of the International Conference on Pattern Recognition, vol. 2, pp. 115–120 (1990)Google Scholar
  9. 9.
    Johnson, J., Alahi, A., Fei-Fei, L.: Perceptual losses for real-time style transfer and super-resolution. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9906, pp. 694–711. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-46475-6_43CrossRefGoogle Scholar
  10. 10.
    Keys, R.: Cubic convolution interpolation for digital image processing. IEEE Trans. Acoust. Speech Signal Process. 29(6), 1153–1160 (1981)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Kim, J., Kwon Lee, J., Mu Lee, K.: Accurate image super-resolution using very deep convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1646–1654 (2016)Google Scholar
  12. 12.
    Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
  13. 13.
    Kumar, V., Gupta, P.: Importance of statistical measures in digital image processing. Int. J. Emerg. Technol. Adv. Eng. 2(8), 56–62 (2012)Google Scholar
  14. 14.
    Ledig, C., et al.: Photo-realistic single image super-resolution using a generative adversarial network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4681–4690 (2017)Google Scholar
  15. 15.
    Li, X., Hu, Y., Gao, X., Tao, D., Ning, B.: A multi-frame image super-resolution method. Sig. Process. 90(2), 405–414 (2010)CrossRefGoogle Scholar
  16. 16.
    Lim, B., Son, S., Kim, H., Nah, S., Mu Lee, K.: Enhanced deep residual networks for single image super-resolution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 136–144 (2017)Google Scholar
  17. 17.
    Ma, L., Jia, X., Sun, Q., Schiele, B., Tuytelaars, T., Van Gool, L.: Pose guided person image generation. In: Proceedings of the International Conference on Advances in Neural Information Processing Systems, pp. 406–416 (2017)Google Scholar
  18. 18.
    Martin, D., Fowlkes, C., Tal, D., Malik, J., et al.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: Proceedings of the IEEE Conference on Computer Vision (2001)Google Scholar
  19. 19.
    Nguyen, N., Milanfar, P., Golub, G.: A computationally efficient superresolution image reconstruction algorithm. IEEE Trans. Image Process. 10(4), 573–583 (2001)CrossRefGoogle Scholar
  20. 20.
    Park, S.C., Park, M.K., Kang, M.G.: Super-resolution image reconstruction: a technical overview. IEEE Signal Process. Mag. 20(3), 21–36 (2003)CrossRefGoogle Scholar
  21. 21.
    Schulter, S., Leistner, C., Bischof, H.: Fast and accurate image upscaling with super-resolution forests. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3791–3799 (2015)Google Scholar
  22. 22.
    Segall, C.A., Molina, R., Katsaggelos, A.K.: High-resolution images from low-resolution compressed video. IEEE Signal Process. Mag. 20(3), 37–48 (2003)CrossRefGoogle Scholar
  23. 23.
    Shen, H., Peng, L., Yue, L., Yuan, Q., Zhang, L.: Adaptive norm selection for regularized image restoration and super-resolution. IEEE Trans. Cybern. 46(6), 1388–1399 (2015)CrossRefGoogle Scholar
  24. 24.
    Shocher, A., Cohen, N., Irani, M.: “zero-shot” super-resolution using deep internal learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3118–3126 (2018)Google Scholar
  25. 25.
    Su, K., Tian, Q., Xue, Q., Sebe, N., Ma, J.: Neighborhood issue in single-frame image super-resolution. In: Proceedings of the IEEE International Conference on Multimedia and Expo, pp. 4–pp (2005)Google Scholar
  26. 26.
    Timofte, R., Agustsson, E., Van Gool, L., Yang, M.H., Zhang, L.: Ntire 2017 challenge on single image super-resolution: methods and results. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 114–125 (2017)Google Scholar
  27. 27.
    Trinh, D.H., Luong, M., Dibos, F., Rocchisani, J.M., Pham, C.D., Nguyen, T.Q.: Novel example-based method for super-resolution and denoising of medical images. IEEE Trans. Image Process. 23(4), 1882–1895 (2014)MathSciNetCrossRefGoogle Scholar
  28. 28.
    Vural, C., Sethares, W.A.: Blind deconvolution of noisy blurred images via dispersion minimization. In: Proceedings of the International Conference on Digital Signal Processing Proceedings, vol. 2, pp. 787–790 (2002)Google Scholar
  29. 29.
    Willett, R., Jermyn, I., Nowak, R., Zerubia, J.: Wavelet-based superresolution in astronomy. Astronomical Society of the Pacific (2003)Google Scholar
  30. 30.
    Yang, M.C., Chu, C.T., Wang, Y.C.F.: Learning sparse image representation with support vector regression for single-image super-resolution. In: Proceedings of the IEEE International Conference on Image Processing, pp. 1973–1976 (2010)Google Scholar
  31. 31.
    Yang, Q., Yang, R., Davis, J., Nistér, D.: Spatial-depth super resolution for range images. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2007)Google Scholar
  32. 32.
    Yang, W., Zhang, X., Tian, Y., Wang, W., Xue, J.H., Liao, Q.: Deep learning for single image super-resolution: a brief review. IEEE Trans. Multimedia (2019).  https://doi.org/10.1109/TMM.2019.2919431
  33. 33.
    Yin, H., Hussain, I.: Independent component analysis and nongaussianity for blind image deconvolution and deblurring. Integr. Comput.-Aided Eng. 15(3), 219–228 (2008)CrossRefGoogle Scholar
  34. 34.
    Yuan, Q., Zhang, L., Shen, H.: Multiframe super-resolution employing a spatially weighted total variation model. IEEE Trans. Circuits Syst. Video Technol. 22(3), 379–392 (2011)CrossRefGoogle Scholar
  35. 35.
    Zeyde, R., Elad, M., Protter, M.: On single image scale-up using sparse-representations. In: Boissonnat, J.-D., et al. (eds.) Curves and Surfaces 2010. LNCS, vol. 6920, pp. 711–730. Springer, Heidelberg (2012).  https://doi.org/10.1007/978-3-642-27413-8_47CrossRefGoogle Scholar
  36. 36.
    Zhang, X., Tang, M., Tong, R.: Robust super resolution of compressed video. Vis. Comput. 28(12), 1167–1180 (2012)CrossRefGoogle Scholar
  37. 37.
    Zhu, J.Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2223–2232 (2017)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Department of Electrical and Electronic EngineeringThe University of ManchesterManchesterUK

Personalised recommendations