Advertisement

Bi-path network coupling for single image super-resolution

  • Yalin Yang
  • Qiegen Liu
  • Minghui Zhang
  • Yuhao WangEmail author
Article
  • 31 Downloads

Abstract

Recent researches have shown that deep convolutional neural networks can significantly boost the performance of single-image super-resolution (SISR). In particular, residual network and densely convolutional network can improve performance remarkably. The residual network enables feature re-usage and the dense skip connections enables new features exploration, which are both favor for feature extraction. In order to alleviate the vanishing-gradient problem in very deep convolution networks. In this paper, a bi-path network coupling is presented for SISR by combining the residual network and the dense skip connections in a very deep network. More specifically, the feature maps in the proposed network are split into two paths, one path is propagated in the form of residual connections, and another is propagated by dense skip connections. In addition, we input the feature maps obtained from the two paths into the coupling layer for feature fusion. Finally, the deconvolution layers are integrated into the network to upscale the feature map for significantly accelerating the network, that the mapping is learned from the low-resolution image to the high-resolution image directly. The proposed network was evaluated on four benchmark datasets and has achieved competing or even higher peak signal-to-noise ratio (PSNR) than most of state-of-the-art methods.

Keywords

Single-image super-resolution Deep convolutional neural networks Bi-path network Residual network Dense network Coupling 

Notes

Acknowledgements

The authors sincerely thank the anonymous reviewers for their valuable comments and constructive suggestions that are very helpful in the improvement of this paper. This work was supported in part by the National Natural Science Foundation of China under 61661028, 61871206, 61661031, 61463035, and the Natural Science Foundation of Jiangxi Province (20181BAB202003).

References

  1. 1.
    Bevilacqua M, Roumy A, Guillemot C, AlberiMorel ML (2012) Low-complexity single-image super-resolution based on nonnegative neighbor embedding. In BMVCGoogle Scholar
  2. 2.
    Bevilacqua M, Roumy A, Guillemot C, Morel M-LA (2014) Single-image superresolution via linear mapping of interpolated self-examples. IEEE Trans Image Process 23(12):5334–5347MathSciNetCrossRefGoogle Scholar
  3. 3.
    Blu T, Thevenaz P, Unser M (2004) Linear interpolation revitalized. IEEE Trans Image Process 13(6):710–719MathSciNetCrossRefGoogle Scholar
  4. 4.
    Chen Y, Li J, Xiao H et al. (2017) Dual path networks[C]//Advances in Neural Information Processing Systems. 4470-4478Google Scholar
  5. 5.
    Dai S, Han M, Xu W, Wu Y, Gong Y, Katsaggelos AK (2009) SoftCuts: A soft edge smoothness prior for color image super-resolution. IEEE Trans Image Process 18(5):969–981MathSciNetCrossRefGoogle Scholar
  6. 6.
    Deng J, Dong W, Socher R, Li LJ, Li K, Fei-Fei L (2009) Imagenet: A large-scale hierarchical image database, Computer Vision and Pattern Recognition 2009. CVPR 2009. IEEE Conference on, pp. 248-255Google Scholar
  7. 7.
    Dong C, Loy CC, He K, et al. (2014) Learning a deep convolutional network for image super-resolution[C]//European Conference on Computer Vision. Springer, Cham, 184-199Google Scholar
  8. 8.
    Dong C, Loy CC, Tang X (2016) Accelerating the super-resolution convolutional neural network[C]//European Conference on Computer Vision. Springer, Cham, 391-407Google Scholar
  9. 9.
    He K, Zhang X, Ren S et al. (2015) Delving deep into rectifiers: Surpassing human-level performance on imagenet classification[C]//Proceedings of the IEEE international conference on computer vision. 1026-1034Google Scholar
  10. 10.
    He K, Zhang X, Ren S et al. (2016) Deep residual learning for image recognition[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 770-778Google Scholar
  11. 11.
    Huang J, Singh A, Ahuja N (2015) Single image super-resolution from transformed self-exemplars. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Google Scholar
  12. 12.
    Huang G, Liu Z, Weinberger KQ et al (2017) Densely connected convolutional networks[C]. Proc IEEE Conf Comput Vis Pattern Recognit 1(2):3Google Scholar
  13. 13.
    Hung KW, Siu WC (2012) Robust soft-decision interpolation using weighted least squares. IEEE Trans Image Process 21(3):1061–1069MathSciNetCrossRefGoogle Scholar
  14. 14.
    Hung KW, Siu WC (2012) Single image super-resolution using iterative Wiener filter, in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), Kyoto, Japan, Mar. 1269-1272Google Scholar
  15. 15.
    Karras T, Aila T, Laine S, Lehtinen J (2017) Progressive growing of gans for improved quality, stability, and variation. submitted to ICLR 2018Google Scholar
  16. 16.
    Kim J, Kwon LJ, Mu LK (2016) Accurate image super-resolution using very deep convolutional networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1646-1654Google Scholar
  17. 17.
    Kim J, Kwon LJ, Mu LK (2016) Deeply-recursive convolutional network for image super-resolution[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 1637-1645Google Scholar
  18. 18.
    Kingma DP, Ba J (2014) Adam: A method for stochastic optimization[J]. arXiv preprint arXiv:1412.6980.Google Scholar
  19. 19.
    Lai WS, Huang JB, Ahuja N, et al. (2017) Deep laplacian pyramid networks for fast and accurate super-resolution[C]//Proc. IEEE Conf. Comput. Vis. Pattern Recognit. 624-632Google Scholar
  20. 20.
    Ledig C, Theis L, Husz` ar F, Caballero J, Cunningham A, Acosta A, Aitken A, Tejani A, Totz J, Wang Z et al. (2016) Photo-realistic single image super-resolution using a generative adversarial network. arXiv:1609.04802Google Scholar
  21. 21.
    Lim B, Son S, Kim H et al. (2017) Enhanced deep residual networks for single image super-resolution[C]//The IEEE conference on computer vision and pattern recognition (CVPR) workshops. 1(2): 4.Google Scholar
  22. 22.
    Liu H, Feng Y, Li L (2010) Multi-channel fast super-resolution image reconstruction based on matrix observation model[J]. J Harbin Inst Techn 17(2):239–246Google Scholar
  23. 23.
    Mao X, Shen C, Yang Y-B Image restoration using very deep convolutional encoder-decoder networks with symmetric skip connections. In NIPS 2016Google Scholar
  24. 24.
    Marquina A, Osher SJ (2008) Image super-resolution by TV-regularization and Bregman iteration. J Sci Comput 37(3):367–382MathSciNetCrossRefGoogle Scholar
  25. 25.
    Martin D, Fowlkes C, Tal D et al. (2001) A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics[C]//Computer Vision, 2001. ICCV 2001. Proceedings. Eighth IEEE International Conference on. IEEE, 2: 416-423Google Scholar
  26. 26.
    Ronneberger O, Fischer P, Brox T (2015) U-net: Convolutional networks for biomedical image segmentation[C]//International Conference on Medical image computing and computer-assisted intervention. Springer, Cham, 234-241Google Scholar
  27. 27.
    Schulter S, Leistner C, Bischof H (2015) Fast and accurate image upscaling with super-resolution forests[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 3791-3799.Google Scholar
  28. 28.
    Shi W, Caballero J, Ledig C, Zhuang X, Bai W, Bhatia K, Marvao AMSM de, Dawes T, ORegan D, Rueckert D (2013) Cardiac image super-resolution with global correspondence using multi-atlas patchmatch. In MICCAIGoogle Scholar
  29. 29.
    Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z (2016) Rethinking the inception architecture for computer vision. In CVPRGoogle Scholar
  30. 30.
    Tai Y, Yang J, Liu X et al. (2017) Memnet: A persistent memory network for image restoration[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 4539-4547Google Scholar
  31. 31.
    Timofte R, De SV, Van GL (2014) A+: Adjusted anchored neighborhood regression for fast super-resolution[C]//Asian Conference on Computer Vision. Springer, Cham, 111-126Google Scholar
  32. 32.
    Timofte R, Agustsson E, Van Gool L, Yang M-H, Zhang L, Lim B, Son S, Kim H, Nah S, Lee KM et al. (2017) Ntire 2017 challenge on single image super-resolution: Methods and results. In CVPRWGoogle Scholar
  33. 33.
    Tong T, Li G, Liu X et al. (2017) Image Super-Resolution Using Dense Skip Connections[C]//2017 IEEE International Conference on Computer Vision (ICCV). IEEE, 4809-4817Google Scholar
  34. 34.
    Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612CrossRefGoogle Scholar
  35. 35.
    Wang Y, Liu Q, Zhou H, Wang Y (2018) Learning multi-denoising autoencoding priors for image super-resolution. J Vis Commun Image Represent 57:152–162CrossRefGoogle Scholar
  36. 36.
    Xu R, Zhang J, Huang K (2016) Image super-resolution using two-channel convolutional neural networks[J]. J Image Graph 21(5):556–564Google Scholar
  37. 37.
    Yang CY, Yang MH (2013) Fast direct super-resolution by simple functions[C]//Computer Vision (ICCV), 2013 IEEE International Conference on. IEEE, 561-568.Google Scholar
  38. 38.
    Yang J, Wright J, Huang T, et al. (2008) Image super-resolution as sparse representation of raw image patches[C]//Proc. IEEE Conf. on Computer Vision and Pattern Recognition. 1-8Google Scholar
  39. 39.
    Yang J, Wright J, Huang TS et al (2010) Image super-resolution via sparse representation[J]. IEEE Trans Image Process 19(11):2861–2873MathSciNetCrossRefGoogle Scholar
  40. 40.
    Zeyde R, Elad M, Protter M (2010) On single image scale-up using sparse-representations. In Proceedings of the International Conference on Curves and SurfacesGoogle Scholar
  41. 41.
    Zhang K, Zuo W, Zhang L (2018) Learning a single convolutional super-resolution network for multiple degradations, in Proc. IEEE Conf. Comput. Vis. Pattern Recog.Google Scholar
  42. 42.
    Zhao Y, Wang RG, Jia W, Wang W-M, Gao W (2017) Iterative projection reconstruction for fast and efficient image upsampling. Neurocomputing 226:200–211CrossRefGoogle Scholar
  43. 43.
    Zou W, Yuen PC (2012) Very low resolution face recognition problem[J]. IEEE Trans Image Process 21(1):327–340MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.School of Electronic Information EngineeringNanchang UniversityNanchangChina

Personalised recommendations