Advertisement

Remote sensing images super-resolution with deep convolution networks

  • Qiong RanEmail author
  • Xiaodong Xu
  • Shizhi Zhao
  • Wei Li
  • Qian Du
Article
  • 69 Downloads

Abstract

Remote sensing image data have been widely applied in many applications, such as agriculture, military, and land use. It is difficult to obtain remote sensing images in both high spatial and spectral resolutions due to the limitation of implements in image acquisition and the law of energy conservation. Super-resolution (SR) is a technique to improve the resolution from a low-resolution (LR) to a high-resolution (HR). In this paper, a novel deep convolution network (DCN) SR method (SRDCN) is proposed. Based on hierarchical architectures, the proposed SRDCN learns an end-to-end mapping function to reconstruct an HR image from its LR version; furthermore, extensions of SRDCN based on residual learning and multi scale version are investigated for further improvement,namely Developed SRDCN(DSRDCN) and Extensive SRDCN(ESRDCN). Experimental results using different types of remote sensing data (e.g., multispectral and hyperspectral) demonstrate that the proposed methods outperform the traditional sparse representation based methods.

Keywords

Remote sensing imagery Super-resolution Convolution neural network 

Notes

Acknowledgments

This work was supported by the National Natural Science Foundation of China under Grant No. NSFC-61501017, No. NSFC-61571033, and partly by the Fundamental Research Funds for the Central Universities under Grants No. BUCTRC201401, BUCTRC201615, XK1521.

References

  1. 1.
    Anwer RM, Khan FS, van de Weijer J, Molinier M, Laaksonen J (2018) Binary patterns encoded convolutional neural networks for texture recognition and remote sensing scene classification. ISPRS J Photogramm Remote Sens 138:74–85CrossRefGoogle Scholar
  2. 2.
    Burger HC, Schuler CJ, Harmeling S (2012) Image denoising: Can plain neural networks compete with bm3d? In: 2012 IEEE conference on computer vision and pattern recognition (CVPR). IEEE, pp 2392–2399Google Scholar
  3. 3.
    Candès EJ, et al (2006) Compressive sampling. In: Proceedings of the international congress of mathematicians, vol 3, Madrid, Spain, pp 1433–1452Google Scholar
  4. 4.
    Chang H, Yeung D-Y, Xiong Y (2004) Super-resolution through neighbor embedding. In: 2004. CVPR 2004. Proceedings of the 2004 IEEE computer society conference on computer vision and pattern recognition, vol 1. IEEE, pp I–IGoogle Scholar
  5. 5.
    Chen C, Fowler J (2012) Single-image super-resolution using multihypothesis prediction. In: Asilomar conference on signals, and computers, Pacific Grove, CA, pp 608–612Google Scholar
  6. 6.
    Dong C, Loy CC, He K, Tang X (2014) Learning a deep convolutional network for image super-resolution. In: European conference on computer vision. Springer, pp 184–199Google Scholar
  7. 7.
    Dong C, Loy CC, He K, Tang X (2016) Image super-resolution using deep convolutional networks. IEEE Trans Pattern Anal Mach Intell 38(2):295–307CrossRefGoogle Scholar
  8. 8.
    Freedman G, Fattal R (2011) Image and video upscaling from local self-examples. ACM Trans Graph (TOG) 30(2):12CrossRefGoogle Scholar
  9. 9.
    Freeman WT, Pasztor EC, Carmichael OT (2000) Learning low-level vision. Int J Comput Vis 40(1):25–47CrossRefGoogle Scholar
  10. 10.
    Glasner D, Bagon S, Irani M (2009) Super-resolution from a single image. In: 2009 IEEE 12th international conference on computer vision. IEEE, pp 349–356Google Scholar
  11. 11.
    Gou S, Liu S, Yang S, Jiao L (2014) Remote sensing image super-resolution reconstruction based on nonlocal pairwise dictionaries and double regularization. IEEE J Sel Top Appl Earth Observ Remote Sens 7(12):4784–4792CrossRefGoogle Scholar
  12. 12.
    He K, Zhang X, Ren S, Sun J (2015) Deep residual learning for image recognition, arXiv:1512.03385
  13. 13.
    He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778Google Scholar
  14. 14.
    Hinton GE, Salakhutdinov RR (2006) Reducing the dimensionality of data with neural networks. Science 313(5786):504–507MathSciNetCrossRefGoogle Scholar
  15. 15.
    Jain V, Seung S (2009) Natural image denoising with convolutional networks. In: Advances in neural information processing systems, pp 769–776Google Scholar
  16. 16.
    Kasetkasem T, Arora MK, Varshney PK (2005) Super-resolution land cover mapping using a markov random field based approach. Remote Sens Environ 96 (3-4):302–314CrossRefGoogle Scholar
  17. 17.
    Keys R (1981) Cubic convolution interpolation for digital image processing. IEEE Trans Acoust Speech Signal Process 29(6):1153–1160MathSciNetCrossRefGoogle Scholar
  18. 18.
    Kim J, Kwon Lee J, Mu Lee K (2016) Accurate image super-resolution using very deep convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1646–1654Google Scholar
  19. 19.
    Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp 1097–1105Google Scholar
  20. 20.
    Ledig C, Theis L, Huszár F, Caballero J, Cunningham A, Acosta A, Aitken AP, Tejani A, Totz J, Wang Z et al (2017) Photo-realistic single image super-resolution using a generative adversarial network. CVPR 2(3):4Google Scholar
  21. 21.
    Li F, Jia X, Fraser D, Lambert A (2010) Super resolution for remote sensing images based on a universal hidden markov tree model. IEEE Trans Geosci Remote Sens 48(3):1270–1278CrossRefGoogle Scholar
  22. 22.
    Li M, Nguyen TQ (2008) Markov random field model-based edge-directed image interpolation. IEEE Trans Image Process 17(7):1121–1128MathSciNetCrossRefGoogle Scholar
  23. 23.
    Pan Z, Yu J, Huang H, Hu S, Zhang A, Ma H, Sun W (2013) Super-resolution based on compressive sensing and structural self-similarity for remote sensing images. IEEE Trans Geosci Remote Sens 51(9):4864–4876CrossRefGoogle Scholar
  24. 24.
    Pouliot D, Latifovic R, Pasher J, Duffe J (2018) Landsat super-resolution enhancement using convolution neural networks and sentinel-2 for training. Remote Sens 10(3):394CrossRefGoogle Scholar
  25. 25.
    Rhee S, Kang MG (1999) Discrete cosine transform based regularized high-resolution image reconstruction algorithm. Opt Eng 38(8):1348–1356CrossRefGoogle Scholar
  26. 26.
    Shen H, Zhang L, Huang B, Li P (2007) A map approach for joint motion estimation, segmentation, and super resolution. IEEE Trans Image Process 16 (2):479–490MathSciNetCrossRefGoogle Scholar
  27. 27.
    Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition, arXiv:1409.1556
  28. 28.
    Sun W, Yang G, Du B, Zhang L, Zhang L (2017) A sparse and low-rank near-isometric linear embedding method for feature extraction in hyperspectral imagery classification. IEEE Trans Geosci Remote Sens 55(7):4032–4046CrossRefGoogle Scholar
  29. 29.
    Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1–9Google Scholar
  30. 30.
    Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z (2016) Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2818–2826Google Scholar
  31. 31.
    Tsai R, Huang T (1984) Multiframe image restoration and registration. Adv Comput Vis Image Process 1(2):317–339Google Scholar
  32. 32.
    Wang L, Xiang S, Meng G, Wu H, Pan C (2013) Edge-directed single-image super-resolution via adaptive gradient magnitude self-interpolation. IEEE Trans Circ Syst Video Technol 23(8):1289–1299CrossRefGoogle Scholar
  33. 33.
    Wang Z, Liu D, Yang J, Han W, Huang T (2015) Deep networks for image super-resolution with sparse prior. In: Proceedings of the IEEE international conference on computer vision, pp 370–378Google Scholar
  34. 34.
    Yang J, Wright J, Huang T, Ma Y (2008) Image super-resolution as sparse representation of raw image patches. In: 2008. CVPR 2008. IEEE conference on computer vision and pattern recognition. IEEE, pp 1–8Google Scholar
  35. 35.
    Yuan Y, Zheng X, Lu X (2017) Hyperspectral image superresolution by transfer learning. IEEE J Sel Top Appl Earth Observ Remote Sens 10(5):1963–1974CrossRefGoogle Scholar
  36. 36.
    Zeng K, Ding S, Jia W (2018) Single image super-resolution using a polymorphic parallel cnn. Appl Intell 2018:1–9.  https://doi.org/10.1007/s10489-018-1270-7 CrossRefGoogle Scholar
  37. 37.
    Zhang H, Huang B (2011) Scale conversion of multi sensor remote sensing image using single frame super resolution technology. In: 2011 19th international conference on geoinformatics. IEEE, pp 1–5Google Scholar
  38. 38.
    Zhang H, Zhang L, Shen H (2012) A super-resolution reconstruction algorithm for hyperspectral images. Signal Process 92(9):2082–2096CrossRefGoogle Scholar
  39. 39.
    Zhang Y, Wu W, Dai Y, Yang X, Yan B, Lu W (2013) Remote sensing images super-resolution based on sparse dictionaries and residual dictionaries. In: 2013 IEEE 11th international conference on dependable, autonomic and secure computing (DASC). IEEE, pp 318–323Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.College of Information Science and TechnologyBeijing University of Chemical TechnologyBeijingChina
  2. 2.Department of Electrical and Computer EngineeringMississippi State UniversityMississippi StateUSA

Personalised recommendations