A NSST Pansharpening method based on directional neighborhood correlation and tree structure matching

  • Xianghai WangEmail author
  • Jingzhe Tao
  • Yutong Shen
  • Shifu Bai
  • Chuanming SongEmail author


In this paper, we propose a multispectral (MS) remote sensing image pansharpening method based on non-subsampled shearlet transform (NSST). By analyzing the NSST high-frequency coefficients correlation of several datasets which are fromWorldView-2 (WV2) and Quick-Bird (QB), we verified that the high-frequency coefficients based on NSST have strong directional neighborhood correlation within the same sub-band and parent-children correlation between sub-bands in the same direction. In order to combine these two kinds of correlations, we design a type of weighted directional neighborhood templates which can be used for any number of direction sub-bands to depict the direction correlation, and use the tree structure to model the correlation between parent-children coefficients. Experiments show that the proposed method in this paper can provide a fused MS image with high spatial resolution, which can provide convenience for subsequent applications such as classification and target recognition.


Remote sensing image Pansharpening NSST Directional neighborhoods matching degree Binary-tree matching degree 



À trous wavelet transform with model 3


Component substitution


Contourlet and IHS


Directional neighborhood matching


Erreur relative globale a dimensionnelle de synthses


Generalized IHS


High spatial resolution




Low spatial resolution


Multiresolution analysis




Mean structural similarity


Nonlinear IHS


Non-subsampled shearlet transform




Principal component analysis




Root mean square error


Spectral angle mapper


Tree structure matching





This research has been funded by the National Natural Science Foundation of China (Grant Nos. 41671439 and 61402214), and Innovation Team Support Program of Liaoning Higher Education Department (LT2017013).


  1. 1.
    Aiazzi B, Alparone L, Baronti S, Garzelli A, Selva M (2006) MTF-tailored multiscale fusion of high-resolution MS and Pan imagery. Photogramm Eng Remote Sens 72(5):591–596Google Scholar
  2. 2.
    Aiazzi B, Baronti S, Selva M (2007) Improving component substitution pansharpening through multivariate regression of MS+Pan data. IEEE Trans Geosci Remote Sens 45(10):3230–3239Google Scholar
  3. 3.
    Alparone V, Aiazzi B, Baronti S, and Garzelli S (2015) Remote sensing image fusion. CRC PressGoogle Scholar
  4. 4.
    Burtand PJ, Adelson EH (1983) The laplacian pyramid as a compact image code. IEEE Trans Commun 31(4):532–540Google Scholar
  5. 5.
    Candes E, Demanet L, Donoho D (2006) Fast discrete Curvelet transforms. Multiscale Modeling & Simulation Journal 5(3):861–899MathSciNetzbMATHGoogle Scholar
  6. 6.
    Carper W, Lillesand T, Kiefer R (1990) The use of intensity-hue-saturation transformations for merging SPOT panchromatic and multi-spectral image data. Photogramm Eng Remote Sens 56(4):459–467Google Scholar
  7. 7.
    Chavez PS Jr, Kwarteng AW (1989) Extracting spectral contrast in Landsat thematic mapper image data using selective principal component analysis. Photogramm Eng Remote Sens 55(3):339–348Google Scholar
  8. 8.
    Chen C, Jafari R, Kehtarnavaz N (2017) A survey of depth and inertial sensor fusion for human action recognition. Multimedia Tools & Applications 76(3):4405–4425Google Scholar
  9. 9.
    Choi J, Yu K, Kim Y (2011) A new adaptive component-substitution based satellite image fusion by using partial replacement. IEEE Transaction on Geoscience and Remote Sensing 49(1):295–309Google Scholar
  10. 10.
    Do MN, Vetterli M (2005) The contourlet transform: an efficient directional multiresolution image representation. IEEE Trans Image Process 14(12):2091–2106Google Scholar
  11. 11.
    Dou W, Chen Y, Li X, Sui D (2007) A general framework for component substitution image fusion: an implementation using fast image fusion method. Comput Geosci 33(2):219–228Google Scholar
  12. 12.
    Easley GR, Labate D, Lim WQ (2008) Sparse directional image representation using the discrete shearlet transforms. Appl Comput Harmon Anal 25(1):25–46MathSciNetzbMATHGoogle Scholar
  13. 13.
    Ferraris V, Dobigeon N, and Wei Q (2017) Change detection between multi-band images using a robust fusion-based approach. IEEE International Conference on Acoustics, Speech and Signal Processing. IEEE, 3346–3350Google Scholar
  14. 14.
    Garzelli A, Nencini F (2009) Hypercomplex quality assessment of multi−/hyper-spectral images. IEEE Geoscience Remote Sensing Letter 6(4):662–665Google Scholar
  15. 15.
    Ghahremani M, Ghassemian H (2015) Remote sensing image fusion using ripplet transform and compressed sensing. IEEE Geosci Remote Sens Lett 12(3):502–506Google Scholar
  16. 16.
    Ghahremani M, Ghassemian H (2016) Nonlinear IHS: a promising method for Pan-sharpening. IEEE Geoscience and Remote Sensing Letter 13(11):1606–1610Google Scholar
  17. 17.
    Kahraman S, Erturk A (2018) Review and performance comparison of pansharpening algorithms for RASAT images. Istanbul University-Journal of Electrical and Electronics Engineering 18(1):109–120Google Scholar
  18. 18.
    Lim WQ (2013) NonseparableShearlet transform. IEEE Trans Image Process 22(5):2056–2065MathSciNetzbMATHGoogle Scholar
  19. 19.
    Mallat S (1989) A theory for multiresolution signal decomposition: the wavelet representation. IEEE Trans Pattern Anal Mach Intell 11(7):674–693zbMATHGoogle Scholar
  20. 20.
    Massip P, Blanc P, Wald L (2012) A method to better account for modulation transfer functions in ARSIS-based pansharpening methods. IEEE Trans Geosci Remote Sens 50(3):800–808Google Scholar
  21. 21.
    Otazu X, Gonzalez-Audicana M, Fors O, Nunez J (2005) Introduction of sensor spectral response into image fusion methods, application to wavelet-based methods. IEEE Trans Geosci Remote Sens 43(10):2376–2385Google Scholar
  22. 22.
    Otsu N (1979) A threshold selection method from gray-level histograms. IEEE Transactions on Systems, Man and Cybernetics 9(1):62–66Google Scholar
  23. 23.
    Paramanandham N, Rajendiran K (2017) Multi sensor image fusion for surveillance applications using hybrid image fusion algorithm. Multimedia Tools & Applications 76(7):1–32Google Scholar
  24. 24.
    Po DDY, Do MN (2006) Directional multiscale modeling of images using the contourlet transform. IEEE Trans Image Process 15(6):1610–1620MathSciNetGoogle Scholar
  25. 25.
    Rahmani S, Strait M, Merkurjev D, Moeller M, Wittman T (2010) An adaptive IHS pansharpening method. IEEE Geoscience & Remote Sensing Letters 7(4):746–750Google Scholar
  26. 26.
    Ranchin T, Wald L (2000) Fusion of high spatial and spectral resolution images: the ARSIS concept and its implementation. Photogramm Eng Remote Sens 66(1):49–61Google Scholar
  27. 27.
    Shah VP, Younan NH, King PL (2008) An efficient pan-sharpening method via a combined adaptive-PCA approach and contourlets. IEEE Transaction on Geoscience and Remote Sensing 46(5):1323–1335Google Scholar
  28. 28.
    Tu TM, Suand SC, Shyu HC (2001) A new look at IHS-like image fusion methods. Inf Fusion 2(3):177–186Google Scholar
  29. 29.
    Upla KP, Joshi MV, Gajjar PP (2015) An edge preserving multiresolution fusion: use of contourlet transform and MRF prior. IEEE Trans Geosci Remote Sens 53(6):3210–3220Google Scholar
  30. 30.
    Vivone G, Alparone L, Chanussot J, Mura MD, Garzelli A, Licciardi GA, Restaino R, Wald L (2015) A critical comparison among pansharpening algorithms. IEEE Geoscience and Remote Sensing 53(5):2565–2586Google Scholar
  31. 31.
    Wald L (2002) Data fusion: definitions and architectures—fusion of images of different spatial resolutions. Paris, France: Les Presses de l’ Écoledes MinesGoogle Scholar
  32. 32.
    Wald L, Ranchin T, Mangolini M (1997) Fusion of satellite images of different spatial resolutions: assessing the quality of resulting images. Amer Soc Photogramm Remote Sensing 63(6):691–699Google Scholar
  33. 33.
    Wang Z, BovikA C (2002) A universal image quality index. IEEE Signal Process Letter 9(3):81–84Google Scholar
  34. 34.
    Wang XH, Wei TT, Zhou ZG (2010) Remote sensing image fusion method based on the contourlet coefficients’ correlativity of directional region. Journal of Remote Sensing 14(5):905–916Google Scholar
  35. 35.
    Wang XH, Shen YT, Zhou ZG (2015) An image fusion algorithm based on lifting wavelet transform. J Opt 17(5):225–229Google Scholar
  36. 36.
    Yilun C, Lee H, and Kwon H (2017) Enhanced object detection via fusion with prior beliefs from image classification. IEEE International Conference on Image Processing IEEE920–924Google Scholar
  37. 37.
    Yuhas R H, Alexander F H, Goetz, Boardman J W (1992) Discrimination among semi-arid landscape end members using the spectral angle mapper (SAM) algorithm. In Proc. Summaries 3rd Annu. JPL Airborne Geosci. Workshop 147–149Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.School of Computer and Information TechnologyLiaoning Normal UniversityDalianChina
  2. 2.School of Urban and Environmental SciencesLiaoning Normal UniversityDalianChina

Personalised recommendations