Multimedia Tools and Applications

, Volume 77, Issue 17, pp 22649–22670 | Cite as

MRI and PET image fusion using structure tensor and dual ripplet-II transform

  • Hamid Reza ShahdoostiEmail author
  • Adel Mehrabi


Medical image fusion aims at preserving salient image features, reducing the redundancy, and increasing the interpretation quality of images in clinical applications e.g. image-guided surgery. The PET image exhibits functional characteristic with low spatial resolution, while the MRI image exhibits brain tissue anatomy with high spatial resolution. Therefore, the image fusion task is carried out to inject the structural and anatomical information of the high-resolution MRI image into the metabolic information of the PET image. This paper firstly introduces the dual ripplet-II transform (DRT) to overcome the shift variance problem caused by the ripplet-II transform. The proposed transform incorporates the dual-tree complex wavelet into the traditional ripplet-II transform. Secondly, the proposed method takes advantage of the structure tensor and DRT to effectively merge the MRI and PET images. To this end, an objective function is proposed which exploits a weighting matrix to preserve more color and spatial information. Visual and statistical analyses show that the proposed method improves the visual quality and increases the quantitative criteria based on mutual information, edge information, spatial frequency, and structural similarity.


Medical image fusion Dual ripplet-II transform Structure tensor Edge detection Weighting matrix 


  1. 1.
    Bhatnagar G, Wu QJ, Liu Z (2015) A new contrast based multimodal medical image fusion framework. Neurocomputing 157:143–152CrossRefGoogle Scholar
  2. 2.
    Bracewell RN (1989) The fourier transform. Sci Am 260(6):86–95CrossRefGoogle Scholar
  3. 3.
    Chen GY, Kégl B (2007) Image denoising with complex ridgelets. Pattern Recogn 40(2):578–585CrossRefzbMATHGoogle Scholar
  4. 4.
    Chen F, Qin F, Peng G, Chen S (2012) Fusion of remote sensing images using improved ICA mergers based on wavelet decomposition. Procedia Engineering 29:2938–2943CrossRefGoogle Scholar
  5. 5.
    Coifman RR, Donoho DL (1995) Translation-invariant de-noising. Springer, New York, pp 125–150zbMATHGoogle Scholar
  6. 6.
    Cormack AM (1981) The radon transform on a family of curves in the plane. Proc Am Math Soc 83(2):325–330MathSciNetCrossRefzbMATHGoogle Scholar
  7. 7.
    Crouse MS, Nowak RD, Baraniuk RG (1998) Wavelet-based statistical signal processing using hidden Markov models. IEEE Trans Signal Process 46(4):886–902MathSciNetCrossRefGoogle Scholar
  8. 8.
    Cui Z, Zhang G, Wu J (2009) Medical image fusion based on wavelet transform and independent component analysis. In Artificial Intelligence, 2009. JCAI'09. International Joint Conference on (pp. 480–483). IEEEGoogle Scholar
  9. 9.
    Da Cunha AL, Zhou J, Do MN (2006) The nonsubsampled contourlet transform: theory, design, and applications. IEEE Trans Image Process 15(10):3089–3101CrossRefGoogle Scholar
  10. 10.
    Dai YH, Yuan Y (1999) A nonlinear conjugate gradient method with a strong global convergence property. SIAM J Optim 10(1):177–182MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Daneshvar S, Ghassemian H (2010) MRI and PET image fusion by combining IHS and retina-inspired models. Information Fusion 11(2):114–123CrossRefGoogle Scholar
  12. 12.
    Deng C, Wang S, Chen X (2009) Remote sensing images fusion algorithm based on shearlet transform. In Environmental Science and Information Application Technology, 2009. ESIAT 2009. International Conference on (Vol. 3, pp. 451–454). IEEEGoogle Scholar
  13. 13.
    Do MN, Vetterli M (2003) The finite ridgelet transform for image representation. IEEE Trans Image Process 12(1):16–28MathSciNetCrossRefzbMATHGoogle Scholar
  14. 14.
    Do MN, Vetterli M (2005) The contourlet transform: an efficient directional multiresolution image representation. IEEE Trans Image Process 14(12):2091–2106CrossRefGoogle Scholar
  15. 15.
    Du J, Li W, Lu K, Xiao B (2016) An overview of multi-modal medical image fusion. Neurocomputing 215:3–20CrossRefGoogle Scholar
  16. 16.
    Easley G, Labate D, Lim WQ (2008) Sparse directional image representations using the discrete shearlet transform. Appl Comput Harmon Anal 25(1):25–46MathSciNetCrossRefzbMATHGoogle Scholar
  17. 17.
    Fowler JE (2005) The redundant discrete wavelet transform and additive noise. IEEE Signal Processing Letters 12(9):629–632CrossRefGoogle Scholar
  18. 18.
    Ganasala P, Kumar V (2014) CT and MR image fusion scheme in nonsubsampled contourlet transform domain. J Digit Imaging 27(3):407–418CrossRefGoogle Scholar
  19. 19.
    Ganasala P, Kumar V (2016) Feature-motivated simplified adaptive PCNN-based medical image fusion algorithm in NSST domain. J Digit Imaging 29(1):73–85CrossRefGoogle Scholar
  20. 20.
    Guo K, Labate D (2007) Optimally sparse multidimensional representation using shearlets. SIAM J Math Anal 39(1):298–318MathSciNetCrossRefzbMATHGoogle Scholar
  21. 21.
    He C, Liu Q, Li H, Wang H (2010) Multimodal medical image fusion based on IHS and PCA. Procedia Engineering 7:280–285CrossRefGoogle Scholar
  22. 22.
    James AP, Dasarathy BV (2014) Medical image fusion: a survey of the state of the art. Information Fusion 19:4–19CrossRefGoogle Scholar
  23. 23.
    Ji X, Zhang G (2015) Image fusion method of SAR and infrared image based on Curvelet transform with adaptive weighting. Multimed Tools Appl 76(17):17633–17649Google Scholar
  24. 24.
    Liu X, Mei W, Du H (2017) Structure tensor and nonsubsampled shearlet transform based algorithm for CT and MRI image fusion. Neurocomputing 235:131-139Google Scholar
  25. 25.
    Miao QG, Shi C, Xu PF, Yang M, Shi YB (2011) A novel algorithm of image fusion using shearlets. Opt Commun 284(6):1540–1547CrossRefGoogle Scholar
  26. 26.
    Patel VM, Easley GR, Healy DM (2008) A new multiresolution generalized directional filter bank design and application in image enhancement. In 2008 15th IEEE International Conference on Image Processing (pp 2816–2819). IEEEGoogle Scholar
  27. 27.
    Perona P, Malik J (1990) Scale-space and edge detection using anisotropic diffusion. IEEE Trans Pattern Anal Mach Intell 12(7):629–639CrossRefGoogle Scholar
  28. 28.
    Petrovic VS, Xydeas CS (2004) Gradient-based multiresolution image fusion. IEEE Trans Image Process 13(2):228–237CrossRefzbMATHGoogle Scholar
  29. 29.
    Piella G (2009) Image fusion for enhanced visualization: a variational approach. Int J Comput Vis 83(1):1–11CrossRefGoogle Scholar
  30. 30.
    Selesnick IW, Baraniuk RG, Kingsbury NC (2005) The dual-tree complex wavelet transform. IEEE Signal Process Mag 22(6):123–151CrossRefGoogle Scholar
  31. 31.
    Shahdoosti HR, Ghassemian H (2015) Fusion of MS and PAN images preserving spectral quality. IEEE Geosci Remote Sens Lett 12(3):611–615CrossRefGoogle Scholar
  32. 32.
    Shahdoosti HR, Ghassemian H (2016) Combining the spectral PCA and spatial PCA fusion methods by an optimal filter. Information Fusion 27:150–160CrossRefGoogle Scholar
  33. 33.
    Shahdoosti HR, Hazavei SM (2017) Image denoising in dual contourlet domain using hidden Markov tree models. Digital Signal Process 67:17-29Google Scholar
  34. 34.
    Shahdoosti HR, Khayat O (2016) Combination of anisotropic diffusion and non-subsampled shearlet transform for image denoising. Journal of Intelligent & Fuzzy Systems vol 30(6):3087–3098CrossRefzbMATHGoogle Scholar
  35. 35.
    Singh R, Khare A (2014) Fusion of multimodal medical images using Daubechies complex wavelet transform–a multiresolution approach. Information Fusion 19:49–60CrossRefGoogle Scholar
  36. 36.
    Tu TM, Su SC, Shyu HC, Huang PS (2001) A new look at IHS-like image fusion methods. Information fusion 2(3):177–186CrossRefGoogle Scholar
  37. 37.
    Velisavljevic V, Beferull-Lozano B, Vetterli M, Dragotti PL (2006) Directionlets: anisotropic multidirectional representation with separable filtering. IEEE Trans Image Process 15(7):1916–1933CrossRefGoogle Scholar
  38. 38.
    Wang Z, Simoncelli EP, Bovik AC (2003) Multiscale structural similarity for image quality assessment. In Signals, Systems and Computers, 2004. Conference Record of the Thirty-Seventh Asilomar Conference on (Vol. 2, pp 1398–1402). IEEEGoogle Scholar
  39. 39.
    Wang L, Li B, Tian LF (2014) Multi-modal medical image fusion using the inter-scale and intra-scale dependencies between image shift-invariant shearlet coefficients. Information Fusion 19:20–28CrossRefGoogle Scholar
  40. 40.
    Xiao YH, XI ZH, Hai T, Guo L (2011) Image edge detection based on nonsubsampled contourlet transform. Systems Engineering and Electronics 33(7):1668–1672Google Scholar
  41. 41.
    Xu Z (2014) Medical image fusion using multi-level local extrema. Information Fusion 19:38–48CrossRefGoogle Scholar
  42. 42.
    Xu J, Wu D (2010) Ripplet-II transform for feature extraction. In Visual Communications and Image Processing 2010 (pp. 77441R-77441R). International Society for Optics and PhotonicsGoogle Scholar
  43. 43.
    Xu J, Yang L, Wu D (2010) Ripplet: a new transform for image processing. J Vis Commun Image Represent 21(7):627–639CrossRefGoogle Scholar
  44. 44.
    Xydeas CS, Petrovic V (2000) Objective image fusion performance measure. Electron Lett 36(4):308–309CrossRefGoogle Scholar
  45. 45.
    Zhang X, Li X, Feng Y Image fusion based on simultaneous empirical wavelet transform. Multimed Tools Appl 76(6):8175–8193Google Scholar
  46. 46.
    Zhao W, Xu Z, Zhao J (2016) Gradient entropy metric and p-Laplace diffusion constraint-based algorithm for noisy multispectral image fusion. Information Fusion 27:138–149CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2017

Authors and Affiliations

  1. 1.Department of Electrical EngineeringHamedan University of TechnologyHamedanIran

Personalised recommendations