Structural and functional medical image fusion using an adaptive Fourier analysis

Abstract

Medical image fusion helps in meaningfully combining, the information provided by various imaging sensors, targeting the clinical details of human organs. It is well known that the Magnetic Resonance Imaging (MRI) provides the structural content, whereas Positron Emission Tomography (PET) and Single Photon Emission Computed Tomography (SPECT) provides functional information. The structural and functional details are required in a single image, to accurately identify the abnormalities in the initial stages. We propose a combination of Fourier Decomposition Method (FDM), Principal Component Analysis (PCA) and maximum selection rule-based technique for generating MRI-PET and MRI-SPECT fused images. Here, FDM is used as an analysis tool, whereas PCA and maximum selection are used for generating the fusion mechanism. FDM helps to decompose the source images into meaningful layers called modes, based on the local features of the images. PCA a tool for dimensionality reduction preserves significant information from the image layers. Crucial details from the modes are properly combined using the rule of max selection. The proposed fusion methodology, when compared with the state-of-the-art fusion methods in terms of visual quality and objective metrics, exhibited promising performance.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

References

  1. 1.

    Anderson, CH (1988). U.S. Patent No. 4,718,104. Washington, DC: U.S. Patent and Trademark Office

  2. 2.

    Arif M, Wang G (2020) Fast curvelet transform through genetic algorithm for multimodal medical image fusion. Soft Comput 24(3):1815–1836

    Article  Google Scholar 

  3. 3.

    Bashir R, Junejo R, Qadri NN, Fleury M, Qadri MY (2019) SWT and PCA image fusion methods for multi-modal imagery. Multimed Tools Appl 78(2):1235–1263

    Article  Google Scholar 

  4. 4.

    Bavirisetti DP, Kollu V, Gang X, Dhuli R (2017) Fusion of MRI and CT images using guided image filter and image statistics. Int J Imaging Syst Technol 27(3):227–237

    Article  Google Scholar 

  5. 5.

    Bhateja V, Krishn A, Sahu A (2016) Medical image fusion in curvelet domain employing PCA and maximum selection rule, In Proceedings of the Second International Conference on Computer and Communication Technologies (pp. 1–9). Springer, New Delhi

    Google Scholar 

  6. 6.

    Burt PJ (1992) A gradient pyramid basis for pattern-selective image fusion. Proc SID 1992:467–470

    Google Scholar 

  7. 7.

    Burt P, Adelson E (1983) The Laplacian pyramid as a compact image code. IEEE Trans Commun 31(4):532–540

    Article  Google Scholar 

  8. 8.

    Candes EJ, Donoho DL (2000) Curvelets: a surprisingly effective nonadaptive representation for objects with edges. Stanford Univ Ca Dept of Statistics

  9. 9.

    Du J, Li W, Lu K, Xiao B (2016) An overview of multi-modal medical image fusion. Neurocomputing 215:3–20

    Article  Google Scholar 

  10. 10.

    Du J, Li W, Xiao B, Nawaz Q (2016) Union Laplacian pyramid with multiple features for medical image fusion. Neurocomputing 194:326–339

    Article  Google Scholar 

  11. 11.

    Du J, Li W, Xiao B (2017) Anatomical-functional image fusion by information of interest in local Laplacian filtering domain. IEEE Trans Image Process 26(12):5855–5866

    MathSciNet  Article  Google Scholar 

  12. 12.

    Guorong G, Luping X, Dongzhu F (2013) Multi-focus image fusion based on non-subsampled shearlet transform. IET Image Process 7(6):633–639

    Article  Google Scholar 

  13. 13.

    Hall DL, Llinas J (1997) An introduction to multisensor data fusion. Proc IEEE 85(1):6–23

    Article  Google Scholar 

  14. 14.

    Hall DL, Llinas J (2017) Multisensor data fusion. In: Handbook of multisensor data fusion (pp. 21-34). CRC press

  15. 15.

    He C, Liu Q, Li H, Wang H (2010) Multimodal medical image fusion based on IHS and PCA. Procedia Engineering 7:280–285

    Article  Google Scholar 

  16. 16.

    Hossny M, Nahavandi S, Creighton D (2008) Comments on'Information measure for performance of image fusion'. Electron Lett 44(18):1066–1067

    Article  Google Scholar 

  17. 17.

    James AP, Dasarathy BV (2014) Medical image fusion: a survey of the state of the art. Information fusion 19:4–19

    Article  Google Scholar 

  18. 18.

    Lewis JJ, O’Callaghan RJ, Nikolov SG, Bull DR, Canagarajah N (2007) Pixel-and region-based image fusion with complex wavelets. Information fusion 8(2):119–130

    Article  Google Scholar 

  19. 19.

    Li, G (2011). Image fusion based on color transfer technique. Image fusion and its applications, 55-72

  20. 20.

    Li H, Manjunath BS, Mitra SK (1995) Multisensor image fusion using the wavelet transform. Graphical models and image processing 57(3):235–245

    Article  Google Scholar 

  21. 21.

    Li X, He M, Roux M (2010) Multifocus image fusion based on redundant wavelet transform. IET Image Process 4(4):283–293

    Article  Google Scholar 

  22. 22.

    Li S, Yang B, Hu J (2011) Performance comparison of different multi-resolution transforms for image fusion. Information Fusion 12(2):74–84

    Article  Google Scholar 

  23. 23.

    Li S, Kang X, Fang L, Hu J, Yin H (2017) Pixel-level image fusion: a survey of the state of the art. information Fusion 33:100–112

    Article  Google Scholar 

  24. 24.

    Liu Y, Liu S, Wang Z (2015) A general framework for image fusion based on multi-scale transform and sparse representation. Information fusion 24:147–164

    Article  Google Scholar 

  25. 25.

    Matsopoulos GK, Marshall S, Brunt JNH (1994) Multiresolution morphological fusion of MR and CT images of the human brain. IEE Proceedings-Vision, Image and Signal Processing 141(3):137–142

    Article  Google Scholar 

  26. 26.

    Miao QG, Shi C, Xu PF, Yang M, Shi YB (2011) A novel algorithm of image fusion using shearlets. Opt Commun 284(6):1540–1547

    Article  Google Scholar 

  27. 27.

    Naidu VPS, Raol JR (2008) Pixel-level image fusion using wavelets and principal component analysis. Def Sci J 58(3):338–352

    Article  Google Scholar 

  28. 28.

    Nencini F, Garzelli A, Baronti S, Alparone L (2007) Remote sensing image fusion using the curvelet transform. Information fusion 8(2):143–156

    Article  Google Scholar 

  29. 29.

    Qu G, Zhang D, Yan P (2001) Medical image fusion by wavelet transform modulus maxima. Opt Express 9(4):184–190

    Article  Google Scholar 

  30. 30.

    Rockinger O (1997) Image sequence fusion using a shift-invariant wavelet transform. In: Proceedings of international conference on image processing (Vol. 3, pp. 288-291). IEEE

  31. 31.

    Sheikh HR, Bovik AC (2006) Image information and visual quality. IEEE Trans Image Process 15(2):430–444

    Article  Google Scholar 

  32. 32.

    Singh, P, & Joshi, SD (2015). Some studies on multidimensional Fourier theory for Hilbert transform, analytic signal and space-time series analysis. arXiv preprint arXiv:1507.08117

  33. 33.

    Singh, P, Joshi, SD, Patney, RK, & Saha, K (2015). The Hilbert spectrum and the energy preserving empirical mode decomposition. arXiv preprint arXiv:1504.04104

  34. 34.

    Singh P, Joshi SD, Patney RK, Saha K (2017) The Fourier decomposition method for nonlinear and non-stationary time series analysis. Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences 473(2199):20160871

    MathSciNet  MATH  Article  Google Scholar 

  35. 35.

    Stokman H, Gevers T (2007) Selection and fusion of color models for image feature detection. IEEE Trans Pattern Anal Mach Intell 29(3):371–381

    Article  Google Scholar 

  36. 36.

    Summers D (2003) Harvard whole brain atlas: www. Med. Harvard. Edu/AANLIB/home. Html. J Neurol Neurosurg Psychiatry 74(3):288–288

    Article  Google Scholar 

  37. 37.

    Tannaz A, Mousa S, Sabalan D, Masoud P (2020) Fusion of multimodal medical images using nonsubsampled shearlet transform and particle swarm optimization. Multidim Syst Sign Process 31(1):269–287

    MATH  Article  Google Scholar 

  38. 38.

    Toet A (1989) A morphological pyramidal image decomposition. Pattern Recogn Lett 9(4):255–261

    MATH  Article  Google Scholar 

  39. 39.

    Toet A (1989) Image fusion by a ration of low-pass pyramid. Pattern Recogn Lett 9(4):245–253

    MATH  Article  Google Scholar 

  40. 40.

    Toet A, Van Ruyven LJ, Valeton JM (1989) Merging thermal and visual images by a contrast pyramid. Opt Eng 28(7):287789

    Article  Google Scholar 

  41. 41.

    Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612

    Article  Google Scholar 

  42. 42.

    Wang Y, Du H, Xu J, Liu Y (2012) A no-reference perceptual blur metric based on complex edge analysis. In: 2012 3rd IEEE International Conference on Network Infrastructure and Digital Content (pp. 487-491). IEEE

  43. 43.

    Xu Z (2014) Medical image fusion using multi-level local Extrema. Information Fusion 19:38–48

    Article  Google Scholar 

  44. 44.

    Xydeas CA, Petrovic V (2000) Objective image fusion performance measure. Electron Lett 36(4):308–309

    Article  Google Scholar 

  45. 45.

    Yang L, Guo BL, Ni W (2008) Multimodality medical image fusion based on multiscale geometric analysis of contourlet transform. Neurocomputing 72(1–3):203–211

    Article  Google Scholar 

  46. 46.

    Yin M, Liu X, Liu Y, Chen X (2018) Medical image fusion with parameter-adaptive pulse coupled neural network in nonsubsampled shearlet transform domain. IEEE Trans Instrum Meas 68(1):49–64

    Article  Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Ravindra Dhuli.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Polinati, S., Dhuli, R. Structural and functional medical image fusion using an adaptive Fourier analysis. Multimed Tools Appl (2020). https://doi.org/10.1007/s11042-020-09017-y

Download citation

Keywords

  • Fourier decomposition method (FDM)
  • Fourier intrinsic band of frequencies (FIBFs)
  • Image fusion
  • Principal component analysis (PCA)
  • Maximum selection rule