Advertisement

Multimedia Tools and Applications

, Volume 78, Issue 22, pp 32605–32629 | Cite as

A fusion algorithm for medical structural and functional images based on adaptive image decomposition

  • Jun Qin
  • Xuanjing Shen
  • Haipeng ChenEmail author
  • Yingda Lv
  • Xiaoli Zhang
Article
  • 58 Downloads

Abstract

Multimodal medical image fusion has been widely used as a powerful tool in the clinical applications because of its ability of enriching information of medical images. In this paper, a novel fusion algorithm dedicated to medical structural and functional image fusion. In the algorithm, textures from functional images are separated from the smooth component in structural images; then we need to segment the source images into two parts function-informative and function-uninformative regions by judging whether each pixel in the functional image contains informative color (black pixel is meaningless); then get the smooth version of fused image by filling the function-informative region with the color from functional image and filling the function-uninformative region with smooth component in structural images; finally, smooth version of fused image and textures from the functional image are combined to get the final fused image. The attractive features of the algorithms include its ability of both color and texture reservation and low time consumption. Experimental results demonstrate that the proposed method can obtain state-of-the-art performance for medical structural and functional image fusion.

Keywords

Medical image fusion Image decomposition Color distortion Texture perseveration 

Notes

Acknowledgements

The work was supported by National Natural Science Foundation of China (Grant No. 61672259, 61602203, 61876070, 61801190), Outstanding Young Talent Foundation of Jilin Province (Grant No. 20180520029JH) and China Postdoctoral Science Foundation (Grant No. 2017 M611323). We would also like to thank http://www.med.harvard.edu/aanlib/home.html for providing us the source medical images.

References

  1. 1.
    Ali F, El-Dokany I, Saad A, Abd El-Samie FE-S (2008) Curvelet fusion of MR and CT images. Prog Electromagn Res 3:215–224zbMATHCrossRefGoogle Scholar
  2. 2.
    Avola D, Bernardi M, Foresti GL (2019) Fusing depth and colour information for human action recognition. Multimed Tools Appl 78(5):5919–5939CrossRefGoogle Scholar
  3. 3.
    Bhatnagar G, Wu Q, Liu Z (2013) Directive contrast based multimodal medical image fusion in NSCT domain. IEEE Trans Multimed 15(5):1014–1024CrossRefGoogle Scholar
  4. 4.
    Bhatnagar G, Wu QMJ, Liu Z (2013) Human visual system inspired multi-modal medical image fusion framework. Expert Syst Appl 40(5):1708–1720CrossRefGoogle Scholar
  5. 5.
    Bhatnagar G, Wu QMJ, Liu Z (2015) A new contrast based multimodal medical image fusion framework. Neurocomputing 157:143–152CrossRefGoogle Scholar
  6. 6.
    Calvi GG, Kisil I, Mandic DP (2018) Feature fusion via tensor network summation. In: 26th European Signal Processing Conference (EUSIPCO), Rome, Italy, pp 2623–2627Google Scholar
  7. 7.
    Chen T, Zhang J, Zhang Y (2005) Remote sensing image fusion based on ridgelet transform. In: IEEE International Geoscience and Remote Sensing Symposium, Coex, Seoul, Korea, pp 1150–1153Google Scholar
  8. 8.
    Choi M (2006) A new intensity-hue-saturation fusion approach to image fusion with a tradeoff parameter. IEEE Trans Geosci Remote Sens 44(6):1672–1682CrossRefGoogle Scholar
  9. 9.
    Daneshvar S, Ghassemian H (2010) MRI and PET image fusion by combining IHS and retina-inspired models. Inf Fusion 11(2):114–123CrossRefGoogle Scholar
  10. 10.
    Das S, Kundu MK (2012) NSCT-based multimodal medical image fusion using pulse-coupled neural network and modified spatial frequency. Med Biol Eng Comput 50(10):1105–1114CrossRefGoogle Scholar
  11. 11.
    Das S, Kundu MK (2013) A neuro-fuzzy approach for medical image fusion. IEEE Trans Biomed Eng 60(12):3347–3353CrossRefGoogle Scholar
  12. 12.
    Ding M, Wei L, Wang B (2015) Research on fusion method for infrared and visible images via compressive sensing. Infrared Phys Technol 57(0):56–67Google Scholar
  13. 13.
    Farbman Z, Fattal R, Lischinski D, Szeliski R (2008) Edge-preserving decompositions for multi-scale tone and detail manipulation. ACM Trans Graph (TOG) 27(3):15–19CrossRefGoogle Scholar
  14. 14.
    Girardi D, Küng J, Kleiser R, Sonnberger M, Csillag D, Trenkler J, Holzinger A (2016) Interactive knowledge discovery with the doctor-in-the-loop: a practical example of cerebral aneurysms research. Brain Informatics 3(3):133–143CrossRefGoogle Scholar
  15. 15.
    González-Audícana M, Saleta JL, Catalán RG, García R (2004) Fusion of multispectral and panchromatic images using improved IHS and PCA mergers based on wavelet decomposition. IEEE Trans Geosci Remote Sens 42(6):1291–1299CrossRefGoogle Scholar
  16. 16.
    Harikumar V, Gajjar PP, Joshi MV, Raval MS (2014) Multiresolution image fusion: use of compressive sensing and graph cuts. IEEE J Sel Top Appl Earth Observ Remote Sens 7(5):1771–1780CrossRefGoogle Scholar
  17. 17.
    Holzinger A (2016) Interactive machine learning for health informatics: when do we need the human-in-the-loop? Brain Informatics 3(2):119–131CrossRefGoogle Scholar
  18. 18.
    Holzinger A, Plass M, Holzinger K, Crişan GC, Pintea C-M, Palade V (2016) Towards interactive Machine Learning (iML): Applying Ant Colony Algorithms to Solve the Traveling Salesman Problem with the Human-in-the-Loop Approach. In: International Conference on Availability, Reliability, and Security, Salzburg, Austria, pp 81–95Google Scholar
  19. 19.
    James AP, Dasarathy BV (2014) Medical image fusion: a survey of the state of the art. Inf Fusion 19:4–19CrossRefGoogle Scholar
  20. 20.
    Jinno T, Okuda M Multiple exposure fusion for high dynamic range image acquisition. IEEE Trans Image Process 21(1):358–365Google Scholar
  21. 21.
    Li HF, Chai Y, Yin HP, Liu GQ (2012) Multifocus image fusion and denoising scheme based on homogeneity similarity. Opt Commun 285(2):91–100CrossRefGoogle Scholar
  22. 22.
    Li S, Kang X (2012) Fast multi-exposure image fusion with median filter and recursive filter. IEEE Trans Consum Electron 58(2):626–632CrossRefGoogle Scholar
  23. 23.
    Li S, Kang X, Hu J (2013) Image fusion with guided filtering. IEEE Trans Image Process 22(7):2864–2875CrossRefGoogle Scholar
  24. 24.
    Li H, Manjunath B, Mitra SK (1995) Multisensor image fusion using the wavelet transform. Graph Models Image Process 57(3):235–245CrossRefGoogle Scholar
  25. 25.
    Li ST, Yin HT, Fang LY (2012) Group-sparse representation with dictionary learning for medical image Denoising and fusion. IEEE Trans Biomed Eng 59(12):3450–3459CrossRefGoogle Scholar
  26. 26.
    Lischinski D, Farbman Z, Uyttendaele M, Szeliski R (2006) Interactive local adjustment of tonal values. ACM Trans Graph (TOG) 25(3):646–653CrossRefGoogle Scholar
  27. 27.
    Liu Y, Liu S, Wang Z (2015) A general framework for image fusion based on multi-scale transform and sparse representation. Inf Fusion 24(0):147–164CrossRefGoogle Scholar
  28. 28.
    Liu Y, Liu SP, Wang ZF (2015) A general framework for image fusion based on multi-scale transform and sparse representation. Inf Fusion 24:147–164CrossRefGoogle Scholar
  29. 29.
    Liu Z, Yin H, Chai Y, Yang SX (2014) A novel approach for multimodal medical image fusion. Expert Syst Appl 41(16):7425–7435CrossRefGoogle Scholar
  30. 30.
    Liu Z, Yin H, Fang B, Chai Y (2013) A novel fusion scheme for visible and infrared images based on compressive sensing. Opt Commun 335(0):168–177Google Scholar
  31. 31.
    Maintz JA, Viergever MA (1998) A survey of medical image registration. Med Image Anal 2(1):1–36CrossRefGoogle Scholar
  32. 32.
    Otsu N (1975) A threshold selection method from gray-level histograms. Automatica 11(285-296):23–27Google Scholar
  33. 33.
    Pajares G, De La Cruz JM (2004) A wavelet-based image fusion tutorial. Pattern Recogn 37(9):1855–1872CrossRefGoogle Scholar
  34. 34.
    Petrovic V (2007) Subjective tests for image fusion evaluation and objective metric validation. Inf Fusion 8(2):208–216CrossRefGoogle Scholar
  35. 35.
    Petrovic V, Xydeas C (2005) Objective evaluation of signal-level image fusion performance. Opt Eng 44(8):1–8Google Scholar
  36. 36.
    Piella G, Heijmans H (2003) A new quality metric for image fusion. In: International Conference on Image Processing, Barcelona, Spain, pp 173–176Google Scholar
  37. 37.
    Qu GH, Zhang DL, Yan PF (2001) Medical image fusion by wavelet transform modulus maxima. Opt Express 9(4):184–190CrossRefGoogle Scholar
  38. 38.
    Qu GH, Zhang DL, Yan PF (2002) Information measure for performance of image fusion. Electron Lett 38(7):313–315CrossRefGoogle Scholar
  39. 39.
    Shutao L, Haitao Y, Leyuan F (2013) Remote sensing image fusion via sparse representations over learned dictionaries. IEEE Trans Geosci Remote Sens 51(9):4779–4789CrossRefGoogle Scholar
  40. 40.
    Singh S, Gupta D, Anand RS, Kumar V (2015) Nonsubsampled shearlet based CT and MR medical image fusion using biologically inspired spiking neural network. Biomed Signal Process Control 18:91–101CrossRefGoogle Scholar
  41. 41.
    Suh JW, Kwon OK, Scheinost D, Sinusas AJ, Cline GW, Papademetris X (2012) CT-PET weighted image fusion for separately scanned whole body rat. Med Phys 39(1):533–542CrossRefGoogle Scholar
  42. 42.
    Tian J, Chen L (2012) Adaptive multi-focus image fusion using a wavelet-based statistical sharpness measure. Signal Process 92(9):2137–2146CrossRefGoogle Scholar
  43. 43.
    Tu T-M, Cheng W-C, Chang C-P, Huang PS, Chang J-C (2007) Best tradeoff for high-resolution image fusion to preserve spatial details and minimize color distortion. Geosci Remote Sens Lett IEEE 4(2):302–306CrossRefGoogle Scholar
  44. 44.
    Wan T, Canagarajah N, Achim A (2008) Compressive image fusion, in International Conference on Image Processing, San Diego, CA, USA, pp 1308–1311Google Scholar
  45. 45.
    Wang Z, Bovik AC (2002) A universal image quality index. Signal Process Lett 9(3):81–84CrossRefGoogle Scholar
  46. 46.
    Wang Z, Bovik AC (2006) Modern image quality assessment. Synth Lectures ImageVideo Multimed Process 2(1):1–156CrossRefGoogle Scholar
  47. 47.
    Wang QZ, Li S, Qin H, Hao AM (2015) Robust multi-modal medical image fusion via anisotropic heat diffusion guided low-rank structural analysis. Inf Fusion 26:103–121CrossRefGoogle Scholar
  48. 48.
    Wang L, Li B, Tian LF (2014) EGGDD: an explicit dependency model for multi-modal medical image fusion in shift-invariant shearlet transform domain. Inf Fusion 19:29–37CrossRefGoogle Scholar
  49. 49.
    Wang L, Li B, Tian LF (2014) Multi-modal medical image fusion using the inter-scale and intra-scale dependencies between image shift-invariant shearlet coefficients. Inf Fusion 19:20–28CrossRefGoogle Scholar
  50. 50.
    Wang ZB, Ma Y (2008) Medical image fusion using m-PCNN. Inf Fusion 9(2):176–185CrossRefGoogle Scholar
  51. 51.
    Wong A, Bishop W (2008) Efficient least squares fusion of MRI and CT images using a phase congruency model. Pattern Recogn Lett 29(3):173–180CrossRefGoogle Scholar
  52. 52.
    Xu ZP (2014) Medical image fusion using multi-level local Extrema. Inf Fusion 19:38–48CrossRefGoogle Scholar
  53. 53.
    Yang L, Guo BL, Ni W (2008) Multimodality medical image fusion based on multiscale geometric analysis of contourlet transform. Neurocomputing 72(1-3):203–211CrossRefGoogle Scholar
  54. 54.
    Zhang X, Li X, Liu Z, Feng Y (2014) Multi-focus image fusion using image-partition-based focus detection. Signal Process 102(0):64–76CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  • Jun Qin
    • 1
    • 2
  • Xuanjing Shen
    • 1
    • 2
  • Haipeng Chen
    • 1
    • 2
    Email author
  • Yingda Lv
    • 1
    • 2
  • Xiaoli Zhang
    • 1
    • 2
  1. 1.Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of EducationJilin UniversityChangchunChina
  2. 2.College of Computer Science and TechnologyJilin UniversityChangchunChina

Personalised recommendations