Improving performance of medical image fusion using histogram, dictionary learning and sparse representation

  • Yi LiEmail author
  • Zhihan LvEmail author
  • Junli Zhao
  • Zhenkuan Pan


Medical image fusion has attracted much attention in recent years, which aims to fuse different medical images into a more informative and clearer one. The fused image is able to help doctors to diagnose diseases rapidly and effectively. Among numerous fusion methods, sparse-representation-based image fusion is a new concept that has emerged over the past several years. However, the high-frequency components of low-resolution and the high-frequency components of source images are obtained equally, and sparse coefficients are solved by a minimization problem. As a result, it ignores the correlation between high-frequency components of low-resolution and the high-frequency components of source images, and solutions to the L0-norm minimization problem. To address these issues, we propose a new image fusion method based on histogram similarity and multi-view weighted sparse representation. By introducing a histogram similarity, different weights are assigned to the high-frequency components of low-resolution and the high-frequency components of source images to efficiently harness the complementary information. In addition, sparse coefficients solved by the L1-norm minimization problem are more accurate. This technique is further incorporated into medical image fusion. Experimental results demonstrate that the proposed method achieves state-of-the-art performance in terms of both visual quality and quantitative evaluation metrics.


Image fusion Histogram similarity Sparse representation 



The authors first sincerely thank the editors and anonymous reviewers for their constructive comments and suggestions, which are of great value to us. The authors would also like to thank Prof. Xiaojun Wu from Jiangnan University and researcher Zhenhua Feng from University of Surrey.

This work was supported in part by the National Natural Science Foundation of China under Grants 61702293.This work was supported in part by the Shandong Provincial Natural Science Foundation of China under Grants ZR2017QF015.


  1. 1.
    Banerjee R, Chatterjee S, Bit SD (2019) Performance of a partial discrete wavelet transform based path merging compression technique for wireless multimedia sensor networks[J]. Wirel Pers Commun 104(1):57–71CrossRefGoogle Scholar
  2. 2.
    Bar-Sinai Y, Brenner M, Getreuer P, et al. (2018) Using image super-resolution techniques as a coarse-graining method for physical systems[J]. Bull Am Phys SocGoogle Scholar
  3. 3.
    Chen S, Lu Y, Gao Q et al. (2018) Image fusion based on morphological component analysis via gradient[C]//2018 2nd IEEE advanced information management, communicates, electronic and automation control conference (IMCEC). IEEE: 1034–1037Google Scholar
  4. 4.
    Cheng B, Powell WB (2018) Co-optimizing battery storage for the frequency regulation and energy arbitrage using multi-scale dynamic programming[J]. IEEE Trans Smart Grid 9(3):1997–2005Google Scholar
  5. 5.
    Chipman LJ, Orr TM, Graham LN (1995) Wavelets and image fusion. Proc of Int Conf on Image Processing Los Aiamitos:IEEE Computer Society,:248–251Google Scholar
  6. 6.
    Dai W, Li Y, Zou J et al (2018) Fully decomposable compressive sampling with joint optimization for multidimensional sparse representation[J]. IEEE Trans Signal Process 66(3):603–616MathSciNetCrossRefzbMATHGoogle Scholar
  7. 7.
    Daniel E (2018) Optimum wavelet based homomorphic medical image fusion using hybrid genetic–Grey wolf optimization algorithm[J]. IEEE Sensors JGoogle Scholar
  8. 8.
    Datta R, Joshi D, Li J, Wang JZ (2008) Image retrieval. ideas, influences, and trends of the new age. ACM Comput Survey 40(2):1–60CrossRefGoogle Scholar
  9. 9.
    Ding R, Wang X, Shang K, et al. (2018) Sparse representation-based intuitionistic fuzzy clustering approach to find the group intra-relations and group leaders for large-scale decision making[J]. IEEE Trans Fuzzy SystGoogle Scholar
  10. 10.
    Ding F, Meng D, Dai J et al (2018) Least squares based iterative parameter estimation algorithm for stochastic dynamical systems with ARMA noise using the model equivalence[J]. Int J Control Autom Syst 16(2):630–639CrossRefGoogle Scholar
  11. 11.
    Ding Z, Zhou D, Nie R et al (2018) Infrared and visible image fusion using modified PCNN and visual saliency detection[C]//2018 international conference on image and video processing, and artificial intelligence. International Society for Optics and Photonics 10836:108360EGoogle Scholar
  12. 12.
    Farid MS, Mahmood A, Al-Maadeed SA (2019) Multi-focus image fusion using content adaptive blurring[J]. Inform Fusion 45:96–112CrossRefGoogle Scholar
  13. 13.
    Fu W, Li S, Fang L et al (2018) Contextual online dictionary learning for hyperspectral image classification[J]. IEEE Trans Geosci Remote Sens 56(3):1336–1347CrossRefGoogle Scholar
  14. 14.
    Gharbia R, Hassanien AE, El-Baz AH et al. (2018) Multi-spectral and panchromatic image fusion approach using stationary wavelet transform and swarm flower pollination optimization for remote sensing applications[J]. Futur Gener Comput SystGoogle Scholar
  15. 15.
    Hagargi PA, Shubhangi DC (2018) Brain tumor MR image fusion using Most dominant features extraction from wavelet and Curvelet transforms[J]. Brain 5(05)Google Scholar
  16. 16.
    Hua G, Zhao L, Zhang H, et al. (2018) Random matching pursuit for image watermarking[J]. IEEE Transactions on Circuits and Systems for Video TechnologyGoogle Scholar
  17. 17.
    Kumar KS, Ram KNNS, Kiranmai K, et al. (2018) Denoising of Iris image using stationary wavelet transform[C]//2018 second international conference on inventive communication and computational technologies (ICICCT). IEEE: 1232–1237Google Scholar
  18. 18.
    Lai W S, Huang J B, Ahuja N, et al. (2018) Fast and accurate image super-resolution with deep laplacian pyramid networks[J]. IEEE Trans Pattern Anal Mach IntellGoogle Scholar
  19. 19.
    Lan X, Zhang S, Yuen PC et al (2018) Learning common and feature-specific patterns: a novel multiple-sparse-representation-based tracker[J]. IEEE Trans Image Process 27(4):2022–2037MathSciNetCrossRefzbMATHGoogle Scholar
  20. 20.
    Li S, Ye W, Liang H, et al. (2018) K-SVD based Denoising algorithm for DoFP polarization image sensors[C]//circuits and systems (ISCAS), 2018 IEEE international symposium on. IEEE : 1–5Google Scholar
  21. 21.
    Li J, Yuan G, Fan H (2019) Multifocus image fusion using wavelet-domain-based deep CNN[J]. Computational Intelligence and Neuroscience 2019Google Scholar
  22. 22.
    Liang RZ, Shi L, Wang H et al. (2016) Optimizing top precision performance measure of content-based image retrieval by learning similarity function[C]//pattern recognition (ICPR), 2016 23rd international conference on. IEEE : 2954–2958Google Scholar
  23. 23.
    Liu S, Chen J, Rahardja S (2019) A new multi-focus image fusion algorithm and its efficient implementation[J]. IEEE Transactions on Circuits and Systems for Video TechnologyGoogle Scholar
  24. 24.
    Lucas A, Lopez-Tapiad S, Molinae R, et al. (2019) Generative adversarial networks and perceptual losses for video super-resolution[J]. IEEE Trans Image ProcessGoogle Scholar
  25. 25.
    Ma K, Zeng K, Wang Z (2015) Perceptual quality assessment for multi-exposure image fusion[J]. IEEE Trans Image Process 24(11):3345–3356MathSciNetCrossRefzbMATHGoogle Scholar
  26. 26.
    Ma J, Yu W, Liang P et al (2019) FusionGAN: a generative adversarial network for infrared and visible image fusion[J]. Inform Fusion 48:11–26CrossRefGoogle Scholar
  27. 27.
    Mancini M, Costante G, Valigi P et al (2018) J-MOD 2: joint monocular obstacle detection and depth estimation[J]. IEEE Robot Auto Lett 3(3):1490–1497CrossRefGoogle Scholar
  28. 28.
    Massa A, Bertolli M, Gottardi G et al. (2018) Compressive sensing as applied to antenna arrays: synthesis, diagnosis, and processing[C]//circuits and systems (ISCAS), 2018 IEEE international symposium on. IEEE : 1–5Google Scholar
  29. 29.
    Nguyen LD, Lin D, Lin Z, et al. (2018) Deep CNNs for microscopic image classification by exploiting transfer learning and feature concatenation[C]//circuits and systems (ISCAS), 2018 IEEE international symposium on. IEEE : 1–5Google Scholar
  30. 30.
    Piella G, Heijmans H (2003) A new quality metric for image fusion. Process IEEE Int Conf Image Process 3:173–176Google Scholar
  31. 31.
    Ra PK, Karmakar G, Eddy FYS et al. (2018) Dual tree complex wavelet transform based detection of power quality disturbances[C]//2018 IEEE innovative smart grid technologies-Asia (ISGT Asia). IEEE : 1177–1182Google Scholar
  32. 32.
    Raj A, Pradhan J, Pal AK, et al. (2018) Multi-scale image fusion scheme based on non-sub sampled contourlet transform and four neighborhood Shannon entropy scheme[C]//2018 4th international conference on recent advances in information technology (RAIT). IEEE : 1–6Google Scholar
  33. 33.
    Srinivasan A, Battacharjee P, Prasad A, et al. (2018) Brain MR image analysis using discrete wavelet transform with fractal feature analysis[C]//2018 second international conference on electronics, communication and aerospace technology (ICECA). IEEE : 1660–1664Google Scholar
  34. 34.
    Wang Z, Bovik AC (2002) A universal image quality index. IEEE Signal Process Lett 9(3):81–84CrossRefGoogle Scholar
  35. 35.
    Wang X, Yu K, Dong C, et al. (2018) Recovering realistic texture in image super-resolution by deep spatial feature transform[J]. arXiv preprint arXiv:1804.02815Google Scholar
  36. 36.
    Wang Y, Zheng Q, Heng PA (2018) Online robust projective dictionary learning: shape modeling for MR-TRUS registration[J]. IEEE Trans Med Imaging 37(4):1067–1078CrossRefGoogle Scholar
  37. 37.
    Wang M, Zhou S, Yang Z et al (2019) Image fusion based on wavelet transform and gray-level features[J]. J Mod Opt 66(1):77–86CrossRefGoogle Scholar
  38. 38.
    Wen Y, Sheng B, Li P et al (2019) Deep color guided coarse-to-fine convolutional network cascade for depth image super-resolution[J]. IEEE Trans Image Process 28(2):994–1006MathSciNetCrossRefzbMATHGoogle Scholar
  39. 39.
    Yan C, Xie H, Liu S et al (2018) Effective Uyghur language text detection in complex background images for traffic prompt identification[J]. IEEE Trans Intell Transp Syst 19(1):220–229CrossRefGoogle Scholar
  40. 40.
    Yang J, Chen X, Hu Y H, et al. (2018) Adaptive visual target tracking based on label consistent K-Svd sparse coding and kernel particle filter[C]//2018 IEEE international conference on acoustics, speech and signal processing (ICASSP). IEEE: 1633–1637Google Scholar
  41. 41.
    Yazdi SV, Douzal-Chouakria A (2018) Time warp invariant kSVD: sparse coding and dictionary learning for time series under time warp[J]. Pattern Recogn Lett 112:1–8CrossRefGoogle Scholar
  42. 42.
    Ye Q, Zhao H, Li Z et al (2018) L1-norm distance minimization-based fast robust twin support vector $ k $-plane clustering[J]. IEEE Trans Neural Netw Learn Syst 29(9):4494–4503CrossRefGoogle Scholar
  43. 43.
    Yin H (2018) Tensor sparse representation for 3-D medical image fusion using weighted average rule[J]. IEEE Trans Biomed EngGoogle Scholar
  44. 44.
    Yin H, Li S, Fang L (2013) Simultaneous image fusion and super-resolution using sparse representation[J]. Inform Fusion 14(3):229–240CrossRefGoogle Scholar
  45. 45.
    Zhang Q, Guo BL (2009) Multifocus image fusion using the nonsubsampled contourlet transform. Signal Process 89(7):1334–1346CrossRefzbMATHGoogle Scholar
  46. 46.
    Zhang M, Li W, Du Q (2018) Diverse region-based CNN for hyperspectral image classification[J]. IEEE Trans Image Process 27(6):2623–2634MathSciNetCrossRefzbMATHGoogle Scholar
  47. 47.
    Zhang Y, Chandler DM, Mou X (2018) Quality assessment of screen content images via convolutional-neural-network-based synthetic/natural segmentation[J]. IEEE Trans Image Process 27(10):5113–5128MathSciNetCrossRefzbMATHGoogle Scholar
  48. 48.
    Zhao W, Lu H, Wang D (2018) Multisensor image fusion and enhancement in spectral total variation domain[J]. IEEE Trans Multimed 20(4):866–879CrossRefGoogle Scholar
  49. 49.
    Zhou T, Liu F, Bhaskar H et al (2018) Robust visual tracking via online discriminative and low-rank dictionary learning[J]. IEEE Trans Cybernet 48(9):2643–2655CrossRefGoogle Scholar
  50. 50.
    Zhu Z, Zheng M, Qi G, et al. (2019) A phase congruency and local Laplacian energy based multi-modality medical image fusion method in NSCT domain[J]. IEEE AccessGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.College of Data Science Software EngineeringQingdao UniversityQingdaoPeople’s Republic of China
  2. 2.College of Computer Science & TechnologyQingdao UniversityQingdaoPeople’s Republic of China

Personalised recommendations