Structural Similarity Based Anatomical and Functional Brain Imaging Fusion
Multimodal medical image fusion helps in combining contrasting features from two or more input imaging modalities to represent fused information in a single image. One of the pivotal clinical applications of medical image fusion is the merging of anatomical and functional modalities for fast diagnosis of malign tissues. In this paper, we present a novel end-to-end unsupervised learning based Convolutional neural network (CNN) for fusing the high and low frequency components of MRI-PET grayscale image pairs publicly available at ADNI by exploiting Structural Similarity Index (SSIM) as the loss function during training. We then apply color coding for the visualization of the fused image by quantifying the contribution of each input image in terms of the partial derivatives of the fused image. We find that our fusion and visualization approach results in better visual perception of the fused image, while also comparing favorably to previous methods when applying various quantitative assessment metrics.
KeywordsMedical image fusion MRI-PET Convolutional neural networks (CNN) Structural similarity index (SSIM)
This work was supported by the European Social Fund (project no. 100312752) and the Saxonian Ministry of Science and Art.
- 13.Liu, Y., Chen, X., Cheng, J., Peng, H.: A medical image fusion method based on convolutional neural networks. In: 20th International Conference on Information Fusion (Fusion), Xi’an, pp. 1–7 (2017). https://doi.org/10.23919/ICIF.2017.8009769
- 17.Johnson, K., Becker, J.: http://www.med.harvard.edu/AANLIB/home.html
- 18.Wang, Q., Shen, Y.: Performances evaluation of image fusion techniques based on nonlinear correlation measurement. In: 21st IEEE Instrumentation and Measurement Technology Conference (IEEE Cat. No. 04CH37510), vol. 1, pp. 472–475 (2004)Google Scholar