Abstract
Multi-modal structural MRI has been widely used for presurgical glioma grading for treatment planning. Despite providing complementary information, a complete set of high-resolution multi-modality data is costly and often impossible to acquire in clinical settings (although T1 MRI is more commonly acquired). To leverage more comprehensive multimodality information for better glioma grading instead of doing so with T1 MRI data only, we introduce a three-dimensional common feature learning-based context-aware generative adversarial network (CoCa-GAN) for multimodal MRI data synthesis based on T1 MRI and use the comprehensive features from a common feature space to achieve a clinically feasible glioma grading with limited imaging modality. The common feature space is first learned by simultaneously utilizing four MRI modalities with the adversarial learning and context-aware learning, where the inter-modality relationships and lesion-specific features can be explicitly encoded. Then, the domain (modality) invariant information represented in the common space is leveraged to synthesize the missing modalities for a joint prediction of glioma grades (high- vs. low-grade). Furthermore, Gradient-weighted Class Activation Mapping (GradCAM) is utilized to provide interpretability to the factors that contribute to the grading, for potential clinical usage. Results demonstrate that the common feature learning achieves more accurate glioma grading than simply using single modality data and leads to a comparable performance to that with complete modalities as inputs. Our method offers a highly feasible solution to clinical practice where multi-modality data is often unavailable.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Sengupta, A., Ramaniharan, A.K., Gupta, R.K., Agarwal, S., Singh, A.: Glioma grading using a machine-learning framework based on optimized features obtained from T1 perfusion mri and volumes of tumor components. J. Magn. Reson. Imaging 50, 1295–1306 (2019)
Goodfellow, I., Pouget-Abadie, J., Mirza, M., et al.: Generative adversarial nets. In: Advances in Neural Information Processing Systems (2014)
Wang, Y., Yu, B., Wang, L., et al.: 3D conditional generative adversarial networks for high-quality PET image estimation at low dose. Neuroimage 174, 550–562 (2018)
Nie, D., Trullo, R., Lian, J., et al.: Medical image synthesis with deep convolutional adversarial networks. IEEE Trans. Biomed. Eng. 65(12), 2720–2730 (2018)
Pan, Y., Liu, M., Lian, C., Zhou, T., Xia, Y., Shen, D.: Synthesizing missing PET from MRI with cycle-consistent generative adversarial networks for alzheimer’s disease diagnosis. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11072, pp. 455–463. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00931-1_52
Chartsias, A., Joyce, T., Giuffrida, M.V., Tsaftaris, S.A.: Multimodal MR synthesis via modality-invariant latent representation. IEEE Trans. Med. Imaging 37(3), 803–814 (2018)
Yu, B., Zhou, L., Wang, L., Shi, Y., Fripp, J., Bourgeat, P.: Ea-GANs: edge-aware Generative Adversarial Networks for Cross-modality MR Image Synthesis. IEEE Trans. Med. Imaging 38(7), 1750–1762 (2019)
Pereira, S., Meier, R., Alves, V., Reyes, M., Silva, C.A.: Automatic brain tumor grading from mri data using convolutional neural networks and quality assessment. In: Stoyanov, D., et al. (eds.) MLCN/DLF/IMIMIC -2018. LNCS, vol. 11038, pp. 106–114. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-02628-8_12
Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., Batra, D.: Grad-CAM: visual explanations from deep networks via gradient-based localization. In: ICCV (2017)
Menze, B.H., Jakab, A., Bauer, S., et al.: The multimodal brain tumor image segmentation benchmark (BRATS). IEEE Trans. Med. Imaging 34(10), 1993–2024 (2015)
Acknowledgements
ZJ, HZ, and DS were supported in part by an NIH grant AG041721. PH and DL were supported in part by the Taishan Scholars Project of Shandong Province (Tsqn20161023) and the Primary Research and Development Plan of Shandong Province (No. 2018GGX101018).
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Huang, P. et al. (2019). CoCa-GAN: Common-Feature-Learning-Based Context-Aware Generative Adversarial Network for Glioma Grading. In: Shen, D., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2019. MICCAI 2019. Lecture Notes in Computer Science(), vol 11766. Springer, Cham. https://doi.org/10.1007/978-3-030-32248-9_18
Download citation
DOI: https://doi.org/10.1007/978-3-030-32248-9_18
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-32247-2
Online ISBN: 978-3-030-32248-9
eBook Packages: Computer ScienceComputer Science (R0)