Postoperative glioma segmentation in CT image using deep feature fusion model guided by multi-sequence MRIs
- 123 Downloads
Computed tomography (CT) and magnetic resonance imaging (MRI) are the most commonly selected methods for imaging gliomas. Clinically, radiotherapists always delineate the CT glioma region with reference to multi-modal MR image information. On this basis, we develop a deep feature fusion model (DFFM) guided by multi-sequence MRIs for postoperative glioma segmentation in CT images.
DFFM is a multi-sequence MRI–guided convolutional neural network (CNN) that iteratively learns the deep features from CT images and multi-sequence MR images simultaneously by utilizing a multi-channel CNN architecture, and then combines these two deep features together to produce the segmentation result. The whole network is optimized together via a standard back-propagation. A total of 59 CT and MRI datasets (T1/T2-weighted FLAIR, T1-weighted contrast-enhanced, T2-weighted) of postoperative gliomas as tumor grade II (n = 24), grade III (n = 18), or grade IV (n = 17) were included. Dice coefficient (DSC), precision, and recall were used to measure the overlap between automated segmentation results and manual segmentation. The Wilcoxon signed-rank test was used for statistical analysis.
DFFM showed a significantly (p < 0.01) higher DSC of 0.836 than U-Net trained by single CT images and U-Net trained by stacking the CT and multi-sequence MR images, which yielded 0.713 DSC and 0.818 DSC, respectively. The precision values showed similar behavior as DSC. Moreover, DSC and precision values have no significant statistical difference (p > 0.01) with difference grades.
DFFM enables the accurate automated segmentation of CT postoperative gliomas of profit guided by multi-sequence MR images and may thus improve and facilitate radiotherapy planning.
• A fully automated deep learning method was developed to segment postoperative gliomas on CT images guided by multi-sequence MRIs.
• CT and multi-sequence MR image integration allows for improvements in deep learning postoperative glioma segmentation method.
• This deep feature fusion model produces reliable segmentation results and could be useful in delineating GTV in postoperative glioma radiotherapy planning.
KeywordsGlioma Machine learning Magnetic resonance imaging Radiotherapy
Convolutional neural networks
Deep feature fusion model
Fluid-attenuated inversion recovery
Gross tumor volume
U-Net trained by single CT images
U-Net trained by stacking the CT and multi-sequence MR images
The authors would like to thank the reviewers for their fruitful comments.
This study has received funding by the National Natural Science Foundation of China under Grant Nos. 61671230 and 31271067, the Science and Technology Program of Guangdong Province under Grant No. 2017A020211012, the Guangdong Provincial Key Laboratory of Medical Image Processing under Grant No. 2014B030301042, and the Science and Technology Program of Guangzhou under Grant No. 201607010097.
Compliance with ethical standards
The scientific guarantor of this publication is Yu Zhang.
Conflict of interest
The authors of this manuscript declare no relationships with any companies, whose products or services may be related to the subject matter of the article.
Statistics and biometry
No complex statistical methods were necessary for this paper.
Written informed consent was waived by the Institutional Review Board.
Institutional Review Board approval was obtained.
• performed at one institution
- 2.Louis DN, Ohgaki H, Wiestler OD et al (2007) The 2007 WHO classification of tumours of the central nervous system. Acta Neuropathol 114(2):97–109Google Scholar
- 4.Warmuth-Metz M (2003) Postoperative imaging after brain tumor resection. In: Westphal M, Tonn JC, Ram Z (eds) Local therapies for glioma present status and future developments. Springer, Vienna, pp 13–20Google Scholar
- 5.Yahyanejad S, Granton PV, Lieuwes NG et al (2015) Complementary use of bioluminescence imaging and contrast-enhanced micro-computed tomography in an orthotopic brain tumor model. Mol Imaging. https://doi.org/10.2310/7290.2014.00038
- 6.Lee CH, Murtha A, Murtha A, Bistritz A, Greiner R (2005) Segmenting brain tumors with conditional random fields and support vector machines. In: Liu Y, Jiang T, Zhang C (eds) Computer Vision for Biomedical Image Applications. CVBIA 2005. Lecture Notes in Computer Science, vol 3765. Springer, Berlin, Heidelberg, pp 469–478Google Scholar
- 7.Li H, Song M, Fan Y (2010) Segmentation of brain tumors in multi-parametric MR images via robust statistic information propagation. In: Kimmel R, Klette R, Sugimoto A (eds) Computer Vision – ACCV 2010. ACCV 2010. Lecture Notes in Computer Science, vol 6495. Springer, Berlin, pp 606–617Google Scholar
- 8.Zikic D, Glocker B, Konukoglu E et al (2012) Decision forests for tissue-specific segmentation of high-grade gliomas in multi-channel MR. In: Ayache N, Delingette H, Golland P, Mori K (eds) Medical Image Computing and Computer-Assisted Intervention – MICCAI 2012. MICCAI 2012. Lecture Notes in Computer Science, vol 7512. Springer, Berlin, pp 369–376Google Scholar
- 9.Meier R, Bauer S, Slotboom J, Wiest R, Reyes M (2014) Appearance-and context-sensitive features for brain tumor segmentation. Proceedings of MICCAI BRATS Challenge, pp 20–26Google Scholar
- 13.Zikic D, Ioannou Y, Brown M, Criminisi A (2014) Segmentation of brain tumor tissues with convolutional neural networks. Proceedings of MICCAI BRATS Challenge, pp 036–039Google Scholar
- 16.Wang G, Li W, Ourselin S, Vercauteren T (2017) Automatic brain tumor segmentation using cascaded anisotropic convolutional neural networks. In: Crimi A, Bakas S, Kuijf H, Menze B, Reyes M (eds) Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries. BrainLes 2017. Lecture Notes in Computer Science, vol 10670. Springer, Cham, pp 178–190Google Scholar
- 17.Krizhevsky A, Sutskever I, Hinton GE (2012) ImageNet classification with deep convolutional neural networks. In: International Conference on Neural Information Processing Systems (NIPS), pp 1097–1105Google Scholar
- 18.Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556Google Scholar
- 21.Jones D (1994) ICRU report 50-prescribing, recording and reporting photon beam therapy. Med Phys 21(6):833–834Google Scholar
- 22.Landberg T, Chavaudra J, Dobbs J et al (1999) ICRU report 62-prescribing, recording and reporting photon beam therapy (supplement to ICRU report 50). J ICRU os32(1):1–52Google Scholar
- 23.National Comprehensive Cancer Network (2018) Clinical Practice Guidelines in Oncology for central nervous system cancers Version 1. National Comprehensive Cancer Network. Available via https://www.nccn.org/professionals/physician_gls/pdf/cns.pdf. Accessed 15 Mar 2018
- 25.Lyu S, Simoncelli EP (2008) Nonlinear image representation using divisive normalization. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, Piscataway, pp 1–8Google Scholar
- 27.Ronneberger O, Fischer P, Brox T (2015) U-Net: convolutional networks for biomedical image segmentation. In: Ayache N, Delingette H, Golland P, Mori K (eds) Medical Image Computing and Computer-Assisted Intervention – MICCAI 2012. MICCAI 2012. Lecture Notes in Computer Science, vol 7512. Springer, Berlin, pp 234–241Google Scholar
- 28.Zhang Z, Luo P, Loy CC, Tang X (2014) Facial landmark detection by deep multi-task learning. In: Fleet D, Pajdla T, Schiele B, Tuytelaars T (eds) Computer Vision – ECCV 2014. ECCV 2014. Lecture Notes in Computer Science, vol 8694.Springer, Cham, pp 94–108Google Scholar
- 31.Kingma DP, Ba J (2014) Adam: a method for stochastic optimization. arXiv:1412.6980Google Scholar
- 32.Chen L, Price RA Jr, Wang L et al (2004) MRI-based treatment planning for radiotherapy: dosimetric verification for prostate IMRT. Int J Radiat Oncol Biol Phys 60(2):636–647Google Scholar