European Radiology

, Volume 30, Issue 2, pp 823–832 | Cite as

Postoperative glioma segmentation in CT image using deep feature fusion model guided by multi-sequence MRIs

  • Fan Tang
  • Shujun Liang
  • Tao Zhong
  • Xia Huang
  • Xiaogang Deng
  • Yu ZhangEmail author
  • Linghong ZhouEmail author
Head and Neck



Computed tomography (CT) and magnetic resonance imaging (MRI) are the most commonly selected methods for imaging gliomas. Clinically, radiotherapists always delineate the CT glioma region with reference to multi-modal MR image information. On this basis, we develop a deep feature fusion model (DFFM) guided by multi-sequence MRIs for postoperative glioma segmentation in CT images.


DFFM is a multi-sequence MRI–guided convolutional neural network (CNN) that iteratively learns the deep features from CT images and multi-sequence MR images simultaneously by utilizing a multi-channel CNN architecture, and then combines these two deep features together to produce the segmentation result. The whole network is optimized together via a standard back-propagation. A total of 59 CT and MRI datasets (T1/T2-weighted FLAIR, T1-weighted contrast-enhanced, T2-weighted) of postoperative gliomas as tumor grade II (n = 24), grade III (n = 18), or grade IV (n = 17) were included. Dice coefficient (DSC), precision, and recall were used to measure the overlap between automated segmentation results and manual segmentation. The Wilcoxon signed-rank test was used for statistical analysis.


DFFM showed a significantly (p < 0.01) higher DSC of 0.836 than U-Net trained by single CT images and U-Net trained by stacking the CT and multi-sequence MR images, which yielded 0.713 DSC and 0.818 DSC, respectively. The precision values showed similar behavior as DSC. Moreover, DSC and precision values have no significant statistical difference (p > 0.01) with difference grades.


DFFM enables the accurate automated segmentation of CT postoperative gliomas of profit guided by multi-sequence MR images and may thus improve and facilitate radiotherapy planning.

Key Points

• A fully automated deep learning method was developed to segment postoperative gliomas on CT images guided by multi-sequence MRIs.

• CT and multi-sequence MR image integration allows for improvements in deep learning postoperative glioma segmentation method.

• This deep feature fusion model produces reliable segmentation results and could be useful in delineating GTV in postoperative glioma radiotherapy planning.


Glioma Machine learning Magnetic resonance imaging Radiotherapy 



Convolutional neural networks


Deep feature fusion model


Fluid-attenuated inversion recovery


Gross tumor volume


T1-weighted contrast-enhanced


U-Net trained by single CT images


U-Net trained by stacking the CT and multi-sequence MR images



The authors would like to thank the reviewers for their fruitful comments.

Funding information

This study has received funding by the National Natural Science Foundation of China under Grant Nos. 61671230 and 31271067, the Science and Technology Program of Guangdong Province under Grant No. 2017A020211012, the Guangdong Provincial Key Laboratory of Medical Image Processing under Grant No. 2014B030301042, and the Science and Technology Program of Guangzhou under Grant No. 201607010097.

Compliance with ethical standards


The scientific guarantor of this publication is Yu Zhang.

Conflict of interest

The authors of this manuscript declare no relationships with any companies, whose products or services may be related to the subject matter of the article.

Statistics and biometry

No complex statistical methods were necessary for this paper.

Informed consent

Written informed consent was waived by the Institutional Review Board.

Ethical approval

Institutional Review Board approval was obtained.


• retrospective

• experimental

• performed at one institution

Supplementary material

330_2019_6441_MOESM1_ESM.docx (22 kb)
ESM 1 (DOCX 22 kb)


  1. 1.
    Goodenberger ML, Jenkins RB (2012) Genetics of adult glioma. Cancer Genet 205(12):613–621CrossRefGoogle Scholar
  2. 2.
    Louis DN, Ohgaki H, Wiestler OD et al (2007) The 2007 WHO classification of tumours of the central nervous system. Acta Neuropathol 114(2):97–109Google Scholar
  3. 3.
    Evans PM (2008) Anatomical imaging for radiotherapy. Phys Med Biol 53(12):R151–R191CrossRefGoogle Scholar
  4. 4.
    Warmuth-Metz M (2003) Postoperative imaging after brain tumor resection. In: Westphal M, Tonn JC, Ram Z (eds) Local therapies for glioma present status and future developments. Springer, Vienna, pp 13–20Google Scholar
  5. 5.
    Yahyanejad S, Granton PV, Lieuwes NG et al (2015) Complementary use of bioluminescence imaging and contrast-enhanced micro-computed tomography in an orthotopic brain tumor model. Mol Imaging.
  6. 6.
    Lee CH, Murtha A, Murtha A, Bistritz A, Greiner R (2005) Segmenting brain tumors with conditional random fields and support vector machines. In: Liu Y, Jiang T, Zhang C (eds) Computer Vision for Biomedical Image Applications. CVBIA 2005. Lecture Notes in Computer Science, vol 3765. Springer, Berlin, Heidelberg, pp 469–478Google Scholar
  7. 7.
    Li H, Song M, Fan Y (2010) Segmentation of brain tumors in multi-parametric MR images via robust statistic information propagation. In: Kimmel R, Klette R, Sugimoto A (eds) Computer Vision – ACCV 2010. ACCV 2010. Lecture Notes in Computer Science, vol 6495. Springer, Berlin, pp 606–617Google Scholar
  8. 8.
    Zikic D, Glocker B, Konukoglu E et al (2012) Decision forests for tissue-specific segmentation of high-grade gliomas in multi-channel MR. In: Ayache N, Delingette H, Golland P, Mori K (eds) Medical Image Computing and Computer-Assisted Intervention – MICCAI 2012. MICCAI 2012. Lecture Notes in Computer Science, vol 7512. Springer, Berlin, pp 369–376Google Scholar
  9. 9.
    Meier R, Bauer S, Slotboom J, Wiest R, Reyes M (2014) Appearance-and context-sensitive features for brain tumor segmentation. Proceedings of MICCAI BRATS Challenge, pp 20–26Google Scholar
  10. 10.
    Liang S, Tang F, Huang X et al (2019) Deep-learning-based detection and segmentation of organs at risk in nasopharyngeal carcinoma computed tomographic images for radiotherapy planning. Eur Radiol 29(4):1961–1967CrossRefGoogle Scholar
  11. 11.
    Huang X, Wang J, Tang F, Zhong T, Zhang Y (2018) Metal artifact reduction on cervical CT images by deep residual learning. Biomed Eng Online 17(1):175CrossRefGoogle Scholar
  12. 12.
    Shen D, Wu G, Suk H (2017) Deep learning in medical image analysis. Annu Rev Biomed Eng 19(1):221–248CrossRefGoogle Scholar
  13. 13.
    Zikic D, Ioannou Y, Brown M, Criminisi A (2014) Segmentation of brain tumor tissues with convolutional neural networks. Proceedings of MICCAI BRATS Challenge, pp 036–039Google Scholar
  14. 14.
    Pereira S, Pinto A, Alves V, Silva CA (2016) Brain tumor segmentation using convolutional neural networks in MRI images. IEEE Trans Med Imaging 35(5):1240–1251CrossRefGoogle Scholar
  15. 15.
    Havaei M, Davy A, Warde-Farley D et al (2017) Brain tumor segmentation with deep neural net-works. Med Image Anal 35:18–31CrossRefGoogle Scholar
  16. 16.
    Wang G, Li W, Ourselin S, Vercauteren T (2017) Automatic brain tumor segmentation using cascaded anisotropic convolutional neural networks. In: Crimi A, Bakas S, Kuijf H, Menze B, Reyes M (eds) Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries. BrainLes 2017. Lecture Notes in Computer Science, vol 10670. Springer, Cham, pp 178–190Google Scholar
  17. 17.
    Krizhevsky A, Sutskever I, Hinton GE (2012) ImageNet classification with deep convolutional neural networks. In: International Conference on Neural Information Processing Systems (NIPS), pp 1097–1105Google Scholar
  18. 18.
    Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556Google Scholar
  19. 19.
    Menze BH, Jakab A, Bauer S et al (2015) The Multimodal Brain Tumor Image Segmentation Benchmark (BRATS). IEEE Trans Med Imaging 34(10):1993–2024CrossRefGoogle Scholar
  20. 20.
    Bakas S, Akbari H, Sotiras A et al (2017) Advancing the cancer genome atlas glioma MRI collections with expert segmentation labels and radiomic features. Sci Data 4:170117CrossRefGoogle Scholar
  21. 21.
    Jones D (1994) ICRU report 50-prescribing, recording and reporting photon beam therapy. Med Phys 21(6):833–834Google Scholar
  22. 22.
    Landberg T, Chavaudra J, Dobbs J et al (1999) ICRU report 62-prescribing, recording and reporting photon beam therapy (supplement to ICRU report 50). J ICRU os32(1):1–52Google Scholar
  23. 23.
    National Comprehensive Cancer Network (2018) Clinical Practice Guidelines in Oncology for central nervous system cancers Version 1. National Comprehensive Cancer Network. Available via Accessed 15 Mar 2018
  24. 24.
    Tustison NJ, Avants BB, Cook PA et al (2010) N4ITK: improved N3 bias correction. IEEE Trans Med Imaging 29(6):1310–1320CrossRefGoogle Scholar
  25. 25.
    Lyu S, Simoncelli EP (2008) Nonlinear image representation using divisive normalization. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, Piscataway, pp 1–8Google Scholar
  26. 26.
    Pinto N, Cox DD, DiCarlo JJ (2008) Why is real-world visual object recognition hard? PLoS Comput Biol 4(1):e27CrossRefGoogle Scholar
  27. 27.
    Ronneberger O, Fischer P, Brox T (2015) U-Net: convolutional networks for biomedical image segmentation. In: Ayache N, Delingette H, Golland P, Mori K (eds) Medical Image Computing and Computer-Assisted Intervention – MICCAI 2012. MICCAI 2012. Lecture Notes in Computer Science, vol 7512. Springer, Berlin, pp 234–241Google Scholar
  28. 28.
    Zhang Z, Luo P, Loy CC, Tang X (2014) Facial landmark detection by deep multi-task learning. In: Fleet D, Pajdla T, Schiele B, Tuytelaars T (eds) Computer Vision – ECCV 2014. ECCV 2014. Lecture Notes in Computer Science, vol 8694.Springer, Cham, pp 94–108Google Scholar
  29. 29.
    Lin TY, Goyal P, Girshick R, He K, Dollar P (2017) Focal loss for dense object detection. In: The IEEE International Conference on Computer Vision (ICCV). IEEE, Piscataway, pp 2999–3007CrossRefGoogle Scholar
  30. 30.
    He K, Zhang X, Ren S, Jian S (2015) Delving deep into rectifiers: surpassing human-level performance on ImageNet classification. In: The IEEE International Conference on Computer Vision (ICCV). IEEE, Piscataway, pp 1026–1034CrossRefGoogle Scholar
  31. 31.
    Kingma DP, Ba J (2014) Adam: a method for stochastic optimization. arXiv:1412.6980Google Scholar
  32. 32.
    Chen L, Price RA Jr, Wang L et al (2004) MRI-based treatment planning for radiotherapy: dosimetric verification for prostate IMRT. Int J Radiat Oncol Biol Phys 60(2):636–647Google Scholar
  33. 33.
    Walker A, Liney G, Metcalfe P, Holloway L (2014) MRI distortion: considerations for MRI based radiotherapy treatment planning. Australas Phys Eng Sci Med 37(1):103–113CrossRefGoogle Scholar
  34. 34.
    Khoo VS, Dearnaley DP, Finnigan DJ, Padhani A, Tanner SF, Leach MO (1997) Magnetic resonance imaging (MRI): considerations and applications in radiotherapy treatment planning. Radiother Oncol 42(1):1–15CrossRefGoogle Scholar
  35. 35.
    Belhawi SM, Hoefnagels FW, Baaijen JC et al (2011) Early postoperative MRI overestimates residual tumour after resection of gliomas with no or minimal enhancement. Eur Radiol 21(7):1526–1534CrossRefGoogle Scholar

Copyright information

© European Society of Radiology 2019

Authors and Affiliations

  1. 1.School of Biomedical EngineeringSouthern Medical UniversityGuangzhouChina
  2. 2.Guangdong Provincial Key Laboratory of Medical Image ProcessingSouthern Medical UniversityGuangzhouChina
  3. 3.Department of Radiation Oncology, Nanfang HospitalSouthern Medical UniversityGuangzhouChina
  4. 4.Department of Medical Imaging Center, Nanfang HospitalSouthern Medical UniversityGuangzhouChina

Personalised recommendations