Abstract
Glioblastoma is a high-grade invasive astrocytoma tumor. The highly invasive nature makes timely detection and characterization of the tumor critical for the survivability prediction of patients. This work proposes MRI- and clinical information-based automated pipeline that implements various state-of-the-art image processing, machine learning, and deep learning techniques to obtain robust tumor segmentation and patient survival estimation. We use 163 cases from the training dataset, and 28 cases from the validation dataset provided by the BraTS 2018 challenge for the evaluation of our model. We achieve an accuracy of 0.679 using the validation dataset and that of 0.519 for the test dataset.
Z. A. Shboul, M. Alam, L. Vidyaratne and L. Pei––The authors have equal contribution.
You have full access to this open access chapter, Download conference paper PDF
Similar content being viewed by others
1 Introduction
High-grade glioblastoma (HGG) or glioblastoma represents tumors arising from the gluey or supportive tissue of the brain. HGG is considered the most aggressive type of brain tumor. According to the American Brain Tumor Association (ABTA) [1] HGGs represent 74.6% of all malignant tumors and 24.7% of all primary brain tumors. World Health Organization (WHO) categorize HGGs as stage IV brain cancer [2]. Typically, the survival duration of patients with HGG tumor is less than two years [3, 4]. Therefore, accurate and timely detection of HGG tumor is essential for devising an appropriate treatment plan that may improve patient survival duration.
Recent works [5,6,7,8,9,10] have focused on developing automated survival prediction techniques for patients with HGG tumor. Different studies analyze tumor heterogeneity using different types of imaging [11] such as Magnetic Resonance Imaging (MRI) [12, 13]. This suggests MR as a potential non-invasive imaging biomarker for Glioblastoma diagnostic, prognostic and survival prediction. Jain et al. [14] extract morphological imaging features represented by Visually Accessible Rembrandt Images (VASRAI) [15] from the non-enhancing region of GBM and then correlate these features to the relative cerebral blood volume of a non-enhancing region and non-enhancing region crossing the midline. Gutman et al. [16] utilize four VASARI imaging features that describe the size of the contrast-enhanced, necrosis, non-enhance, edema, and the size of the whole tumor. Then these imaging features are associated with genetic mutation, survival prediction and Verhaak subtypes [17]. Nicolasjilwan et al. [18] combine clinical factors, VASARI imaging features, and genomics in a stepwise multivariate Cox model to predict overall survival time. Prasanna et al. [19] extract radiomic texture features that characterize three tumor regions; enhancing tumor, peritumoral brain zone, and necrosis from MR images. These features are assessed to overall survival prediction. Itakura et al. [20] utilize quantitative MRI features that describe tumor histogram statistics, texture, edge sharpness, compactness, and roughness. The authors cluster the features into three MRI phenotypic imaging subtypes; pre-multifocal, spherical, and rim-enhancing tumor. The three distinct subtypes are then correlated with overall survival and associated with molecular pathways.
Our proposed deep learning and machine learning based survival prediction technique have shown the best performance in the BraTS 2017 survival prediction challenge on the validation and test dataset, respectively [9]. This work proposes a sophisticated computational modeling-based survival prediction method. Specifically, we approach the survival prediction task as a two-step process, brain tumor segmentation followed by survival prediction, considering the segmentation output as an input to the second step. The brain tumor segmentation task is performed by utilizing two state-of-the-art convolutional neural networks (CNN) models, U-Net and fully convolutional neural network (FCN). The outcome of these two models is fused together to achieve the final segmentation output. This segmentation output along with the original MRI volumes are considered as input to the survival prediction step. Several radiomics features such as texture, topological, histogram etc. are extracted from the raw MRI sequences and the segmented tumor volume. Furthermore, a state-of-the-art 3D CNN architecture is utilized to extract additional features useful for the survival prediction task. These features are then processed using a gradient boosting-based regression technique known as extended gradient boosting (XGBoost) to obtain the survival estimation of patients.
2 Dataset
In this study, we use MR images of 163 high-grade GBM patients from BtaTS18 training dataset [21,22,23,24,25]. The dataset provides the ground truth segmentation of tumor tissues which comprises of enhancing tumor (ET), edema (ED), and the necrosis and non-enhancing tumor (NCR/NET). The training dataset provides age and overall survival (in days) data. The available scans of the MRI are native (T1), post-contrast T1-weighted (T1Gd), T2-weighted (T2), and T2 Fluid Attenuated Inversion Recovery (FLAIR) volumes. The dataset is co-registered, re-sampled to 1 mm3 and skull-stripped. In addition, for overall survival validation proposes we use the 28 cases of BraTS18 validation dataset.
3 Methodology
3.1 Brain Tumor Segmentation
This work utilizes two state-of-the-art CNN architectures, U-Net and fully convolutional neural network (FCN) to perform the brain tumor segmentation task. The following two sub-sections provide a very brief outline of these two models.
Brain Tumor Segmentation Using Deep CNNs.
This work utilizes a CNN based U-Net model [26, 27] to perform the brain tumor segmentation task. Unlike patch based CNN segmentation pipeline where the model only sees a localized region of the brain, the U-Net based segmentation model captures the global information from a different region of the brain tissues which is essential to achieve robust segmentation performance. Moreover, U-Net based model allows achieving an end-to-end segmentation framework rather than a pixel-wise classification technique. The U-Net architecture utilized in this work is implemented following [26]. Rather than using regular cross-entropy based loss function, we utilize a soft dice similarity coefficient (DSC) metric based loss function to train the U-Net model [28]. The U-Net model is trained using mini-batch gradient descent (GD) technique which minimizes the soft dice cost function. This work also uses a fully convolutional network (FCNs) [29, 30] for tumor segmentation. We adapt VGG-11 [31] as a pre-trained model. The overall FCN architecture contains an encode and a decode stage. The encoding stage contains convolution and max-pooling steps whereas the decoding stage contains a deconvolution step to obtain the same output size as the input. The final segmentation output is achieved by fusing the outcomes obtained from the above mentioned CNN based tumor segmentation pipelines as shown in Fig. 1.
3.2 Survival Prediction
The proposed survival prediction pipeline essentially involves three stages, (1) feature extraction stage, (2) feature selection stage, and (3) feature classification/regression stage.
Feature Extraction.
Overall, approximately 31 thousand features representing texture, volume, area, and Euler characteristics are extracted from the tumor and the sub-regions (edema, enhancing tumor, and tumor core). Each feature type is selected to emphasize different characteristics of the tumor that may be relevant for survival prediction. For instance, texture features define the heterogeneity of the different tumor tissues, and Volumetric and Euler characteristic features define tumor shape.
We extract forty-one representative features [32] from three raw MRI sequences, and from eight texture representations of the tumor volume that includes several Texton filters [33], and fractal characterizations using algorithms such as PTPSA [34], mBm [35], and Holder exponent [36]. The features obtained from these representations include histograms, co-occurrence matrix, grey-tone difference matrix, and several other statistical measures. Furthermore, histogram-based features are extracted from the different modalities of different histogram graphs of the tumor tissue regions (edema, enhancing tumor, and necrosis). We also extract many representative volumetric features from the different tumor tissues with respect to the brain and whole tumor regions. In addition, we compute the Euler characteristics of the whole tumor, edema, enhancing and necrosis, for each slice as feature vectors. Euler characteristic [37] identifies the shape of a tumor by computing tumor vertices, edges, and faces. Combining feature vectors from all the above-mentioned methods constitute a total 31,000 feature points for the survival prediction task.
Feature Selection.
We perform recursive feature selection on the Euler features alone, another recursive feature selection on the other features (texture, volumetric, histogram-graph based).
Feature Classification and Regression.
Extreme Gradient Boosting (XGBoost) [38] is a tree boosting supervised machine learning technique that is highly effective. XGBoost is widely used in classification and regression tasks. In our study, XGBoost is utilized for classification and regression overall survival prediction on the selected features. The trained models are tuned to their optimized hyper-parameters when a tuned grid (search grid) is created by the different combination of the hyper-parameters. The complete pipeline for classification and regression overall survival is illustrated in Fig. 2.
4 Experimental Results
Following the proposed pipeline in Fig. 1, we first perform the FCN and U-Net fused tumor segmentation task. Figure 3 shows an example from the BraTS 2018 training dataset of the fused segmentation outcome of FCN and U-Net. We perform leave-one-out cross-validation analysis on the BraTS 2018 training dataset using the proposed survival prediction pipeline. We evaluate the performance of our proposed method using the root mean square error (RMSE), and the classification accuracy for a three-class setting defined as follows: (1) long – more than 15 months, (2) Medium – between 10 to 15 months, (3) short – less than 10 months. The best model is picked from the analysis of training data to be used to process the validation dataset of the BraTS 2018 competition. The ground truth is considered for the segmented tumor in the survival analysis of the training dataset, while the segmented tumor obtained in stage one of the proposed pipeline is used as input in the evaluation of the validation dataset. The leave-one-out cross-validation RMSE of the training dataset is 391.25, and the three-class leave-one-out cross-validated survival classification accuracy is 0.73. Table 1 shows the online evaluation results we achieve with the BraTS18 validation dataset and the test dataset.
5 Conclusions
This work proposes a robust automated glioblastoma survival prediction using state-of-the-art computational modeling techniques. The survival prediction task is performed in two steps: tumor segmentation and survival prediction. A combination of hand-crafted and learned features are used in a regression technique to obtain the final survival prediction output. The performance of the proposed pipeline is evaluated using BraTS 2018 challenge training and validation datasets. Our results show a leave-one-out cross-validated classification accuracy of 0.679 for the validation dataset and that of 0.519 for the test dataset.
References
A.B.T. Association: Brain tumor statistics. vol. 2 (2016). Accessed May
Louis, D.N., et al.: The 2007 WHO classification of tumours of the central nervous system. Acta Neuropathol. 114(2), 97–109 (2007)
Holland, E.C.: Progenitor cells and glioma formation. Curr. Opin. Neurol. 14(6), 683–688 (2001)
Ohgaki, H., Kleihues, P.: Population-based studies on incidence, survival rates, and genetic alterations in astrocytic and oligodendroglial gliomas. J. Neuropathol. Exp. Neurol. 64(6), 479–489 (2005)
Lao, J., et al.: A deep learning-based radiomics model for prediction of survival in glioblastoma multiforme. Sci. Rep. 7(1), 10353 (2017)
Nie, D., Zhang, H., Adeli, E., Liu, L., Shen, D.: 3D deep learning for multi-modal imaging-guided survival time prediction of brain tumor patients. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9901, pp. 212–220. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46723-8_25
Huang, C., Zhang, A., Xiao, G.: Deep Integrative Analysis for Survival Prediction (2017)
Chato, L., Latifi, S.: Machine learning and deep learning techniques to predict overall survival of brain tumor patients using MRI images. In: 2017 IEEE 17th International Conference on Bioinformatics and Bioengineering (BIBE), pp. 9–14. IEEE (2017)
Shboul, Z.A., Vidyaratne, L., Alam, M., Iftekharuddin, K.M.: Glioblastoma and survival prediction. In: Crimi, A., Bakas, S., Kuijf, H., Menze, B., Reyes, M. (eds.) BrainLes 2017. LNCS, vol. 10670, pp. 358–368. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-75238-9_31
Vidyaratne, L., Alam, M., Shboul, Z., Iftekharuddin, K.: Deep learning and texture-based semantic label fusion for brain tumor segmentation. In: Medical Imaging 2018: Computer-Aided Diagnosis, vol. 10575, p. 105750D. International Society for Optics and Photonics (2018)
Cyran, C.C., et al.: Visualization, imaging and new preclinical diagnostics in radiation oncology. Radiat. Oncol. 9(1), 3 (2014)
Ellingson, B.M.: Radiogenomics and imaging phenotypes in glioblastoma: novel observations and correlation with molecular characteristics. Curr. Neurol. Neurosci. Rep. 15(1), 506 (2015)
Kickingereder, P., et al.: Radiogenomics of glioblastoma: machine learning–based classification of molecular characteristics by using multiparametric and multiregional MR imaging features. Radiology 281(3), 907–918 (2016)
Jain, R., et al.: Outcome prediction in patients with glioblastoma by using imaging, clinical, and genomic biomarkers: focus on the nonenhancing component of the tumor. Radiology 272(2), 484–493 (2014)
VASARI Research Project - Cancer Imaging Archive Wiki. https://wiki.cancerimagingarchive.net/display/Public/VASARI+Research+Project
Gutman, D.A., et al.: MR imaging predictors of molecular profile and survival: multi-institutional study of the TCGA glioblastoma data set. Radiology 267(2), 560–569 (2013)
Verhaak, R.G., et al.: Integrated genomic analysis identifies clinically relevant subtypes of glioblastoma characterized by abnormalities in PDGFRA, IDH1, EGFR, and NF1. Cancer Cell 17(1), 98–110 (2010)
Nicolasjilwan, M., et al.: Addition of MR imaging features and genetic biomarkers strengthens glioblastoma survival prediction in TCGA patients. J. Neuroradiol. 42(4), 212–221 (2015)
Prasanna, P., Patel, J., Partovi, S., Madabhushi, A., Tiwari, P.: Radiomic features from the peritumoral brain parenchyma on treatment-naive multi-parametric MR imaging predict long versus short-term survival in glioblastoma multiforme: preliminary findings. Eur. Radiol. 27(10), 4188–4197 (2017)
Itakura, H., et al.: Magnetic resonance image features identify glioblastoma phenotypic subtypes with distinct molecular pathway activities. Sci. Transl. Med. 7(303), 303ra138 (2015)
Bakas, S., et al.: Advancing The Cancer Genome Atlas glioma MRI collections with expert segmentation labels and radiomic features. Sci. Data 4, 170117 (2017)
Bakas, S., et al.: Segmentation labels and radiomic features for the pre-operative scans of the TCGA-GBM collection. The Cancer Imaging Archive (2017)
Bakas, S., et al.: Segmentation labels and radiomic features for the pre-operative scans of the TCGA-LGG collection. The Cancer Imaging Archive (2017)
Menze, B.H., et al.: The multimodal brain tumor image segmentation benchmark (BRATS). IEEE Trans. Med. Imaging 34(10), 1993–2024 (2015)
Bakas, S., et al.: Identifying the Best Machine Learning Algorithms for Brain Tumor Segmentation, Progression Assessment, and Overall Survival Prediction in the BRATS Challenge. arXiv preprint arXiv:1811.02629 (2018)
Dong, H., Yang, G., Liu, F., Mo, Y., Guo, Y.: Automatic brain tumor detection and segmentation using U-Net based fully convolutional networks. In: Valdés Hernández, M., González-Castro, V. (eds.) MIUA 2017. CCIS, vol. 723, pp. 506–517. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-60964-5_44
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Milletari, F., Navab, N., Ahmadi, S.-A.: V-Net: Fully convolutional neural networks for volumetric medical image segmentation. In: 2016 Fourth International Conference on 3D Vision (3DV), pp. 565–571. IEEE (2016)
Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3431–3440 (2015)
Zhao, X., Wu, Y., Song, G., Li, Z., Fan, Y., Zhang, Y.: Brain tumor segmentation using a fully convolutional neural network with conditional random fields. In: Crimi, A., Menze, B., Maier, O., Reyes, M., Winzeck, S., Handels, H. (eds.) BrainLes 2016. LNCS, vol. 10154, pp. 75–87. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-55524-9_8
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)
Vallières, M., Freeman, C.R., Skamene, S.R., El Naqa, I.: A radiomics model from joint FDG-PET and MRI texture features for the prediction of lung metastases in soft-tissue sarcomas of the extremities. Phys. Med. Biol. 60(14), 5471 (2015)
Leung, T., Malik, J.: Representing and recognizing the visual appearance of materials using three-dimensional textons. Int. J. Comput. Vis. 43(1), 29–44 (2001)
Iftekharuddin, K.M., Jia, W., Marsh, R.: Fractal analysis of tumor in brain MR images. Mach. Vis. Appl. 13(5–6), 352–362 (2003)
Islam, A., Iftekharuddin, K.M., Ogg, R.J., Laningham, F.H., Sivakumar, B.: Multifractal modeling, segmentation, prediction, and statistical validation of posterior fossa tumors. In: Medical Imaging 2008: Computer-Aided Diagnosis, vol. 6915, p. 69153C. International Society for Optics and Photonics (2008)
Ayache, A., Véhel, J.L.: Generalized multifractional Brownian motion: definition and preliminary results. In: Dekking, M., Véhel, J.L., Lutton, E., Tricot, C. (eds.) Fractals, pp. 17–32. Springer, London (1999). https://doi.org/10.1007/978-1-4471-0873-3_2
Turner, K., Mukherjee, S., Boyer, D.M.: Persistent homology transform for modeling shapes and surfaces. Inf. Infer. J. IMA 3(4), 310–344 (2014)
Chen, T., Guestrin, C.: XGBoost: a scalable tree boosting system. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 785–794. ACM (2016)
Acknowledgements
This work was funded by NIBIB/NIH grant# R01 EB020683.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Shboul, Z.A., Alam, M., Vidyaratne, L., Pei, L., Iftekharuddin, K.M. (2019). Glioblastoma Survival Prediction. In: Crimi, A., Bakas, S., Kuijf, H., Keyvan, F., Reyes, M., van Walsum, T. (eds) Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries. BrainLes 2018. Lecture Notes in Computer Science(), vol 11384. Springer, Cham. https://doi.org/10.1007/978-3-030-11726-9_45
Download citation
DOI: https://doi.org/10.1007/978-3-030-11726-9_45
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-11725-2
Online ISBN: 978-3-030-11726-9
eBook Packages: Computer ScienceComputer Science (R0)