Segmentation of Gliomas in Pre-operative and Post-operative Multimodal Magnetic Resonance Imaging Volumes Based on a Hybrid Generative-Discriminative Framework

  • Ke ZengEmail author
  • Spyridon BakasEmail author
  • Aristeidis Sotiras
  • Hamed Akbari
  • Martin Rozycki
  • Saima Rathore
  • Sarthak Pati
  • Christos Davatzikos
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10154)


We present an approach for segmenting both low- and high-grade gliomas in multimodal magnetic resonance imaging volumes. The proposed framework is an extension of our previous work [6, 7], with an additional component for segmenting post-operative scans. The proposed approach is based on a hybrid generative-discriminative model. Firstly, a generative model based on a joint segmentation-registration framework is used to segment the brain scans into cancerous and healthy tissues. Secondly, a gradient boosting classification scheme is used to refine tumor segmentation based on information from multiple patients. We evaluated our approach in 218 cases during the training phase of the BRAin Tumor Segmentation (BRATS) 2016 challenge and report promising results. During the testing phase, the proposed approach was ranked among the top performing methods, after being additionally evaluated in 191 unseen cases.


Segmentation Brain tumor Glioma Multimodal MRI Gradient boosting Expectation maximization Probabilistic model BRATS challenge 


  1. 1.
    Agn, M., Puonti, O., Rosenschöld, P.M., Law, I., Leemput, K.: Brain tumor segmentation using a generative model with an RBM prior on tumor shape. In: Crimi, A., Menze, B., Maier, O., Reyes, M., Handels, H. (eds.) BrainLes 2015. LNCS, vol. 9556, pp. 168–180. Springer, Cham (2016). doi: 10.1007/978-3-319-30858-6_15 CrossRefGoogle Scholar
  2. 2.
    Akbari, H., Macyszyn, L., Da, X., Wolf, R.L., Bilello, M., Verma, R., O’Rourke, D.M., Davatzikos, C.: Pattern analysis of dynamic susceptibility contrast-enhanced MR imaging demonstrates peritumoral tissue heterogeneity. Radiology 273(2), 502–510 (2014)CrossRefGoogle Scholar
  3. 3.
    Ayachi, R., Ben Amor, N.: Brain tumor segmentation using support vector machines. In: Sossai, C., Chemello, G. (eds.) ECSQARU 2009. LNCS (LNAI), vol. 5590, pp. 736–747. Springer, Heidelberg (2009). doi: 10.1007/978-3-642-02906-6_63 CrossRefGoogle Scholar
  4. 4.
    Bakas, S., Chatzimichail, K., Hunter, G., Labbe, B., Sidhu, P.S., Makris, D.: Fast semi-automatic segmentation of focal liver lesions in contrast-enhanced ultrasound, based on a probabilistic model. In: Computer Methods in Biomechanics and Biomedical Engineering: Imaging & Visualization, pp. 1–10 (2015)Google Scholar
  5. 5.
    Bakas, S., Labbe, B., Hunter, G.J.A., Sidhu, P., Chatzimichail, K., Makris, D.: Fast segmentation of focal liver lesions in contrast-enhanced ultrasound data. In: Proceedings of the 18th Annual Conference on Medical Image Understanding and Analysis (MIUA), pp. 73–78 (2014)Google Scholar
  6. 6.
    Bakas, S., Zeng, K., Sotiras, A., Rathore, S., Akbari, H., Gaonkar, B.,Rozycki, M., Pati, S., Davazikos, C.: Segmentation of gliomas in multimodal magnetic resonance imaging volumes based on a hybrid generative-discriminative framework. In: Proceedings of the Multimodal Brain Tumor Image Segmentation Challenge (BRATS) 2015, pp. 5–12 (2015)Google Scholar
  7. 7.
    Bakas, S., Zeng, K., Sotiras, A., Rathore, S., Akbari, H., Gaonkar, B., Rozycki, M., Pati, S., Davatzikos, C.: GLISTRboost: combining multimodal MRI segmentation, registration, and biophysical tumor growth modeling with gradient boosting machines for glioma segmentation. In: Crimi, A., Menze, B., Maier, O., Reyes, M., Handels, H. (eds.) BrainLes 2015. LNCS, vol. 9556, pp. 144–155. Springer, Cham (2016). doi: 10.1007/978-3-319-30858-6_13 CrossRefGoogle Scholar
  8. 8.
    Deeley, M.A., Chen, A., Datteri, R., Noble, J.H., Cmelak, A.J., Donnelly, E.F., Malcolm, A.W., Moretti, L., Jaboin, J., Niermann, K., Yang, E.S., Yu, D.S., Yei, F., Koyama, T., Ding, G.X., Dawant, B.M.: Comparison of manual and automatic segmentation methods for brain structures in the presence of space-occupying lesions: a multi-expert study. Phys. Med. Biol. 56(14), 4557–4577 (2011)CrossRefGoogle Scholar
  9. 9.
    Deschamps, T., Cohen, L.D.: Fast extraction of minimal paths in 3D images and applications to virtual endoscopy. Med. Image Anal. 5(4), 281–299 (2001)CrossRefGoogle Scholar
  10. 10.
    Friedman, J.H.: Greedy function approximation: a gradient boosting machine. Ann. Stat. 29(5), 1189–1232 (2001)MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Friedman, J.H.: Stochastic gradient boosting. Comput. Stat. Data Anal. 38(4), 367–378 (2002)MathSciNetCrossRefzbMATHGoogle Scholar
  12. 12.
    Gaonkar, B., Macyszyn, L., Bilello, M., Sadaghiani, M.S., Akbari, H., Attiah, M.A., Ali, Z.S., Da, X., Zhan, Y., O’Rourke, D., Grady, S.M., Davatzikos, C.: Automated tumor volumetry using computer-aided image segmentation. Acad. Radiol. 22(5), 653–661 (2015)CrossRefGoogle Scholar
  13. 13.
    Gering, D.T., Grimson, W.E.L., Kikinis, R.: Recognizing deviations from normalcy for brain tumor segmentation. In: Dohi, T., Kikinis, R. (eds.) MICCAI 2002. LNCS, vol. 2488, pp. 388–395. Springer, Heidelberg (2002). doi: 10.1007/3-540-45786-0_48 CrossRefGoogle Scholar
  14. 14.
    Gooya, A., Biros, G., Davatzikos, C.: Deformable registration of glioma images using EM algorithm and diffusion reaction modeling. IEEE Trans. Med. Imaging 30(2), 375–390 (2011)CrossRefGoogle Scholar
  15. 15.
    Gooya, A., Pohl, K.M., Bilello, M., Biros, G., Davatzikos, C.: Joint segmentation and deformable registration of brain scans guided by a tumor growth model. Med. Image Comput. Comput.-Assist. Interventions 14(2), 532–540 (2011)Google Scholar
  16. 16.
    Gooya, A., Pohl, K.M., Bilello, M., Cirillo, L., Biros, G., Melhem, E.R., Davatzikos, C.: GLISTR: glioma image segmentation and registration. IEEE Trans. Med. Imaging 31(10), 1941–1954 (2012)CrossRefGoogle Scholar
  17. 17.
    Haralick, R.M., Shanmugam, K., Dinstein, I.H.: Textural features for image classification. IEEE Trans. Syst. Man Cybern. 6, 610–621 (1973)CrossRefGoogle Scholar
  18. 18.
    Havaei, M., Davy, A., Warde-Farley, D., Biard, A., Courville, A., Bengio, Y., Pal, C., Jodoin, P.M., Larochelle, H.: Brain tumor segmentation with deep neural networks. Med. Image Anal. 35, 18–31 (2017). CrossRefGoogle Scholar
  19. 19.
    Hogea, C., Davatzikos, C., Biros, G.: An image-driven parameter estimation problem for a reaction-diffusion glioma growth model with mass effects. J. Math. Biol. 56(6), 793–825 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  20. 20.
    Kamnitsas, K., Ferrante, E., Parisot, S., Ledig, C., Nori, A., Criminisi, A., Rueckert, D., Glocker, B.: Deepmedic on brain tumor segmentationGoogle Scholar
  21. 21.
    Kamnitsas, K., Ledig, C., Newcombe, V.F., Simpson, J.P., Kane, A.D., Menon, D.K., Rueckert, D., Glocker, B.: Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation. Med. Image Anal. 36, 61–78 (2017). CrossRefGoogle Scholar
  22. 22.
    Kistler, M., Bonaretti, S., Pfahrer, M., Niklaus, R., Büchler, P.: The virtual skeleton database: an open access repository for biomedical research and collaboration. J. Med. Internet Res. 15(11), e245 (2013)CrossRefGoogle Scholar
  23. 23.
    Kwon, D., Akbari, H., Da, X., Gaonkar, B., Davatzikos, C.: Multimodal brain tumor image segmentation using GLISTR. In: MICCAI Brain Tumor Segmentation (BraTS) Challenge Manuscripts, pp. 18–19 (2014)Google Scholar
  24. 24.
    Kwon, D., Niethammer, M., Akbari, H., Bilello, M., Davatzikos, C., Pohl, K.M.: Portr: pre-operative and post-recurrence brain tumor registration. IEEE Trans. Med. Imaging 33(3), 651–667 (2014)CrossRefGoogle Scholar
  25. 25.
    Kwon, D., Shinohara, R.T., Akbari, H., Davatzikos, C.: Combining generative models for multifocal glioma segmentation and registration. Med. Image Comput. Comput.-Assist. Interventions 17(1), 763–770 (2014)Google Scholar
  26. 26.
    Kwon, D., Zeng, K., Bilello, M., Davatzikos, C.: Estimating patient specific templates for pre-operative and follow-up brain tumor registration. Med. Image Comput. Comput.-Assist. Interventions 2, 222–229 (2015)Google Scholar
  27. 27.
    Lee, C.-H., Wang, S., Murtha, A., Brown, M.R.G., Greiner, R.: Segmenting brain tumors using pseudo–conditional random fields. In: Metaxas, D., Axel, L., Fichtinger, G., Székely, G. (eds.) MICCAI 2008. LNCS, vol. 5241, pp. 359–366. Springer, Heidelberg (2008). doi: 10.1007/978-3-540-85988-8_43 CrossRefGoogle Scholar
  28. 28.
    Louis, D.N.: Molecular pathology of malignant gliomas. Ann. Rev. Pathol. Mech. Dis. 1, 97–117 (2006)CrossRefGoogle Scholar
  29. 29.
    Menze, B.H., et al.: The Multimodal Brain Tumor Image Segmentation Benchmark (BRATS). IEEE Trans. Med. Imaging 34(10), 1993–2024 (2015). doi: 10.1109/TMI.2014.2377694 CrossRefGoogle Scholar
  30. 30.
    Moon, N., Bullitt, E., van Leemput, K., Gerig, G.: Model-based brain and tumor segmentation. In: Object Recognition Supported by User Interaction for Service Robots. vol. 1, pp. 528–531 (2002)Google Scholar
  31. 31.
    Nyul, L.G., Udupa, J.K., Zhang, X.: New variants of a method of mri scale standardization. IEEE Trans. Med. Imaging 19(2), 143–150 (2000)CrossRefGoogle Scholar
  32. 32.
    Parisot, S., Duffau, H., Chemouny, S., Paragios, N.: Joint tumor segmentation and dense deformable registration of brain MR images. In: Ayache, N., Delingette, H., Golland, P., Mori, K. (eds.) MICCAI 2012. LNCS, vol. 7511, pp. 651–658. Springer, Heidelberg (2012). doi: 10.1007/978-3-642-33418-4_80 CrossRefGoogle Scholar
  33. 33.
    Pati, S., Rathore, S., Kalarot, R., Sridharan, P., Bergman, M., Shinohara, T., Yushkevich, P., Fan, Y., Verma, R., Kontos, D., Davatzikos, C.: Cancer and Phenomics Toolkit (CAPTk): a software suite for computational oncology and radiomics. In: Radiological Society of North America 2016 Scientific Assembly and Annual Meeting, November 27 - December 2, 2016, Chicago IL (2016).
  34. 34.
    Pereira, S., Pinto, A., Alves, V., Silva, C.A.: Brain tumor segmentation using convolutional neural networks in MRI images. IEEE Trans. Med. Imaging 35(5), 1240–1251 (2016)CrossRefGoogle Scholar
  35. 35.
    Prastawa, M., Bullitt, E., Ho, S., Gerig, G.: A brain tumor segmentation framework based on outlier detection. Med. Image Anal. 8(3), 275–283 (2004). CrossRefGoogle Scholar
  36. 36.
    Sethian, J.A.: A fast marching level set method for monotonically advancing fronts. Proc. Nat. Acad. Sci. U.S.A. 93(4), 1591–1595 (1996)MathSciNetCrossRefzbMATHGoogle Scholar
  37. 37.
    Smith, S.M., Brady, J.M.: SUSAN - a new approach to low level image processing. Int. J. Comput. Vis. 23(1), 45–78 (1997)CrossRefGoogle Scholar
  38. 38.
    Wen, P.Y., Kesari, S.: Malignant gliomas in adults. New England J. Med. 359(5), 492–507 (2008)CrossRefGoogle Scholar
  39. 39.
    Zikic, D., Glocker, B., Konukoglu, E., Criminisi, A., Demiralp, C., Shotton, J., Thomas, O.M., Das, T., Jena, R., Price, S.J.: Decision forests for tissue-specific segmentation of high-grade gliomas in multi-channel MR. In: Ayache, N., Delingette, H., Golland, P., Mori, K. (eds.) MICCAI 2012. LNCS, vol. 7512, pp. 369–376. Springer, Heidelberg (2012). doi: 10.1007/978-3-642-33454-2_46 CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  • Ke Zeng
    • 1
    Email author
  • Spyridon Bakas
    • 1
    Email author
  • Aristeidis Sotiras
    • 1
  • Hamed Akbari
    • 1
  • Martin Rozycki
    • 1
  • Saima Rathore
    • 1
  • Sarthak Pati
    • 1
  • Christos Davatzikos
    • 1
  1. 1.Section of Biomedical Image Analysis, Perelman School of Medicine, Center for Biomedical Image Computing and AnalyticsUniversity of PennsylvaniaPhiladelphiaUSA

Personalised recommendations