Brain Tumor Segmentation Using a Fully Convolutional Neural Network with Conditional Random Fields

  • Xiaomei ZhaoEmail author
  • Yihong WuEmail author
  • Guidong Song
  • Zhenye Li
  • Yong Fan
  • Yazhuo Zhang
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10154)


Deep learning techniques have been widely adopted for learning task-adaptive features in image segmentation applications, such as brain tumor segmentation. However, most of existing brain tumor segmentation methods based on deep learning are not able to ensure appearance and spatial consistency of segmentation results. In this study we propose a novel brain tumor segmentation method by integrating a Fully Convolutional Neural Network (FCNN) and Conditional Random Fields (CRF), rather than adopting CRF as a post-processing step of the FCNN. We trained our network in three stages based on image patches and slices respectively. We evaluated our method on BRATS 2013 dataset, obtaining the second position on its Challenge dataset and first position on its Leaderboard dataset. Compared with other top ranking methods, our method could achieve competitive performance with only three imaging modalities (Flair, T1c, T2), rather than four (Flair, T1, T1c, T2), which could reduce the cost of data acquisition and storage. Besides, our method could segment brain images slice-by-slice, much faster than the methods patch-by-patch. We also took part in BRATS 2016 and got satisfactory results. As the testing cases in BRATS 2016 are more challenging, we added a manual intervention post-processing system during our participation.


Brain tumor segmentation Magnetic resonance image Fully Convolutional Neural Network Conditional Random Fields Recurrent Neural Network 



This work was supported by the National High Technology Research and Development Program of China (2015AA020504) and the National Natural Science Foundation of China under Grant No. 61572499, 61421004.


  1. 1.
    Menze, B.H., Jakab, A., Bauer, S., Kalpathy-Cramer, J., Farahani, K., Kirby, J., Burren, Y., Porz, N., Slotboom, J., Wiest, R., et al.: The multimodal brain tumor image segmentation benchmark (BRATS). IEEE Trans. Med. Imaging 34, 1993–2024 (2015)CrossRefGoogle Scholar
  2. 2.
    Bauer, S., Wiest, R., Nolte, L.-P., Reyes, M.: A survey of MRI-based medical image analysis for brain tumor studies. Phys. Med. Biol. 58, 97–129 (2013)CrossRefGoogle Scholar
  3. 3.
    Goetz, M., Weber, C., Binczyk, F., Polanska, J., Tarnawski, R., Bobek-Billewicz, B., Koethe, U., Kleesiek, J., Stieltjes, B., Maier-Hein, K.H.: DALSA: domain adaptation for supervised learning from sparsely annotated MR images. IEEE Trans. Med. Imaging 35, 184–196 (2016)CrossRefGoogle Scholar
  4. 4.
    Prastawa, M., Bullitt, E., Ho, S., Gerig, G.: A brain tumor segmentation framework based on outlier detection. Med. Image Anal. 8, 275–283 (2004)CrossRefGoogle Scholar
  5. 5.
    Cobzas, D., Birkbeck, N., Schmidt, M., Jagersand, M., Murtha, A.: 3D variational brain tumor segmentation using a high dimensional feature set. In: IEEE 11th International Conference on Computer Vision, pp. 1–8 (2007)Google Scholar
  6. 6.
    Goetz, M., Weber, C., Bloecher, J., Stieltjes, B., Meinzer, H.-P., Maier-Hein, K.: Extremely randomized trees based brain tumor segmentation. In: Proceedings MICCAI BraTS (Brain Tumor Segmentation Challenge), pp. 6–11 (2014)Google Scholar
  7. 7.
    Kleesiek, J., Biller, A., Urban, G., Kothe, U., Bendszus, M., Hamprecht, F.: Ilastik for multi-modal brain tumor segmentation. In: Proceedings MICCAI BraTS (Brain Tumor Segmentation Challenge), pp. 12–17 (2014)Google Scholar
  8. 8.
    Davy, A., Havaei, M., Warde-farley, D., Biard, A., Tran, L., Jodoin, P.-M., Courville, A., Larochelle, H., Pal, C., Bengio, Y.: Brain tumor segmentation with deep neural networks. In: Proceedings MICCAI BraTS (Brain Tumor Segmentation Challenge), pp. 1–5 (2014)Google Scholar
  9. 9.
    Urban, G., Bendszus, M., Hamprecht, F., Kleesiek, J.: Multi-modal brain tumor segmentatioin using deep convolutional neural networks. In: Proceedings MICCAI BraTS (Brain Tumor Segmentation Challenge), pp. 31–35 (2014)Google Scholar
  10. 10.
    Zikic, D., Ioannou, Y., Brown, M., Criminisi, A.: Segmentation of brain tumor tissues with convolutional neural networks. In: Proceedings MICCAI BraTS (Brain Tumor Segmentation Challenge), pp. 36–39 (2014)Google Scholar
  11. 11.
    Dvorak, P., Menze, B.H.: Structured prediction with convolutional neural networks for multimodal brain tumor segmentation. In: Proceedings MICCAI BraTS (Brain Tumor Segmentation Challenge), pp. 13–24 (2015)Google Scholar
  12. 12.
    Havaei, M., Davy, A., Warde-Farley, D., Biard, A., Courville, A., Bengio, Y., Pal, C., Jodoin, P.-M., Larochelle, H.: Brain tumor segmentation with deep neural networks. Med. Image Anal. 35, 18–31 (2017)CrossRefGoogle Scholar
  13. 13.
    Vaidhya, K., Thirunavukkarasu, S., Alex, V., Krishnamurthi, G.: Multi-modal brain tumor segmentation using stacked denoising autoencoders. In: Proceedings MICCAI BraTS (Brain Tumor Segmentation Challenge), pp. 60–64 (2015)Google Scholar
  14. 14.
    Pereira, S., Pinto, A., Alves, V., Silva, C.A.: Brain tumor segmentation using convolutional neural networks in MRI images. IEEE Trans. Med. Imaging 35, 1240–1251 (2016)CrossRefGoogle Scholar
  15. 15.
    Zheng, S., Jayasumana, S., Romera-Paredes, B., Vineet, V., Su, Z., Du, D., Huang, C., Torr, P.H.: Conditional random fields as recurrent neural networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1529–1537 (2015)Google Scholar
  16. 16.
    Tustison, N.J., Avants, B.B., Cook, P.A., Zheng, Y., Egan, A., Yushkevich, P.A., Gee, J.C.: N4ITK: improved N3 bias correction. IEEE Trans. Med. Imaging 29, 1310–1320 (2010)CrossRefGoogle Scholar
  17. 17.
    Krähenbühl, P., Koltun, V.: Efficient inference in fully connected CRFs with Gaussian edge potentials. NIPS (2011)Google Scholar
  18. 18.
    Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., Darrell, T.: Caffe: convolutional architecture for fast feature embedding. In: Proceedings of the 22nd ACM International Conference on Multimedia, pp. 675–678 (2014)Google Scholar

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  1. 1.National Laboratory of Pattern Recognition, Institute of AutomationChinese Academy of SciencesBeijingChina
  2. 2.Beijing Neurosurgical InstituteCapital Medical UniversityBeijingChina
  3. 3.Department of Neurosurgery, Beijing Tiantan HospitalCapital Medical UniversityBeijingChina
  4. 4.Department of Radiology, Perelman School of MedicineUniversity of PennsylvaniaPhiladelphiaUSA
  5. 5.Beijing Institute for Brain Disorders Brain Tumor CenterBeijingChina
  6. 6.China National Clinical Research Center for Neurological DiseasesBeijingChina

Personalised recommendations