voxel-GAN: Adversarial Framework for Learning Imbalanced Brain Tumor Segmentation

  • Mina RezaeiEmail author
  • Haojin Yang
  • Christoph Meinel
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11384)


We propose a new adversarial network, named voxel-GAN, to mitigate imbalanced data problem in brain tumor semantic segmentation where the majority of voxels belong to a healthy region and few belong to tumor or non-health region. We introduce a 3D conditional generative adversarial network (cGAN) comprises two components: a segmentor and a discriminator. The segmentor is trained on 3D brain MR or CT images to learn the segmentation label’s in voxel-level, while the discriminator is trained to distinguish a segmentor output, coming from the ground truth or generated artificially. The segmentor and discriminator networks simultaneously train with new weighted adversarial loss to mitigate imbalanced training data issue. We show evidence that the proposed framework is applicable to different types of brain images of varied sizes. In our experiments on BraTS-2018 and ISLES-2018 benchmarks, we find improved results, demonstrating the efficacy of our approach.


3D generative adversarial network Learning imbalanced data 


  1. 1.
    Abadi, M., et al.: Tensorflow: a system for large-scale machine learning. In: OSDI, vol. 16, pp. 265–283 (2016)Google Scholar
  2. 2.
    Afshin, M., et al.: Regional assessment of cardiac left ventricular myocardial function via MRI statistical features. IEEE Trans. Med. Imaging 33(2), 481–494 (2014)CrossRefGoogle Scholar
  3. 3.
    Bakas, S., Akbari, H.: Segmentation labels and radiomic features for the pre-operative scans of the TCGA-GBM collection. Cancer Imaging Arch. 286 (2017).
  4. 4.
    Bakas, S., et al.: Segmentation labels and radiomic features for the pre-operative scans of the TCGA-LGG collection. Cancer Imaging Arch. 286 (2017).
  5. 5.
    Bakas, S., et al.: Advancing The Cancer Genome Atlas glioma MRI collections with expert segmentation labels and radiomic features. Nat. Sci. Data (2017). Scholar
  6. 6.
    Bakas, S., Reyes, M., Menze, B. et al.: Identifying the best machine learning algorithms for brain tumor segmentation, progression assessment, and overall survival prediction in the brats challenge. arXiv preprint arXiv:1811.02629 (2018)
  7. 7.
    Chollet, F., et al.: Keras (2015)Google Scholar
  8. 8.
    Christ, P.F., et al.: Automatic liver and tumor segmentation of CT and MRI volumes using cascaded fully convolutional neural networks. CoRR abs/1702.05970 (2017).
  9. 9.
    Ciecholewski, M.: Support vector machine approach to cardiac SPECT diagnosis. In: Aggarwal, J.K., Barneva, R.P., Brimkov, V.E., Koroutchev, K.N., Korutcheva, E.R. (eds.) IWCIA 2011. LNCS, vol. 6636, pp. 432–443. Springer, Heidelberg (2011). Scholar
  10. 10.
    Douzas, G., Bacao, F.: Effective data generation for imbalanced learning using conditional generative adversarial networks. Expert Syst. Appl. 91, 464–471 (2018)CrossRefGoogle Scholar
  11. 11.
    Eslami, A., Karamalis, A., Katouzian, A., Navab, N.: Segmentation by retrieval with guided random walks: application to left ventricle segmentation in MRI. Med. Image Anal. 17(2), 236–253 (2013)CrossRefGoogle Scholar
  12. 12.
    Fidon, L., et al.: Generalised wasserstein dice score for imbalanced multi-class segmentation using holistic convolutional networks. In: Crimi, A., Bakas, S., Kuijf, H., Menze, B., Reyes, M. (eds.) BrainLes 2017. LNCS, vol. 10670, pp. 64–76. Springer, Cham (2018). Scholar
  13. 13.
    Fischl, B., et al.: Sequence-independent segmentation of magnetic resonance images. Neuroimage 23, S69–S84 (2004)CrossRefGoogle Scholar
  14. 14.
    Gholami, A., et al.: A novel domain adaptation framework for medical image segmentation. CoRR abs/1810.05732 (2018).
  15. 15.
    Goodfellow, I.J., et al.: Generative Adversarial Networks. ArXiv e-prints (2014)Google Scholar
  16. 16.
    Hashemi, S.R., Salehi, S.S.M., Erdogmus, D., Prabhu, S.P., Warfield, S.K., Gholipour, A.: Tversky as a loss function for highly unbalanced image segmentation using 3D fully convolutional deep networks. CoRR abs/1803.11078 (2018).
  17. 17.
    Inda, M.d.M., Bonavia, R., Seoane, J., et al.: Glioblastoma multiforme: a look inside its heterogeneous nature. Cancers 6(1), 226–239 (2014)CrossRefGoogle Scholar
  18. 18.
    Isola, P., Zhu, J.Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017Google Scholar
  19. 19.
    Jang, J., et al.: Medical image matching using variable randomized undersampling probability pattern in data acquisition. In: 2014 International Conference on Electronics, Information and Communications (ICEIC), pp. 1–2, January 2014.
  20. 20.
    Kistler, M., Bonaretti, S., Pfahrer, M., Niklaus, R., Büchler, P.: The virtual skele-ton database: an open access repository for biomedical research and collaboration. J. Med. Internet Res. 15(11) (2013)CrossRefGoogle Scholar
  21. 21.
    Kohli, M.D., Summers, R.M., Geis, J.R.: Medical image data and datasets in the era of machine learning—whitepaper from the 2016 C-MIMI meeting dataset session. J. Digit. Imaging 30(4), 392–399 (2017)CrossRefGoogle Scholar
  22. 22.
    LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436–444 (2015)CrossRefGoogle Scholar
  23. 23.
    Mahapatra, D.: Automatic cardiac segmentation using semantic information from random forests. J. Digit. Imaging 27(6), 794–804 (2014)CrossRefGoogle Scholar
  24. 24.
    Maier, O., et al.: ISLES 2015-a public evaluation benchmark for ischemic stroke lesion segmentation from multispectral MRI. Med. Image Anal. 35, 250–269 (2017)CrossRefGoogle Scholar
  25. 25.
    Menze, B.H., et al.: The multimodal brain tumor image segmentation benchmark (brats). IEEE transactions on medical imaging 34(10), 1993–2024 (2015)CrossRefGoogle Scholar
  26. 26.
    Mirza, M., Osindero, S.: Conditional generative adversarial nets. CoRR abs/1411.1784 (2014)Google Scholar
  27. 27.
    Moeskops, P., Veta, M., Lafarge, M.W., Eppenhof, K.A.J., Pluim, J.P.W.: Adversarial training and dilated convolutions for brain MRI segmentation. CoRR abs/1707.03195 (2017).
  28. 28.
    Myronenko, A.: 3D MRI brain tumor segmentation using autoencoder regularization. arXiv preprint arXiv:1810.11654 (2018)
  29. 29.
    Nasr, G.E., Badr, E., Joun, C.: Cross entropy error function in neural networks: forecasting gasoline demand. In: FLAIRS Conference, pp. 381–384 (2002)Google Scholar
  30. 30.
    Nyúl, L.G., Udupa, J.K., Zhang, X.: New variants of a method of MRI scale standardization. IEEE Trans. Med. Imaging 19(2), 143–150 (2000)CrossRefGoogle Scholar
  31. 31.
    Palade, V., Neagu, D.-C., Patton, R.J.: Interpretation of trained neural networks by rule extraction. In: Reusch, B. (ed.) Fuzzy Days 2001. LNCS, vol. 2206, pp. 152–161. Springer, Heidelberg (2001). Scholar
  32. 32.
    Pohl, K.M., Fisher, J., Grimson, W.E.L., Kikinis, R., Wells, W.M.: A Bayesian model for joint segmentation and registration. NeuroImage 31(1), 228–239 (2006)CrossRefGoogle Scholar
  33. 33.
    Rezaei, M., Yang, H., Meinel, C.: Instance tumor segmentation using multitask convolutional neural network. In: 2018 International Joint Conference on Neural Networks (IJCNN), pp. 1–8, July 2018.
  34. 34.
    Rezaei, M., et al.: A conditional adversarial network for semantic segmentation of brain tumor. In: Crimi, A., Bakas, S., Kuijf, H., Menze, B., Reyes, M. (eds.) BrainLes 2017. LNCS, vol. 10670, pp. 241–252. Springer, Cham (2018). Scholar
  35. 35.
    Rezaei, M., Yang, H., Meinel, C.: Deep neural network with l2-norm unit for brain lesions detection. In: Liu, D., Xie, S., Li, Y., Zhao, D., El-Alfy, E.S. (eds.) ICONIP 2017. LNCS, vol. 10637, pp. 798–807. Springer, Cham (2017). Scholar
  36. 36.
    Rezaei, M., Yang, H., Meinel, C.: Whole heart and great vessel segmentation with context-aware of generative adversarial networks. In: Maier, A., Deserno, T., Handels, H., Maier-Hein, K., Palm, C., Tolxdorff, T. (eds.) Bildverarbeitung für die Medizin 2018. Informatik aktuell, pp. 353–358. Springer, Heidelberg (2018). Scholar
  37. 37.
    Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). Scholar
  38. 38.
    Sudre, C.H., Li, W., Vercauteren, T., Ourselin, S., Jorge Cardoso, M.: Generalised dice overlap as a deep learning loss function for highly unbalanced segmentations. In: Cardoso, M., et al. (eds.) DLMIA/ML-CDS -2017. LNCS, vol. 10553, pp. 240–248. Springer, Cham (2017). Scholar
  39. 39.
    Wang, G., Li, W., Ourselin, S., Vercauteren, T.: Automatic brain tumor segmentation using convolutional neural networks with test-time augmentation. arXiv preprint arXiv:1810.07884 (2018)
  40. 40.
    Xu, J., Schwing, A.G., Urtasun, R.: Tell me what you see and i will show you where it is. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3190–3197 (2014)Google Scholar
  41. 41.
    Xue, Y., Xu, T., Zhang, H., Long, L.R., Huang, X.: Segan: adversarial network with multi-scalel1 loss for medical image segmentation. CoRR abs/1706.01805 (2017)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Hasso Plattner InstituteBerlinGermany

Personalised recommendations