Ensemble of Deep Convolutional Neural Networks for Prognosis of Ischemic Stroke

  • Youngwon Choi
  • Yongchan Kwon
  • Hanbyul Lee
  • Beom Joon Kim
  • Myunghee Cho Paik
  • Joong-Ho WonEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10154)


We propose an ensemble of deep neural networks for the two tasks of automated prognosis of post-treatment ischemic stroke, as imposed by the ISLES 2016 Challenge. For lesion outcome prediction, we employ an ensemble of three-dimensional multiscale residual U-Net and a fully convolutional network, trained using image patches. In order to handle class imbalance, we devise a multi-step training strategy. For clinical outcome prediction, we combine a convolutional neural network (CNN) and a logistic regression model. To overcome the small sample size and the need for whole brain image, we use the CNN trained using patches as a feature extractor and trained a shallow network based on the extracted features. Our ensemble approach demonstrated an appealing performance on both problems, and is ranked among the top entries in the Challenge.


Fully convolutional networks U-Net 3D convolutional kernels Patchwise learning Multi-phase training Class imbalance Ensemble 



This research was supported by the National Research Foundation of Korea (NRF) grant funded by the Korean government (MSIP, No. 2016012002). Joong-Ho Won’s research was partially supported by the National Research Foundation of Korea (NRF) grant funded by the Korean government (MSIP, Nos. 2013R1A1A1057949 and 2014R1A4A1007895).


  1. 1.
    ISLES 2016 challenge.
  2. 2.
    Chollet, F.: Keras (2015).
  3. 3.
    Çiçek, Ö., Abdulkadir, A., Lienkamp, S.S., Brox, T., Ronneberger, O.: 3D u-net: learning dense volumetric segmentation from sparse annotation. CoRR abs/1606.06650 (2016).
  4. 4.
    Clevert, D., Unterthiner, T., Hochreiter, S.: Fast and accurate deep network learning by exponential linear units (ELUs). CoRR abs/1511.07289 (2015).
  5. 5.
    Dieleman, S., Schlter, J., Raffel, C., Olson, E., Snderby, S.K., Nouri, D., Maturana, D., Thoma, M., Battenberg, E., Kelly, J., Fauw, J.D., Heilman, M., de Almeida, D.M., McFee, B., Weideman, H., Takcs, G., de Rivaz, P., Crall, J., Sanders, G., Rasul, K., Liu, C., French, G., Degrave, J.: Lasagne: first release.
  6. 6.
    Goodfellow, I.J., Warde-Farley, D., Mirza, M., Courville, A.C., Bengio, Y.: Maxout networks. ICML 3(28), 1319–1327 (2013)Google Scholar
  7. 7.
    Havaei, M., Davy, A., Warde-Farley, D., Biard, A., Courville, A., Bengio, Y., Pal, C., Jodoin, P.-M., Larochelle, H.: Brain tumor segmentation with deep neural networks. Med. Image Anal. 35, 18–31 (2017). ElsevierCrossRefGoogle Scholar
  8. 8.
    Havaei, M., Guizard, N., Larochelle, H., Jodoin, P.: Deep learning trends for focal brain pathology segmentation in MRI. CoRR abs/1607.05258 (2016).
  9. 9.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)Google Scholar
  10. 10.
    He, K., Zhang, X., Ren, S., Sun, J.: Delving deep into rectifiers: surpassing human-level performance on imagenet classification. In: Proceedings of the IEEE International Conference on Computer Vision (2015)Google Scholar
  11. 11.
    He, K., Zhang, X., Ren, S., Sun, J.: Identity mappings in deep residual networks. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9908, pp. 630–645. Springer, Cham (2016). doi: 10.1007/978-3-319-46493-0_38 CrossRefGoogle Scholar
  12. 12.
    Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. CoRR abs/1502.03167 (2015).
  13. 13.
    Kamnitsas, K., Ledig, C., Newcombe, V.F.J., Simpson, J.P., Kane, A.D., Menon, D.K., Rueckert, D., Glocker, B.: Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation. CoRR abs/1603.05959 (2016).
  14. 14.
    Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. CoRR abs/1412.6980 (2014).
  15. 15.
    Lai, M.: Deep learning for medical image segmentation. CoRR abs/1505.02000 (2015).
  16. 16.
    Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3431–3440 (2015)Google Scholar
  17. 17.
    Menze, B.H., Jakab, A., Bauer, S., Kalpathy-Cramer, J., Farahani, K., Kirby, J., Burren, Y., Porz, N., Slotboom, J., Wiest, R., et al.: The multimodal brain tumor image segmentation benchmark (brats). IEEE Trans. Med. Imaging 34(10), 1993–2024 (2015)CrossRefGoogle Scholar
  18. 18.
    Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. CoRR abs/1505.04597 (2015).
  19. 19.
    Saver, J.L., Goyal, M., van der Lugt, A., Menon, B.K., Majoie, C.B., Dippel, D.W., Campbell, B.C., Nogueira, R.G., Demchuk, A.M., Tomasello, A., et al.: Time to treatment with endovascular thrombectomy and outcomes from ischemic stroke: a meta-analysis. Jama 316(12), 1279–1288 (2016)CrossRefGoogle Scholar
  20. 20.
    Shah, A., Kadam, E., Shah, H., Shinde, S.: Deep residual networks with exponential linear unit. In: Proceedings of the Third International Symposium on Computer Vision and the Internet. ACM (2016)Google Scholar
  21. 21.
    Zeiler, M.D.: Adadelta: an adaptive learning rate method. arXiv preprint arXiv:1212.5701 (2012)

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  • Youngwon Choi
    • 1
  • Yongchan Kwon
    • 1
  • Hanbyul Lee
    • 1
  • Beom Joon Kim
    • 2
  • Myunghee Cho Paik
    • 1
  • Joong-Ho Won
    • 1
    Email author
  1. 1.Department of StatisticsSeoul National UniversitySeoulKorea
  2. 2.Department of Neurology and Cerebrovascular CenterSeoul National University Bundang HospitalSeongnamKorea

Personalised recommendations