Advertisement

Scalable Multimodal Convolutional Networks for Brain Tumour Segmentation

  • Lucas FidonEmail author
  • Wenqi Li
  • Luis C. Garcia-Peraza-Herrera
  • Jinendra Ekanayake
  • Neil Kitchen
  • Sebastien Ourselin
  • Tom Vercauteren
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10435)

Abstract

Brain tumour segmentation plays a key role in computer-assisted surgery. Deep neural networks have increased the accuracy of automatic segmentation significantly, however these models tend to generalise poorly to different imaging modalities than those for which they have been designed, thereby limiting their applications. For example, a network architecture initially designed for brain parcellation of monomodal T1 MRI can not be easily translated into an efficient tumour segmentation network that jointly utilises T1, T1c, Flair and T2 MRI. To tackle this, we propose a novel scalable multimodal deep learning architecture using new nested structures that explicitly leverage deep features within or across modalities. This aims at making the early layers of the architecture structured and sparse so that the final architecture becomes scalable to the number of modalities. We evaluate the scalable architecture for brain tumour segmentation and give evidence of its regularisation effect compared to the conventional concatenation approach.

Notes

Acknowledgements

This work was supported by the Wellcome Trust (WT101957, 203145Z/16/Z, HICF-T4-275, WT 97914), EPSRC (NS/A000027/1, EP/H046410/1, EP/J020990/1, EP/K005278, NS/A000050/1), the NIHR BRC UCLH/UCL, a UCL ORS/GRS Scholarship and a hardware donation from NVidia.

References

  1. 1.
    Chen, H., Dou, Q., Yu, L., Qin, J., Heng, P.A.: Voxresnet: Deep voxelwise residual networks for brain segmentation from 3D MR images. NeuroImage (2017). http://www.sciencedirect.com/science/article/pii/S1053811917303348
  2. 2.
    Havaei, M., Davy, A., Warde-Farley, D., Biard, A., Courville, A., Bengio, Y., Pal, C., Jodoin, P.M., Larochelle, H.: Brain tumor segmentation with deep neural networks. Med. Image Anal. 35, 18–31 (2017)CrossRefGoogle Scholar
  3. 3.
    Havaei, M., Guizard, N., Chapados, N., Bengio, Y.: HeMIS: hetero-modal image segmentation. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9901, pp. 469–477. Springer, Cham (2016). doi: 10.1007/978-3-319-46723-8_54 CrossRefGoogle Scholar
  4. 4.
    Ioannou, Y., Robertson, D., Cipolla, R., Criminisi, A.: Deep roots: Improving CNN efficiency with hierarchical filter groups. In: CVPR 2017 (2017)Google Scholar
  5. 5.
    Ioannou, Y., Robertson, D., Zikic, D., Kontschieder, P., Shotton, J., Brown, M., Criminisi, A.: Decision forests, convolutional networks and the models in-between arXiv:1603.01250 (2016)
  6. 6.
    Kamnitsas, K., Ledig, C., Newcombe, V.F., Simpson, J.P., Kane, A.D., Menon, D.K., Rueckert, D., Glocker, B.: Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation. Med. Image Anal. 36, 61–78 (2017)CrossRefGoogle Scholar
  7. 7.
    Kingma, D., Ba, J.: Adam: A method for stochastic optimization arXiv:1412.6980 (2014)
  8. 8.
    Li, W., Wang, G., Fidon, L., Ourselin, S., Cardoso, M.J., Vercauteren, T.: On the compactness, efficiency, and representation of 3D convolutional networks: brain parcellation as a pretext task. In: Niethammer, M., Styner, M., Aylward, S., Zhu, H., Oguz, I., Yap, P.-T., Shen, D. (eds.) IPMI 2017. LNCS, vol. 10265, pp. 348–360. Springer, Cham (2017). doi: 10.1007/978-3-319-59050-9_28 CrossRefGoogle Scholar
  9. 9.
    Menze, B.H., Jakab, A., Bauer, S., Kalpathy-Cramer, J., Farahani, K., Kirby, J., Burren, Y., Porz, N., Slotboom, J., Wiest, R., et al.: The multimodal brain tumor image segmentation benchmark (BraTS). IEEE Trans. Med. Imag. 34(10), 1993–2024 (2015)CrossRefGoogle Scholar
  10. 10.
    Milletari, F., Navab, N., Ahmadi, S.A.: V-Net: fully convolutional neural networks for volumetric medical image segmentation. In: Proceeding of 3DV 2016, pp. 565–571 (2016)Google Scholar
  11. 11.
    Nyul, L.G., Udupa, J.K., Zhang, X.: New variants of a method of MRI scale standardization. IEEE Trans. Med. Imag. 19(2), 143–150 (2000)CrossRefGoogle Scholar
  12. 12.
    Pereira, S., Pinto, A., Alves, V., Silva, C.A.: Brain tumor segmentation using convolutional neural networks in MRI images. IEEE Trans. Med. Imag. 35(5), 1240–1251 (2016)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Lucas Fidon
    • 1
    Email author
  • Wenqi Li
    • 1
  • Luis C. Garcia-Peraza-Herrera
    • 1
  • Jinendra Ekanayake
    • 2
    • 3
  • Neil Kitchen
    • 2
  • Sebastien Ourselin
    • 1
    • 3
  • Tom Vercauteren
    • 1
    • 3
  1. 1.TIG, CMICUniversity College LondonLondonUK
  2. 2.NHNNUniversity College London HospitalsLondonUK
  3. 3.Wellcome/EPSRC Centre for Surgical and Interventional ScienceUCLLondonUK

Personalised recommendations