Transfer Learning from Partial Annotations for Whole Brain Segmentation

  • Chengliang DaiEmail author
  • Yuanhan Mo
  • Elsa Angelini
  • Yike Guo
  • Wenjia Bai
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11795)


Brain MR image segmentation is a key task in neuroimaging studies. It is commonly conducted using standard computational tools, such as FSL, SPM, multi-atlas segmentation etc, which are often registration-based and suffer from expensive computation cost. Recently, there is an increased interest using deep neural networks for brain image segmentation, which have demonstrated advantages in both speed and performance. However, neural networks-based approaches normally require a large amount of manual annotations for optimising the massive amount of network parameters. For 3D networks used in volumetric image segmentation, this has become a particular challenge, as a 3D network consists of many more parameters compared to its 2D counterpart. Manual annotation of 3D brain images is extremely time-consuming and requires extensive involvement of trained experts. To address the challenge with limited manual annotations, here we propose a novel multi-task learning framework for brain image segmentation, which utilises a large amount of automatically generated partial annotations together with a small set of manually created full annotations for network training. Our method yields a high performance comparable to state-of-the-art methods for whole brain segmentation.



This research is independent research funded by the NIHR Imperial Biomedical Research Centre (BRC). The views expressed in this publication are those of the author(s) and not necessarily those of the NHS, NIHR or Department of Health. The research is conducted using the UK Biobank Resource under Application Number 18545. We gratefully acknowledge the support of NVIDIA Corporation with the donation of the GPU used for this research.


  1. 1.
    Douaud, G., Groves, A.R., Tamnes, C.K., Westlye, L.T., Duff, E.P., et al.: A common brain network links development, aging, and vulnerability to disease. Proc. Natl. Acad. Sci. 111(49), 17648–17653 (2014)CrossRefGoogle Scholar
  2. 2.
    Jenkinson, M., Beckmann, C.F., Behrens, T.E., Woolrich, M.W., Smith, S.M.: FSL. NeuroImage 62(2), 782–790 (2012)CrossRefGoogle Scholar
  3. 3.
    Ashburner, J., Friston, K.J.: Voxel-based morphometry - the methods. NeuroImage 11(6), 805–821 (2000)CrossRefGoogle Scholar
  4. 4.
    Ledig, C., et al.: Robust whole-brain segmentation: application to traumatic brain injury. Med. Image Anal. 21(1), 40–58 (2015)CrossRefGoogle Scholar
  5. 5.
    Rajchl, M., Pawlowski, N., Rueckert, D., Matthews, P.M., Glocker, B.: NeuroNet: fast and robust reproduction of multiple brain image segmentation pipelines. In: International Conference on Medical Imaging with Deep Learning (MIDL) (2018)Google Scholar
  6. 6.
    Tajbakhsh, N., et al.: Convolutional neural networks for medical image analysis: full training or fine tuning? IEEE Trans. Med. Imaging 35(5), 1299–1312 (2016)CrossRefGoogle Scholar
  7. 7.
    Roy, A.G., Conjeti, S., Navab, N., Wachinger, C., Alzheimer’s Disease Neuroimaging Initiative: QuickNAT: a fully convolutional network for quick and accurate segmentation of neuroanatomy. NeuroImage 186, 713–727 (2019)Google Scholar
  8. 8.
    Moeskops, P., et al.: Deep learning for multi-task medical image segmentation in multiple modalities. In: International Conference on Medical Image Computing and Computer Assisted Intervention, pp. 478–486 (2016)Google Scholar
  9. 9.
    Shin, H.C., et al.: Medical image synthesis for data augmentation and anonymization using generative adversarial networks. In: International Workshop on Simulation and Synthesis in Medical Imaging, pp. 1–11 (2018)Google Scholar
  10. 10.
    Hammers, A., et al.: Three-dimensional maximum probability atlas of the human brain, with particular reference to the temporal lobe. Hum. Brain Mapp. 19(4), 224–247 (2003)CrossRefGoogle Scholar
  11. 11.
    Gousias, I.S., et al.: Automatic segmentation of brain MRIs of 2-year-olds into 83 regions of interest. NeuroImage 40(2), 672–684 (2008)CrossRefGoogle Scholar
  12. 12.
    Landman, B., Warfield, S.: MICCAI 2012 workshop on multi-atlas labeling. In: Medical Image Computing and Computer Assisted Intervention Conference (2012)Google Scholar
  13. 13.
    Huo, Y., et al.: 3D whole brain segmentation using spatially localized atlas network tiles. NeuroImage 194, 105–119 (2019)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Chengliang Dai
    • 1
    Email author
  • Yuanhan Mo
    • 1
  • Elsa Angelini
    • 2
  • Yike Guo
    • 1
  • Wenjia Bai
    • 1
    • 3
  1. 1.Data Science InstituteImperial College LondonLondonUK
  2. 2.ITMAT Data Science GroupImperial College LondonLondonUK
  3. 3.Department of Brain SciencesImperial College LondonLondonUK

Personalised recommendations