Brain Image Labeling Using Multi-atlas Guided 3D Fully Convolutional Networks

  • Longwei Fang
  • Lichi Zhang
  • Dong Nie
  • Xiaohuan Cao
  • Khosro Bahrami
  • Huiguang HeEmail author
  • Dinggang ShenEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10530)


Automatic labeling of anatomical structures in brain images plays an important role in neuroimaging analysis. Among all methods, multi-atlas based segmentation methods are widely used, due to their robustness in propagating prior label information. However, non-linear registration is always needed, which is time-consuming. Alternatively, the patch-based methods have been proposed to relax the requirement of image registration, but the labeling is often determined independently by the target image information, without getting direct assistance from the atlases. To address these limitations, in this paper, we propose a multi-atlas guided 3D fully convolutional networks (FCN) for brain image labeling. Specifically, multi-atlas based guidance is incorporated during the network learning. Based on this, the discriminative of the FCN is boosted, which eventually contribute to accurate prediction. Experiments show that the use of multi-atlas guidance improves the brain labeling performance.


  1. 1.
    Jia, H., Yap, P.T., Shen, D.: Iterative multi-atlas-based multi-image segmentation with tree-based registration. NeuroImage 59(1), 422–430 (2012)CrossRefGoogle Scholar
  2. 2.
    Wolz, R., Aljabar, P., Hajnal, J.V., Hammers, A., Rueckert, D., The Alzheimer’s Disease Neuroimaging Initiative: LEAP: learning embeddings for atlas propagation. NeuroImage 49(2), 1316–1325 (2010)Google Scholar
  3. 3.
    Langerak, T.R., van der Heide, U.A., Kotte, A.N., Viergever, M.A., Van Vulpen, M., Pluim, J.P.: Label fusion in atlas-based segmentation using a selective and iterative method for performance level estimation (SIMPLE). IEEE Trans. Med. Imaging 29(12), 2000–2008 (2010)CrossRefGoogle Scholar
  4. 4.
    Iglesias, J.E., Sabuncu, M.R.: Multi-atlas segmentation of biomedical images: a survey. Med. Image Anal. 24(1), 205–219 (2015)CrossRefGoogle Scholar
  5. 5.
    Coupé, P., Manjón, J.V., Fonov, V., Pruessner, J., Robles, M., Collins, D.L.: Patchbased segmentation using expert priors: application to hippocampus and ventricle segmentation. NeuroImage 54(2), 940–954 (2011)CrossRefGoogle Scholar
  6. 6.
    Wang, H., Suh, J.W., Das, S.R., Pluta, J.B., Craige, C., Yushkevich, P.A.: Multiatlas segmentation with joint label fusion. IEEE Trans. Pattern Anal. Mach. Intell. 35(3), 611–623 (2013)CrossRefGoogle Scholar
  7. 7.
    Wu, G., Kim, M., Sanroma, G., Wang, Q., Munsell, B.C., Shen, D., The Alzheimer’s Disease Neuroimaging Initiative: Hierarchical multi-atlas label fusion with multi-scale feature representation and labelspecific patch partition. NeuroImage 106, 34–46 (2015)Google Scholar
  8. 8.
    Tu, Z., Bai, X.: Auto-context and its application to high-level vision tasks and 3D brain image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 32(10), 1744–1757 (2010)CrossRefGoogle Scholar
  9. 9.
    Hao, Y., Wang, T., Zhang, X., Duan, Y., Yu, C., Jiang, T., Fan, Y.: Local label learning (LLL) for subcortical structure segmentation: application to hippocampus segmentation. Hum. Brain Mapp. 35(6), 2674–2697 (2014)CrossRefGoogle Scholar
  10. 10.
    Zikic, D., Glocker, B., Criminisi, A.: Encoding atlases by randomized classification forests for efficient multi-atlas label propagation. Med. Image Anal. 18(8), 1262–1273 (2014)CrossRefGoogle Scholar
  11. 11.
    Zhang, L., Wang, Q., Gao, Y., Wu, G., Shen, D.: Automatic labeling of MR brain images by hierarchical learning of atlas forests. Med. Phys. 43(3), 1175–1186 (2016)CrossRefGoogle Scholar
  12. 12.
    Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3431–3440 (2015)Google Scholar
  13. 13.
    Nie, D., Wang, L., Gao, Y., Sken, D.: Fully convolutional networks for multimodality isointense infant brain image segmentation. In: 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI), pp. 1342–1345. IEEE (2016)Google Scholar
  14. 14.
    Woolrich, M.W., Jbabdi, S., Patenaude, B., Chappell, M., Makni, S., Behrens, T., Beckmann, C., Jenkinson, M., Smith, S.M.: Bayesian analysis of neuroimaging data in FSL. Neuroimage 45(1), S173–S186 (2009)CrossRefGoogle Scholar
  15. 15.
    Landman, B., Warfield, S.: Miccai 2012 workshop on multi-atlas labeling. In: Medical Image Computing and Computer Assisted Intervention Conference 2012: MICCAI 2012 Grand Challenge and Workshop on Multi-Atlas Labeling Challenge Results (2012)Google Scholar
  16. 16.
    Moeskops, P., Viergever, M.A., Mendrik, A.M., de Vries, L.S., Benders, M.J., Iˇsgum, I.: Automatic segmentation of MR brain images with a convolutional neural network. IEEE Trans. Med. Imaging 35(5), 1252–1261 (2016)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Longwei Fang
    • 1
    • 2
    • 4
  • Lichi Zhang
    • 4
  • Dong Nie
    • 4
  • Xiaohuan Cao
    • 4
    • 5
  • Khosro Bahrami
    • 4
  • Huiguang He
    • 1
    • 2
    • 3
    Email author
  • Dinggang Shen
    • 4
    Email author
  1. 1.Research Center for Brain-Inspired Intelligence, Institute of AutomationChinese Academy of SciencesBeijingChina
  2. 2.University of Chinese Academy of SciencesBeijingChina
  3. 3.Center for Excellence in Brain Science and Intelligence TechnologyChinese Academy of SciencesBeijingChina
  4. 4.Department of Radiology and BRICUniversity of North Carolina at Chapel HillChapel HillUSA
  5. 5.School of AutomationNorthwestern Polytechnical UniversityXi’anChina

Personalised recommendations