Cognitive Computation

, Volume 11, Issue 5, pp 630–643 | Cite as

Multi-target Interactive Neural Network for Automated Segmentation of the Hippocampus in Magnetic Resonance Imaging

  • Beibei Hou
  • Guixia KangEmail author
  • Ningbo Zhang
  • Kui Liu


The hippocampus has been recognized as an important biomarker for the diagnosis and assessment of neurological diseases. Convenient and accurate automated segmentation of the hippocampus facilitates the analysis of large-scale neuroimaging studies. This work describes a novel technique for hippocampus segmentation in magnetic resonance images, in which interactive neural network (Inter-Net) is based on 3D convolutional operations. Inter-Net achieves the interaction through two aspects: one is the compartments, which builds an exponential ensemble network that integrates numerous short networks together when forward propagation. The other is the pathways, which realizes inter-connection between feature extraction and restoration. In addition, a multi-target architecture is proposed by designing multiple objective functions in terms of evaluation index, information theory, and data distribution. The proposed architecture is validated in fivefold cross-validation on the Alzheimer’s Disease Neuroimaging Initiative (ADNI) dataset, where the mean Dice similarity indices of 0.919 (± 0.023) and precision of 0.926 (± 0.032) for the hippocampus segmentation. The running time is approximately 42.1 s from reading the image to outputting the segmentation result in our computer configuration. We compare the experimental results of a variety of methods to prove the effectiveness of the Inter-Net and contrast integrated architectures with different objective functions to illustrate the robustness of the fusion. The proposed framework is general and can be easily extended to numerous tissue segmentation tasks while it is tailored for the hippocampus.


Hippocampus Interactive neural network Magnetic resonance images Multi-target Objective function 



This work was supported by National Natural Science Foundation of China (61471064), National Science and Technology Major Project of China (No.2017ZX03001022), and BUPT Excellent Ph.D. Students Foundation (No.CX2019309).

Compliance with Ethical Standards

Conflict of Interest

The authors declare that they have no conflict of interest.

Ethical Approval

This article does not contain any studies with human participants or animals performed by any of the authors.


  1. 1.
    Czepielewski LS, Wang L, Gama CS, et al. The relationship of intellectual functioning and cognitive performance to brain structure in schizophrenia. Schizophr Bull. 2017;43(2):355–64.PubMedGoogle Scholar
  2. 2.
    Steiger VR, Brühl AB, Weidt S, Delsignore A, Rufer M, Jäncke L, et al. Pattern of structural brain changes in social anxiety disorder after cognitive behavioral group therapy: a longitudinal multimodal MRI study. Mol Psychiatry. 2017;22(8):1164–71.PubMedCrossRefGoogle Scholar
  3. 3.
    den Heijer T, van der Lijn F, Vernooij MW, et al. Structural and diffusion MRI measures of the hippocampus and memory performance. Neuroimage. 2012;63(4):1782–9.CrossRefGoogle Scholar
  4. 4.
    Wixted JT, Squire LR. The medial temporal lobe and the attributes of memory. Trends Cogn Sci. 2011;15(5):210–7.PubMedPubMedCentralCrossRefGoogle Scholar
  5. 5.
    Jeneson A, Squire LR. Working memory, long-term memory, and medial temporal lobe function. Learn Mem. 2012;19(1):15–25.PubMedPubMedCentralCrossRefGoogle Scholar
  6. 6.
    Bobinski M, Wegiel J, Wisniewski HM, Tarnawski M, Bobinski M, Reisberg B, et al. Neurofibrillary pathology—correlation with hippocampal formation atrophy in Alzheimer disease. Neurobiol Aging. 1996;17(6):909–19.PubMedGoogle Scholar
  7. 7.
    Geuze E, Vermetten E, Bremner JD. MR-based in vivo hippocampal volumetrics: 1. Review of methodologies currently employed. Mol Psychiatry. 2005;10(2):147–59.PubMedCrossRefGoogle Scholar
  8. 8.
    Knickmeyer RC, Gouttard S, Kang C, Evans D, Wilber K, Smith JK, et al. A structural MRI study of human brain development from birth to 2 years. J Neurosci. 2008;28(47):12176–82.PubMedPubMedCentralCrossRefGoogle Scholar
  9. 9.
    Filippi M, Rocca MA, Ciccarelli O, De Stefano N, Evangelou N, Kappos L, et al. MRI criteria for the diagnosis of multiple sclerosis: MAGNIMS consensus guidelines. Lancet Neurol. 2016;15(3):292–303.PubMedPubMedCentralCrossRefGoogle Scholar
  10. 10.
    Jacobsen C, Hagemeier J, Myhr KM, Nyland H, Lode K, Bergsland N, et al. Brain atrophy and disability progression in multiple sclerosis patients: a 10-year follow-up study. J Neurol Neurosurg Psychiatry. 2014;85(10):1109–15.PubMedCrossRefGoogle Scholar
  11. 11.
    Andreasen NC, Liu D, Ziebell S, Vora A, Ho BC. Relapse duration, treatment intensity, and brain tissue loss in schizophrenia: a prospective longitudinal MRI study. Am J Psychiatr. 2013;170(6):609–15.PubMedCrossRefGoogle Scholar
  12. 12.
    Scheenstra AEH, van de Ven RCG, van der Weerd L, van den Maagdenberg AM, Dijkstra J, Reiber JH. Automated segmentation of in vivo and ex vivo mouse brain magnetic resonance images. Mol Imaging. 2009;8(1):35–44.PubMedCrossRefGoogle Scholar
  13. 13.
    Carmichael OT, Aizenstein HA, Davis SW, Becker JT, Thompson PM, Meltzer CC, et al. Atlas-based hippocampus segmentation in Alzheimer’s disease and mild cognitive impairment. Neuroimage. 2005;27(4):979–90.PubMedPubMedCentralCrossRefGoogle Scholar
  14. 14.
    Chupin M, Mukuna-Bantumbakulu AR, Hasboun D, Bardinet E, Baillet S, Kinkingnéhun S, et al. Anatomically constrained region deformation for the automated segmentation of the hippocampus and the amygdala: method and validation on controls and patients with Alzheimer’s disease. Neuroimage. 2007;34(3):996–1019.PubMedCrossRefGoogle Scholar
  15. 15.
    Fischl B, Salat DH, Busa E, Albert M, Dieterich M, Haselgrove C, et al. Whole brain segmentation: automated labeling of neuroanatomical structures in the human brain. Neuron. 2002;33(3):341–55.PubMedPubMedCentralCrossRefGoogle Scholar
  16. 16.
    Zandifar A, Fonov V, Coupé P, Pruessner J, Collins DL, Alzheimer’s Disease Neuroimaging Initiative. A comparison of accurate automatic hippocampal segmentation methods. NeuroImage. 2017;155:383–93.PubMedCrossRefGoogle Scholar
  17. 17.
    Hosseini MP, Nazem Zadeh MR, Pompili D, Jafari-Khouzani K, Elisevich K, Soleanian-Zadeh H. Comparative performance evaluation of automated segmentation methods of hippocampus from magnetic resonance images of temporal lobe epilepsy patients. Med Phys. 2016;43(1):538–53.PubMedPubMedCentralCrossRefGoogle Scholar
  18. 18.
    Dill V, Franco AR, Pinho MS. Automated methods for hippocampus segmentation: the evolution and a review of the state of the art. Neuroinformatics. 2015;13(2):133–50.PubMedCrossRefGoogle Scholar
  19. 19.
    Birenbaum A, Greenspan H. Multi-view longitudinal CNN for multiple sclerosis lesion segmentation. Eng Appl Artif Intell. 2017;65:111–8.CrossRefGoogle Scholar
  20. 20.
    Kwak K, Yoon U, Lee DK, Kim GH, Seo SW, Na DL. Fully-automated approach to hippocampus segmentation using a graph-cuts algorithm combined with atlas-based segmentation and morphological opening. Magn Reson Imaging. 2013;31(7):1190–6.CrossRefGoogle Scholar
  21. 21.
    Pipitone J, Park MTM, Winterburn J, Lett TA, Lerch JP, Pruessner JC, et al. Multi-atlas segmentation of the whole hippocampus and subfields using multiple automatically generated templates. Neuroimage. 2014;101:494–512.PubMedCrossRefGoogle Scholar
  22. 22.
    Sabuncu MR, Yeo BTT, Van Leemput K, Fischl B, Golland P. A generative model for image segmentation based on label fusion. IEEE Trans Med Imaging. 2010;29(10):1714–29.PubMedPubMedCentralCrossRefGoogle Scholar
  23. 23.
    Van der Lijn F, De Bruijne M, Klein S, Den Heijer T, Hoogendam YY, Van der Lugt A, et al. Automated brain structure segmentation based on atlas registration and appearance models. IEEE Trans Med Imaging. 2012;31(2):276–86.PubMedCrossRefGoogle Scholar
  24. 24.
    Kim M, Wu G, Li W, Wang L, Son YD, Cho ZH, et al. Automatic hippocampus segmentation of 7.0 Tesla MR images by combining multiple atlases and auto-context models. NeuroImage. 2013;83:335–45.PubMedPubMedCentralCrossRefGoogle Scholar
  25. 25.
    Hao Y, Wang T, Zhang X, Duan Y, Yu C, Jiang T, et al. Local label learning (LLL) for subcortical structure segmentation: application to hippocampus segmentation. Hum Brain Mapp. 2014;35(6):2674–97.CrossRefGoogle Scholar
  26. 26.
    Moghaddam MJ, Soltanian-Zadeh H. Automatic segmentation of brain structures using geometric moment invariants and artificial neural networks//International conference on Information Processing in Medical Imaging. Berlin: Springer; 2009. p. 326–37.Google Scholar
  27. 27.
    Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation. Proceedings of the IEEE conference on computer vision and pattern recognition, 2015. pp. 3431–3440.Google Scholar
  28. 28.
    Ronneberger O, Fischer P, Brox T. U-net: convolutional networks for biomedical image segmentation. International conference on medical image computing and computer-assisted intervention. Cham: Springer; 2015. p. 234–41.Google Scholar
  29. 29.
    Kamnitsas K, Ledig C, Newcombe VF, Simpson JP, Kane AD, Menon DK, et al. Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation. Med Image Anal. 2017;36:61–78.PubMedCrossRefGoogle Scholar
  30. 30.
    Liu X, Deng Z. Segmentation of drivable road using deep fully convolutional residual network with pyramid pooling. Cogn Comput. 2017:1–10.Google Scholar
  31. 31.
    Liu W, Tao D. Multiview Hessian regularization for image annotation. IEEE Trans Image Process. 2013;22(7):2676–87.PubMedCrossRefGoogle Scholar
  32. 32.
    Liu W, Yang X, Tao D, Cheng J, Tang Y. Multiview dimension reduction via Hessian multiset canonical correlations. Information Fusion. 2018;41:119–28.CrossRefGoogle Scholar
  33. 33.
    Yuan Y, Xun G, Ma F, et al. Muvan: a multi-view attention network for multivariate temporal data. 2018 IEEE International Conference on Data Mining (ICDM). Piscataway: IEEE; 2018. p. 717–26.Google Scholar
  34. 34.
    Kang G, Liu K, Hou B, Zhang N. 3D multi-view convolutional neural networks for lung nodule classification. PloS one, Public Library of Science. 2017;12(11):e0188290.CrossRefGoogle Scholar
  35. 35.
    Setio AAA, Ciompi F, Litjens G, Gerke P, Jacobs C, van Riel SJ, et al. Pulmonary nodule detection in CT images: false positive reduction using multi-view convolutional networks. IEEE Trans Med Imaging. 2016;35(5):1160–9.PubMedCrossRefGoogle Scholar
  36. 36.
    Fischl B. FreeSurfer. Neuroimage. 2012;62(2):774–81.PubMedPubMedCentralCrossRefGoogle Scholar
  37. 37.
    Chen Y, Shi B, Wang Z, Zhang P, Smith CD, Liu J. Hippocampus segmentation through multi-view ensemble ConvNets[C]//Biomedical Imaging (ISBI 2017), 2017 IEEE 14th International Symposium on. IEEE, 2017. pp. 192–196.Google Scholar
  38. 38.
    Jack CR Jr, Bernstein MA, Fox NC, et al. The Alzheimer’s disease neuroimaging initiative (ADNI): MRI methods. J Magn Reson Imaging. 2008;27(4):685–91.PubMedPubMedCentralCrossRefGoogle Scholar
  39. 39.
    Wen G, Hou Z, Li H, Li D, Jiang L, Xun E. Ensemble of deep neural networks with probability-based fusion for facial expression recognition. Cogn Comput. 2017;9(5):597–610.CrossRefGoogle Scholar
  40. 40.
    Brosch T, Tang LY, Yoo Y, Li DK. Deep 3D convolutional encoder networks with shortcuts for multiscale feature integration applied to multiple sclerosis lesion segmentation. IEEE Trans Med Imaging. 2016;35(5):1229–39.PubMedCrossRefGoogle Scholar
  41. 41.
    Veit A, Wilber M, Belongie S. Residual networks are exponential ensembles of relatively shallow networks. arXiv preprint. arXiv preprint arXiv:1605.06431. 2016;1(2):3.Google Scholar
  42. 42.
    He K, Zhang X, Ren S, Sun J. Identity mappings in deep residual networks. European Conference on Computer Vision. Cham: Springer; 2016. p. 630–45.Google Scholar
  43. 43.
    Nair V, Hinton G E. Rectified linear units improve restricted boltzmann machines. Proceedings of the 27th international conference on machine learning (ICML-10). 2010. pp. 807–814.Google Scholar
  44. 44.
    Zeiler MD. ADADELTA: an adaptive learning rate method. arXiv preprint arXiv:1212.5701. 2012.Google Scholar
  45. 45.
    Kingma DP, Ba J. Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980. 2014.Google Scholar
  46. 46.
    Dauphin Y, de Vries H, Bengio Y. Equilibrated adaptive learning rates for non-convex optimization[C]. Adv Neural Inf Proces Syst. 2015:1504–12.Google Scholar
  47. 47.
    Srivastava N, Hinton G, Krizhevsky A, Stuskever I, Salakhutdinov R. Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res. 2014;15(1):1929–58.Google Scholar
  48. 48.
    Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556. 2014.Google Scholar
  49. 49.
    Zeng D, Zhao F, Shen W, Ge S. Compressing and accelerating neural network for facial point localization. Cogn Comput. 2017:1–9.Google Scholar
  50. 50.
    Dice LR. Measures of the amount of ecologic association between species. Ecology. 1945;26(3):297–302.CrossRefGoogle Scholar
  51. 51.
    Cabezas M, Oliver A, Lladó X, Freixenet J, Cuadra MB. A review of atlas-based segmentation for magnetic resonance brain images. Comput Methods Prog Biomed. 2011;104(3):e158–77.CrossRefGoogle Scholar
  52. 52.
    Ghanei A, Soltanian-Zadeh H, Windham JP. A 3D deformable surface model for segmentation of objects from volumetric data in medical images. Comput Biol Med. 1998;28(3):239–2.PubMedCrossRefGoogle Scholar
  53. 53.
    Lötjönen JMP, Wolz R, Koikkalainen JR, Thurfjell L, Waldemar G, Soininen H, et al. Fast and robust multi-atlas segmentation of brain magnetic resonance images. Neuroimage. 2010;49(3):2352–65.PubMedCrossRefGoogle Scholar
  54. 54.
    He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. Proceedings of the IEEE conference on computer vision and pattern recognition. 2016. pp. 770–778.Google Scholar
  55. 55.
    Wolz R, Aljabar P, Hajnal JV, Hammers A, Rueckert D. Alzheimer’s Disease Neuroimaging Initiative. LEAP: learning embeddings for atlas propagation. NeuroImage. 2010;49(2):1316–25.PubMedCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  • Beibei Hou
    • 1
    • 2
  • Guixia Kang
    • 1
    • 2
    Email author
  • Ningbo Zhang
    • 1
  • Kui Liu
    • 1
    • 2
  1. 1.Beijing University of Posts and TelecommunicationsBeijingChina
  2. 2.Wuxi BUPT Sensory Technology and Industry Institute CO.LTDWuxiChina

Personalised recommendations