4D Multi-atlas Label Fusion Using Longitudinal Images

  • Yuankai HuoEmail author
  • Susan M. Resnick
  • Bennett A. Landman
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10530)


Longitudinal reproducibility is an essential concern in automated medical image segmentation, yet has proven to be an elusive objective as manual brain structure tracings have shown more than 10% variability. To improve reproducibility, longitudinal segmentation (4D) approaches have been investigated to reconcile temporal variations with traditional 3D approaches. In the past decade, multi-atlas label fusion has become a state-of-the-art segmentation technique for 3D image and many efforts have been made to adapt it to a 4D longitudinal fashion. However, the previous methods were either limited by using application specified energy function (e.g., surface fusion and multi model fusion) or only considered temporal smoothness on two consecutive time points (t and t + 1) under sparsity assumption. Therefore, a 4D multi-atlas label fusion theory for general label fusion purpose and simultaneously considering temporal consistency on all time points is appealing. Herein, we propose a novel longitudinal label fusion algorithm, called 4D joint label fusion (4DJLF), to incorporate the temporal consistency modeling via non-local patch-intensity covariance models. The advantages of 4DJLF include: (1) 4DJLF is under the general label fusion framework by simultaneously incorporating the spatial and temporal covariance on all longitudinal time points. (2) The proposed algorithm is a longitudinal generalization of a leading joint label fusion method (JLF) that has proven adaptable to a wide variety of applications. (3) The spatial temporal consistency of atlases is modeled in a probabilistic model inspired from both voting based and statistical fusion. The proposed approach improves the consistency of the longitudinal segmentation while retaining sensitivity compared with original JLF approach using the same set of atlases. The method is available online in open-source.



This research was supported by NSF CAREER 1452485, NIH 5R21EY024036, NIH 1R21NS064534, NIH 2R01EB006136, NIH 1R03EB012461, and supported by the Intramural Research Program, National Institute on Aging, NIH. This project was supported in part by the National Center for Research Resources, Grant UL1 RR024975-01, and is now at the National Center for Advancing Translational Sciences, Grant 2 UL1 TR000445-06. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH.


  1. 1.
    Roy, S., Carass, A., Pacheco, J., Bilgel, M., Resnick, S.M., Prince, J.L., Pham, D.L.: Temporal filtering of longitudinal brain magnetic resonance images for consistent segmentation. NeuroImage Clin. 11, 264–275 (2016)CrossRefGoogle Scholar
  2. 2.
    Pham, D.L.: Spatial models for fuzzy clustering. Comput. Vis. Image Underst. 84, 285–297 (2001)CrossRefzbMATHGoogle Scholar
  3. 3.
    Iglesias, J.E., Sabuncu, M.R.: Multi-atlas segmentation of biomedical images: a survey. Med. Image Anal. 24, 205–219 (2015)CrossRefGoogle Scholar
  4. 4.
    Huo, Y., Asman, A.J., Plassard, A.J., Landman, B.A.: Simultaneous total intracranial volume and posterior fossa volume estimation using multi-atlas label fusion. Hum. Brain Mapp. 38, 599–616 (2017)CrossRefGoogle Scholar
  5. 5.
    Huo, Y., Plassard, A.J., Carass, A., Resnick, S.M., Pham, D.L., Prince, J.L., Landman, B.A.: Consistent cortical reconstruction and multi-atlas brain segmentation. NeuroImage 138, 197–210 (2016)CrossRefGoogle Scholar
  6. 6.
    Li, G., Wang, L., Shi, F., Lin, W., Shen, D.: Multi-atlas based simultaneous labeling of longitudinal dynamic cortical surfaces in infants. In: Mori, K., Sakuma, I., Sato, Y., Barillot, C., Navab, N. (eds.) MICCAI 2013. LNCS, vol. 8149, pp. 58–65. Springer, Heidelberg (2013). doi: 10.1007/978-3-642-40811-3_8 CrossRefGoogle Scholar
  7. 7.
    Guo, Y., Wu, G., Yap, P.-T., Jewells, V., Lin, W., Shen, D.: Segmentation of infant hippocampus using common feature representations learned for multimodal longitudinal data. In: Navab, N., Hornegger, J., Wells, William M., Frangi, Alejandro F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 63–71. Springer, Cham (2015). doi: 10.1007/978-3-319-24574-4_8 CrossRefGoogle Scholar
  8. 8.
    Wang, L., Guo, Y., Cao, X., Wu, G., Shen, D.: Consistent multi-atlas hippocampus segmentation for longitudinal MR brain images with temporal sparse representation. In: Wu, G., Coupé, P., Zhan, Y., Munsell, Brent C., Rueckert, D. (eds.) Patch-MI 2016. LNCS, vol. 9993, pp. 34–42. Springer, Cham (2016). doi: 10.1007/978-3-319-47118-1_5 CrossRefGoogle Scholar
  9. 9.
    Ourselin, S., Roche, A., Subsol, G., Pennec, X., Ayache, N.: Reconstructing a 3D structure from serial histological sections. Image Vis. Comput. 19, 25–31 (2001)CrossRefGoogle Scholar
  10. 10.
    Wang, H.Z., Suh, J.W., Das, S.R., Pluta, J.B., Craige, C., Yushkevich, P.A.: Multi-atlas segmentation with joint label fusion. IEEE Trans. Pattern Anal. 35, 611–623 (2013)CrossRefGoogle Scholar
  11. 11.
    Resnick, S.M., Pham, D.L., Kraut, M.A., Zonderman, A.B., Davatzikos, C.: Longitudinal magnetic resonance imaging studies of older adults: a shrinking brain. J. Neurosci.: Off. J. Soc. Neurosci. 23, 3295–3301 (2003)Google Scholar
  12. 12.
    Avants, B.B., Epstein, C.L., Grossman, M., Gee, J.C.: Symmetric diffeomorphic image registration with cross-correlation: evaluating automated labeling of elderly and neurodegenerative brain. Med. Image Anal. 12, 26–41 (2008)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Yuankai Huo
    • 1
    Email author
  • Susan M. Resnick
    • 2
  • Bennett A. Landman
    • 1
  1. 1.Electrical EngineeringVanderbilt UniversityNashvilleUSA
  2. 2.Laboratory of Behavioral NeuroscienceNational Institute on AgingBaltimoreUSA

Personalised recommendations