, Volume 60, Issue 6, pp 617–634 | Cite as

Multi-center reproducibility of structural, diffusion tensor, and resting state functional magnetic resonance imaging measures

  • S. Deprez
  • Michiel B. de Ruiter
  • S. Bogaert
  • R. Peeters
  • J. Belderbos
  • D. De Ruysscher
  • S. Schagen
  • S. Sunaert
  • P. Pullens
  • E. Achten
Functional Neuroradiology



The aim of this study is to assess multi-center reproducibility and longitudinal consistency of MRI imaging measurements, as part of a phase III longitudinal multi-center study comparing the neurotoxic effect following prophylactic cranial irradiation with hippocampal avoidance (HA-PCI), in comparison with conventional PCI in patients with small-cell lung cancer.


Harmonized MRI acquisition protocols from six participating sites and two different vendors were compared using both physical and human phantoms. We assessed variability across sites and time points by evaluating various phantoms and data including hippocampal volume, diffusion metrics, and resting-state fMRI, from two healthy volunteers.


We report average coefficients of variation (CV) below 5% for intrascanner, intravendor, and intervendor reproducibility for both structural and diffusion imaging metrics, except for diffusion metrics obtained from tractography with average CVs ranging up to 7.8%. Additionally, resting-state fMRI showed stable temporal SNR and reliable generation of subjects DMN across vendors and time points.


These findings indicate that the presented multi-site MRI acquisition protocol can be used in a longitudinal study design and that pooling of the acquired data as part of the phase III longitudinal HA-PCI project is possible with careful monitoring of the results of the half-yearly QA assessment to follow-up on potential scanner-related longitudinal changes in image quality.


Multi-center Reproducibility Structural MRI Diffusion tensor imaging Resting state fMRI 



We are grateful to Dr. Bram Stieltjes (Universitätsspital Basel, CH) for his advice and help with the DTI anisotropic phantom.

Compliance with ethical standards


This study was funded by the Institute for the Promotion of Innovation by Science and Technology in Flanders (Grant Number IWT 130262), the Vlaamse Liga Tegen Kanker (VLK) and the Dutch Cancer Society.

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed consent

Informed consent was obtained from all individual participants included in the study.

Supplementary material

234_2018_2017_Fig6_ESM.gif (126 kb)

(GIF 125 kb)

234_2018_2017_MOESM1_ESM.tif (1 mb)
High Resolution Image (TIFF 1033 kb)
234_2018_2017_Fig7_ESM.gif (9 kb)

(GIF 8 kb)

234_2018_2017_MOESM2_ESM.tif (99 kb)
High Resolution Image (TIFF 98 kb)
234_2018_2017_Fig8_ESM.gif (147 kb)

(GIF 147 kb)

234_2018_2017_MOESM3_ESM.tif (1.4 mb)
High Resolution Image (TIFF 1480 kb)


  1. 1.
    Van Horn JD, Toga AW (2009) Multisite neuroimaging trials. Curr Opin Neurol 22:370–378. CrossRefPubMedPubMedCentralGoogle Scholar
  2. 2.
    Friedman L, Glover GH, Fbirn C (2006) Reducing interscanner variability of activation in a multicenter fMRI study: controlling for signal-to-fluctuation-noise-ratio (SFNR) differences. NeuroImage 33:471–481. CrossRefPubMedGoogle Scholar
  3. 3.
    Fox RJ, Sakaie K, Lee JC, Debbins JP, Liu Y, Arnold DL, Melhem ER, Smith CH, Philips MD, Lowe M, Fisher E (2012) A validation study of multicenter diffusion tensor imaging: reliability of fractional anisotropy and diffusivity values. AJNR Am J Neuroradiol 33:695–700. CrossRefPubMedGoogle Scholar
  4. 4.
    Jovicich J, Minati L, Marizzoni M, Marchitelli R, Sala-Llonch R, Bartrés-Faz D, Arnold J, Benninghoff J, Fiedler U, Roccatagliata L, Picco A, Nobili F, Blin O, Bombois S, Lopes R, Bordet R, Sein J, Ranjeva JP, Didic M, Gros-Dagnac H, Payoux P, Zoccatelli G, Alessandrini F, Beltramello A, Bargalló N, Ferretti A, Caulo M, Aiello M, Cavaliere C, Soricelli A, Parnetti L, Tarducci R, Floridi P, Tsolaki M, Constantinidis M, Drevelegas A, Rossini PM, Marra C, Schönknecht P, Hensch T, Hoffmann KT, Kuijer JP, Visser PJ, Barkhof F, Frisoni GB, PharmaCog Consortium (2016) Longitudinal reproducibility of default-mode network connectivity in healthy elderly participants: a multicentric resting-state fMRI study. NeuroImage 124:442–454. CrossRefPubMedGoogle Scholar
  5. 5.
    Jovicich J, Marizzoni M, Sala-Llonch R, Bosch B, Bartrés-Faz D, Arnold J, Benninghoff J, Wiltfang J, Roccatagliata L, Nobili F, Hensch T, Tränkner A, Schönknecht P, Leroy M, Lopes R, Bordet R, Chanoine V, Ranjeva JP, Didic M, Gros-Dagnac H, Payoux P, Zoccatelli G, Alessandrini F, Beltramello A, Bargalló N, Blin O, Frisoni GB, PharmaCog Consortium (2013) Brain morphometry reproducibility in multi-center 3T MRI studies: a comparison of cross-sectional and longitudinal segmentations. NeuroImage 83:472–484. CrossRefPubMedGoogle Scholar
  6. 6.
    Chalavi S, Simmons A, Dijkstra H, Barker GJ, Reinders AATS (2012) Quantitative and qualitative assessment of structural magnetic resonance imaging data in a two-center study. BMC Med Imaging 12:27. CrossRefPubMedPubMedCentralGoogle Scholar
  7. 7.
    Weiskopf N, Suckling J, Williams G, Correia MM, Inkster B, Tait R, Ooi C, Bullmore ET, Lutti A (2013) Quantitative multi-parameter mapping of R1, PD(*), MT, and R2(*) at 3T: a multi-center validation. Front Neurosci 7:95. CrossRefPubMedPubMedCentralGoogle Scholar
  8. 8.
    Keshavan A, Paul F, Beyer MK, Zhu AH, Papinutto N, Shinohara RT, Stern W, Amann M, Bakshi R, Bischof A, Carriero A, Comabella M, Crane JC, D'Alfonso S, Demaerel P, Dubois B, Filippi M, Fleischer V, Fontaine B, Gaetano L, Goris A, Graetz C, Gröger A, Groppa S, Hafler DA, Harbo HF, Hemmer B, Jordan K, Kappos L, Kirkish G, Llufriu S, Magon S, Martinelli-Boneschi F, McCauley J, Montalban X, Mühlau M, Pelletier D, Pattany PM, Pericak-Vance M, Cournu-Rebeix I, Rocca MA, Rovira A, Schlaeger R, Saiz A, Sprenger T, Stecco A, Uitdehaag BMJ, Villoslada P, Wattjes MP, Weiner H, Wuerfel J, Zimmer C, Zipp F, International Multiple Sclerosis Genetics Consortium. Electronic address: AIVINSON@PARTNERS.ORG, Hauser SL, Oksenberg JR, Henry RG (2016) Power estimation for non-standardized multisite studies. NeuroImage 134:281–294. CrossRefPubMedPubMedCentralGoogle Scholar
  9. 9.
    Jovicich J, Marizzoni M, Bosch B, Bartrés-Faz D, Arnold J, Benninghoff J, Wiltfang J, Roccatagliata L, Picco A, Nobili F, Blin O, Bombois S, Lopes R, Bordet R, Chanoine V, Ranjeva JP, Didic M, Gros-Dagnac H, Payoux P, Zoccatelli G, Alessandrini F, Beltramello A, Bargalló N, Ferretti A, Caulo M, Aiello M, Ragucci M, Soricelli A, Salvadori N, Tarducci R, Floridi P, Tsolaki M, Constantinidis M, Drevelegas A, Rossini PM, Marra C, Otto J, Reiss-Zimmermann M, Hoffmann KT, Galluzzi S, Frisoni GB, PharmaCog Consortium (2014) Multisite longitudinal reliability of tract-based spatial statistics in diffusion tensor imaging of healthy elderly subjects. NeuroImage 101:390–403. CrossRefPubMedGoogle Scholar
  10. 10.
    Teipel SJ, Reuter S, Stieltjes B, Acosta-Cabronero J, Ernemann U, Fellgiebel A, Filippi M, Frisoni G, Hentschel F, Jessen F, Klöppel S, Meindl T, Pouwels PJW, Hauenstein KH, Hampel H (2011) Multicenter stability of diffusion tensor imaging measures: a European clinical and physical phantom study. Psychiatry Res Neuroimaging 194:363–371. CrossRefGoogle Scholar
  11. 11.
    Zhu T, Hu R, Qiu X, Taylor M, Tso Y, Yiannoutsos C, Navia B, Mori S, Ekholm S, Schifitto G, Zhong J (2011) Quantification of accuracy and precision of multi-center DTI measurements: a diffusion phantom and human brain study. NeuroImage 56:1398–1411. CrossRefPubMedPubMedCentralGoogle Scholar
  12. 12.
    Huang L, Wang X, Baliki MN, Wang L, Apkarian AV, Parrish TB (2012) Reproducibility of structural, resting-state BOLD and DTI data between identical scanners. PLoS One 7:e47684. CrossRefPubMedPubMedCentralGoogle Scholar
  13. 13.
    Takao H, Hayashi N, Ohtomo K (2011) Effect of scanner in asymmetry studies using diffusion tensor imaging. NeuroImage 54:1053–1062. CrossRefPubMedGoogle Scholar
  14. 14.
    Vollmar C, O’Muircheartaigh J, Barker GJ et al (2010) Identical, but not the same: intra-site and inter-site reproducibility of fractional anisotropy measures on two 3.0T scanners. NeuroImage 51:1384–1394. CrossRefPubMedPubMedCentralGoogle Scholar
  15. 15.
    Belli G, Busoni S, Ciccarone A, Coniglio A, Esposito M, Giannelli M, Mazzoni LN, Nocetti L, Sghedoni R, Tarducci R, Zatelli G, Anoja RA, Belmonte G, Bertolino N, Betti M, Biagini C, Ciarmatori A, Cretti F, Fabbri E, Fedeli L, Filice S, Fulcheri CPL, Gasperi C, Mangili PA, Mazzocchi S, Meliadò G, Morzenti S, Noferini L, Oberhofer N, Orsingher L, Paruccini N, Princigalli G, Quattrocchi M, Rinaldi A, Scelfo D, Freixas GV, Tenori L, Zucca I, Luchinat C, Gori C, Gobbi G, for the Italian Association of Physics in Medicine (AIFM) Working Group on MR Intercomparison (2016) Quality assurance multicenter comparison of different MR scanners for quantitative diffusion-weighted imaging. J Magn Reson Imaging 43:213–219. CrossRefPubMedGoogle Scholar
  16. 16.
    Laun FB, Huff S, Stieltjes B (2009) On the effects of dephasing due to local gradients in diffusion tensor imaging experiments: relevance for diffusion tensor imaging fiber phantoms. Magn Reson Imaging 27:541–548. CrossRefPubMedGoogle Scholar
  17. 17.
    Jones DK (2011) Diffusion MRI: theory, methods, and applications. Oxford University Press, OxfordGoogle Scholar
  18. 18.
    Dowell NG, Tofts PS (2008) Simple reliable and precise quantitative quality assurance of in-vivo brain ADC. Proc. Int. Soc. Magn. Reson. Med. 16th Annu. Meet. 3152Google Scholar
  19. 19.
    Jack CR Jr, Bernstein MA, Fox NC et al (2008) The Alzheimer’s Disease Neuroimaging Initiative (ADNI): MRI methods. J Magn Reson Imaging 27:685–691. CrossRefPubMedPubMedCentralGoogle Scholar
  20. 20.
    Achten E, Deblaere K, De Wagter C et al (1998) Intra- and interobserver variability of MRI-based volume measurements of the hippocampus and amygdala using the manual ray-tracing method. Neuroradiology 40:558–566CrossRefPubMedGoogle Scholar
  21. 21.
    Reuter M, Schmansky NJ, Rosas HD, Fischl B (2012) Within-subject template estimation for unbiased longitudinal image analysis. NeuroImage 61:1402–1418. CrossRefPubMedPubMedCentralGoogle Scholar
  22. 22.
    Jones R, Payne B (1997) Clinical investigation and statistics in laboratory medicine. ACB Venture Publications, LondonGoogle Scholar
  23. 23.
    Leemans A (2009) ExploreDTI: a graphical toolbox for processing, analyzing, and visualizing diffusion MR data. 17th Annu Meet Intl Soc Mag Reson Med 3537Google Scholar
  24. 24.
    Otsu N (1979) A threshold selection method from gray-level histograms. IEEE Trans Syst Man Cybern 9:62–66CrossRefGoogle Scholar
  25. 25.
    De Santis S, Evans CJ, Jones DK (2013) RAPID: a routine assurance pipeline for imaging of diffusion. Magn Reson Med 70:490–496. CrossRefPubMedGoogle Scholar
  26. 26.
    Holz M, Heil SR, Sacco A (2000) Temperature-dependent self-diffusion coefficients of water and six selected molecular liquids for calibration in accurate 1H NMR PFG measurements. Phys Chem Chem Phys 2:4740–4742. CrossRefGoogle Scholar
  27. 27.
    Wakana S, Caprihan A, Panzenboeck MM, Fallon JH, Perry M, Gollub RL, Hua K, Zhang J, Jiang H, Dubey P, Blitz A, van Zijl P, Mori S (2007) Reproducibility of quantitative tractography methods applied to cerebral white matter. NeuroImage 36:630–644. CrossRefPubMedPubMedCentralGoogle Scholar
  28. 28.
    Lebel C, Walker L, Leemans A, Phillips L, Beaulieu C (2008) Microstructural maturation of the human brain from childhood to adulthood. NeuroImage 40:1044–1055. CrossRefPubMedGoogle Scholar
  29. 29.
    Song XW, Dong ZY, Long XY, Li SF, Zuo XN, Zhu CZ, He Y, Yan CG, Zang YF (2011) REST: a toolkit for resting-state functional magnetic resonance imaging data processing. PLoS One 6:e25031. CrossRefPubMedPubMedCentralGoogle Scholar
  30. 30.
    Calhoun VD, Adali T, Pearlson GD, Pekar JJ (2001) A method for making group inferences from functional MRI data using independent component analysis. Hum Brain Mapp 14:140–151CrossRefPubMedGoogle Scholar
  31. 31.
    Chen CC, Wan YL, Wai YY, Liu HL (2004) Quality assurance of clinical MRI scanners using ACR MRI phantom: preliminary results. J Digit Imaging 17:279–284. CrossRefPubMedPubMedCentralGoogle Scholar
  32. 32.
    Tropp J (2004) Image brightening in samples of high dielectric constant. J Magn Reson 167:12–24. CrossRefPubMedGoogle Scholar
  33. 33.
    Wenger E, Mårtensson J, Noack H, Bodammer NC, Kühn S, Schaefer S, Heinze HJ, Düzel E, Bäckman L, Lindenberger U, Lövdén M (2014) Comparing manual and automatic segmentation of hippocampal volumes: reliability and validity issues in younger and older brains. Hum Brain Mapp 35:4236–4248. CrossRefPubMedGoogle Scholar
  34. 34.
    Maclaren J, Han Z, Vos SB, Fischbein N, Bammer R (2014) Reliability of brain volume measurements: a test-retest dataset. Sci Data 1:140037. CrossRefPubMedPubMedCentralGoogle Scholar
  35. 35.
    Jovicich J, Czanner S, Han X, Salat D, van der Kouwe A, Quinn B, Pacheco J, Albert M, Killiany R, Blacker D, Maguire P, Rosas D, Makris N, Gollub R, Dale A, Dickerson BC, Fischl B (2009) MRI-derived measurements of human subcortical, ventricular and intracranial brain volumes: reliability effects of scan sessions, acquisition sequences, data analyses, scanner upgrade, scanner vendors and field strengths. NeuroImage 46:177–192. CrossRefPubMedPubMedCentralGoogle Scholar
  36. 36.
    Malyarenko D, Galban CJ, Londy FJ et al (2013) Multi-system repeatability and reproducibility of apparent diffusion coefficient measurement using an ice-water phantom. J Magn Reson Imaging 37:1238–1246. CrossRefPubMedGoogle Scholar
  37. 37.
    Malyarenko DI, Newitt D, JW L et al (2016) Demonstration of nonlinearity bias in the measurement of the apparent diffusion coefficient in multicenter trials. Magn Reson Med 75:1312–1323. CrossRefPubMedGoogle Scholar
  38. 38.
    Wu YC, Alexander AL (2007) A method for calibrating diffusion gradients in diffusion tensor imaging. J Comput Assist Tomogr 31:984–993. CrossRefPubMedPubMedCentralGoogle Scholar
  39. 39.
    Grech-Sollars M, Hales PW, Miyazaki K, Raschke F, Rodriguez D, Wilson M, Gill SK, Banks T, Saunders DE, Clayden JD, Gwilliam MN, Barrick TR, Morgan PS, Davies NP, Rossiter J, Auer DP, Grundy R, Leach MO, Howe FA, Peet AC, Clark CA (2015) Multi-centre reproducibility of diffusion MRI parameters for clinical sequences in the brain. NMR Biomed 28:468–485. CrossRefPubMedPubMedCentralGoogle Scholar
  40. 40.
    Marenco S, Rawlings R, Rohde GK, Barnett AS, Honea RA, Pierpaoli C, Weinberger DR (2006) Regional distribution of measurement error in diffusion tensor imaging. Psychiatry Res 147:69–78. CrossRefPubMedPubMedCentralGoogle Scholar
  41. 41.
    Parrish TB, Gitelman DR, LaBar KS, Mesulam MM (2000) Impact of signal-to-noise on functional MRI. Magn Reson Med 44:925–932CrossRefPubMedGoogle Scholar
  42. 42.
    Damoiseaux JS, Rombouts SA, Barkhof F et al (2006) Consistent resting-state networks across healthy subjects. Proc Natl Acad Sci U S A 103:13848–13853. CrossRefPubMedPubMedCentralGoogle Scholar
  43. 43.
    Rosazza C, Minati L, Ghielmetti F, Mandelli ML, Bruzzone MG (2012) Functional connectivity during resting-state functional MR imaging: study of the correspondence between independent component analysis and region-of-interest-based methods. AJNR Am J Neuroradiol 33:180–187. CrossRefPubMedGoogle Scholar
  44. 44.
    Jovicich J, Czanner S, Greve D et al (2006) Reliability in multi-site structural MRI studies: effects of gradient non-linearity correction on phantom and human data. NeuroImage 30:436–443CrossRefPubMedGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2018

Authors and Affiliations

  • S. Deprez
    • 1
    • 2
  • Michiel B. de Ruiter
    • 3
  • S. Bogaert
    • 4
  • R. Peeters
    • 1
    • 2
  • J. Belderbos
    • 5
  • D. De Ruysscher
    • 6
    • 7
  • S. Schagen
    • 3
  • S. Sunaert
    • 1
    • 2
  • P. Pullens
    • 8
  • E. Achten
    • 4
  1. 1.Department of RadiologyUniversity Hospitals LeuvenLeuvenBelgium
  2. 2.Department of Imaging and PathologyKU LeuvenLeuvenBelgium
  3. 3.Division of Psychosocial Research and EpidemiologyNetherlands Cancer InstituteAmsterdamThe Netherlands
  4. 4.Department of RadiologyGhent University HospitalGhentBelgium
  5. 5.Department of Radiation OncologyNetherlands Cancer InstituteAmsterdamThe Netherlands
  6. 6.Department of Radiation Oncology (Maastro clinic), GROW SchoolMaastricht University Medical CenterMaastrichtThe Netherlands
  7. 7.Radiation OncologyKU LeuvenLeuvenBelgium
  8. 8.Department of RadiologyUniversity Hospital Antwerp & Antwerp UniversityEdegemBelgium

Personalised recommendations