Skip to main content
SpringerLink
Log in

Multi-center reproducibility of structural, diffusion tensor, and resting state functional magnetic resonance imaging measures

  • Functional Neuroradiology
  • Published:
Neuroradiology Aims and scope Submit manuscript

Abstract

Purpose

The aim of this study is to assess multi-center reproducibility and longitudinal consistency of MRI imaging measurements, as part of a phase III longitudinal multi-center study comparing the neurotoxic effect following prophylactic cranial irradiation with hippocampal avoidance (HA-PCI), in comparison with conventional PCI in patients with small-cell lung cancer.

Methods

Harmonized MRI acquisition protocols from six participating sites and two different vendors were compared using both physical and human phantoms. We assessed variability across sites and time points by evaluating various phantoms and data including hippocampal volume, diffusion metrics, and resting-state fMRI, from two healthy volunteers.

Results

We report average coefficients of variation (CV) below 5% for intrascanner, intravendor, and intervendor reproducibility for both structural and diffusion imaging metrics, except for diffusion metrics obtained from tractography with average CVs ranging up to 7.8%. Additionally, resting-state fMRI showed stable temporal SNR and reliable generation of subjects DMN across vendors and time points.

Conclusion

These findings indicate that the presented multi-site MRI acquisition protocol can be used in a longitudinal study design and that pooling of the acquired data as part of the phase III longitudinal HA-PCI project is possible with careful monitoring of the results of the half-yearly QA assessment to follow-up on potential scanner-related longitudinal changes in image quality.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Van Horn JD, Toga AW (2009) Multisite neuroimaging trials. Curr Opin Neurol 22:370–378. https://doi.org/10.1097/WCO.0b013e32832d92de

    Article  PubMed  PubMed Central  Google Scholar 

  2. Friedman L, Glover GH, Fbirn C (2006) Reducing interscanner variability of activation in a multicenter fMRI study: controlling for signal-to-fluctuation-noise-ratio (SFNR) differences. NeuroImage 33:471–481. https://doi.org/10.1016/j.neuroimage.2006.07.012

    Article  PubMed  Google Scholar 

  3. Fox RJ, Sakaie K, Lee JC, Debbins JP, Liu Y, Arnold DL, Melhem ER, Smith CH, Philips MD, Lowe M, Fisher E (2012) A validation study of multicenter diffusion tensor imaging: reliability of fractional anisotropy and diffusivity values. AJNR Am J Neuroradiol 33:695–700. https://doi.org/10.3174/ajnr.A2844

    Article  PubMed  CAS  Google Scholar 

  4. Jovicich J, Minati L, Marizzoni M, Marchitelli R, Sala-Llonch R, Bartrés-Faz D, Arnold J, Benninghoff J, Fiedler U, Roccatagliata L, Picco A, Nobili F, Blin O, Bombois S, Lopes R, Bordet R, Sein J, Ranjeva JP, Didic M, Gros-Dagnac H, Payoux P, Zoccatelli G, Alessandrini F, Beltramello A, Bargalló N, Ferretti A, Caulo M, Aiello M, Cavaliere C, Soricelli A, Parnetti L, Tarducci R, Floridi P, Tsolaki M, Constantinidis M, Drevelegas A, Rossini PM, Marra C, Schönknecht P, Hensch T, Hoffmann KT, Kuijer JP, Visser PJ, Barkhof F, Frisoni GB, PharmaCog Consortium (2016) Longitudinal reproducibility of default-mode network connectivity in healthy elderly participants: a multicentric resting-state fMRI study. NeuroImage 124:442–454. https://doi.org/10.1016/j.neuroimage.2015.07.010

    Article  PubMed  Google Scholar 

  5. Jovicich J, Marizzoni M, Sala-Llonch R, Bosch B, Bartrés-Faz D, Arnold J, Benninghoff J, Wiltfang J, Roccatagliata L, Nobili F, Hensch T, Tränkner A, Schönknecht P, Leroy M, Lopes R, Bordet R, Chanoine V, Ranjeva JP, Didic M, Gros-Dagnac H, Payoux P, Zoccatelli G, Alessandrini F, Beltramello A, Bargalló N, Blin O, Frisoni GB, PharmaCog Consortium (2013) Brain morphometry reproducibility in multi-center 3T MRI studies: a comparison of cross-sectional and longitudinal segmentations. NeuroImage 83:472–484. https://doi.org/10.1016/j.neuroimage.2013.05.007

    Article  PubMed  Google Scholar 

  6. Chalavi S, Simmons A, Dijkstra H, Barker GJ, Reinders AATS (2012) Quantitative and qualitative assessment of structural magnetic resonance imaging data in a two-center study. BMC Med Imaging 12:27. https://doi.org/10.1186/1471-2342-12-27

    Article  PubMed  PubMed Central  Google Scholar 

  7. Weiskopf N, Suckling J, Williams G, Correia MM, Inkster B, Tait R, Ooi C, Bullmore ET, Lutti A (2013) Quantitative multi-parameter mapping of R1, PD(*), MT, and R2(*) at 3T: a multi-center validation. Front Neurosci 7:95. https://doi.org/10.3389/fnins.2013.00095

    Article  PubMed  PubMed Central  Google Scholar 

  8. Keshavan A, Paul F, Beyer MK, Zhu AH, Papinutto N, Shinohara RT, Stern W, Amann M, Bakshi R, Bischof A, Carriero A, Comabella M, Crane JC, D'Alfonso S, Demaerel P, Dubois B, Filippi M, Fleischer V, Fontaine B, Gaetano L, Goris A, Graetz C, Gröger A, Groppa S, Hafler DA, Harbo HF, Hemmer B, Jordan K, Kappos L, Kirkish G, Llufriu S, Magon S, Martinelli-Boneschi F, McCauley J, Montalban X, Mühlau M, Pelletier D, Pattany PM, Pericak-Vance M, Cournu-Rebeix I, Rocca MA, Rovira A, Schlaeger R, Saiz A, Sprenger T, Stecco A, Uitdehaag BMJ, Villoslada P, Wattjes MP, Weiner H, Wuerfel J, Zimmer C, Zipp F, International Multiple Sclerosis Genetics Consortium. Electronic address: AIVINSON@PARTNERS.ORG, Hauser SL, Oksenberg JR, Henry RG (2016) Power estimation for non-standardized multisite studies. NeuroImage 134:281–294. https://doi.org/10.1016/j.neuroimage.2016.03.051

    Article  PubMed  PubMed Central  Google Scholar 

  9. Jovicich J, Marizzoni M, Bosch B, Bartrés-Faz D, Arnold J, Benninghoff J, Wiltfang J, Roccatagliata L, Picco A, Nobili F, Blin O, Bombois S, Lopes R, Bordet R, Chanoine V, Ranjeva JP, Didic M, Gros-Dagnac H, Payoux P, Zoccatelli G, Alessandrini F, Beltramello A, Bargalló N, Ferretti A, Caulo M, Aiello M, Ragucci M, Soricelli A, Salvadori N, Tarducci R, Floridi P, Tsolaki M, Constantinidis M, Drevelegas A, Rossini PM, Marra C, Otto J, Reiss-Zimmermann M, Hoffmann KT, Galluzzi S, Frisoni GB, PharmaCog Consortium (2014) Multisite longitudinal reliability of tract-based spatial statistics in diffusion tensor imaging of healthy elderly subjects. NeuroImage 101:390–403. https://doi.org/10.1016/j.neuroimage.2014.06.075

    Article  PubMed  Google Scholar 

  10. Teipel SJ, Reuter S, Stieltjes B, Acosta-Cabronero J, Ernemann U, Fellgiebel A, Filippi M, Frisoni G, Hentschel F, Jessen F, Klöppel S, Meindl T, Pouwels PJW, Hauenstein KH, Hampel H (2011) Multicenter stability of diffusion tensor imaging measures: a European clinical and physical phantom study. Psychiatry Res Neuroimaging 194:363–371. https://doi.org/10.1016/j.pscychresns.2011.05.012

    Article  Google Scholar 

  11. Zhu T, Hu R, Qiu X, Taylor M, Tso Y, Yiannoutsos C, Navia B, Mori S, Ekholm S, Schifitto G, Zhong J (2011) Quantification of accuracy and precision of multi-center DTI measurements: a diffusion phantom and human brain study. NeuroImage 56:1398–1411. https://doi.org/10.1016/j.neuroimage.2011.02.010

    Article  PubMed  PubMed Central  Google Scholar 

  12. Huang L, Wang X, Baliki MN, Wang L, Apkarian AV, Parrish TB (2012) Reproducibility of structural, resting-state BOLD and DTI data between identical scanners. PLoS One 7:e47684. https://doi.org/10.1371/journal.pone.0047684

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  13. Takao H, Hayashi N, Ohtomo K (2011) Effect of scanner in asymmetry studies using diffusion tensor imaging. NeuroImage 54:1053–1062. https://doi.org/10.1016/j.neuroimage.2010.09.023

    Article  PubMed  Google Scholar 

  14. Vollmar C, O’Muircheartaigh J, Barker GJ et al (2010) Identical, but not the same: intra-site and inter-site reproducibility of fractional anisotropy measures on two 3.0T scanners. NeuroImage 51:1384–1394. https://doi.org/10.1016/j.neuroimage.2010.03.046

    Article  PubMed  PubMed Central  Google Scholar 

  15. Belli G, Busoni S, Ciccarone A, Coniglio A, Esposito M, Giannelli M, Mazzoni LN, Nocetti L, Sghedoni R, Tarducci R, Zatelli G, Anoja RA, Belmonte G, Bertolino N, Betti M, Biagini C, Ciarmatori A, Cretti F, Fabbri E, Fedeli L, Filice S, Fulcheri CPL, Gasperi C, Mangili PA, Mazzocchi S, Meliadò G, Morzenti S, Noferini L, Oberhofer N, Orsingher L, Paruccini N, Princigalli G, Quattrocchi M, Rinaldi A, Scelfo D, Freixas GV, Tenori L, Zucca I, Luchinat C, Gori C, Gobbi G, for the Italian Association of Physics in Medicine (AIFM) Working Group on MR Intercomparison (2016) Quality assurance multicenter comparison of different MR scanners for quantitative diffusion-weighted imaging. J Magn Reson Imaging 43:213–219. https://doi.org/10.1002/jmri.24956

    Article  PubMed  Google Scholar 

  16. Laun FB, Huff S, Stieltjes B (2009) On the effects of dephasing due to local gradients in diffusion tensor imaging experiments: relevance for diffusion tensor imaging fiber phantoms. Magn Reson Imaging 27:541–548. https://doi.org/10.1016/j.mri.2008.08.011

    Article  PubMed  Google Scholar 

  17. Jones DK (2011) Diffusion MRI: theory, methods, and applications. Oxford University Press, Oxford

    Google Scholar 

  18. Dowell NG, Tofts PS (2008) Simple reliable and precise quantitative quality assurance of in-vivo brain ADC. Proc. Int. Soc. Magn. Reson. Med. 16th Annu. Meet. 3152

  19. Jack CR Jr, Bernstein MA, Fox NC et al (2008) The Alzheimer’s Disease Neuroimaging Initiative (ADNI): MRI methods. J Magn Reson Imaging 27:685–691. https://doi.org/10.1002/jmri.21049

    Article  PubMed  PubMed Central  Google Scholar 

  20. Achten E, Deblaere K, De Wagter C et al (1998) Intra- and interobserver variability of MRI-based volume measurements of the hippocampus and amygdala using the manual ray-tracing method. Neuroradiology 40:558–566

    Article  PubMed  CAS  Google Scholar 

  21. Reuter M, Schmansky NJ, Rosas HD, Fischl B (2012) Within-subject template estimation for unbiased longitudinal image analysis. NeuroImage 61:1402–1418. https://doi.org/10.1016/j.neuroimage.2012.02.084

    Article  PubMed  PubMed Central  Google Scholar 

  22. Jones R, Payne B (1997) Clinical investigation and statistics in laboratory medicine. ACB Venture Publications, London

    Google Scholar 

  23. Leemans A (2009) ExploreDTI: a graphical toolbox for processing, analyzing, and visualizing diffusion MR data. 17th Annu Meet Intl Soc Mag Reson Med 3537

  24. Otsu N (1979) A threshold selection method from gray-level histograms. IEEE Trans Syst Man Cybern 9:62–66

    Article  Google Scholar 

  25. De Santis S, Evans CJ, Jones DK (2013) RAPID: a routine assurance pipeline for imaging of diffusion. Magn Reson Med 70:490–496. https://doi.org/10.1002/mrm.24465

    Article  PubMed  Google Scholar 

  26. Holz M, Heil SR, Sacco A (2000) Temperature-dependent self-diffusion coefficients of water and six selected molecular liquids for calibration in accurate 1H NMR PFG measurements. Phys Chem Chem Phys 2:4740–4742. https://doi.org/10.1039/B005319H

    Article  CAS  Google Scholar 

  27. Wakana S, Caprihan A, Panzenboeck MM, Fallon JH, Perry M, Gollub RL, Hua K, Zhang J, Jiang H, Dubey P, Blitz A, van Zijl P, Mori S (2007) Reproducibility of quantitative tractography methods applied to cerebral white matter. NeuroImage 36:630–644. https://doi.org/10.1016/j.neuroimage.2007.02.049

    Article  PubMed  PubMed Central  Google Scholar 

  28. Lebel C, Walker L, Leemans A, Phillips L, Beaulieu C (2008) Microstructural maturation of the human brain from childhood to adulthood. NeuroImage 40:1044–1055. https://doi.org/10.1016/j.neuroimage.2007.12.053

    Article  PubMed  CAS  Google Scholar 

  29. Song XW, Dong ZY, Long XY, Li SF, Zuo XN, Zhu CZ, He Y, Yan CG, Zang YF (2011) REST: a toolkit for resting-state functional magnetic resonance imaging data processing. PLoS One 6:e25031. https://doi.org/10.1371/journal.pone.0025031

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  30. Calhoun VD, Adali T, Pearlson GD, Pekar JJ (2001) A method for making group inferences from functional MRI data using independent component analysis. Hum Brain Mapp 14:140–151

    Article  PubMed  CAS  Google Scholar 

  31. Chen CC, Wan YL, Wai YY, Liu HL (2004) Quality assurance of clinical MRI scanners using ACR MRI phantom: preliminary results. J Digit Imaging 17:279–284. https://doi.org/10.1007/s10278-004-1023-5

    Article  PubMed  PubMed Central  Google Scholar 

  32. Tropp J (2004) Image brightening in samples of high dielectric constant. J Magn Reson 167:12–24. https://doi.org/10.1016/j.jmr.2003.11.003

    Article  PubMed  CAS  Google Scholar 

  33. Wenger E, Mårtensson J, Noack H, Bodammer NC, Kühn S, Schaefer S, Heinze HJ, Düzel E, Bäckman L, Lindenberger U, Lövdén M (2014) Comparing manual and automatic segmentation of hippocampal volumes: reliability and validity issues in younger and older brains. Hum Brain Mapp 35:4236–4248. https://doi.org/10.1002/hbm.22473

    Article  PubMed  Google Scholar 

  34. Maclaren J, Han Z, Vos SB, Fischbein N, Bammer R (2014) Reliability of brain volume measurements: a test-retest dataset. Sci Data 1:140037. https://doi.org/10.1038/sdata.2014.37

    Article  PubMed  PubMed Central  Google Scholar 

  35. Jovicich J, Czanner S, Han X, Salat D, van der Kouwe A, Quinn B, Pacheco J, Albert M, Killiany R, Blacker D, Maguire P, Rosas D, Makris N, Gollub R, Dale A, Dickerson BC, Fischl B (2009) MRI-derived measurements of human subcortical, ventricular and intracranial brain volumes: reliability effects of scan sessions, acquisition sequences, data analyses, scanner upgrade, scanner vendors and field strengths. NeuroImage 46:177–192. https://doi.org/10.1016/j.neuroimage.2009.02.010

    Article  PubMed  PubMed Central  Google Scholar 

  36. Malyarenko D, Galban CJ, Londy FJ et al (2013) Multi-system repeatability and reproducibility of apparent diffusion coefficient measurement using an ice-water phantom. J Magn Reson Imaging 37:1238–1246. https://doi.org/10.1002/jmri.23825

    Article  PubMed  Google Scholar 

  37. Malyarenko DI, Newitt D, JW L et al (2016) Demonstration of nonlinearity bias in the measurement of the apparent diffusion coefficient in multicenter trials. Magn Reson Med 75:1312–1323. https://doi.org/10.1002/mrm.25754

    Article  PubMed  Google Scholar 

  38. Wu YC, Alexander AL (2007) A method for calibrating diffusion gradients in diffusion tensor imaging. J Comput Assist Tomogr 31:984–993. https://doi.org/10.1097/rct.0b013e31805152fa

    Article  PubMed  PubMed Central  Google Scholar 

  39. Grech-Sollars M, Hales PW, Miyazaki K, Raschke F, Rodriguez D, Wilson M, Gill SK, Banks T, Saunders DE, Clayden JD, Gwilliam MN, Barrick TR, Morgan PS, Davies NP, Rossiter J, Auer DP, Grundy R, Leach MO, Howe FA, Peet AC, Clark CA (2015) Multi-centre reproducibility of diffusion MRI parameters for clinical sequences in the brain. NMR Biomed 28:468–485. https://doi.org/10.1002/nbm.3269

    Article  PubMed  PubMed Central  Google Scholar 

  40. Marenco S, Rawlings R, Rohde GK, Barnett AS, Honea RA, Pierpaoli C, Weinberger DR (2006) Regional distribution of measurement error in diffusion tensor imaging. Psychiatry Res 147:69–78. https://doi.org/10.1016/j.pscychresns.2006.01.008

    Article  PubMed  PubMed Central  Google Scholar 

  41. Parrish TB, Gitelman DR, LaBar KS, Mesulam MM (2000) Impact of signal-to-noise on functional MRI. Magn Reson Med 44:925–932

    Article  PubMed  CAS  Google Scholar 

  42. Damoiseaux JS, Rombouts SA, Barkhof F et al (2006) Consistent resting-state networks across healthy subjects. Proc Natl Acad Sci U S A 103:13848–13853. https://doi.org/10.1073/pnas.0601417103

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  43. Rosazza C, Minati L, Ghielmetti F, Mandelli ML, Bruzzone MG (2012) Functional connectivity during resting-state functional MR imaging: study of the correspondence between independent component analysis and region-of-interest-based methods. AJNR Am J Neuroradiol 33:180–187. https://doi.org/10.3174/ajnr.A2733

    Article  PubMed  CAS  Google Scholar 

  44. Jovicich J, Czanner S, Greve D et al (2006) Reliability in multi-site structural MRI studies: effects of gradient non-linearity correction on phantom and human data. NeuroImage 30:436–443

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

We are grateful to Dr. Bram Stieltjes (Universitätsspital Basel, CH) for his advice and help with the DTI anisotropic phantom.

Author information

Authors and Affiliations

  1. Department of Radiology, University Hospitals Leuven, Leuven, Belgium

    S. Deprez, R. Peeters & S. Sunaert

  2. Department of Imaging and Pathology, KU Leuven, Leuven, Belgium

    S. Deprez, R. Peeters & S. Sunaert

  3. Division of Psychosocial Research and Epidemiology, Netherlands Cancer Institute, Plesmanlaan 121, 1066 CX, Amsterdam, The Netherlands

    Michiel B. de Ruiter & S. Schagen

  4. Department of Radiology, Ghent University Hospital, Ghent, Belgium

    S. Bogaert & E. Achten

  5. Department of Radiation Oncology, Netherlands Cancer Institute, Amsterdam, The Netherlands

    J. Belderbos

  6. Department of Radiation Oncology (Maastro clinic), GROW School, Maastricht University Medical Center, Maastricht, The Netherlands

    D. De Ruysscher

  7. Radiation Oncology, KU Leuven, Leuven, Belgium

    D. De Ruysscher

  8. Department of Radiology, University Hospital Antwerp & Antwerp University, Edegem, Belgium

    P. Pullens

Authors
  1. S. Deprez
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Michiel B. de Ruiter
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. S. Bogaert
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. R. Peeters
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. J. Belderbos
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. D. De Ruysscher
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. S. Schagen
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. S. Sunaert
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. P. Pullens
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. E. Achten
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Michiel B. de Ruiter.

Ethics declarations

Funding

This study was funded by the Institute for the Promotion of Innovation by Science and Technology in Flanders (Grant Number IWT 130262), the Vlaamse Liga Tegen Kanker (VLK) and the Dutch Cancer Society.

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed consent

Informed consent was obtained from all individual participants included in the study.

Electronic supplementary material

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Deprez, S., de Ruiter, M.B., Bogaert, S. et al. Multi-center reproducibility of structural, diffusion tensor, and resting state functional magnetic resonance imaging measures. Neuroradiology 60, 617–634 (2018). https://doi.org/10.1007/s00234-018-2017-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00234-018-2017-1

Keywords

Search

Navigation