MRIQC is a quality control tool that predicts the binary rating (accept/exclude) that human experts would assign to T1-weighted MR images of the human brain. For such prediction, a random forests classifier performs on a vector of image quality metrics (IQMs) extracted from each image. Although MRIQC achieved an out-of-sample accuracy of \(\sim \)76%, we concluded that this performance on new, unseen datasets would likely improve after addressing two problems. First, we found that IQMs show “site-effects” since they are highly correlated with the acquisition center and imaging parameters. Second, the high inter-rater variability suggests the presence of annotation errors in the labels of both training and test data sets. Annotation errors may be accentuated by some preprocessing decisions. Here, we confirm the “site-effects” in our IQMs using t-student Stochastic Neighbour Embedding (t-SNE). We also improve by a \(\sim \)10% accuracy increment on the out-of-sample prediction of MRIQC by revising a label binarization step in MRIQC. Reliable and automated QC of MRI is in high demand for the increasingly large samples currently being acquired. We show here one iteration to improve the performance of MRIQC on this task, by investigating two challenging problems: site-effects and noise in the labels assigned by human experts.


MRIQC Site-effect Labels noise 


  1. 1.
    Power, J.D., Barnes, K.A., Snyder, A.Z., Schlaggar, B.L., Petersen, S.E.: Spurious but systematic correlations in functional connectivity MRI networks arise from subject motion. NeuroImage 59, 2142–2154 (2012). Scholar
  2. 2.
    Yendiki, A., Koldewyn, K., Kakunoori, S., Kanwisher, N., Fischl, B.: Spurious group differences due to head motion in a diffusion MRI study. NeuroImage 88, 79–90 (2014). Scholar
  3. 3.
    Reuter, M., et al.: Head motion during MRI acquisition reduces gray matter volume and thickness estimates. NeuroImage 107, 107–115 (2015). Scholar
  4. 4.
    Alexander-Bloch, A., et al.: Subtle in-scanner motion biases automated measurement of brain anatomy from in vivo MRI. Human Brain Mapping 37, 2385–2397 (2016). Scholar
  5. 5.
    Esteban, O., et al.: MRIQC: advancing the automatic prediction of image quality in MRI from unseen sites. PLOS ONE 12, e0184661 (2017). Scholar
  6. 6.
    Di Martino, A., et al.: The autism brain imaging data exchange: towards a large-scale evaluation of the intrinsic brain architecture in autism. Mol. Psychiatry 19, 659–667 (2014). Scholar
  7. 7.
    Poldrack, R.A., et al.: A phenome-wide examination of neural and cognitive function. Sci. Data 3, 160110 (2016). Scholar
  8. 8.
    Gorgolewski, K.J., Durnez, J., Poldrack, R.A.: Preprocessed consortium for neuropsychiatric phenomics dataset. F1000Research 6, 1262 (2017). Scholar
  9. 9.
    Leek, J.T., et al.: Tackling the widespread and critical impact of batch effects in high-throughput data. Nature Rev. Genet. 11, 733–739 (2010). Scholar
  10. 10.
    Woodard, J.P., Carley-Spencer, M.P.: No-Reference image quality metrics for structural MRI. Neuroinformatics 4, 243–262 (2006). Scholar
  11. 11.
    Mortamet, B., et al.: Automatic quality assessment in structural brain magnetic resonance imaging. Magn. Reson. Med. 62, 365–372 (2009). Scholar
  12. 12.
    Alfaro-Almagro, F., et al.: Image processing and Quality Control for the first 10,000 brain imaging datasets from UK Biobank. NeuroImage (2017). Scholar
  13. 13.
    Pizarro, R.A., et al.: Automated quality assessment of structural magnetic resonance brain images based on a supervised machine learning algorithm. Front. Neuroinformatics 10 (2016).
  14. 14.
    Shehzad, Z., et al.: The Preprocessed Connectomes Project Quality Assessment Protocol - a resource for measuring the quality of MRI data. In Front. Neurosci. Conference Abstract: Neuroinformatics 2015, Cairns, Australia, 2015.
  15. 15.
    Pedregosa, F., et al.: Scikit-learn: machine learning in python. J. Mach. Learn. Res. 12, 2825–2830 (2011)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Oscar Esteban
    • 1
    Email author
  • Russell A. Poldrack
    • 1
  • Krzysztof J. Gorgolewski
    • 1
  1. 1.Stanford UniversityStanfordUSA

Personalised recommendations