Advertisement

Deep Learning on Brain Images in Autism: What Do Large Samples Reveal of Its Complexity?

  • Matthew LemingEmail author
  • John Suckling
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11486)

Abstract

Deep learning models for image classification face two recurring problems: they are typically limited by low sample size and are abstracted by their own complexity (the “black box problem”). We address these problems with the largest functional MRI connectome dataset ever compiled, classifying it across gender and Task vs rest (no task) to ascertain its performance, and then apply the model to a cross-sectional comparison of autism vs typically developing (TD) controls that has proved difficult to characterise with inferential statistics. Employing class-balancing to build a training set, a convolutional neural network was classified fMRI connectivity with overall accuracies of 76.35% (AUROC 0.8401), 90.71% (AUROC 0.9573), and 67.65% (AUROC 0.7162) for gender, task vs rest, and autism vs TD, respectively. Salience maps demonstrated that the deep learning model is capable of distinguishing complex patterns across either wide networks or localized areas of the brain, and, by analyzing maximal activations of the hidden layers, that the deep learning model partitions data at an early stage in its classification.

Keywords

Autism Big data Functional connectivity Deep learning 

Notes

Acknowledgements

This study used publicly available datasets, each with their own acknowledgements. For brevity, we have not included the full text, but recognise the contributions of the Alzheimer’s Disease Neuroimaging Initiative, International Consortium for Brain Mapping, National Database for Autism Research, NIH Pediatric MRI Data Repository, National Database for Clinical Trials, Research Domain Criteria Database, Adolescent Brain Cognitive Development Study, UK Biobank Resource, 1000 Functional Connectomes Project, ABIDE I and II, and Open fMRI. This research was co-funded by the NIHR Cambridge Biomedical Research Centre and Marmaduke Sheild. ML is supported by a Gates Cambridge Scholarship from the University of Cambridge.

References

  1. 1.
    Arbabshirani, M., Havlicek, M., Kiehl, K., Pearlson, G., Calhoun, V.: Functional network connectivity during rest and task conditions: a comparative study. Hum. Brain Mapp. 34, 2959–2971 (2012).  https://doi.org/10.1002/hbm.22118CrossRefGoogle Scholar
  2. 2.
    Blanken, L., et al.: A prospective study of fetal head growth, autistic traits and autism spectrum disorder. Autism Res. 11, 602–612 (2018).  https://doi.org/10.1002/aur.1921CrossRefGoogle Scholar
  3. 3.
    Brown, C., Kawahara, J., Hamarneh, G.: Connectome priors in deep neural networks to predict autism. In: 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018) (2018).  https://doi.org/10.1109/ISBI.2018.8363534
  4. 4.
    Bruna, J., Zaremba, W., Szlam, A., LeCun, Y.: Spectral networks and locally connected networks on graphs. In: ICLR (2014)Google Scholar
  5. 5.
    Casey, B., Dale, A.: The adolescent brain cognitive development (ABCD) study: imaging acquisition across 21 sites. Dev. Cogn. Neurosci. 32, 43–54 (2018).  https://doi.org/10.1016/j.dcn.2018.03.001CrossRefGoogle Scholar
  6. 6.
    Cauda, F., et al.: Grey matter abnormality in autism spectrum disorder: an activation likelihood estimation meta-analysis study. J. Neurol. Neurosurg. Psychiatry 82, 1304–1313 (2011).  https://doi.org/10.1136/jnnp.2010.239111CrossRefGoogle Scholar
  7. 7.
    Chollet, F.: Keras (2015). https://github.com/fchollet/keras
  8. 8.
    Courchesne, E., Carper, R., Akshoomoff, N.: Evidence of brain overgrowth in the first year of life in autism. JAMA 290, 337–344 (2003).  https://doi.org/10.1001/jama.290.3.337CrossRefGoogle Scholar
  9. 9.
    Defferrard, M., Bresson, P., Vandergheynst, X.: Convolutional neural networks on graphs with fast localized spectral filtering. In: NIPS, pp. 3844–3852 (2016)Google Scholar
  10. 10.
    DeRamus, T., Kana, R.: Anatomical likelihood estimation meta-analysis of grey and white matter anomalies in autism spectrum disorders author links open overlay panel. NeuroImage: Clin. 7, 525–536 (2015).  https://doi.org/10.1016/j.nicl.2014.11.004CrossRefGoogle Scholar
  11. 11.
    Di Martino, A., et al.: Enhancing studies of the connectome in autism using the autism brain imaging data exchange II. Sci. Data 4, 170010 (2017).  https://doi.org/10.1038/sdata.2017.10CrossRefGoogle Scholar
  12. 12.
    Di Martino, A., et al.: The autism brain imaging data exchange: towards a large-scale evaluation of the intrinsic brain architecture in autism. Mol. Psychiatry 19, 659–67 (2014).  https://doi.org/10.1038/mp.2013.78CrossRefGoogle Scholar
  13. 13.
    Dinstein, I., Haar, S., Atsmon, S., Schtaerman, H.: No evidence of early head circumference enlargements in children later diagnosed with autism in Israel. Mol. Autism 8 (2018).  https://doi.org/10.1186/s13229-017-0129-9
  14. 14.
    Dolgin, E.: This is your brain online: the functional connectomes project. Nat. Med. 16, 351 (2010).  https://doi.org/10.1038/nm0410-351bCrossRefGoogle Scholar
  15. 15.
    Erhan, D., Bengio, Y., Courville, A., Vincent, P.: Visualizing higher-layer features of a deep network. Technical report 1341, University of Montreal (2009)Google Scholar
  16. 16.
    Finn, E., et al.: Functional connectome fingerprinting: identifying individuals using patterns of brain connectivity. Nat Neurosci. 18, 1664–1671 (2015).  https://doi.org/10.1038/nn.4135CrossRefGoogle Scholar
  17. 17.
    Haar, S., Berman, S., Behrmann, M., Dinstein, I.: Anatomical abnormalities in autism? Cereb. Cortex 26, 1440–1452 (2016).  https://doi.org/10.1093/cercor/bhu242CrossRefGoogle Scholar
  18. 18.
    Hall, D., Huerta, M., McAuliffe, M., Farber, G.: Sharing heterogeneous data: the national database for autism research. Neuroinformatics 10, 331–339 (2012).  https://doi.org/10.1007/s12021-012-9151-4CrossRefGoogle Scholar
  19. 19.
    Hamilton, W., Ying, R., Leskovec, J.: Representation learning on graphs: methods and applications. Bulletin of the IEEE Computer Society Technical Committee on Data Engineering (2017)Google Scholar
  20. 20.
    Hasson, U., Nusbaum, H., Small, S.: Task-dependent organization of brain regions active during rest. PNAS 106, 10841–10846 (2009).  https://doi.org/10.1073/pnas.0903253106CrossRefGoogle Scholar
  21. 21.
    Hechtlinger, Y., Chakravarti, P., Qin, J.: A generalization of convolutional neural networks to graph-structured data. arXiv (2017)Google Scholar
  22. 22.
    Heinsfeld, A., Franco, A., Craddock, R., Buchweitz, A., Meneguzzia, F.: Identification of autism spectrum disorder using deep learning and the abide dataset. NeuroImage: Clin. 17, 16–23 (2018).  https://doi.org/10.1016/j.nicl.2017.08.017CrossRefGoogle Scholar
  23. 23.
    Kawahara, J., et al.: BrainNetCNN: convolutional neural networks for brain networks; towards predicting neurodevelopment. NeuroImage 146, 1038–1049 (2017).  https://doi.org/10.1016/j.neuroimage.2016.09.046CrossRefGoogle Scholar
  24. 24.
    Keown, C., Datko, M., Chen, C., Maximo, J., Jahedi, A., Müller, R.: Network organization is globally atypical in autism: a graph theory study of intrinsic functional connectivity. Biol. Psychiatry: Cogn. Neurosci. Neuroimaging 2, 66–75 (2017).  https://doi.org/10.1016/j.bpsc.2016.07.008CrossRefGoogle Scholar
  25. 25.
    Khundrakpam, B., Lewis, J., Kostopoulos, P., Carbonell, F., Evans, A.: Cortical thickness abnormalities in autism spectrum disorders through late childhood, adolescence, and adulthood: a large-scale MRI study. Cereb. Cortex 27, 1721–1731 (2017).  https://doi.org/10.1093/cercor/bhx038CrossRefGoogle Scholar
  26. 26.
    Kipf, T., Welling, M.: Semi-supervised classification with graph convolutional neural networks. In: ICLR 2017 (2017)Google Scholar
  27. 27.
    Kotikalapudi, R., Contributors: keras-vis (2017). https://github.com/raghakot/keras-vis
  28. 28.
    Khosla, M., Jamison, K., Kuceyeski, A., Sabuncu, M.R.: 3D convolutional neural networks for classification of functional connectomes. In: Stoyanov, D., et al. (eds.) DLMIA/ML-CDS-2018. LNCS, vol. 11045, pp. 137–145. Springer, Cham (2018).  https://doi.org/10.1007/978-3-030-00889-5_16CrossRefGoogle Scholar
  29. 29.
    Nikolentzos, G., Meladianos, P., Tixier, A.J.-P., Skianis, K., Vazirgiannis, M.: Kernel graph convolutional neural networks. In: Kůrková, V., Manolopoulos, Y., Hammer, B., Iliadis, L., Maglogiannis, I. (eds.) ICANN 2018. LNCS, vol. 11139, pp. 22–32. Springer, Cham (2018).  https://doi.org/10.1007/978-3-030-01418-6_3CrossRefGoogle Scholar
  30. 30.
    Patel, A., Bullmore, E.: A wavelet-based estimator of the degrees of freedom in denoised fmri time series for probabilistic testing of functional connectivity and brain graphs. NeuroImage 142, 14–26 (2016).  https://doi.org/10.1016/j.neuroimage.2015.04.052CrossRefGoogle Scholar
  31. 31.
    Piven, J., Arndt, S., Bailey, J., Havercamp, S., Andreasen, N., Palmer, P.: An MRI study of brain size in autism. Am. J. Psychiatry 152, 1145–1149 (1995).  https://doi.org/10.1176/ajp.152.8.1145CrossRefGoogle Scholar
  32. 32.
    Plitt, M., Barnes, K., Martin, A.: Functional connectivity classification of autism identifies highly predictive brain features but falls short of biomarker standards. NeuroImage Clin. 7, 359–66 (2015).  https://doi.org/10.1016/j.nicl.2014.12.013CrossRefGoogle Scholar
  33. 33.
    Poldrack, R., et al.: Toward open sharing of task-based fMRI data: the OpenfMRI project. Front. Neuroinform. 7 (2013).  https://doi.org/10.3389/fninf.2013.00012
  34. 34.
    Poldrack, R., Gorgolewski, K.: OpenfMRI: open sharing of task fMRI data. NeuroImage 144, 259–261 (2017).  https://doi.org/10.1016/j.neuroimage.2015.05.073CrossRefGoogle Scholar
  35. 35.
    Redcay, E., Courchesne, E.: Biol. Psychiatry 58, 1–9 (2005).  https://doi.org/10.1016/j.biopsych.2005.03.026CrossRefGoogle Scholar
  36. 36.
    Russakovsky, O., et al.: ImageNet large scale visual recognition challenge. Int. J. Comput. Vis. 115, 211–252 (2015).  https://doi.org/10.1007/s11263-015-0816-yMathSciNetCrossRefGoogle Scholar
  37. 37.
    Satterthwaite, T., et al.: Linked sex differences in cognition and functional connectivity in youth. Cereb. Cortex 25, 2383–2394 (2015).  https://doi.org/10.1093/cercor/bhu036CrossRefGoogle Scholar
  38. 38.
    Simonyan, K., Vedaldi, A., Zisserman, A.: Deep inside convolutional networks: visualising image classification models and saliency maps. In: Workshop at International Conference on Learning Representations (2014)Google Scholar
  39. 39.
    Tomasi, D., Volkow, N.: Gender differences in brain functional connectivity density. Hum. Brain Mapp. 33, 849–860 (2013).  https://doi.org/10.1002/hbm.21252CrossRefGoogle Scholar
  40. 40.
    Wang, W., et al.: Altered resting-state functional activity in patients with autism spectrum disorder: a quantitative meta-analysis. Front. Neurol. 9, 556 (2018).  https://doi.org/10.3389/fneur.2018.00556CrossRefGoogle Scholar
  41. 41.
    Yang, J., Hofmann, J.: Action observation and imitation in autism spectrum disorders: an ALE meta-analysis of fMRI studies. Brain Imaging Behav. 10, 960–969 (2016).  https://doi.org/10.1007/s11682-015-9456-7CrossRefGoogle Scholar
  42. 42.
    Zhang, W., Groen, W., Mennes, M., Greven, C., Buitelaar, J., Rommelse, N.: Revisiting subcortical brain volume correlates of autism in the abide dataset: effects of age and sex. Psychol. Med. 48, 654–668 (2018).  https://doi.org/10.1017/S003329171700201XCrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Department of PsychiatryCambridge UniversityCambridgeUK

Personalised recommendations