Deep Learning on Brain Images in Autism: What Do Large Samples Reveal of Its Complexity?
Deep learning models for image classification face two recurring problems: they are typically limited by low sample size and are abstracted by their own complexity (the “black box problem”). We address these problems with the largest functional MRI connectome dataset ever compiled, classifying it across gender and Task vs rest (no task) to ascertain its performance, and then apply the model to a cross-sectional comparison of autism vs typically developing (TD) controls that has proved difficult to characterise with inferential statistics. Employing class-balancing to build a training set, a convolutional neural network was classified fMRI connectivity with overall accuracies of 76.35% (AUROC 0.8401), 90.71% (AUROC 0.9573), and 67.65% (AUROC 0.7162) for gender, task vs rest, and autism vs TD, respectively. Salience maps demonstrated that the deep learning model is capable of distinguishing complex patterns across either wide networks or localized areas of the brain, and, by analyzing maximal activations of the hidden layers, that the deep learning model partitions data at an early stage in its classification.
KeywordsAutism Big data Functional connectivity Deep learning
This study used publicly available datasets, each with their own acknowledgements. For brevity, we have not included the full text, but recognise the contributions of the Alzheimer’s Disease Neuroimaging Initiative, International Consortium for Brain Mapping, National Database for Autism Research, NIH Pediatric MRI Data Repository, National Database for Clinical Trials, Research Domain Criteria Database, Adolescent Brain Cognitive Development Study, UK Biobank Resource, 1000 Functional Connectomes Project, ABIDE I and II, and Open fMRI. This research was co-funded by the NIHR Cambridge Biomedical Research Centre and Marmaduke Sheild. ML is supported by a Gates Cambridge Scholarship from the University of Cambridge.
- 3.Brown, C., Kawahara, J., Hamarneh, G.: Connectome priors in deep neural networks to predict autism. In: 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018) (2018). https://doi.org/10.1109/ISBI.2018.8363534
- 4.Bruna, J., Zaremba, W., Szlam, A., LeCun, Y.: Spectral networks and locally connected networks on graphs. In: ICLR (2014)Google Scholar
- 7.Chollet, F.: Keras (2015). https://github.com/fchollet/keras
- 9.Defferrard, M., Bresson, P., Vandergheynst, X.: Convolutional neural networks on graphs with fast localized spectral filtering. In: NIPS, pp. 3844–3852 (2016)Google Scholar
- 13.Dinstein, I., Haar, S., Atsmon, S., Schtaerman, H.: No evidence of early head circumference enlargements in children later diagnosed with autism in Israel. Mol. Autism 8 (2018). https://doi.org/10.1186/s13229-017-0129-9
- 15.Erhan, D., Bengio, Y., Courville, A., Vincent, P.: Visualizing higher-layer features of a deep network. Technical report 1341, University of Montreal (2009)Google Scholar
- 19.Hamilton, W., Ying, R., Leskovec, J.: Representation learning on graphs: methods and applications. Bulletin of the IEEE Computer Society Technical Committee on Data Engineering (2017)Google Scholar
- 21.Hechtlinger, Y., Chakravarti, P., Qin, J.: A generalization of convolutional neural networks to graph-structured data. arXiv (2017)Google Scholar
- 24.Keown, C., Datko, M., Chen, C., Maximo, J., Jahedi, A., Müller, R.: Network organization is globally atypical in autism: a graph theory study of intrinsic functional connectivity. Biol. Psychiatry: Cogn. Neurosci. Neuroimaging 2, 66–75 (2017). https://doi.org/10.1016/j.bpsc.2016.07.008CrossRefGoogle Scholar
- 25.Khundrakpam, B., Lewis, J., Kostopoulos, P., Carbonell, F., Evans, A.: Cortical thickness abnormalities in autism spectrum disorders through late childhood, adolescence, and adulthood: a large-scale MRI study. Cereb. Cortex 27, 1721–1731 (2017). https://doi.org/10.1093/cercor/bhx038CrossRefGoogle Scholar
- 26.Kipf, T., Welling, M.: Semi-supervised classification with graph convolutional neural networks. In: ICLR 2017 (2017)Google Scholar
- 27.Kotikalapudi, R., Contributors: keras-vis (2017). https://github.com/raghakot/keras-vis
- 28.Khosla, M., Jamison, K., Kuceyeski, A., Sabuncu, M.R.: 3D convolutional neural networks for classification of functional connectomes. In: Stoyanov, D., et al. (eds.) DLMIA/ML-CDS-2018. LNCS, vol. 11045, pp. 137–145. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00889-5_16CrossRefGoogle Scholar
- 29.Nikolentzos, G., Meladianos, P., Tixier, A.J.-P., Skianis, K., Vazirgiannis, M.: Kernel graph convolutional neural networks. In: Kůrková, V., Manolopoulos, Y., Hammer, B., Iliadis, L., Maglogiannis, I. (eds.) ICANN 2018. LNCS, vol. 11139, pp. 22–32. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01418-6_3CrossRefGoogle Scholar
- 33.Poldrack, R., et al.: Toward open sharing of task-based fMRI data: the OpenfMRI project. Front. Neuroinform. 7 (2013). https://doi.org/10.3389/fninf.2013.00012
- 38.Simonyan, K., Vedaldi, A., Zisserman, A.: Deep inside convolutional networks: visualising image classification models and saliency maps. In: Workshop at International Conference on Learning Representations (2014)Google Scholar