Generalizability and reproducibility of functional connectivity in autism
Autism is hypothesized to represent a disorder of brain connectivity, yet patterns of atypical functional connectivity show marked heterogeneity across individuals.
We used a large multi-site dataset comprised of a heterogeneous population of individuals with autism and typically developing individuals to compare a number of resting-state functional connectivity features of autism. These features were also tested in a single site sample that utilized a high-temporal resolution, long-duration resting-state acquisition technique.
No one method of analysis provided reproducible results across research sites, combined samples, and the high-resolution dataset. Distinct categories of functional connectivity features that differed in autism such as homotopic, default network, salience network, long-range connections, and corticostriatal connectivity, did not align with differences in clinical and behavioral traits in individuals with autism. One method, lag-based functional connectivity, was not correlated to other methods in describing patterns of resting-state functional connectivity and their relationship to autism traits.
Overall, functional connectivity features predictive of autism demonstrated limited generalizability across sites, with consistent results only for large samples. Different types of functional connectivity features do not consistently predict different symptoms of autism. Rather, specific features that predict autism symptoms are distributed across feature types.
KeywordsAutism spectrum conditions Resting-state fMRI Functional connectivity MRI Replicability Reproducibility
Autism Brain Imaging Data Exchange
Autism Diagnostic Interview-Revised
Autism Diagnostic Observation Schedule
Analysis of Functional NeuroImages
Diffusion tensor imaging
False discovery rate
functional magnetic resonance imaging
Multi-echo independent component analysis
Magnetic resonance imaging
Region of interest
Social Responsiveness Scale
Atypical resting-state functional connectivity has been proposed as a metric for the pathophysiology of autism [1, 2]. While abnormal functional connectivity has been well-documented in the literature in autism, there are inconsistencies across studies with respect to the spatial distribution of connectivity atypicalities in the brain and even whether abnormal connectivity is too high or too low. There is no consensus on what types of brain connections are abnormal beyond idiosyncratic or atypical connectivity compared to typical development, or how features may evolve with age. Thus, researchers have developed a number of complementary resting-state fMRI analysis methods resulting in multiple approaches to quantifying functional connectivity in autism.
Many of these approaches describe patterns of under- or over-connectivity, in autism compared to controls, between multiple brain regions or networks including corticostriatal [3, 4, 5, 6, 7, 8], thalamocortical regions [4, 9], and default mode and salience networks [3, 10, 11, 12, 13, 14, 15, 16, 17]. Long-range underconnectivity and short-range overconnectivity have both been hypothesized as a brain abnormality in autism [1, 18, 19, 20]. Other models suggest abnormal segregation and integration of resting-state networks [21, 22, 23, 24, 25] and idiosyncrasy of connectivity [26, 27]. Aberrant homotopic connectivity in autism compared to controls, measured by interhemispheric correlations, has also been replicated in the literature [28, 29].
Despite this evolution of investigative methods, there are a number of limitations. These approaches are usually examined in isolation, limiting information about overlap or divergence of the information obtained from different methods. Due to conflicting results, there is no accepted role in the clinical practice of brain imaging to constrain diagnosis, prognosis, or therapeutic choices. Converging resting-state functional connectivity evidence is also complicated by the variability of study acquisition parameters, sample selection, preprocessing methods, and analysis methods. As a result, both the extent to which different approaches to functional connectivity in autism are found in similar cohorts of autism participants and whether they correspond to different phenotypic patterns of autistic symptoms remains unclear.
Fortunately, in tandem with advances in resting-state analysis methods comes the growth of publicly available datasets allowing for data collected from many sources to be processed using a uniform analysis pipeline. The Autism Brain Imaging Data Exchange (ABIDE) is one such publicly available dataset that now provides structural and functional brain imaging data collected from more than 25 brain imaging laboratories across the globe [29, 30]. With the availability of multi-site resting-state data comes the ability to test multiple resting-state functional connectivity theories related to autism brain function. This study attempts to distill published differences between individuals with autism and control individuals across a large sample, that includes multiple age ranges and varying cohort characteristics, using proposed methods for resting-state functional magnetic resonance imaging (fMRI) analysis. We do this by addressing the following questions:
1) Which functional connectivity features are reproducible across a large multi-site sample of participants with autism?
2) Can significant features of autism from a large sample that are heterogeneous across sites be identified within a single sample using high-temporal resolution, long-duration modern acquisition techniques?
3) To what extent do distinct functional connectivity features track together in the same participants, or do different features represent different aspects or endophenotypes of autism?
We present these findings across individual research sites included in the ABIDE dataset, the combined ABIDE I data release, the combined ABIDE II data release, and the full ABIDE data sample. Our goal is to compare and quantify the generalizability and reproducibility of functional connectivity methods in individuals with autism and identify the extent to which different methodological approaches identify complementary information.
The ABIDE dataset included an initial (ABIDE I) and a second data release (ABIDE II). A total of 17 sites contributed 1112 individuals in the ABIDE I release including 539 individuals with autism and 573 controls. Data from ABIDE II was compiled across 19 sites with a total of 1114 individual datasets comprised of 521 individuals with autism and 593 controls. For site-specific details on both diagnostic criteria and more detailed characterization of clinical and behavioral phenotypes, see . Both ABIDE I and ABIDE II contain data supplied by the University of Utah. The current study also includes a replication sample (Utah cohort) that consisted of 52 males with autism and 38 control males with high-temporal resolution and long duration resting-state scan data acquired as part of a larger longitudinal study of autism aimed at investigating brain development across the adolescent and adult lifespan.
Structural data from the ABIDE dataset were processed using FreeSurfer (v6.0.0). A detailed description of the FreeSurfer pipeline can be found on the FreeSurfer website . Preprocessing of the ABIDE fMRI blood-oxygen-level-dependent (BOLD) data was performed in MATLAB (MathWorks, Natick, MA, USA) using SPM12 (Wellcome Department of Imaging Neuroscience, London, UK). All images were corrected for motion using a realign and unwarp procedure. Each participant’s BOLD images were coregistered to their individual MPRAGE anatomic image sequence. Phase-shifted soft tissue correction (PSTCor)  was used to regress participant motion parameters, eroded white matter, eroded cerebral spinal fluid, and soft tissues of the face and calvarium. Eroded masks were obtained by removing all voxels from white matter and CSF masks that were adjacent to a voxel not in the mask. Volume censoring (scrubbing) was performed with removal of volumes before and after mean framewise displacement head motion greater than 0.3 mm . Only participants with ≥ 50% volumes remaining after scrubbing were considered for further analysis. In the ABIDE I data release, 1112 participants were analyzed and 419 were removed (220 due to motion, 199 due to other quality issues). In the ABIDE II data release, 1114 participants were analyzed and 405 were removed (217 due to motion, 188 due to other quality issues). Individual research sites with less than 10 participants remaining after initial quality control were removed from further analysis. During volume censoring, 14 ± 1% of volumes were censored for neurotypical subjects and 18 ± 1% of volumes were censored for autism subjects across sites (t (1400) = − 5.12, p < .001, 95% CI [−.05–.02]). Following preprocessing, quality control was completed by imaging experts that consisted of visual inspection of each participant’s neuroimaging data for successful completion of the preprocessing pipeline and image quality. Specifically, each image was checked for brain coverage and any scanner artifacts such as ghosting and indications of high movement. This was completed by viewing representative images on each plane. Images with obvious quality issues were immediately rejected. Images with suspected quality issues were subjected to further review requiring the reviewer to scroll through each volume in the image. A second instance of quality control consisted of verifying normalization by overlaying T1 images on processed BOLD data. Finally, the FreeSurfer subcortical segmentation was overlaid on representative images and assessed for quality. No edits were made to subcortical segmentations. Data not meeting visual inspection was removed from further analysis.
Detailed information regarding the acquisition and processing of this cohort can be found in King et al. . Briefly, resting-state functional images were acquired using a multi-band multi-echo echo-planar sequence (TR = 1553 milliseconds; flip angle = 65°; inplane acceleration factor = 2; fields of view = 208 mm; 72 axial slices; resolution = 2.0 mm isotropic; multi-band acceleration factor = 4; partial Fourier = 6/8; bandwidth = 1850 Hz; 3 echoes with TEs of 12.4 ms, 34.28 ms, and 56.16 ms; and effective TE spacing = 22 ms) with two acquisitions of 590 images (15 min, 27 s) each (one left-to-right and one right-to-left), each with 3 volumes representing different echo times. Structural images consisted of an MP2RAGE sequence with isotropic 1 mm resolution (TR = 5000 ms, TE = 2.91 ms). Structural data were processed using FreeSurfer (v6.0.0) using the default processing pipeline with the input image derived by multiplying the MP2RAGE uniform image by the proton-density-weighted image . Visual inspection of the finished product was completed to ensure subcortical segmentation quality. No subjects included in this cohort required any alterations to segmentation. Analysis of resting-state data was conducted using a multi-echo independent component analysis (ME-ICA) pipeline included in the Analysis of Functional NeuroImages (AFNI) open-source environment [36, 37].
Human connectome project
In order to establish normalized measures of resting-state functional connectivity patterns following typical development, FIX ICA processed fcMRI data for 1003 participants and diffusion tensor imaging (DTI) data from 1021 participants from the Human Connectome Project (ages 22–35)  were included in this analysis. As data in the ABIDE dataset includes a variety of acquisition parameters and data quality, some not yielding a very high resolution, relying on normative data created from dissimilar resting-state acquisitions was not ideal. Due to the high quality of the Human Connectome Project dataset, it was selected in order to model a representative typically developing the brain in a number of functional connectivity measures. Similarly, the DTI data included in the Human Connectome Project dataset provide high-quality data from which structural path lengths can be estimated. Regrettably, the ABIDE dataset does not include DTI data at this time.
Resting-state fMRI regions of interest
Parcellation of the brain regions of interest (ROI) from which functional connectivity values were derived was conducted as previously described [34, 39]. This process was completed for all three datasets. Briefly, for each participant, time series data were extracted from 333 cortical regions , 14 participant-specific subcortical regions from FreeSurfer-derived segmentation  (bilateral thalamus, caudate, putamen, amygdala, hippocampus, pallidum, and nucleus accumbens), and 14 bilateral cerebellar representations of a 7-network parcellation  with each network treated as a single region of interest. When combined, this parcellation scheme incorporates major cortical, subcortical, and cerebellar gray matter ROIs numbering 361 regions in total [39, 43].
Resting-state data analysis
Between-group analyses were conducted for each research site within the ABIDE dataset, for participants included in the ABIDE I (A1) and ABIDE II (A2) data releases, as well as the full ABIDE sample (ABIDE). All analyses were then duplicated in the Utah replication cohort.
General linear models were used to compare group differences in functional connectivity while controlling for age and mean head motion (both cohorts) as well as sex and site when applicable (ABIDE). Correction for multiple comparisons was completed using the false discovery rate (q (FDR) < .05). Unless stated otherwise, FDR corrections presented in figures were conducted on the combined p values across sites, A1, A2, ABIDE, and the Utah cohort. Pearson correlations were used to test for associations between neuroimaging findings and both behavioral and cognitive factors.
BrainNet Viewer was used to create brain images included in figures . For display purposes, a smoothed gray matter mask was used, and each gray matter voxel was assigned to one of the 333 regions based on nearest proximity. Statistical analyses were performed in the MATLAB computing environment (MathWorks, Natick, MA, USA) and SPSS software (version 25) for Mac OS X.
Resting-state functional connectivity analysis methods
A total of eight resting-state functional connectivity analysis methods were selected to compare connectivity differences between individuals with autism and controls. Methods were selected based on an extensive review of the literature.
Positive versus negative functional connectivity
Specific differences in task fMRI have been identified in brain regions responsible for control of inhibition , and specific abnormalities have been observed in autism for negatively correlated (anticorrelated) connections  and in negative BOLD responses in networks . In order to establish normalized connectivity strength values for each of the 361 Gy matter region pairs from an independent sample of young adults, functional connectivity MRI values from the Human Connectome Project dataset were averaged across participants. These values were then grouped into 40 bins of .02 ranging between −.2 and .6. The four most negative bins contained no data and were removed. Mean functional connectivity values for each participant in the ABIDE dataset were extracted across the same 361 ROI pairs and average values for the connections in each bin were computed for each participant. These values were then compared between individuals with autism and controls in the ABIDE and Utah cohorts for each bin.
Short- and long-range functional connectivity
Between-group differences in short- and long-range functional connectivity in the ABIDE sample were calculated using two methods. First, normalized path lengths previously established using DTI data from the Human Connectome Project dataset were calculated between each pair of ROIs for which a structural connection was identified. A group average DTI template was constructed from 1021 participants using q-space diffeomorphic reconstruction with deterministic fiber tracking algorithm with 50,000 whole brain seeding in DSI studio [47, 48]. Data from 47,903 fibers were intersected with the 361-region atlas previously described to obtain structurally connected regions. Connections were binned into 10 bins of 50 mm ranging from 0 to 500 mm. Functional connectivity region pair values from the ABIDE dataset were then compared between individuals with autism and controls for each of the path length bins for region pairs identified from the Human Connectome Project dataset.
Second, this process was repeated using the Euclidian distance between ROI centroids in the Human Connectome Project dataset for all ROI pairs. Bins were created representing these distances ranging between 0 and 165 mm for every 5 mm. The first bin was empty resulting in a total of 32 bins. Values from the ABIDE dataset were then compared between individuals with autism and controls for each of the Euclidian distance path length bins for region pairs identified from the Human Connectome Project dataset. This process was replicated in the Utah cohort.
Region of interest centroids were used to identify homotopic region pairs by first inverting the x coordinate from each of the 361 ROI centroids and determining which of the non-inverted ROIs represented the ROI with the minimum distance. The same homotopic region pairs were used for all the datasets in the analysis.
Ipsilateral corticostriatal connectivity was evaluated by extracting functional connectivity values between participant-specific, FreeSurfer-derived left and right caudate, putamen, globus pallidus, and nucleus accumbens subcortical gray matter ROIs to all intrahemispheric cortical ROIs. Cortical ROIs numbered 161 in the left hemisphere and 172 in the right hemisphere. Between-group differences in corticostriatal connectivity were then explored for each of the eight subcortical gray matter ROIs for both the ABIDE and Utah cohorts.
Thalamocortical connectivity was evaluated by extracting functional connectivity values between left and right thalamus to all ipsilateral cortical ROIs as described above. Between-group differences in thalamocortical connectivity were then explored for both the ABIDE and Utah cohorts.
These variance values were then compared between individuals with autism and controls.
Network modularity was used to assess segregation while integration was assessed utilizing global efficiency. In order to compute modularity for each participant in the ABIDE and Utah cohorts, time series data were first normalized to fall between −1 and 1. Next, the MATLAB script modularity_und.m, from the Brain Connectivity Toolbox, was used to modularize the data . Each participant’s maximized modularity value was stored for between-group analyses. Participant-specific global efficiency was then computed on inverted normalized values from the previous step using the Brain Connectivity Toolbox’s efficiency_wei.m script. Modularity and global efficiency were then compared between individuals with autism and controls in the ABIDE and Utah cohorts.
Default mode and salience network connectivity
For each participant in the ABIDE and Utah cohorts, functional connectivity values from the 333 ROI Gordon et al.  parcellation that had been previously assigned to the default mode (41 cortical ROIs) and salience (36 total cortical ROIs including 4 from the salience network and 32 from the dorsal attention network) networks in the referenced study were evaluated for within and between network connectivity.
Recent reports have suggested that alterations in lagged functional connectivity may be abnormal in autism [34, 50]. A closely related finding that the shape and parameters of the hemodynamic response function may be altered in autism [51, 52], similar to alterations in the HRF in stroke , may be related to both neural and non-neural factors. A study of the width of the autocorrelation function in resting state fMRI, determined by lagged connectivity, found a close mutual relationship between parameters of the HRF, autocorrelation width, cognitive processing speed, and reaction times on functional tasks . A neural contribution to lagged connectivity in autism may relate to prolonged neural activity in autism and could be associated with recent reports of brain hyperstability in autism [54, 55, 56, 57].
Based on findings detailed in King et al.  that found between-group differences in a measure of lag-based functional connectivity, these lag-based findings were also included in feature selection in order to compare method similarities. Lag-based connectivity values considered for feature selection from King et al.  represented functional connectivity values at 6 (ABIDE) or 6.212 (Utah cohort) second lags. Refer to King et al.  for details related to processing. Briefly, cross-correlation curves were computed using the repetition time (TR) from each individual research site, with cubic spline interpolation to identify correlation at zero lag and positive and negative lags (seconds).
In order to further determine which of the methods provide analogous findings in individuals with autism compared to controls, a number of relevant features were selected for further analysis. Feature selection consisted of identifying functional connectivity features that demonstrated a significance value of p < .05 (uncorrected) for each of the examined methods. Methods used to evaluate idiosyncrasy and abnormal segregation and integration provided only one p value for each participant and were not included in feature selection.
Feature selection was conducted on p values in the combined ABIDE sample. Region pairs identified using feature selection were also evaluated in the Utah cohort. A linear regression model was employed to regress out effects of site, age, sex, and mean head motion (root-mean-square) in the ABIDE sample, and age and mean head motion in the Utah cohort. As the proportion of censored volumes was significantly different between individuals with autism and controls, a post hoc analysis was also completed using percent motion-free volumes as a regressor in place of mean head motion. In order to compare similarities in findings related to methods, Pearson correlations were calculated between all features as well as features averaged across methods in individuals with autism from the combined ABIDE sample. In the combined ABIDE autism sample, Pearson correlations (unthresholded) were also calculated between available reported performance, verbal, and full-scale intelligence quotient (IQ) scores (see http://fcon_1000.projects.nitrc.org/indi/abide/ for details related to IQ measures), Autism Diagnostic Observation Schedule (ADOS) severity, total and subscale scores (Gotham score from ABIDE I; ADOS-2 scores from ABIDE II) [58, 59], Social Responsiveness Scale (SRS) raw total and subscale scores , and Autism Diagnostic Interview-Revised (ADI) total, subscale, and onset scores .
Resting-state functional connectivity
Demographic information for the ABIDE dataset and the Utah cohort
Male % (total sample)
16.7 ± 7.3
16.5 ± 6.6
.11 ± .05
.09 ± .05
13.4 ± 5.9
13.7 ± 6.7
.11 ± .06
.10 ± .05
15.1 ± 6.9
15.1 ± 6.8
.11 ± .06
.10 ± .05
27.7 ± 8.7
27.1 ± 7.5
.10 ± .04
.07 ± .02
Comparison of resting-state functional connectivity methods
No one method demonstrated wholly reproducible findings across sites and datasets when controlling for age, sex, mean head motion, and site.
Positive versus negative functional connectivity
Short- and long-range connectivity
Default mode and salience network connectivity
Correlation with autism symptoms
As an additional investigation into what factors may be related to limited reproducibility, we performed a more systematic investigation of demographic and technical factors that might underlie limited replication of functional connectivity results. We evaluated the combined ABIDE data by removing one site at a time and comparing the site-specific results for that site across features (vector of autism vs. control t statistics for 1229 features) to the results from the remaining ABIDE sample, yielding a measurement of “generalizability” for data from each site. We compared this measure using partial correlation across sites to seven factors: age, sex ratio, mean head motion, percent of motion free data, eyes open vs. eyes closed status, temporal resolution (repetition time), and number of subjects per site. Of these, only the number of subjects per site showed significant correlation to generalizability across sites (r = .65, p = .0024), although the other 6 factors showed variable partial correlation across sites that may contribute to limited reproducibility (see Additional file 1: Figure S12). Nevertheless, almost all of the data in the ABIDE sample was acquired with temporal resolutions of at least 2 s, and it is possible that faster acquisitions that can freeze effects of breathing and mitigate aliasing of heart rate artifacts may offer nontrivial improvement in results.
This study examined whether different aspects of functional MRI connectivity described in the literature represent distinct symptoms of autism or cohorts of individuals with autism and the extent to which these functional connectivity methods exhibit reproducibility across individuals and datasets. For all functional connectivity methods tested, results showed poor generalizability across sites and participants rather than clear reproducibility, with no method demonstrating highly reproducible results when compared to the entire multi-site ABIDE sample (see Fig. 13 for summary figure with p < .05 (uncorrected) results; (q [FDR] < .05 corrected results can be found in Additional file 1: Figure S8). When functional connectivity features were compared to behavioral symptoms, distinct types of analysis, such as corticostriatal, homotopic, or default mode connectivity, did not correlate with different types of autism symptoms. Rather, individual connections or features that tracked with different symptoms of autism were distributed across methodological approaches without any clear pattern.
In the current study, few sites demonstrated significant between-group differences in positive vs. negative functional connectivity assessed using bins of connections with similar connectivity in the independent Human Connectome Project sample. Short-range and long-range connectivity results were also inconsistent across sites, consistent with a recent analysis demonstrating only region-specific local overconnectivity using a regional homogeneity approach, with different subgroups of subjects demonstrating variably higher or lower long-distance connectivity . Variability in local connectivity has also been demonstrated with cohorts differing in fMRI acquisitions with eyes open vs. eyes closed . Theoretical proposals of long-range under-connectivity and short-range over-connectivity [1, 18, 19, 20] have been variably defined in terms of distance ranging from cortical columns to many centimeters, and studies examining distant connections have produced variable over- and under-connectivity. The analysis in the current sample may be limited by the use of an independent dataset (Human Connectome Project) not matched for sex and age to define distances.
Encouragingly, the current study found decreased homotopic connectivity which has been reported and replicated in the literature [28, 29]. Though there were some sites that demonstrated increased homotopic connectivity, findings reaching multiple comparison correction were nearly all decreased in individuals with autism compared to controls.
Consistent with the literature, this study also found both hypo- and hyper-connectivity in corticostriatal connections [3, 4, 5, 6, 7]; however, the direction of these findings was not consistent between research sites, and it appears to be predominantly decreased in individuals with autism compared to controls when larger sample sizes are assessed. Similar incongruities were found with thalamocortical connectivity which also varies in the literature with respect to directionality [4, 9].
In the current study, idiosyncrasy was estimated by calculating the variance between an individual’s time series data for each ROI and an averaged value based on an independent dataset. Two sites showed decreased idiosyncrasy in individuals with autism compared to controls. This finding is in contrast to outcomes in the literature that report increased idiosyncrasy in individuals with autism compared to controls [26, 27]. This inconsistency is likely attributable to methodological differences as the majority of studies in the literature utilize machine learning techniques to establish idiosyncratic values.
Similarly, the current study found no between-group differences in modularity for any research site or combined dataset; however, increased global efficiency was found in individuals with autism compared to controls for two research sites. Both increased and decreased global efficiency in individuals with autism compared to controls have been demonstrated in the literature [21, 22]. With regard to within and between default mode and salience networks, the findings in the current study closely mirror those from the literature [3, 10, 11, 12, 13, 14, 15]; however, it is important to note that few research sites demonstrated multiple comparison corrected findings (see Additional file 1). Indeed, widespread decreased connectivity meeting FDR correction was only evident in the larger combined ABIDE dataset with respect to inter-default mode network connectivity.
Overall, we found poor generalizability across sites when testing which functional connectivity features predict autism, with consistent results only for samples of hundreds of participants. Furthermore, different types of functional connectivity features (homotopic, thalamocortical, corticostriatal, specific networks) seem to not consistently predict different features of autism. Rather, specific features that predict autism symptoms seem to be distributed across feature types. Interestingly, there is a web of interrelationships between which features are high in which participants, with only lagged connectivity not showing correlation across individuals with autism with other feature types. As more features are added together, consistent results are obtained regardless of which feature types are added. It may be that these findings reflect global connectivity, which predicts ADOS and ADI scores but not SRS scores. Indeed, measures of global connectivity have been found to decrease in individuals with autism compared to controls [34, 64]. Even when using modern acquisition strategies (multi-band data, 30-min acquisition times per participant), heterogeneity and modest prediction rates for autism are seen, although findings were very consistent with those obtained from the entire ABIDE sample, suggesting that long-duration, high-temporal resolution acquisitions may improve replicability of results. Holiga and colleagues used data from 4 separate datasets including ABIDE I and ABIDE II and examined reproducibility of degree centrality as a metric distinguishing autism from control individuals . While effect sizes were large in the EU-AIMS and InFoR datasets (Cohen’s d > .8 for some measures), effect sizes were much smaller in ABIDE datasets (d = .2), possibly indicating that technical parameters of acquisition may contribute to reproducibility of the results, given that EU-AIMS data were acquired with more volumes and using a multi-echo technique. Similarly, a recent report imaging participants with autism and low cognitive and verbal performance identified similar connectivity differences to the entire ABIDE sample in this report within a single site’s data . While none of the individual features tested show promise in this analysis as sensitive and specific biomarkers, consistent with recent reviews [66, 67], the individual features demonstrated a rich web of interrelationships across subjects as well as differences across subjects that may inform efforts to identify clinical subtypes  or use multi-parametric deep-learning approaches to arrive at more sensitive and specific imaging markers .
A number of limitations should be considered. Though we consider all participant data being processed using the same parameters a strength of this study, certain aspects of that model could act as a confound. For example, differences in acquisition parameters, volume numbers, and spatial scale may benefit from preprocessing pipelines more suited to the nuances of each study site with the ABIDE dataset. Second, while efforts were made to minimize variance due to differences in the research site, statistical controls are likely not able to account for more nuanced between-site variance. Third, while we attempted to replicate methods identified in the literature, all method tests were based on a common parcellation scheme that was created using imaging data from adult participants. Many of the participants included in the ABIDE dataset are children or adolescents. Thus, it may be that lack of reproducibility across methods reported in this study are tied to differences in cortical parcellations, nuanced atlas registration, or changes across development; though, many of the methods tested do not require extremely precise cerebral region assignment (long-range vs. short-range, positive vs. negative, etc.), and age was included as a covariate in all analyses. Finally, it cannot be overstated that strategic choices in image postprocessing have a clear effect on functional connectivity results , and different choices in postprocessing may have resulted in improved or poorer reproducibility.
Functional connectivity in autism is characterized more by variability than consistent reproducibility across participants and sites, which may be attributed to multi-factorial demographic and technical differences including recruitment strategy across autism and neurotypical cohorts, age of participants, male to female ratios, duration of acquisition, and technical acquisition parameters. Different functional connectivity methods did not identify distinct behavioral correlations for participant cohorts; however, most functional connectivity methods covary with ADOS, ADI, and SRS scores, indicating a relationship to autism symptomatology. We did not observe that different approaches to functional connectivity (e.g., homotopic connectivity, long-range vs. short-range connectivity, corticostriatal connectivity), nor connections associated with distinct spatial regions or networks, cleanly identify with different behavioral features of autism. Extensive variability and limited reproducibility were not overcome by averaging results of many features of a similar type or by extensive postprocessing to mitigate physiological and technical artifacts. Many functional connectivity features do show differences in symptom profiles; methods from deep-learning and big data analysis, combined with large feature sets from functional connectivity data, may be promising approaches to prognosis, outcomes monitoring, and treatment effects.
We would like to thank the contributors and organizers of the Autism Brain Imaging Data Exchange (http://fcon_1000.projects.nitrc.org/indi/abide/). We would also like to thank our study participants and their families for their involvement and ongoing support of our research.
This study was supported by the National Institute of Health (R01 MH080826; S10 OD18482).
JBK, JEL, and JSA designed the study. MDBP, CKK, and JM recruited participants, administered assessments, and were responsible for data collection including neuroimaging data. MDBP, KLK, and JEL contributed to clinical diagnosis consensus. JBK and JSA performed statistical analyses and drafted the manuscript. FW and JCF contributed to neuroimaging data analysis. All authors participated in data interpretation and manuscript preparation, and have given final approval of the version to be published.
Ethics approval and consent to participate
This study was approved by the Institutional Review Board at the University of Utah. All study participants provided written informed assent or consent prior to study participation with parental or legal guardian consent required of all study participants under the age of 18.
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
- 8.Holiga S, Hipp JF, Chatham CH, Garces P, Spooren W, D'Ardhuy XL, Bertolino A, Bouquet C, Buitelaar JK, Bours C, et al. Patients with autism spectrum disorders display reproducible functional connectivity alterations. Sci Transl Med. 2019(481):11. https://doi.org/10.1126/scitranslmed.aat9223.CrossRefGoogle Scholar
- 23.Rudie JD, Shehzad Z, Hernandez LM, Colich NL, Bookheimer SY, Iacoboni M, Dapretto M. Reduced functional integration and segregation of distributed neural systems underlying social and emotional information processing in autism spectrum disorders. Cereb Cortex. 2012;22(5):1025–37.CrossRefGoogle Scholar
- 31.Autism Brain Imaging Data Exchange, [http://fcon_1000.projects.nitrc.org/indi/abide/].
- 32.FreeSurfer [https://surfer.nmr.mgh.harvard.edu/].
- 37.Analysis of functional NeuroImages [afni.nimh.nih.gov].
- 48.DSI Studio [http://dsi-studio.labsolver.org].
- 54.Rashid B, Blanken LME, Muetzel RL, Miller R, Damaraju E, Arbabshirani MR, Erhardt EB, Verhulst FC, van der Lugt A, Jaddoe VWV, et al. Connectivity dynamics in typical development and its relationship to autistic traits and autism spectrum disorder. Hum Brain Mapp. 2018;39(8):3127–42.CrossRefGoogle Scholar
- 57.Watanabe T, Rees G, Masuda N. Atypical intrinsic neural timescale in autism. Elife. 2019;8. https://doi.org/10.7554/eLife.42256.
- 59.Lord C, DiLavore PC, Gotham K, Guthrie W, Luyster RJ, Risi S, Rutter M, Western Psychological S. Autism diagnostic observation schedule, second edition (ADOS-2) manual (part I): modules 1–4. Torrance: Western Psychological Services; 2012.Google Scholar
- 60.Constantino JN. The social responsiveness scale. Los Angeles: Western Psychological Services; 2002.Google Scholar
- 62.Jao Keehn RJ, Nair S, Pueschel EB, Linke AC, Fishman I, Muller RA. Atypical local and distal patterns of occipito-frontal functional connectivity are related to symptom severity in autism. Cereb Cortex. 2018.Google Scholar
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.