Journal of Autism and Developmental Disorders

, Volume 44, Issue 11, pp 2921–2930 | Cite as

Are We Under-Estimating the Association Between Autism Symptoms?: The Importance of Considering Simultaneous Selection When Using Samples of Individuals Who Meet Diagnostic Criteria for an Autism Spectrum Disorder

  • Aja Louise Murray
  • Karen McKenzie
  • Renate Kuenssberg
  • Michael O’Donnell
Original Paper


The magnitude of symptom inter-correlations in diagnosed individuals has contributed to the evidence that autism spectrum disorders (ASD) is a fractionable disorder. Such correlations may substantially under-estimate the population correlations among symptoms due to simultaneous selection on the areas of deficit required for diagnosis. Using statistical simulations of this selection mechanism, we provide estimates of the extent of this bias, given different levels of population correlation between symptoms. We then use real data to compare domain inter-correlations in the Autism Spectrum Quotient, in those with ASD versus a combined ASD and non-ASD sample. Results from both studies indicate that samples restricted to individuals with a diagnosis of ASD potentially substantially under-estimate the magnitude of association between features of ASD.


Autism spectrum disorder Range restriction Fractionable triad Simultaneous selection Sampling 



Aja Louise Murray is supported by a studentship from The University of Edinburgh Centre for Cognitive Ageing and Cognitive Epidemiology, (, part of the cross council Lifelong Health and Wellbeing Initiative (G0700704/84698). Funding from the Biotechnology and Biological Sciences Research Council, Engineering and Physical Sciences Research Council, Economic and Social Science Research Council, and the Medical Research Council is gratefully acknowledged.

Conflict of interest

The authors have no conflict of interest to declare.

Supplementary material

10803_2014_2154_MOESM1_ESM.docx (15 kb)
Supplementary material 1 (DOCX 14 kb)


  1. Aitken, A. C. (1935). Note on selection from a multivariate normal population. Proceedings of the Edinburgh Mathematical Society (Series 2), 4(02), 106–110.CrossRefGoogle Scholar
  2. Alexander, R. A., Carson, K. P., Alliger, G. M., & Cronshaw, S. F. (1989). Empirical distributions of range restricted SD x in validity studies. Journal of Applied Psychology, 74(2), 253.CrossRefGoogle Scholar
  3. Allison, C., Auyeung, B., & Baron-Cohen, S. (2012). Toward brief “red flags” for autism screening: The short autism spectrum quotient and the short quantitative checklist in 1,000 cases and 3,000 controls. Journal of the American Academy of Child and Adolescent Psychiatry, 51(2), 202–212.PubMedCrossRefGoogle Scholar
  4. American Psychiatric Association. (1994). Diagnostic and statistical manual of mental disorders (4th ed.). Washington, DC: American Psychiatric Association.Google Scholar
  5. American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). Washington, DC: American Psychiatric Association.Google Scholar
  6. Austin, E. J. (2005). Personality correlates of the broader autism phenotype as assessed by the Autism Spectrum Quotient (AQ). Personality and Individual Differences, 38(2), 451–460.CrossRefGoogle Scholar
  7. Baird, G., Simonoff, E., Pickles, A., Chandler, S., Loucas, T., Meldrum, D., et al. (2006). Prevalence of disorders of the autism spectrum in a population cohort of children in South Thames: The Special Needs and Autism Project (SNAP). The Lancet, 368(9531), 210–215.CrossRefGoogle Scholar
  8. Baron-Cohen, S., Scott, F. J., Allison, C., Williams, J., Bolton, P., Matthews, F. E., et al. (2009). Prevalence of autism-spectrum conditions: UK school-based population study. The British Journal of Psychiatry, 194(6), 500–509.PubMedCrossRefGoogle Scholar
  9. Baron-Cohen, S., Wheelwright, S., Skinner, R., Martin, J., & Clubley, E. (2001). The autism-spectrum quotient (AQ): Evidence from asperger syndrome/high-functioning autism, malesand females, scientists and mathematicians. Journal of Autism and Developmental Disorders, 31(1), 5–17.PubMedCrossRefGoogle Scholar
  10. Berry, C. M., Clark, M. A., & McClure, T. K. (2011). Racial/ethnic differences in the criterion-related validity of cognitive ability tests: A qualitative and quantitative review. Journal of Applied Psychology, 96(5), 881–906.PubMedCrossRefGoogle Scholar
  11. Booth, T., Murray, A. L., McKenzie, K., Kuenssberg, R., O’Donnell, M., & Burnett, H. (2013). Brief report: An evaluation of the AQ-10 as a brief screening instrument for ASD in adults. Journal of Autism and Developmental Disorders, 43(12), 2997–3000.PubMedCrossRefGoogle Scholar
  12. Brunsdon, V. E., & Happé, F. (2014). Exploring the ‘fractionation’of autism at the cognitive level. Autism, 18(1), 17–30.PubMedCrossRefGoogle Scholar
  13. Constantino, J. N., Gruber, C. P., Davis, S., Hayes, S., Passanante, N., & Przybeck, T. (2004). The factor structure of autistic traits. Journal of Child Psychology and Psychiatry, 45(4), 719–726.PubMedCrossRefGoogle Scholar
  14. Dominicus, A., Palmgren, J., & Pedersen, N. L. (2006). Bias in variance components due to nonresponse in twin studies. Twin Research and Human Genetics, 9(2), 185–193.PubMedCrossRefGoogle Scholar
  15. Dworzynski, K., Happé, F., Bolton, P., & Ronald, A. (2009). Relationship between symptom domains in autism spectrum disorders: A population based twin study. Journal of Autism and Developmental Disorders, 39(8), 1197–1210.PubMedCrossRefGoogle Scholar
  16. Fife, D. A., Mendoza, J. L., & Terry, R. (2012). The assessment of reliability under range restriction: A comparison of α, ω, and test–retest reliability for dichotomous data. Educational and Psychological Measurement, 72(5), 862–888.CrossRefGoogle Scholar
  17. Frazier, T. W., Youngstrom, E. A., Speer, L., Embacher, R., Law, P., Constantino, J., et al. (2012). Validation of proposed DSM 5 criteria for Autism Spectrum Disorder. Journal of the American Academy of Child and Adolescent Psychiatry, 51(1), 28–40.PubMedCrossRefPubMedCentralGoogle Scholar
  18. Happé, F., & Ronald, A. (2008). The ‘fractionable autism triad’: A review of evidence from behavioural, genetic, cognitive and neural research. Neuropsychology Review, 18(4), 287–304.PubMedCrossRefGoogle Scholar
  19. Hobson, R. P. (2014). The coherence of autism. Autism, 18(1), 6–16.PubMedCrossRefGoogle Scholar
  20. Hoekstra, R. A., Bartels, M., Cath, D. C., & Boomsma, D. I. (2008). Factor structure, reliability and criterion validity of the Autism-Spectrum Quotient (AQ): A study in Dutch population and patient groups. Journal of Autism and Developmental Disorders, 38(8), 1555–1566.PubMedCrossRefPubMedCentralGoogle Scholar
  21. Hoekstra, R. A., Vinkhuyzen, A. A., Wheelwright, S., Bartels, M., Boomsma, D. I., Baron-Cohen, S., et al. (2011). The construction and validation of an abridged version of the autism-spectrum quotient (AQ-Short). Journal of Autism and Developmental Disorders, 41(5), 589–596.PubMedCrossRefPubMedCentralGoogle Scholar
  22. Hunter, J. E., Schmidt, F. L., & Le, H. (2006). Implications of direct and indirect range restriction for meta-analysis methods and findings. Journal of Applied Psychology, 91(3), 594.PubMedCrossRefGoogle Scholar
  23. Kolevzon, A., Smith, C. J., Schmeidler, J., Buxbaum, J. D., & Silverman, J. M. (2004). Familial symptom domains in monozygotic siblings with autism. American Journal of Medical Genetics Part B: Neuropsychiatric Genetics, 129(1), 76–81.CrossRefGoogle Scholar
  24. Kuenssberg, R., Murray, A. L., Booth, T., & McKenzie, K. (2014). Structural validation of the abridged Autism Spectrum Quotient-Short Form in a clinical sample of people with autism spectrum disorders. Autism, 18(2), 69–75.PubMedCrossRefGoogle Scholar
  25. Lai, M. C., Lombardo, M. V., Pasco, G., Ruigrok, A. N., Wheelwright, S. J., Sadek, S. A., et al. (2011). A behavioral comparison of male and female adults with high functioning autism spectrum conditions. PLoS ONE, 6(6), e20835.PubMedCrossRefPubMedCentralGoogle Scholar
  26. Lawley, D. N. (1944). A note on Karl Pearson’s selection formulae. Proceedings of the Royal Society of Edinburgh. Section A. Mathematical and Physical Sciences, 62(1), 28–30.CrossRefGoogle Scholar
  27. Lundström, S., Chang, Z., Råstam, M., Gillberg, C., Larsson, H., Anckarsäter, H., et al. (2012). Autism spectrum disorders and autistic-like traits: Similar etiology in the extreme end and the normal variation. Archives of General Psychiatry, 69(1), 46–52.PubMedCrossRefGoogle Scholar
  28. Maenner, M. J., Rice, C. E., Arneson, C. L., Cunniff, C., Schieve, L. A., Carpenter, L. A., et al. (2014). Potential impact of DSM-5 criteria on autism spectrum disorder prevalence estimates. JAMA Psychiatry, 71(3), 292–300.Google Scholar
  29. Mandy, W., Charman, T., Puura, K., & Skuse, D. (2014). Investigating the cross-cultural validity of DSM-5 autism spectrum disorder: Evidence from Finnish and UK samples. Autism, 18(1), 45–54.PubMedCrossRefGoogle Scholar
  30. Mazefsky, C. A., Goin-Kochel, R. P., Riley, B. P., & Maes, H. H. (2008). Genetic and environmental influences on symptom domains in twins and siblings with autism. Research in Autism Spectrum Disorders, 2(2), 320–331.PubMedCrossRefPubMedCentralGoogle Scholar
  31. Murray, A. L., Booth, T., McKenzie, K., Kuenssberg, R., & O’Donnell, M. (2014). Are Autistic traits measured equivalently in individuals with and without an Autism Spectrum Disorder? An invariance analysis of the Autism Spectrum Quotient Short Form. Journal of Autism and Developmental Disorders, 44(1), 55–64.PubMedCrossRefGoogle Scholar
  32. Muthén, B. (1990). Moments of the censored and truncated bivariate normal distribution. British Journal of Mathematical and Statistical Psychology, 43(1), 131–143.CrossRefGoogle Scholar
  33. Pearson, K. (1903). Mathematical contributions to the theory of evolution. XI. On the influence of natural selection on the variability and correlation of organs. Philosophical Transactions of the Royal Society of London. Series A, Containing Papers of a Mathematical or Physical Character, 200, 1–66.CrossRefGoogle Scholar
  34. Rivet, T. T., & Matson, J. L. (2011). Review of gender differences in core symptomatology in autism spectrum disorders. Research in Autism Spectrum Disorders, 5(3), 957–976.CrossRefGoogle Scholar
  35. Robinson, E. B., Koenen, K. C., McCormick, M. C., Munir, K., Hallett, V., Happé, F., et al. (2012). A multivariate twin study of autistic traits in 12-year-olds: Testing the fractionable autism triad hypothesis. Behavior Genetics, 42(2), 245–255.PubMedCrossRefPubMedCentralGoogle Scholar
  36. Rutter, M. (2014). Addressing the issue of fractionation in autism spectrum disorder: A commentary on Brunsdon and Happé, Frazier et al., Hobson and Mandy et al. Autism, 18(1), 55–57.PubMedCrossRefGoogle Scholar
  37. Sackett, P. R., & Yang, H. (2000). Correction for range restriction: An expanded typology. Journal of Applied Psychology, 85(1), 112–118.PubMedCrossRefGoogle Scholar
  38. R Core Team (2013). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing, ISBN 3-900051-07-0, URL:
  39. Schmidt, F. L., & Hunter, J. E. (1977). Development of a general solution to the problem of validity generalization. Journal of Applied Psychology, 62(5), 529–540.CrossRefGoogle Scholar
  40. Takagishi, H., Takahashi, T., Yamagishi, T., Shinada, M., Inukai, K., Tanida, S., et al. (2010). Salivary testosterone levels and autism-spectrum quotient in adults. Neuroendocrinology Letters, 31(6), 101–105.Google Scholar
  41. Taylor, A. (2004). The consequences of selective participation on behavioral-genetic findings: Evidence from simulated and real data. Twin Research, 7(05), 485–504.PubMedCrossRefGoogle Scholar
  42. Wheelwright, S., Auyeung, B., Allison, C., & Baron-Cohen, S. (2010). Research defining the broader, medium and narrow autism phenotype among parents using the Autism Spectrum Quotient (AQ). Molecular Autism, 1(10), 1–9.Google Scholar
  43. Williams, D. M., & Bowler, D. M. (2014). Autism spectrum disorder: Fractionable or coherent? Autism, 18(1), 2–5.PubMedCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  • Aja Louise Murray
    • 1
    • 2
  • Karen McKenzie
    • 3
  • Renate Kuenssberg
    • 4
  • Michael O’Donnell
    • 5
  1. 1.Centre for Cognitive Ageing and Cognitive EpidemiologyUniversity of EdinburghEdinburghUK
  2. 2.Department of PsychologyUniversity of EdinburghEdinburghUK
  3. 3.Community Child HealthNHS LothianEdinburghUK
  4. 4.NHS FifeKirkcaldyUK
  5. 5.School of PsychologyUniversity of Ulster at MageeLondonderryUK

Personalised recommendations