Advertisement

Child Psychiatry & Human Development

, Volume 50, Issue 6, pp 987–1001 | Cite as

Description of Criterion Validity of the Autism Spectrum Rating Scales 6–18 Parent Report: Initial Exploration in a Large Community Sample

  • Amy CamodecaEmail author
Original Article
  • 98 Downloads

Abstract

Research regarding psychometric properties of autism spectrum disorder questionnaires is lacking. This study explored the criterion validity of the ASRS 6–18 parent report (ASRS-6–18-P) in a large, well-characterized, real-world clinical sample of 422 children (\(\bar{X}_{\text{age}} = 10.04;\) autism [AUT] n = 139; non-autism [NOT] n = 283) evaluated with the Autism Diagnostic Observation Schedule-2, a gold-standard measure. Significant mean differences were observed for DSM-5, social, and unusual behaviors. Total, DSM-5, social, and unusual behaviors demonstrated significant correlations with ADOS-2 comparison scores (modules 1–3), but not with module 4 (raw) scores. DSM-5 and Unusual Behaviors demonstrated significant but poor AUCs (0.60). Findings with/without covariates (IQ/age) were overall similar. Sensitivity and specificity could not be optimized. The suggested cutpoint (T-score = 60) demonstrated unacceptably high false positive rates (> 76.33%). While findings suggest limited diagnostic utility of the ASRS-6–18-P, the sample’s complex psychiatric presentation and measurement error inherent in cutoff score application should be considered when generalizing results. Further research is recommended.

Keywords

Autism spectrum disorder Parent report Validity ROC analyses 

Notes

Compliance with Ethical Standards

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical Approval

This article does not contain any studies with animals performed by any of the authors. All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. For this type of study formal consent is not required.

Supplementary material

10578_2019_899_MOESM1_ESM.docx (36 kb)
Supplementary material 1 (DOCX 36 kb)

References

  1. 1.
    American Psychiatric Association (2013) Diagnostic and statistical manual of mental disorders, 5th edn. American Psychiatric Publishing, RosslynCrossRefGoogle Scholar
  2. 2.
    Norris M, Lecavalier L (2010) Screening accuracy of level 2 autism spectrum disorder rating scales. Autism 14:263–284PubMedGoogle Scholar
  3. 3.
    Mckenney ELW, Dorencz J, Bristol RM, Hall LP (2015) Publishing about ASD in school psychology journals. Psychol Schools 52:265–283Google Scholar
  4. 4.
    Falkmer T, Anderson K, Falkmer M, Horlin C (2013) Diagnostic procedures in autism spectrum disorders: a systematic literature review. Eur Child Adolesc Psychiatry 22:329–340PubMedGoogle Scholar
  5. 5.
    Hampton J, Strand PS (2015) A review of level 2 parent-report instruments used to screen children aged 1.5–5 for autism: a meta-analytic update. J Autism Dev Disord 45:2519–2530PubMedGoogle Scholar
  6. 6.
    Hus V, Bishop S, Gotham K, Huerta M, Lord C (2013) Factors influencing scores on the social responsiveness scale. J Child Psychol Psychiatry 54:216–224PubMedGoogle Scholar
  7. 7.
    Rutter M, LeCouteur A, Lord C (2003) Autism diagnostic interview-revised. Western Psychological Services, TorranceGoogle Scholar
  8. 8.
    Lord C, Rutter M, DiLavore PC, Risi S, Gotham K, Bishop SL (2012) ADOS-2: autism diagnostic observation schedule, 2nd edn, (part I): modules 1-4. Western Psychological Services, TorranceGoogle Scholar
  9. 9.
    Allen RA, Robins DL, Decker SL (2008) Autism spectrum disorders: neurobiology and current assessment practices. Psychol Schools 45:905–917Google Scholar
  10. 10.
    Cook JR, Hausman EM, Jensen-Doss A, Hawley KM (2017) Assessment practices of child clinicians: results from a national survey. Assessment 24:210–221PubMedGoogle Scholar
  11. 11.
    Pugliese CE, Kenworthy L, Bal VH, Wallace GL, Yerys BE, Maddox BB et al (2015) Replication and comparison of the newly proposed ADOS-2, module 4 algorithm in ASD without ID: a multi-site study. J Autism Dev Disord 45:3919–3931PubMedPubMedCentralGoogle Scholar
  12. 12.
    Goldstein S, Naglieri JA (2013) Autism spectrum rating scales (ASRS) technical manual. Multi-Health Systems, New YorkGoogle Scholar
  13. 13.
    Goldstein S, Naglieri JA (2014) Autism spectrum rating scales (ASRS) technical report #2. Multi-Health Systems, New YorkGoogle Scholar
  14. 14.
    Zhou H, Zhang L, Luo X, Wu L, Zou X, Xia K et al (2017) Modifying the autism spectrum rating scale (6–18 years) to a chinese context: an exploratory factor analysis. Neurosci Bull 33:175–182PubMedPubMedCentralGoogle Scholar
  15. 15.
    Borenstein M, Hedges LV, Higgins JPT, Rothstein HR (2009) Introduction to meta-analysis. Wiley, New YorkGoogle Scholar
  16. 16.
    Kreiser NL, White SW (2015) ASD traits and co-occurring psychopathology: the moderating role of gender. J Autism Dev Disord 45:3932–3938PubMedGoogle Scholar
  17. 17.
    Tureck K, Matson JL, May A, Davis TE, Whiting SE (2013) Investigation of the rates of comorbid symptoms in children with ADHD compared to children with ASD. J Dev Phys Disabil 25:405–417Google Scholar
  18. 18.
    Youngstrom EA (2014) A primer on receiver operating characteristic analysis and diagnostic efficiency statistics for pediatric psychology: we are ready to ROC. J Pediatr Psychol 39:204–221PubMedGoogle Scholar
  19. 19.
    Swets JA, Dawes RM, Monahan J, Bjork RA (2000) Psychological science can improve diagnostic decisions. Psychol Sci Public Interest 1:1–26PubMedGoogle Scholar
  20. 20.
    Cheng D, Branscum AJ, Johnson WO (2012) Sample size calculations for ROC studies: parametric robustness and bayesian nonparametrics. Stat Med 31:131–142PubMedGoogle Scholar
  21. 21.
    Tabachnick BG, Fidell LS (2012) Using multivariate statistics, 6th edn. Pearson, LondonGoogle Scholar
  22. 22.
    Wechsler D (2014) Technical and interpretative manual of the wechsler intelligence scale for children, 5th edn. PsychCorp, San AntonioGoogle Scholar
  23. 23.
    Wechsler D (2011) Technical and interpretative manual of the wechsler abbreviated scale of intelligence, 2nd edn. PsychCorp, San AntonioGoogle Scholar
  24. 24.
    Kaufman AS, Kaufman NL (2004) Kaufman assessment battery for children: technical manual, 2nd edn. American Guidance Service, Circle PinesGoogle Scholar
  25. 25.
    Roid GH (2003) Stanford–binet intelligence scales, 5th edn. Riverside Publishing, ItascaGoogle Scholar
  26. 26.
    Wechsler D (2003) Technical and interpretative manual of the wechsler intelligence scale for children, 4th edn. PsychCorp, San AntonioGoogle Scholar
  27. 27.
    Wechsler D (2012) Technical and interpretative manual of the wechsler preschool and primary scale of intelligence, 2nd edn. PsychCorp, San AntonioGoogle Scholar
  28. 28.
    Wechsler D (2008) Technical and interpretive manual of the wechsler adult intelligence scale, 4th edn. PsychCorp, San AntonioGoogle Scholar
  29. 29.
    Bracken B, McCallum S (2016) Universal nonverbal intelligence test, 2nd edn. Western Psychological Publishers, TorranceGoogle Scholar
  30. 30.
    Reynolds CR, Kamphaus RW (2003) Handbook of psychological and educational assessment of children: intelligence, aptitude, and achievement, 2nd edn. Guilford Press, New YorkGoogle Scholar
  31. 31.
    Riccio CA, Sullivan JR, Cohen MJ (2010) Neuropsychological assessment and intervention for childhood and adolescent disorders. Wiley, New YorkGoogle Scholar
  32. 32.
    Langmann A, Becker J, Poustka L, Becker K, Kamp-Becker I (2017) Diagnostic utility of the autism diagnostic observation schedule in a clinical sample of adolescents and adults. Res Autism Spectr Disord 34:34–43.  https://doi.org/10.1016/j.rasd.2016.11.012 CrossRefGoogle Scholar
  33. 33.
    Zander E, Willfors C, Berggren S, Choque-Olsson N, Coco C, Elmund A et al (2016) The objectivity of the autism diagnostic observation schedule (ADOS) in naturalistic clinical settings. Eur Child Adolesc Psychiatry 25:769–780PubMedGoogle Scholar
  34. 34.
    Chojnicka I, Pisula E (2017) Adaptation and validation of the ADOS-2, polish version. Front Psychol 8:1916–1930PubMedGoogle Scholar
  35. 35.
    Medda JE, Cholemkery H, Freitag CM (2019) Sensitivity and specificity of the ADOS-2 algorithm in a large german sample. J Autism Dev Disord 49:750–761PubMedGoogle Scholar
  36. 36.
    Oosterling IJ, Roos S, Bildt AD, Lambregts-Rommelse NNJ, Jonge MD, Visser J et al (2010) Improved diagnostic validity of the ADOS revised algorithms: a replication study in an independent sample. J Autism Dev Disord 40:689–703PubMedGoogle Scholar
  37. 37.
    Fusar-Poli L, Brondino N, Rocchetti M, Panisi C, Provenzani U, Damiani S, Politi P (2017) Diagnosing ASD in adults without ID: accuracy of the ADOS-2 and the ADI-R. J Autism Dev Disord 47:3370–3379PubMedGoogle Scholar
  38. 38.
    Kamp-Becker I, Albertowski K, Becker J, Ghareman M, Langmann A, Mingebach T et al (2018) Diagnostic accuracy of the ADOS and ADOS-2 in clinical practice. Eur Child Adolesc Psychiatry 27:1193–1207PubMedGoogle Scholar
  39. 39.
    Louwerse I, Eussen M, Ende J, Nijs P, Gool A, Dekker LP et al (2015) ASD symptom severity in adolescence of individuals diagnosed with PDD-NOS in childhood: stability and the relation with psychiatric comorbidity and societal participation. J Autism Dev Disord 45:3908–3918PubMedGoogle Scholar
  40. 40.
    Camodeca A (2018) Utility of three N-item scales of the child behavior checklist 6–18 in autism diagnosis. Res Autism Spectrum Disord 51:75–85Google Scholar
  41. 41.
    Duchan JF (1998) Describing the unusual behavior of children with autism. J Commun Disord 31:93–112PubMedGoogle Scholar
  42. 42.
    Hedley D, Brewer N, Nevill R, Uljarević M, Butter E, Mulick JA (2016) The relationship between clinicians’ confidence and accuracy, and the influence of child characteristics, in the screening of autism spectrum disorder. J Autism Dev Disord 46:2340–2348PubMedGoogle Scholar
  43. 43.
    Dolan BK, Van Hecke AV, Carson AM, Karst JS, Stevens S, Schohl KA et al (2016) Brief report: assessment of intervention effects on in vivo peer interactions in adolescents with autism spectrum disorder (ASD). J Autism Dev Disord 46:2251–2259PubMedPubMedCentralGoogle Scholar
  44. 44.
    Green JL, Sciberras E, Anderson V, Efron D, Rinehart N (2016) Association between autism symptoms and functioning in children with ADHD. Arch Dis Child 101:922–928PubMedGoogle Scholar
  45. 45.
    Hinshaw SP, Carte ET, Sami N, Treuting JJ, Zupan BA (2002) Preadolescent girls with attention-deficit/hyperactivity disorder—II: neuropsychological performance in relation to subtypes and individual classification. J Consult Clin Psychol 70:1099–1111PubMedGoogle Scholar
  46. 46.
    Perugini EM, Harvey EA, Lovejoy DW, Sandstrom K, Webb AH (2000) The predictive power of combined neuropsychological measures for attention-deficit/hyperactivity disorder in children. Child Neuropsychol 6:101–114PubMedGoogle Scholar
  47. 47.
    Grzadzinski R, Dick C, Lord C, Bishop S (2016) Parent-reported and clinician-observed autism spectrum disorder (ASD) symptoms in children with attention deficit/hyperactivity disorder (ADHD): implications for practice under DSM-5. Mol Autism 7:7PubMedPubMedCentralGoogle Scholar
  48. 48.
    Berry K, Russell K, Frost K (2018) Restricted and repetitive behaviors in autism spectrum disorder: a review of associated features and presentation across clinical populations. Curr Dev Disord Rep 5:108–115Google Scholar
  49. 49.
    Green D, Chandler S, Charman T, Simonoff E, Baird G (2016) Brief report: DSM-5 sensory behaviours in children with and without an autism spectrum disorder. J Autism Dev Disord 46:3597–3606PubMedGoogle Scholar
  50. 50.
    Lam KSL, Bodfish JW, Piven J (2008) Evidence for three subtypes of repetitive behavior in autism that differ in familiality and association with other symptoms. J Child Psychol Psychiatry 49:1193–1200PubMedPubMedCentralGoogle Scholar
  51. 51.
    Esbensen AJ, Seltzer MM, Lam KSL, Bodfish JW (2009) Age-related differences in restricted repetitive behaviors in autism spectrum disorders. J Autism Dev Disord 39:57–66PubMedGoogle Scholar
  52. 52.
    Bishop SL, Richler J, Lord C (2006) Association between restricted and repetitive behaviors and nonverbal IQ in children with autism spectrum disorders. Child Neuropsychol 12:247–267PubMedGoogle Scholar
  53. 53.
    Harrop C, McConachie H, Emsley R, Leadbitter K, Green J, PACT Consortium, The PACT Consortium (2014) Restricted and repetitive behaviors in autism spectrum disorders and typical development: cross-sectional and longitudinal comparisons. J Autism Dev Disord 44:1207–1219Google Scholar
  54. 54.
    Magyar CI, Pandolfi V (2017) Utility of the CBCL DSM-oriented scales in assessing emotional disorders in youth with autism. Res Autism Spectrum Disord 37:11–20Google Scholar
  55. 55.
    Joshi G, Petty C, Wozniak J, Henin A, Fried R, Galdo M et al (2010) The heavy burden of psychiatric comorbidity in youth with autism spectrum disorders: a large comparative study of a psychiatrically referred population. J Autism Dev Disord 40:1361–1370PubMedGoogle Scholar
  56. 56.
    Rieske RD, Matson JL, Beighley JS, Cervantes PE, Goldin RL, Jang J (2015) Comorbid psychopathology rates in children diagnosed with autism spectrum disorders according to the DSM-IV-TR and the proposed DSM-5. Dev Neurorehabil 18:218PubMedGoogle Scholar
  57. 57.
    Bent CA, Barbaro J, Dissanayake C (2017) Change in autism diagnoses prior to and following the introduction of DSM-5. J Autism Dev Disord 47:163–171PubMedGoogle Scholar
  58. 58.
    Aldridge FJ, Gibbs VM, Schmidhofer K, Williams M (2012) Investigating the clinical usefulness of the social responsiveness scale (SRS) in a tertiary level, autism spectrum disorder specific assessment clinic. J Autism Dev Disord 42:294–300PubMedGoogle Scholar
  59. 59.
    Barnard-Brak L, Brewer A, Chesnut S, Richman D, Schaeffer AM (2016) The sensitivity and specificity of the social communication questionnaire for autism spectrum with respect to age. Autism Res 9:838–845PubMedGoogle Scholar
  60. 60.
    Mazefsky CA, Oswald DP (2006) The discriminative ability and diagnostic utility of the ADOS-G, ADI-R, and GARS for children in a clinical setting. Autism 10:533–549PubMedGoogle Scholar
  61. 61.
    Sizoo BB, Horwitz EH, Teunisse JPWM, Kan CC, Vissers CTWM, Forceville EJM et al (2015) Predictive validity of self-report questionnaires in the assessment of autism spectrum disorders in adults. Autism 19:842–849PubMedGoogle Scholar
  62. 62.
    Blader JC, Pliszka SR, Kafantaris V, Sauder C, Posner J, Foley CA et al (2016) Prevalence and treatment outcomes of persistent negative mood among children with attention-deficit/hyperactivity disorder and aggressive behavior. J Child Adolesc Psychopharmacol 26:164–173PubMedPubMedCentralGoogle Scholar
  63. 63.
    Esbjørn BH, Hoeyer M, Dyrborg J, Leth I, Kendall PC (2010) Prevalence and co-morbidity among anxiety disorders in a national cohort of psychiatrically referred children and adolescents. J Anxiety Disord 24:866–872PubMedGoogle Scholar
  64. 64.
    Vasiliadis H, Diallo FB, Rochette L, Smith M, Langille D, Lin E et al (2017) Temporal trends in the prevalence and incidence of diagnosed ADHD in children and young adults between 1999 and 2012 in Canada: a data linkage study. Can J Psychiatry 62:818–826PubMedPubMedCentralGoogle Scholar
  65. 65.
    Warner LA, Song NK, Pottick KJ (2014) Outpatient psychotropic medication use in the US: a comparison based on foster care status. J Child Fam Stud 23:652–665Google Scholar
  66. 66.
    Pandolfi V, Magyar CI, Dill CA (2012) An initial psychometric evaluation of the CBCL 6–18 in a sample of youth with autism spectrum disorders. Res Autism Spectrum Disord 6:96–108Google Scholar
  67. 67.
    Ooi YP, Rescorla L, Ang RP, Woo B, Fung DSS (2011) Identification of autism spectrum disorders using the child behavior checklist in Singapore. J Autism Dev Disord 41:1147–1156PubMedGoogle Scholar
  68. 68.
    So P, Greaves-Lord K, van der Ende J, Verhulst FC, Rescorla L, de Nijs PF (2012) Using the child behavior checklist and the teacher’s report form for identification of children with autism spectrum disorders. Autism 17:595–607PubMedGoogle Scholar
  69. 69.
    Achenbach TM, Rescorla LA (2001) Manual for the ASEBA school-age forms & profiles. University of Vermont, Research Center for Children, Youth, and Families, BurlingtonGoogle Scholar
  70. 70.
    Reynolds CR, Kamphaus RW (2015) Behavior assessment system for children, 3rd edn. Pearson, BloomingtonGoogle Scholar
  71. 71.
    Gilliam JE (2014) Gilliam Autism Rating Scale, 3rd edn. Pro-Ed, AustinGoogle Scholar
  72. 72.
    Myles B, Bock S, Simpson R (2001) Asperger Syndrome Diagnostic Scale manual. MHS, North TonawandaGoogle Scholar
  73. 73.
    Constantino JN, Gruber CP (2012) Social Responsiveness Scale, 2nd edn. Western Psychological Services, TorranceGoogle Scholar
  74. 74.
    Kotov R, Krueger RF, Watson D, Achenbach TM, Althoff RR, Bagby RM et al (2017) The hierarchical taxonomy of psychopathology (HiTOP): a dimensional alternative to traditional nosologies. J Abnorm Psychol 126:454–477PubMedGoogle Scholar
  75. 75.
    Ingersoll B, Hopwood CJ, Wainer A, Donnellan MB (2011) A comparison of three self-report measures of the Broad Autism Phenotype in a non-clinical sample. J Autism Dev Dis 12:1646–1657Google Scholar
  76. 76.
    Ruzich E, Allison C, Smith P, Watson P, Auyeung B, Ring H, Baron-Cohen S (2015) Measuring autistic traits in the general population: a systematic review of the autism-spectrum quotient (AQ) in a nonclinical population sample of 6,900 typical adult males and females. Mol Autism 6:2PubMedGoogle Scholar
  77. 77.
    Wainer AL, Ingersoll BR, Hopwood CJ (2011) The structure and nature of the broader autism phenotype in a non-clinical sample. J Psychopathol Behav 33:459–469Google Scholar
  78. 78.
    Altman DG, Royston P (2006) The cost of dichotomizing continuous variables. BMJ Br Med J 332:1080Google Scholar
  79. 79.
    Royston P, Altman DG, Sauerbrei W (2006) Dichotimizing continuous predictors in multiple regression: a bad idea. Stat Med 25:127–141PubMedGoogle Scholar
  80. 80.
    Barnwell-Ménard J, Cohen AA (2015) Effects of categorization method, regression type, and variable distribution on the inflation of Type-1 error rate when categorizing a confounding variable. Stat Med 34:936–949PubMedGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.The Pennsylvania State UniversityMonacaUSA

Personalised recommendations