Advertisement

Contribution of the Behavioral Observation of Students in Schools to ADHD Assessment

  • Yuanyuan JiangEmail author
  • Matthew Capriotti
  • Allyson Beaulieu
  • Mary Rooney
  • Keith McBurnett
  • Linda J. Pfiffner
Original Paper

Abstract

Classroom observations have long been considered a necessary component of Attention-Deficit/Hyperactivity Disorder (ADHD) evaluations. Yet, research evaluating the utility of observational ratings in ADHD assessment is limited. This study examined the contributions of the Behavioral Observation of Students in Schools (BOSS) to ADHD assessment by investigating associations between BOSS scores with ADHD symptom clusters and symptoms of frequently co-occurring externalizing and internalizing disorders. The utility of BOSS scores in predicting future ADHD-related impairment beyond standard parent and teacher ratings was also examined. One hundred and thirty-five children in grades 2–5 across 23 public schools participated in a randomized controlled trial examining a psychosocial treatment for ADHD. BOSS ratings were collected at baseline. Parent and teacher ratings of child symptoms and impairment were collected at baseline, post-treatment (3–4 months later), and follow-up (8–12 months later). Multiple regressions investigated the associations between the BOSS subscale of Task Engagement (TE) and parent and teacher ratings of ADHD symptoms and related disorders. Multi-level modeling accounted for school cluster effects. Results showed that lower BOSS TE was related to higher teacher-rated inattention but not hyperactivity/impulsivity symptoms. Lower BOSS TE was also associated with higher teacher-rated Oppositional Defiant Disorder (ODD) and depression symptoms, but not anxiety symptoms. Further, BOSS TE predicted higher future impairment beyond baseline teacher and parent ratings of ADHD symptoms and impairment, controlling for treatment. The BOSS appears sensitive to symptoms of child inattention, ODD, and depression, and may have utility in informing future impairment beyond standard informant ratings of ADHD.

Keywords

Attention-Deficit/Hyperactivity Disorder (ADHD) Behavioral Observation of Students in Schools (BOSS) Task Engagement Systematic direct observations 

Notes

Acknowledgements

The authors thank John DuBois and Melissa Plageman for their contributions to data entry and management, as well as many research assistants and study staff who served as coders.

Funding

This work was supported by the Institute of Education Sciences, U.S. Department of Education Award Grant R324A120358 (Principal Investigator: Linda J. Pfiffner).

Compliance with Ethical Standards

Conflict of interest

Dr. McBurnett has received research support from Akili Interactive and Shire Pharmaceuticals. Dr. Capriotti receives grant funding from the American Academy of Neurology, Tourette Association of America (Clinical Research Training Fellowship in Tourette Syndrome), and Center for California Studies at Sacramento State University. Dr. Capriotti has received honoraria from Sanofi Genzyme for presenting on mindfulness at events for individuals with Multiple Sclerosis and from the Tourette Association of America for leading workshops on behavioral treatment of Tourette Syndrome. Drs. Jiang, Beaulieu, Rooney, and Pfiffner report no potential conflicts of interest.

Ethical Approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards.

Informed Consent

Informed consent was obtained from all individual participants included in the study.

References

  1. Abikoff, H., & Gittelman, R. (1985). Hyperactive children treated with stimulants: Is cognitive training a useful adjunct? Archives of General Psychiatry, 42, 953–961.  https://doi.org/10.1001/archpsyc.1985.01790330033004.CrossRefGoogle Scholar
  2. Abikoff, H., Gittelman, R., & Klein, D. F. (1980). Classroom observation code for hyperactive children: A replication of validity. Journal of Consulting and Clinical Psychology, 48, 555.  https://doi.org/10.1037//0022-006X.48.5.555.CrossRefGoogle Scholar
  3. Abikoff, H., Gittelman-Klein, R., & Klein, D. F. (1977). Validation of a classroom observation code for hyperactive children. Journal of Consulting and Clinical Psychology, 45, 772.  https://doi.org/10.1037/0022-006X.45.5.772.CrossRefGoogle Scholar
  4. Abikoff, H. B., Jensen, P. S., Arnold, L. E., Hoza, B., Hechtman, L., Pollack, S., et al. (2002). Observed classroom behavior of children with ADHD: Relationship to gender and comorbidity. Journal of Abnormal Child Psychology, 30, 349–359.  https://doi.org/10.1023/A:1015713807297.CrossRefGoogle Scholar
  5. Achenbach, T. M. (2009). The Achenbach system of empirically based assessment (ASEBA): Development, findings, theory, and applications. Burlington, VT: University of Vermont Research Center for Children, Youth, & Families.Google Scholar
  6. American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). Washington, DC: America Psychiatric Association.CrossRefGoogle Scholar
  7. Atkins, M. S., & Pelham, W. E. (1991). School-based assessment of attention deficit-hyperactivity disorder. Journal of Learning Disabilities, 24, 197–204.  https://doi.org/10.1177/002221949102400403.CrossRefGoogle Scholar
  8. Austerman, J. (2015). ADHD and behavioral disorders: Assessment, management, and an update from DSM-5. Cleveland Clinic Journal of Medicine, 82, 2–7.  https://doi.org/10.3949/ccjm.82.s1.01.CrossRefGoogle Scholar
  9. Barkley, R. A. (1997). Behavioral inhibition, sustained attention, and executive functions: Constructing a unifying theory of ADHD. Psychological Bulletin, 121, 65.CrossRefGoogle Scholar
  10. Benjamini, Y., & Hochberg, Y. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society, 57, 289–300.Google Scholar
  11. Biederman, J., Monuteaux, M. C., Doyle, A. E., Seidman, L. J., Wilens, T. E., Ferrero, F., et al. (2004). Impact of executive function deficits and attention-deficit/hyperactivity disorder (ADHD) on academic outcomes in children. Journal of Consulting and Clinical Psychology, 72, 757.  https://doi.org/10.1037/0022-006X.72.5.757.CrossRefGoogle Scholar
  12. Conners, C. K. (2008). Conners 3rd edition manual. New York, NY: Multi-health Systems Inc.Google Scholar
  13. Daley, D., & Birchwood, J. (2010). ADHD and academic performance: Why does ADHD impact on academic performance and what can be done to support ADHD children in the classroom? Child: Care, Health and Development, 36, 455–464.  https://doi.org/10.1111/j.1365-2214.2009.01046.x.Google Scholar
  14. Dodge, K. A., & Crick, N. R. (1990). Social information-processing bases of aggressive behavior in children. Personality and Social Psychology Bulletin, 16, 8–22.  https://doi.org/10.1177/0146167290161002.CrossRefGoogle Scholar
  15. DuPaul, G. J., & Stoner, G. (2014). ADHD in the schools: Assessment and intervention strategies. New York: Guilford Publications.Google Scholar
  16. DuPaul, G. J., Volpe, R. J., Jitendra, A. K., Lutz, J. G., Lorah, K. S., & Gruber, R. (2004). Elementary school students with AD/HD: Predictors of academic achievement. Journal of School Psychology, 42, 285–301.  https://doi.org/10.1016/j.jsp.2004.05.001.CrossRefGoogle Scholar
  17. Fabiano, G. A., Pelham, W. E., Gnagy, E. M., Waschbusch, D. A., Lahey, B. B., Chronis, A. M., et al. (2006). A practical measure of impairment: Psychometric properties of the impairment rating scale in samples of children with attention deficit hyperactivity disorder and two school-based samples. Journal of Clinical Child and Adolescent Psychology, 35, 369–385.  https://doi.org/10.1207/s15374424jccp3503_3.CrossRefGoogle Scholar
  18. Gadow, K. D., Nolan, E. E., & Sverd, J. (1992). Methylphenidate in hyperactive boys with comorbid tic disorder: II. Short-term behavioral effects in school settings. Journal of the American Academy of Child and Adolescent Psychiatry, 31, 462–471.  https://doi.org/10.1097/00004583-199205000-00012.CrossRefGoogle Scholar
  19. Gadow, K. D., & Sprafkin, J. N. (2002). Child symptom inventory 4: Screening and norms manual. Stony Brook, NY: Checkmate Plus.Google Scholar
  20. Gadow, K. D., Sprafkin, J., & Nolan, E. E. (1996). ADHD school observation code. Stony Brook, NY: Checkmate Plus.Google Scholar
  21. Greenwood, C. R., Horton, B. T., & Utley, C. A. (2002). Academic engagement: Current perspectives on research and practice. School Psychology Review, 31, 328.Google Scholar
  22. Handler, M. W., & DuPaul, G. J. (2005). Assessment of ADHD: Differences across psychology specialty areas. Journal of Attention Disorders, 9, 402–412.  https://doi.org/10.1177/1087054705278762.CrossRefGoogle Scholar
  23. Hintze, J. M. (2005). Psychometrics of direct observation. School Psychology Review, 34, 507.Google Scholar
  24. Junod, R. E. V., DuPaul, G. J., Jitendra, A. K., Volpe, R. J., & Cleary, K. S. (2006). Classroom observations of students with and without ADHD: Differences across types of engagement. Journal of School Psychology, 44, 87–104.  https://doi.org/10.1016/j.jsp.2005.12.004.CrossRefGoogle Scholar
  25. Kofler, M. J., Rapport, M. D., & Alderson, R. M. (2008). Quantifying ADHD classroom inattentiveness, its moderators, and variability: A meta-analytic review. Journal of Child Psychology and Psychiatry, 49, 59–69.  https://doi.org/10.1111/j.1469-7610.2007.01809.x.CrossRefGoogle Scholar
  26. Lahey, B. B., Pelham, W. E., Loney, J., Kipp, H., Ehrhardt, A., Lee, S. S., et al. (2004). Three-year predictive validity of DSM-IV attention-deficit hyperactivity disorder in children diagnosed at 4–6 years of age. American Journal of Psychiatry, 161, 2014–2020.  https://doi.org/10.1176/appi.ajp.161.11.2014.CrossRefGoogle Scholar
  27. Lee, S. S., Lahey, B. B., Owens, E. B., & Hinshaw, S. P. (2008). Few preschool boys and girls with ADHD are well-adjusted during adolescence. Journal of Abnormal Child Psychology, 36, 373–383.  https://doi.org/10.1007/s10802-007-9184-6.CrossRefGoogle Scholar
  28. Loe, I. M., & Feldman, H. M. (2007). Academic and educational outcomes of children with ADHD. Journal of Pediatric Psychology, 32, 643–654.  https://doi.org/10.1093/jpepsy/jsl054.CrossRefGoogle Scholar
  29. Luk, S. L. (1985). Direct observation studies of hyperactive behaviors. Journal of the American Academy of Child Psychiatry, 24, 338–344.  https://doi.org/10.1016/S0002-7138(09)61096-7.CrossRefGoogle Scholar
  30. McConaughy, S. H., Harder, V. S., Antshel, K. M., Gordon, M., Eiraldi, R., & Dumenci, L. (2010). Incremental validity of test session and classroom observations in a multimethod assessment of attention deficit/hyperactivity disorder. Journal of Clinical Child and Adolescent Psychology, 39, 650–666.  https://doi.org/10.1080/15374416.2010.501287.CrossRefGoogle Scholar
  31. MTA Cooperative Group. (1999). 14-month randomized clinical trial of treatment strategies for attention deficit hyperactivity disorder. Archives of General Psychiatry, 56, 1073–1086.  https://doi.org/10.1001/archpsyc.56.12.1073.CrossRefGoogle Scholar
  32. Nock, M. K., & Kurtz, S. M. (2005). Direct behavioral observation in school settings: Bringing science to practice. Cognitive and Behavioral Practice, 12, 359–370.  https://doi.org/10.1016/S1077-7229(05)80058-6.CrossRefGoogle Scholar
  33. Orban, S. A., Rapport, M. D., Friedman, L. M., Eckrich, S. J., & Kofler, M. J. (2018). Inattentive behavior in boys with ADHD during classroom instruction: The mediating role of working memory processes. Journal of Abnormal Child Psychology, 46, 713–727.  https://doi.org/10.1007/s10802-017-0338-x.CrossRefGoogle Scholar
  34. Ota, K. R., & DuPaul, G. J. (2002). Task engagement and mathematics performance in children with attention-deficit hyperactivity disorder: Effects of supplemental computer instruction. School Psychology Quarterly, 17, 242.  https://doi.org/10.1521/scpq.17.3.242.20881.CrossRefGoogle Scholar
  35. Owens, M., Stevenson, J., Hadwin, J. A., & Norgate, R. (2012). Anxiety and depression in academic performance: An exploration of the mediating factors of worry and working memory. School Psychology International, 33, 433–449.  https://doi.org/10.1177/0143034311427433.CrossRefGoogle Scholar
  36. Parker, A., & Corkum, P. (2016). ADHD diagnosis: As simple as administering a questionnaire or a complex diagnostic process? Journal of Attention Disorders, 20, 478–486.  https://doi.org/10.1177/1087054713495736.CrossRefGoogle Scholar
  37. Pekrun, R., Elliot, A. J., & Maier, M. A. (2009). Achievement goals and achievement emotions: Testing a model of their joint relations with academic performance. Journal of Educational Psychology, 101, 115.  https://doi.org/10.1037/a0013383.CrossRefGoogle Scholar
  38. Pekrun, R., & Linnenbrink-Garcia, L. (2012). Academic emotions and student engagement. In S. L. Christenson, A. L. Reschly, & C. Wylie (Eds.), Handbook of research on student engagement (pp. 259–282). New York, NY: Springer.CrossRefGoogle Scholar
  39. Pelham, W. E., Jr., Fabiano, G., & Massetti, G. (2005). Evidence-based assessment of attention deficit hyperactivity disorder in children and adolescents. Journal of Clinical Child & Adolescent Psychology, 34, 449–476.  https://doi.org/10.1207/s15374424jccp3403_5.CrossRefGoogle Scholar
  40. Pfiffner, L. J., Rooney, M., Haack, L., Villodas, M., Delucchi, K., & McBurnett, K. (2016). A randomized controlled trial of a school-implemented school–home intervention for attention-deficit/hyperactivity disorder symptoms and impairment. Journal of the American Academy of Child and Adolescent Psychiatry, 55, 762–770.  https://doi.org/10.1016/j.jaac.2016.05.023.CrossRefGoogle Scholar
  41. Platzman, K. A., Stoy, M. R., Brown, R. T., Coles, C. D., Smith, I. E., & Falek, A. (1992). Review of observational methods in attention deficit hyperactivity disorder (ADHD): Implications for diagnosis. School Psychology Quarterly, 7, 155.CrossRefGoogle Scholar
  42. Pliszka, S. R. (1992). Comorbidity of attention-deficit hyperactivity disorder and overanxious disorder. Journal of the American Academy of Child and Adolescent Psychiatry, 28, 882–887.  https://doi.org/10.1097/00004583-199203000-00003.CrossRefGoogle Scholar
  43. Quay, H. C. (1997). Inhibition and attention deficit hyperactivity disorder. Journal of Abnormal Child Psychology, 25, 7–13.  https://doi.org/10.1023/A:1025799122529.CrossRefGoogle Scholar
  44. Reynolds, C. R., & Kamphaus, R. W. (2004). BASC-2: Behavior assessment system for children (2nd ed.): Manual. Circle Pines, MN: AGS Printing.Google Scholar
  45. Rogers, M., Hwang, H., Toplak, M., Weiss, M., & Tannock, R. (2011). Inattention, working memory, and academic achievement in adolescents referred for attention deficit/hyperactivity disorder (ADHD). Child Neuropsychology, 17, 444–458.  https://doi.org/10.1080/09297049.2010.544648.CrossRefGoogle Scholar
  46. Saudargas, R. A. (1997). State-event classroom observation system (SECOS): Observation manual. Knoxville, TN: University of Tennessee.Google Scholar
  47. Scahill, L., Schwab-Stone, M., Merikangas, K. R., Leckman, J. F., Zhang, H., & Kasl, S. (1999). Psychosocial and clinical correlates of ADHD in a community sample of school-age children. Journal of the American Academy of Child and Adolescent Psychiatry, 38, 976–984.  https://doi.org/10.1097/00004583-199908000-00013.CrossRefGoogle Scholar
  48. Schatz, D. B., & Rostain, A. L. (2006). ADHD with comorbid anxiety: A review of the current literature. Journal of Attention Disorders, 10, 141–149.  https://doi.org/10.1177/1087054706286698.CrossRefGoogle Scholar
  49. Shapiro, E. S. (2004). Direct observation: Manual for the behavioral observation of students in schools (BOSS). New York, NY: Guilford Press.Google Scholar
  50. Shapiro, E. S. (2014). Behavioral assessment in school psychology. New York, NY: Psychology Press.CrossRefGoogle Scholar
  51. Shapiro, E. S., & Heick, P. F. (2004). School psychologist assessment practices in the evaluation of students referred for social/behavioral/emotional problems. Psychology in the Schools, 41, 551–561.  https://doi.org/10.1002/pits.10176.CrossRefGoogle Scholar
  52. Skansgaard, E. P., & Burns, G. L. (1998). Comparison of DSM-IV ADHD combined and predominantly inattention types: Correspondence between teacher ratings and direct observations of inattentive, hyperactivity/impulsivity, slow cognitive tempo, oppositional defiant, and overt conduct disorder symptoms. Child & Family Behavior Therapy, 20, 1–14.  https://doi.org/10.1300/J019v20n01_01.CrossRefGoogle Scholar
  53. Steiner, H., & Remsing, L. (2007). Practice parameter for the assessment and treatment of children and adolescents with oppositional defiant disorder. Journal of the American Academy of Child and Adolescent Psychiatry, 46, 126–141.  https://doi.org/10.1097/01.chi.0000246060.62706.af.CrossRefGoogle Scholar
  54. Thorell, L. B. (2007). Do delay aversion and executive function deficits make distinct contributions to the functional impact of ADHD symptoms? A study of early academic skill deficits. Journal of Child Psychology and Psychiatry, 48, 1061–1070.  https://doi.org/10.1111/j.1469-7610.2007.01777.x.CrossRefGoogle Scholar
  55. Volpe, R. J., DiPerna, J. C., Hintze, J. M., & Shapiro, E. S. (2005). Observing students in classroom settings: A review of seven coding schemes. School Psychology Review, 34, 454.Google Scholar
  56. Walker, H. M., Severson, H. H., & Feil, E. G. (2014). Systematic screening for behavior disorders (SSBD): Technical manual. Eugene, OR: Pacific Northwest Publishing.Google Scholar
  57. Wechsler, D. (2011). Wechsler abbreviated scale of intelligence. San Antonio, TX: Psychological Corporation.Google Scholar
  58. Willcutt, E. G., Hartung, C. M., Lahey, B. B., Loney, J., & Pelham, W. E. (1999). Utility of behavior ratings by examiners during assessments of preschool children with attention-deficit/hyperactivity disorder. Journal of Abnormal Child Psychology, 27, 463–472.  https://doi.org/10.1023/A:1021984126774.CrossRefGoogle Scholar
  59. Wilson, M. S., & Reschly, D. J. (1996). Assessment in school psychology training and practice. School Psychology Review, 25, 923.Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  • Yuanyuan Jiang
    • 1
    • 2
    Email author
  • Matthew Capriotti
    • 1
    • 3
  • Allyson Beaulieu
    • 1
  • Mary Rooney
    • 1
  • Keith McBurnett
    • 1
  • Linda J. Pfiffner
    • 1
  1. 1.Department of PsychiatryUniversity of California, San Francisco401 Parnassus Ave., San FranciscoUSA
  2. 2.Department of Educational PsychologyUniversity of AlbertaEdmontonCanada
  3. 3.Department of PsychologySan José UniversitySan JoseUSA

Personalised recommendations