Skip to main content

Advertisement

SpringerLink
Log in

A Comparison of Virtual Reality Classroom Continuous Performance Tests to Traditional Continuous Performance Tests in Delineating ADHD: a Meta-Analysis

  • Review
  • Published:
Neuropsychology Review Aims and scope Submit manuscript

Abstract

Computerized continuous performance tests (CPTs) are commonly used to characterize attention in attention deficit-hyperactivity disorder (ADHD). Virtual classroom CPTs, designed to enhance ecological validity, are increasingly being utilized. Lacking is a quantitative meta-analysis of clinical comparisons of attention performance in children with ADHD using virtual classroom CPTs. The objective of the present systematic PRISMA review was to address this empirical void and compare three-dimensional (3D) virtual classroom CPTs to traditional two-dimensional (2D) CPTs. The peer-reviewed literature on comparisons of virtual classroom performance between children with ADHD and typically developing children was explored in six databases (e.g., Medline). Published studies using a virtual classroom to compare attentional performance between children with ADHD and typically developing children were included. Given the high heterogeneity with modality comparisons (i.e., computerized CPTs vs. virtual classroom CPTs for ADHD), both main comparisons included only population comparisons (i.e., control vs. ADHD) using each CPT modality. Meta-analytic findings were generally consistent with previous meta-analyses of computerized CPTs regarding the commonly used omission, commission, and hit reaction time variables. Results suggest that the virtual classroom CPTs reliably differentiate attention performance in persons with ADHD. Ecological validity implications are discussed pertaining to subtle meta-analytic outcome differences compared to computerized 2D CPTs. Further, due to an inability to conduct moderator analyses, it remains unclear if modality differences are due to other factors. Suggestions for future research using the virtual classroom CPTs are provided.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  • Adams, R., Finn, P., Moes, E., Flannery, K., & Rizzo, A. S. (2009). Distractibility in attention/deficit/hyperactivity disorder (ADHD): The virtual reality classroom. Child Neuropsychology, 15(2), 120–135.

    Article  PubMed  Google Scholar 

  • Areces, D., Rodríguez, C., García, T., Cueli, M., & González-Castro, P. (2016). Efficacy of a continuous performance test based on virtual reality in the diagnosis of ADHD and its clinical presentations. Journal of Attention Disorders, 1–11. https://doi.org/10.1177/1087054716629711

    Article  PubMed  Google Scholar 

  • Areces, D., Rodríguez, C., García, T., Cueli, M., & González-Castro, P. (2018a). Efficacy of a continuous performance test based on virtual reality in the diagnosis of ADHD and its clinical presentations. Journal of Attention Disorders, 22(11), 1081–1091.

    Article  PubMed  Google Scholar 

  • Ballard, J. C. (2001). Assessing attention: Comparison of response-inhibition and traditional continuous performance tests. Journal of Clinical and Experimental Neuropsychology, 23(3), 331–350.

    Article  CAS  PubMed  Google Scholar 

  • Barkley, R. A. (1991). The ecological validity of laboratory and analogue assessment methods of ADHD symptoms. Journal of Abnormal Child Psychology, 19(2), 149–178.

    Article  CAS  PubMed  Google Scholar 

  • Barkley, R. A., & Murphy, K. R. (2011). The nature of executive function (EF) deficits in daily life activities in adults with ADHD and their relationship to performance on EF tests. Journal of Psychopathology and Behavioral Assessment, 33(2), 137–158.

    Article  Google Scholar 

  • Beale, I. L., Matthew, P. J., Oliver, S., & Corballis, M. C. (1987). Performance of disabled and normal readers on the continuous performance test. Journal of Abnormal Child Psychology, 15(2), 229–238.

    Article  CAS  PubMed  Google Scholar 

  • Berger, I., Slobodin, O., & Cassuto, H. (2017). Usefulness and validity of continuous performance tests in the diagnosis of attention-deficit hyperactivity disorder children. Archives of Clinical Neuropsychology, 32(1), 81–93.

    PubMed  Google Scholar 

  • Bioulac, S., Lallemand, S., Rizzo, A., Philip, P., Fabrigoule, C., & Bouvard, M. P. (2012). Impact of time on task on ADHD patient’s performances in a virtual classroom. European Journal of Paediatric Neurology, 16(5), 514–521.

    Article  PubMed  Google Scholar 

  • Bioulac, S., Micoulaud-Franchi, J. A., Maire, J., Bouvard, M. P., Rizzo, A. A., Sagaspe, P., & Philip, P. (2018). Virtual remediation versus methylphenidate to improve distractibility in children with ADHD: A controlled randomized clinical trial study. Journal of Attention Disorders, 1087054718759751.

  • Bohil, C. J., Alicea, B., & Biocca, F. A. (2011). Virtual reality in neuroscience research and therapy. Nature Reviews Neuroscience, 12(12), 752.

    Article  CAS  PubMed  Google Scholar 

  • Borenstein, M., Higgins, J. P., Hedges, L. V., & Rothstein, H. R. (2017). Basics of meta-analysis: I2 is not an absolute measure of heterogeneity. Research Synthesis Methods, 8(1), 5–18.

    Article  PubMed  Google Scholar 

  • Brosco, J. P., & Bona, A. (2016). Changes in academic demands and attention-deficit/hyperactivity disorder in young children. JAMA Pediatrics, 170(4), 396–397.

    Article  PubMed  Google Scholar 

  • Bruce, K. S. (2011). Academic underachievement: A neurodevelopmental perspective. Revista Médica Clínica Las Condes, 22(2), 211–217.

    Article  Google Scholar 

  • Chaytor, N., Schmitter-Edgecombe, M., & Burr, R. (2006). Improving the ecological validity of executive functioning assessment. Archives of Clinical Neuropsychology, 21(3), 217–227.

    Article  PubMed  Google Scholar 

  • Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). New York: Academic Press.

    Google Scholar 

  • Cohen, J. (1992). A power primer. Psychological Bulletin, 112, 155–159.

    Article  CAS  PubMed  Google Scholar 

  • Cohn, L. D., & Becker, B. J. (2003). How meta-analysis increases statistical power. Psychological Methods, 8, 243–253.

    Article  PubMed  Google Scholar 

  • Conners, C. K., Epstein, J. N., Angold, A., & Klaric, J. (2003). Continuous performance test performance in a normative epidemiological sample. Journal of Abnormal Child Psychology, 31(5), 555–562.

    Article  PubMed  Google Scholar 

  • Corkum, P. V., & Siegel, L. S. (1993). Is the continuous performance task a valuable research tool for use with children with attention-deficit-hyperactivity disorder? Journal of Child Psychology and Psychiatry, 34(7), 1217–1239.

    Article  CAS  PubMed  Google Scholar 

  • Díaz-Orueta, U. (2017). Advances in neuropsychological assessment of attention. In R. Kane & T. Parsons (Eds.), The role of Technology in Clinical Neuropsychology (pp. 103–142). Oxford: Oxford University Press.

    Google Scholar 

  • Díaz-Orueta, U., Garcia-López, C., Crespo-Eguílaz, N., Sánchez-Carpintero, R., Climent, G., & Narbona, J. (2014). AULA virtual reality test as an attention measure: Convergent validity with Conners’ continuous performance test. Child Neuropsychology, 20(3), 328–342.

    Article  PubMed  Google Scholar 

  • Duffield, T., Parsons, T. D., Karam, S., Otero, T., & Hall, T. (2018). Virtual reality as an assessment modality with pediatric ASD populations: A systematic review. Child Neuropsychology.

  • Fasmer, O. B., Mjeldheim, K., Førland, W., Hansen, A. L., Syrstad, V. E. G., Oedegaard, K. J., & Berle, J. Ø. (2016). Linear and non-linear analyses of Conner’s continuous performance test-II discriminate adult patients with attention deficit hyperactivity disorder from patients with mood and anxiety disorders. BMC Psychiatry, 16(1), 284.

    Article  PubMed  PubMed Central  Google Scholar 

  • Faul, F., Erdfelder, E., Buchner, A., & Lang, A.-G. (2009). Statistical power analyses using G*power 3.1: Tests for correlation and regression analyses. Behavior Research Methods, 41, 1149–1160.

    Article  PubMed  Google Scholar 

  • Franzen, M. D., & Wilhelm, K. L. (1996). Conceptual foundations of ecological validity in neuropsychology. In R. J. Sbordone & C. J. Long (Eds.), Ecological validity of neuropsychological testing (pp. 91–112). Delray Beach, FL: GR Press/St. Lucie Press.

    Google Scholar 

  • Gilboa, Y., Kerrouche, B., Longaud-Vales, A., Kieffer, V., Tiberghien, A., Aligon, D., … Paule Chevignard, M. (2015). Describing the attention profile of children and adolescents with acquired brain injury using the virtual classroom. Brain Injury, 29(13–14), 1691–1700.

    Article  PubMed  Google Scholar 

  • Gilboa, Y., Rosenblum, S., Fattal-Valevski, A., Toledano-Alhadef, H., & Josman, N. (2011). Using a virtual classroom environment to describe the attention deficits profile of children with neurofibromatosis type 1. Research in Developmental Disabilities, 32(6), 2608–2613.

    Article  PubMed  Google Scholar 

  • Gutiérrez Maldonado, J., Letosa Porta, A., RusCalafell, M., & Peñaloza Salazar, C. (2009). The assessment of attention deficit hyperactivity disorder in children using continous performance tasks in virtual environments. Anuario de Psicología, 40(2), 211–222.

    Google Scholar 

  • Hale, J. B., Chen, S. A., Tan, S. C., Poon, K., Fitzer, K. R., & Boyd, L. A. (2016). Reconciling individual differences with collective needs: The juxtaposition of sociopolitical and neuroscience perspectives on remediation and compensation of student skill deficits. Trends in Neuroscience and Education, 5(2), 41–51.

    Article  Google Scholar 

  • Hedges, L. V. (1981). Distribution theory for Glass's estimator of effect size and related estimators. Journal of Educational Statistics, 6(2), 107–128.

    Article  Google Scholar 

  • Hedges, L. V. (1984). Advances in statistical methods for meta-analysis. New Directions for Program Evaluation, 24, 25–42.

    Article  Google Scholar 

  • Hedges, L. V., & Olkin, I. (1985). Statistical methods for meta-analysis. New York: Academic Press.

    Google Scholar 

  • Higgins, J. P., Thompson, S. G., Deeks, J. J., & Altman, D. G. (2003). Measuring inconsistency in meta-analyses. BMJ: British Medical Journal, 327(7414), 557–560.

    Article  PubMed  Google Scholar 

  • Huang-Pollock, C. L., Karalunas, S. L., Tam, H., & Moore, A. N. (2012). Evaluating vigilance deficits in ADHD: A meta-analysis of CPT performance. Journal of Abnormal Psychology, 121(2), 360.

    Article  PubMed  PubMed Central  Google Scholar 

  • Huedo-Medina, T. B., Sanchez-Meca, J., Marin-Martinez, F., & Botella, J. (2006). Assessing heterogeneity in meta-analysis: Q statistic or I2 index? Psychological Methods, 11, 193–206.

    Article  PubMed  Google Scholar 

  • Hunter, J. E., & Schmidt, F. L. (2004). Methods of meta-analysis: Correcting error and bias in research findings (2nd ed.). Thousand Oaks, CA: Sage.

    Book  Google Scholar 

  • Iriarte, Y., Diaz-Orueta, U., Cueto, E., Irazustabarrena, P., Banterla, F., & Climent, G. (2016). AULA—Advanced virtual reality tool for the assessment of attention: Normative study in Spain. Journal of Attention Disorders, 20(6), 542–568.

    Article  PubMed  Google Scholar 

  • Kofler, M. J., Rapport, M. D., & Matt Alderson, R. (2008). Quantifying ADHD classroom inattentiveness, its moderators, and variability: a meta-analytic review. Journal of Child Psychology and Psychiatry, 49(1), 59–69.

    Article  PubMed  Google Scholar 

  • Kofler, M. J., Raiker, J. S., Sarver, D. E., Wells, E. L., & Soto, E. F. (2016). Is hyperactivity ubiquitous in ADHD or dependent on environmental demands? Evidence from meta-analysis. Clinical Psychology Review, 46, 12–24.

    Article  PubMed  PubMed Central  Google Scholar 

  • Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33(1), 159–174.

    Article  CAS  PubMed  Google Scholar 

  • Losier, B. J., McGrath, P. J., & Klein, R. M. (1996). Error patterns on the continuous performance test in non-medicated and medicated samples of children with and without ADHD: A meta-analytic review. Journal of Child Psychology and Psychiatry, 37(8), 971–987.

    Article  CAS  PubMed  Google Scholar 

  • Moreau, G., Guay, M., Achim, A., Rizzo, A., & Lageix, P. (2006). The virtual classroom: An ecological version of the continuous performance test–A pilot study. Annual Review of Cybertherapy and Telemedicine, 4, 59–66.

    Google Scholar 

  • Mühlberger, A., Jekel, K., Probst, T., Schecklmann, M., Conzelmann, A., Andreatta, M., ... & Romanos, M. (2016). The influence of methylphenidate on hyperactivity and attention deficits in children with ADHD: A virtual classroom test. Journal of Attention Disorders, 1–13 https://doi.org/10.1177/1087054716647480.

  • Neguț, A., Matu, S. A., Sava, F. A., & David, D. (2016). Virtual reality measures in neuropsychological assessment: A meta-analytic review. The Clinical Neuropsychologist, 30(2), 165–184.

    Article  PubMed  Google Scholar 

  • Nigg, J. T. (2005). Neuropsychologic theory and findings in attention-deficit/hyperactivity disorder: The state of the field and salient challenges for the coming decade. Biological Psychiatry, 57(11), 1424–1435.

    Article  PubMed  Google Scholar 

  • Nolin, P., Martin, C., & Bouchard, S. (2009). Assessment of inhibition deficits with the virtual classroom in children with traumatic brain injury: a pilot-study. Annual Review of Cybertherapy and Telemedicine, 7, 240–242.

    Google Scholar 

  • Nolin, P., Stipanicic, A., Henry, M., Joyal, C. C., & Allain, P. (2012). Virtual reality as a screening tool for sports concussion in adolescents. Brain Injury, 26(13–14), 1564–1573.

    Article  PubMed  Google Scholar 

  • Nolin, P., Stipanicic, A., Henry, M., Lachapelle, Y., Lussier-Desrochers, D., & Allain, P. (2016). ClinicaVR: Classroom-CPT: A virtual reality tool for assessing attention and inhibition in children and adolescents. Computers in Human Behavior, 59, 327–333.

    Article  Google Scholar 

  • Ogles, B. M., Lunnen, K. M., & Bonesteel, K. (2001). Clinical significance: History, application, and current practice. Clinical Psychology Review, 21(3), 421–446.

    Article  CAS  PubMed  Google Scholar 

  • Parsons, T. D., Bowerly, T., Buckwalter, J. G., & Rizzo, A. A. (2007). A controlled clinical comparison of attention performance in children with ADHD in a virtual reality classroom compared to standard neuropsychological methods. Child Neuropsychology, 13(4), 363–381.

    Article  PubMed  Google Scholar 

  • Parsons, T. D., Carlew, A. R., Magtoto, J., & Stonecipher, K. (2017). The potential of function-led virtual environments for ecologically valid measures of executive function in experimental and clinical neuropsychology. Neuropsychological Rehabilitation, 37(5), 777–807.

    Article  Google Scholar 

  • Parsons, T. D., & Rizzo, A. A. (2018). A virtual classroom for ecologically valid assessment of attention-deficit/hyperactivity disorder. In P. Sharkey (Ed.), Virtual reality Technologies for Health and Clinical Applications: Psychological and neurocognitive interventions (pp. ##- ##). Germany: Springer-Verlag.

    Google Scholar 

  • Pelham, W. E., Waschbusch, D. A., Hoza, B., Gnagy, E. M., Greiner, A. R., Sams, S. E., … Carter, R. L. (2011). Music and video as distractors for boys with ADHD in the classroom: Comparison with controls, individual differences, and medication effects. Journal of Abnormal Child Psychology, 39(8), 1085–1098.

    Article  PubMed  Google Scholar 

  • Pievsky, M. A., & McGrath, R. E. (2017). The neurocognitive profile of attention-deficit/hyperactivity disorder: A review of meta-analyses. Archives of Clinical Neuropsychology, 33(2), 143–157.

    Article  Google Scholar 

  • Pollak, Y., Shomaly, H. B., Weiss, P. L., Rizzo, A. A., & Gross-Tsur, V. (2010). Methylphenidate effect in children with ADHD can be measured by an ecologically valid continuous performance test embedded in virtual reality. CNS Spectrums, 15(2), 125–130.

    Article  PubMed  Google Scholar 

  • Pollak, Y., Weiss, P. L., Rizzo, A. A., Weizer, M., Shriki, L., Shalev, R. S., & Gross-Tsur, V. (2009). The utility of a continuous performance test embedded in virtual reality in measuring ADHD-related deficits. Journal of Developmental & Behavioral Pediatrics, 30(1), 2–6.

    Article  Google Scholar 

  • Rapport, M. D., Chung, K. M., Shore, G., Denney, C. B., & Isaacs, P. (2000). Upgrading the science and technology of assessment and diagnosis: Laboratory and clinic-based assessment of children with ADHD. Journal of Clinical Child Psychology, 29(4), 555–568.

    Article  CAS  PubMed  Google Scholar 

  • Review Manager (RevMan) [Computer program]. (2014) Version 5.3. Copenhagen: The Nordic Cochrane Centre, The Cochrane Collaboration.

  • Rizzo, A., Bowerly, T., Buckwalter, J., Klimchuk, D., Mitura, R., & Parsons, T. D. (2006). A virtual reality scenario for all seasons: The virtual classroom. CNS Spectrums, 11(1), 35–44.

    Article  PubMed  Google Scholar 

  • Rosenthal, R. (1979). The file drawer problem and tolerance for null results. Psychological Bulletin, 86(3), 638.

    Article  Google Scholar 

  • Shaddish, W. R., & Haddock, C. K. (1994). Combining estimates of effect size. In H. Cooper & L. V. Hedges (Eds.), The handbook of research synthesis (pp. 226–285). New York: Russell Sage Foundation.

    Google Scholar 

  • Silverstein, M. L., Weinstein, M., & Turnbull, A. (2004). Nonpatient CPT performance varying target frequency and interstimulus interval on five response measures. Archives of Clinical Neuropsychology, 19(8), 1017–1025.

    Article  PubMed  Google Scholar 

  • Sonuga-Barke, E. J., Sergeant, J. A., Nigg, J., & Willcutt, E. (2008). Executive dysfunction and delay aversion in attention deficit hyperactivity disorder: Nosologic and diagnostic implications. Child and Adolescent Psychiatric Clinics of North America, 17(2), 367–384.

    Article  PubMed  Google Scholar 

  • Spooner, D. M., & Pachana, N. A. (2006). Ecological validity in neuropsychological assessment: A case for greater consideration in research with neurologically intact populations. Archives of Clinical Neuropsychology, 21(4), 327–337.

    Article  PubMed  Google Scholar 

  • U.S. Department of Education, (2007-2008) National Center for Education Statistics, Schools and Staffing Survey (SASS), "Public School Data File," 2007–08.

  • Willcutt, E. G., Doyle, A. E., Nigg, J. T., Faraone, S. V., & Pennington, B. F. (2005). Validity of the executive function theory of attention-deficit/hyperactivity disorder: A meta-analytic review. Biological Psychiatry, 57(11), 1336–1346.

    Article  PubMed  Google Scholar 

  • Witmer, B. G., & Singer, M. J. (1998). Measuring presence in virtual environments: A presence questionnaire. Presence, 7(3), 225–240.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computational Neuropsychology and Simulation (CNS) Laboratory, University of North Texas, 3940 N. Elm, G150, Denton, TX, 76207, USA

    Thomas D. Parsons & Justin Asbee

  2. NetDragon Digital Research Center, University of North Texas, 3940 N. Elm, G150, Denton, TX, 76207, USA

    Thomas D. Parsons

  3. Learning Technologies, College of Information, University of North Texas, 3940 N. Elm, G150, Denton, TX, 76207, USA

    Thomas D. Parsons

  4. Oregon Health and Science University, Portland, OR, USA

    Tyler Duffield

Authors
  1. Thomas D. Parsons
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tyler Duffield
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Justin Asbee
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Thomas D. Parsons.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Parsons, T.D., Duffield, T. & Asbee, J. A Comparison of Virtual Reality Classroom Continuous Performance Tests to Traditional Continuous Performance Tests in Delineating ADHD: a Meta-Analysis. Neuropsychol Rev 29, 338–356 (2019). https://doi.org/10.1007/s11065-019-09407-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11065-019-09407-6

Keywords

Search

Navigation