Advertisement

Development and Initial Validation of a Reading-Specific Performance Validity Test: the College Assessment of Reading Effort (CARE)

  • Robert WeisEmail author
  • Sarah J. Droder
Article

Abstract

Although performance validity tests (PVTs) are routinely used in neuropsychological assessment to detect malingering or low-effort test-taking, they are seldom administered to college students seeking academic accommodations and other benefits for reading disabilities. Previous research indicates that between 9.5 and 31% of students seeking learning disability evaluations at university-based clinics provide noncredible test scores indicative of symptom exaggeration or low effort. We developed a brief reading–specific PVT designed for college students participating in reading disability testing: the College Assessment of Reading Effort (CARE). We administered the CARE and standardized reading tests to three groups of students: honest controls, students with documented reading disabilities, and students coached to simulate reading disabilities. Simulators displayed normative deficits on standardized reading measures, similar to the scores earned by students with actual reading disabilities and lower than the scores earned by honest controls. In contrast, CARE scores differentiated simulators from honest examinees with and without disabilities. ROC curve analysis showed that CARE composite scores could be used diagnostically to detect low effort with sensitivity, specificity, and predictive power ≥ 0.90. The CARE offers a time- and cost-effective way to assess performance validity during reading disability testing for postsecondary students.

Keywords

Assessment College students Malingering Reading disabilities Symptom validity Performance validity 

Notes

Compliance with Ethical Standards

Conflict of Interest

The authors declare that they have no conflict of interest.

Informed Consent

All procedures followed were in accordance with the ethical standards of the responsible committee on human experimentation (national and institutional). Approval was obtained from the ethics board at Denison University.

Animal Rights

No animal studies were carried out by the authors for this article.

References

  1. Alfano, K., & Boone, K. B. (2007). The use of effort tests in the context of actual versus feigned attention-deficit/hyperactivity disorder and learning disability. In K. B. Boone (Ed.), Assessment of feigned cognitive impairment (pp. 366–383). New York: Guilford.Google Scholar
  2. American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). Washington, DC: American Psychiatric Publishing.Google Scholar
  3. Berger, C. L., & Lewandowski, L. (2013). The effect of a word processor as an accommodation for students with learning disabilities. Journal of Writing Research, 4, 261–280.Google Scholar
  4. Boone, K. B. (2017). Self-deception in somatoform conditions. In K. B. Boone (Ed.), Neuropsychological evaluation of somatoform and other functional somatic conditions (pp. 3–42). New York: Routledge.Google Scholar
  5. Bouck, E. C. (2009). Calculating the value of graphing calculators for seventh-grade students with and without disabilities. Remedial and Special Education, 30, 207–215.Google Scholar
  6. Bouck, E. C., & Bouck, M. K. (2008). Does it add up? Calculators as accommodations for sixth grade students with disabilities. Journal of Special Education Technology, 23, 17–32.Google Scholar
  7. Bouck, E. C., & Yadav, A. (2008). Assessing calculators as assessment accommodations for students with disabilities. Assistive Technology Outcomes and Benefits, 5, 19–28.Google Scholar
  8. Brueggemann, A. E., Kamphaus, R. W., & Dombrowski, S. C. (2008). An impairment model of learning disability diagnosis. Professional Psychology: Research and Practice, 39, 424–430.Google Scholar
  9. Bush, S. S., Heilbronner, R. L., & Ruff, R. M. (2014). Psychological assessment of symptom and performance validity, response bias, and malingering: Official position of the Association for Scientific Advancement in Psychological Injury and Law. Psychological Injury and Law, 7, 197–205.Google Scholar
  10. Chafetz, M. D., Williams, M. A., Ben-Porath, Y. S., Bianchini, K. J., Boone, K. B., Kirkwood, M. W., Larrabee, G. J., & Ord, J. S. (2015). Official position of the American Academy of Clinical Neuropsychology Social Security Administration policy on validity testing. The Clinical Neuropsychologist, 29, 723–740.Google Scholar
  11. Dombrowski, S. C., Kamphaus, R. W., & Reynolds, C. R. (2004). After the demise of the discrepancy: Proposed learning disabilities diagnostic criteria. Professional Psychology: Research and Practice, 35, 364–372.Google Scholar
  12. DuPaul, G. J., Pinho, T. D., Pollack, B. L., Gormley, M. J., & Laracy, S. D. (2017). First-year college students with ADHD and/or LD. Journal of Learning Disabilities, 50, 238–251.Google Scholar
  13. Erdodi, L., & Roth, R. (2017). Low scores on BDAE Complex Ideational Material are associated with invalid performance in adults without aphasia. Applied Neuropsychology, 24, 264–274.Google Scholar
  14. Flanagan, D. P., Fiorello, C. A., & Ortiz, S. O. (2010). Enhancing practice through application of Cattell–Horn–Carroll theory and research: A third method approach to specific learning disability identification. Psychology in the Schools, 47, 739–760.Google Scholar
  15. Flanagan, D. P., Ortiz, S. O., & Alfonso, V. C. (2013). Essentials of cross-battery assessment. New York: Wiley.Google Scholar
  16. Frazier, T. W., Frazier, A. R., Busch, R. M., Kerwood, M. A., & Demaree, H. (2008). Detection of simulated ADHD and reading disorder using symptom validity measures. Archives of Clinical Neuropsychology, 23, 501–509.Google Scholar
  17. Green, P. (2005). Green’s Word Memory Test user’s manual. Edmonton, Alberta, Canada: Author.Google Scholar
  18. Green, P., & Flaro, L. (2003). Word Memory Test performance in children. Child Neuropsychology, 9, 189–207.Google Scholar
  19. Gregg, N. (2007). Underserved and unprepared: Postsecondary learning disabilities. Learning Disabilities Research & Practice, 22, 219–228.Google Scholar
  20. Gregg, N. (2012). Increasing access to learning for the adult basic education learner with learning disabilities. Journal of Learning Disabilities, 45, 47–63.Google Scholar
  21. Greher, M. R., & Wodushek, T. R. (2017). Performance validity testing in neuropsychology. Journal of Psychiatric Practice, 23, 134–140.Google Scholar
  22. Harrison, A. G., & Edwards, M. J. (2010). Symptom exaggeration in post-secondary students. Applied Neuropsychology, 17, 135–143.Google Scholar
  23. Harrison, A. G., Edwards, M. J., & Parker, K. C. (2008). Identifying students feigning dyslexia. Dyslexia, 14, 228–246.Google Scholar
  24. Harrison, A. G., Edwards, M. J., Armstrong, I., & Parker, K. C. (2010). An investigation of methods to detect feigned reading disabilities. Archives of Clinical Neuropsychology, 25, 89–98.Google Scholar
  25. Harrison, A. G., Green, P., & Flaro, L. (2012). The importance of symptom validity testing in adolescents and young adults undergoing assessments for learning or attention difficulties. Canadian Journal of School Psychology, 27, 98–113.Google Scholar
  26. Heilbronner, R. L., Sweet, J. J., Morgan, J. E., Larrabee, G. J., Millis, S. R., & Participants, C. (2009). American Academy of Clinical Neuropsychology Consensus Conference Statement on the neuropsychological assessment of effort, response bias, and malingering. The Clinical Neuropsychologist, 23, 1093–1129.Google Scholar
  27. Hurtubise, J. L., Scavone, A., Sagar, S., & Erdodi, L. A. (2017). Psychometric markers of genuine and feigned neurodevelopmental disorders in the context of applying for academic accommodations. Psychological Injury and Law, 10, 121–137.Google Scholar
  28. Internal Revenue Service. (2017a). Publication 502: Medical and dental expenses. Washington, DC: US Treasury Department.Google Scholar
  29. Internal Revenue Service. (2017b). Publication 907: Tax highlights for persons with disabilities. Washington, DC: US Treasury Department.Google Scholar
  30. Joy, J. A., Julius, R. J., Akter, R., & Baron, D. A. (2010). Assessment of ADHD documentation from candidates requesting Americans with Disabilities Act (ADA) accommodations for the National Board of Osteopathic Medical Examiners COMLEX exam. Journal of Attention Disorders, 14, 104–108.Google Scholar
  31. Kraemer, H. C. (1992). Evaluating medical tests: Objective and quantitative guidelines. Newbury Park, CA: Sage.Google Scholar
  32. Larochette, A. C., & Harrison, A. G. (2012). Word Memory Test performance in Canadian adolescents with learning disabilities. Applied Neuropsychology, 1, 38–47.Google Scholar
  33. Lewandowski, L. J., Lovett, B. J., Codding, R. S., & Gordon, M. (2008a). Symptoms of ADHD and academic concerns in college students with and without ADHD diagnoses. Journal of Attention Disorders, 12, 156–161.Google Scholar
  34. Lewandowski, L. J., Lovett, B. J., & Rogers, C. L. (2008b). Extended time as a testing accommodation for students with reading disabilities: Does a rising tide lift all ships? Journal of Psychoeducational Assessment, 26, 315–324.Google Scholar
  35. Lewandowski, L., Cohen, J., & Lovett, B. J. (2013). Effects of extended time allotments on reading comprehension performance of college students with and without learning disabilities. Journal of Psychoeducational Assessment, 31, 326–336.Google Scholar
  36. Lewandowski, L., Lambert, T. L., Lovett, B. J., Panahon, C. J., & Sytsma, M. R. (2014). College students’ preferences for test accommodations. Canadian Journal of School Psychology, 29, 116–126.Google Scholar
  37. Lewandowski, L. J., Berger, C., Lovett, B. J., & Gordon, M. (2016a). Test-taking skills of high school students with and without learning disabilities. Journal of Psychoeducational Assessment, 34, 566–576.Google Scholar
  38. Lewandowski, L. J., Wood, W., & Miller, L. A. (2016b). Technological applications for individuals with learning disabilities and ADHD. In J. K. Luiselli & A. J. Fischer (Eds.), Computer-assisted and web-based innovations in psychology, special education, and health (pp. 61–93). San Diego, CA: Academic Press.Google Scholar
  39. Lindstrom, W., & Lindstrom, J. H. (2017). College admissions tests and LD and ADHD documentation guidelines. Journal of Disability Policy Studies, 28, 32–42.Google Scholar
  40. Lindstrom, W., Coleman, C., Thomassin, K., Southall, C. M., & Lindstrom, J. H. (2011). Simulated dyslexia in postsecondary students. The Clinical Neuropsychologist, 25, 302–322.Google Scholar
  41. Lovett, B. J. (2010). Extended time testing accommodations for students with disabilities. Review of Educational Research, 80, 611–638.Google Scholar
  42. Lovett, B. J. (2014). Testing accommodations under the Amended Americans with disabilities act. Journal of Disability Policy Studies, 25, 81–90.Google Scholar
  43. Lovett, B. J., & Lewandowski, L. J. (2015). Testing accommodations for students with disabilities. Washington, DC: American Psychological Association.Google Scholar
  44. Lovett, B. J., & Nelson, J. M. (2017). Test anxiety and the Americans with Disabilities Act. Journal of Disability Policy Studies, 28, 99–108.Google Scholar
  45. Lovett, B. J., & Sparks, R. L. (2010). Exploring the diagnosis of “Gifted/LD”. Journal of Psychoeducational Assessment, 28, 91–101.Google Scholar
  46. Lovett, B. J., Nelson, J. M., & Lindstrom, W. (2015). Documenting hidden disabilities in higher education. Journal of Disability Policy Studies, 26, 44–53.Google Scholar
  47. Mather, N., & Wendling, B. J. (2014). Examiner’s manual. Rolling Meadows, IL: Riverside: Woodcock-Johnson IV.Google Scholar
  48. McGregor, K. K., Langenfeld, N., Van Horne, S., Oleson, J., Anson, M., & Jacobson, W. (2016). The university experiences of students with learning disabilities. Learning Disabilities Research and Practice, 31, 90–102.Google Scholar
  49. Meehl, P. E., & Rosen, A. (1955). Antecedent probability and the efficiency of psychometric signs, patterns, or cutting scores. Psychological Bulletin, 52, 194–216.Google Scholar
  50. Nelson, J.M., Whipple, B., Lindstrom, W., & Foels, P.A. (2014). How is ADHD assessed and documented? Journal of Attention Disorders. Online First. https://journals.sagepub.com/doi/10.1177/1087054714561860.
  51. Newman, L., Wagner, M., Huang, T., Shaver, D., Knokey, A., Yu, J., et al. (2011). Secondary school programs and performance of students with disabilities. Washington, DC: National Center for Special Education Research.Google Scholar
  52. Newman, L., Wagner, M., Knokey, A., Marder, C., Nagle, K., Shaver, D.…., & Schwarting, M. (2012). The post-high school outcomes of young adults with disabilities up to 8 years after high school. Menlo Park, CA: SRI.Google Scholar
  53. Odland, A. P., Lammy, A. B., Martin, P. K., Grote, C. L., & Mittenberg, W. (2015). Advanced administration and interpretation of multiple validity tests. Psychological Injury and Law, 8, 46–63.Google Scholar
  54. Osmon, D. C., Plambeck, E., Klein, L., & Mano, Q. (2006). The word reading test of effort in adult learning disability. The Clinical Neuropsychologist, 20, 315–324.Google Scholar
  55. Peterson’s Guide. (2017). Colleges for students with learning disabilities or ADHD. Lawrenceville, NJ: Author.Google Scholar
  56. Phillips, S. E. (1994). High-stakes testing accommodations. Applied Measurement in Education, 7, 93–120.Google Scholar
  57. Princeton Review. (2016). The K&W guide to colleges for students with learning differences. Natick, MA: Author.Google Scholar
  58. Proctor, B., & Prevatt, F. (2003). Agreement among four models used for diagnosing learning disabilities. Journal of Learning Disabilities, 36, 459–466.Google Scholar
  59. Rayner, K., White, S. J., Johnson, R. L., & Liversedge, S. P. (2006). Raeding wrods with jubmled lettres: There is a cost. Psychological Science, 17, 192–193.Google Scholar
  60. Schrank, F.A., McGrew, K.S., & Mather, N. (2014). Woodcock-Johnson IV. Rolling Meadows, IL: Riverside.Google Scholar
  61. Shaywitz, S. (2003). Overcoming dyslexia. New York: Vintage.Google Scholar
  62. Sireci, S. G., Scarpati, S. E., & Li, S. (2005). Test accommodations for students with disabilities. Review of Educational Research, 75, 457–490.Google Scholar
  63. Sparks, R. L., & Lovett, B. J. (2009). Objective criteria for classification of postsecondary students as learning disabled. Journal of Learning Disabilities, 42, 230–239.Google Scholar
  64. Sparks, R. L., & Lovett, B. J. (2013). Applying objective diagnostic criteria to students in a college support program for learning disabilities. Learning Disability Quarterly, 36, 231–241.Google Scholar
  65. Streiner, D. L. (2003). Diagnosing tests: Using and misusing diagnostic and screening tests. Journal of Personality Assessment, 81, 209–219.Google Scholar
  66. Suhr, J. A. (2016). Illness identity and its implications for neuropsychological assessment. National Academy of Neuropsychology Bulletin, 30, 14–16.Google Scholar
  67. Suhr, J. A., & Wei, C. (2013a). Response expectancies and their potential influence in neuropsychological evaluation. In P. A. Arnett (Ed.), Secondary influences on neuropsychological test performance (pp. 182–200). New York: Oxford.Google Scholar
  68. Suhr, J. A., & Wei, C. (2013b). Symptoms as an excuse. Journal of Social and Clinical Psychology, 32, 753–769.Google Scholar
  69. Suhr, J. A., & Wei, C. (2017). Attention deficit/hyperactivity disorder as an illness identity. In K. Boone (Ed.), Neuropsychological evaluation of somatoform and other functional somatic conditions (pp. 251–273). New York: Routledge.Google Scholar
  70. Sullivan, B. K., May, K., & Galbally, L. (2007). Symptom exaggeration by college adults in attention-deficit hyperactivity disorder and learning disorder assessments. Applied Neuropsychology, 14, 189–207.Google Scholar
  71. UD Department of Education. (2018). The civil rights of students with hidden disabilities under Section 504 of the Rehabilitation Act of 1973. Washington, DC: Office for Civil Rights.Google Scholar
  72. US Department of Justice. (2016). Amendment of Americans with Disabilities Act Title II and Title III: Regulations to implement ADA Amendments Act of 2008. Federal Register, 81, 53204–53243.Google Scholar
  73. van den Boer, M., de Bree, E. H., & de Jong, P. F. (2018). Simulation of dyslexia. PLoS One, 13, e0196903.Google Scholar
  74. Weis, R., Sykes, L., & Unadkat, D. (2012). Qualitative differences in learning disabilities across postsecondary institutions. Journal of Learning Disabilities, 45, 491–502.Google Scholar
  75. Weis, R., Speridakos, E. C., & Ludwig, K. (2014). Community college students with learning disabilities. Journal of Learning Disabilities, 47, 556–568.Google Scholar
  76. Weis, R., Dean, E. L., & Osborne, K. J. (2016). Accommodation decision making for postsecondary students with learning disabilities. Journal of Learning Disabilities, 49, 484–498.Google Scholar
  77. Weis, R., Erickson, C. P., & Till, C. H. (2017). When average is not good enough: Students with learning disabilities at selective, private colleges. Journal of Learning Disabilities, 50, 684–700.Google Scholar
  78. Wiederholt, J. L., & Bryant, B. R. (2013). Gray oral reading test: Examiner’s manual (5th ed.). Austin, TX: Pro-Ed.Google Scholar
  79. Ziegler, E. A., & Boone, K. B. (2013). Symptom invalidity on neuropsychological testing. In P. A. Arnett (Ed.), Secondary influences on neuropsychological test performance (pp. 7–38). Oxford: Oxford University Press.Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Department of PsychologyDenison UniversityGranvilleUSA

Personalised recommendations