Skip to main content

Advertisement

SpringerLink
Log in

Understanding patient-reported outcome measures in Huntington disease: at what point is cognitive impairment related to poor measurement reliability?

  • Published:
Quality of Life Research Aims and scope Submit manuscript

Abstract

Purpose

Symptom progression in Huntington disease (HD) is associated with cognitive decline which may interfere with the self-report of symptoms. Unfortunately, data to support or refute the psychometric reliability of patient-reported outcomes (PROs) as HD progresses are limited. This is problematic given that PROs are increasingly recognized as important measures of efficacy for new treatments.

Methods

We examined PRO data from the HDQLIFE Measurement System (Speech Difficulties; Swallowing Difficulties; Chorea) in 509 individuals with premanifest, early-stage, or late-stage HD. Clinician-administered assessments of motor functioning (items from the UHDRS) and standardized objective assessments of cognition (Stroop, Symbol Digit Modalities) were also collected. We examined item bias using differential item functioning (DIF) across HD stage (premanifest, early-, late-) and relative to cognitive performance. We also examined the correlations between self-report and clinician ratings. Regression models that considered total cognitive ability were utilized to determine psychometric reliability of the PROs.

Results

Most PRO items were free from DIF for both staging and cognition. There were modest correlations between PROs and clinician report (ranged from − 0.40 to − 0.60). Modeling analyses indicated that psychometric reliability breaks down with poorer cognition and more progressed disease stage; split-half reliability was compromised (i.e., split-half reliability < 0.80) when scores were < 136 for Chorea, < 109 for Speech Difficulties, and < 179 for Swallowing Difficulties.

Conclusions

Results indicate that the psychometric reliability of PROs can be compromised as HD symptoms progress and cognition declines. Clinicians should consider PROs in conjunction with other types of assessments when total cognition scores exceed critical thresholds.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. The Huntington’s Disease Collaborative Research Group, A novel gene containing a trinucleotide repeat that is expanded and unstable on Huntington’s disease chromosomes. Cell, 1993. 72: 971–983.

    Article  Google Scholar 

  2. Ross, C. A., et al. (2014). Huntington disease: natural history, biomarkers and prospects for therapeutics. Nature Reviews Neurology, 10(4), 204–216.

    Article  CAS  PubMed  Google Scholar 

  3. Squitieri, F., et al.(2015). Epidemiology of Huntington disease: first post-HTT gene analysis of prevalence in Italy. Clinical Genetics, 89(3), 367–370.

    Google Scholar 

  4. Evans, S. J. W., et al. (2013). Prevalence of adult Huntington’s disease in the UK based on diagnoses recorded in general practice records. Journal of Neurology Neurosurgery and Psychiatry, 84(10), 1156–1160.

    Article  Google Scholar 

  5. Paulsen, J. S. Early Detection of Huntington Disease. Future Neurology, 2010. 5(1), 85–104.

    Article  Google Scholar 

  6. Walker, F. O. (2007). Huntington’s disease. Lancet, 369(9557), 218–228.

    Article  CAS  PubMed  Google Scholar 

  7. Ross, C. A., et al. (1997). Huntington disease and the related disorder, dentatorubral-pallidoluysian atrophy (DRPLA). Medicine (Baltimore), 76(5), 305–338.

    Article  CAS  Google Scholar 

  8. Cella, D. F. (1995). Measuring quality of life in palliative care. Seminars in oncology, 22(2 Suppl 3), 73–81.

    CAS  PubMed  Google Scholar 

  9. Basch, E. (2010). The missing voice of patients in drug-safety reporting. New England Journal of Medicine, 362(10), 865–869.

    Article  CAS  PubMed  Google Scholar 

  10. Carlozzi, N. E., et al. (2016). HDQLIFE: Development and assessment of health-related quality of life in Huntington disease (HD). Quality of Life Research, 25(10), 2441–2455.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  11. Cella, D. F., et al. (2011). The Neurology Quality of Life Measurement (Neuro-QOL) initiative. Archives of Physical Medicine and Rehabilitation, Supplement., 92(Suppl 1), S28-S36.

    Google Scholar 

  12. Cella, D., et al. (2012). Neuro-QOL brief measures of health-related quality of life for clinical research in neurology. Neurology, 78(23), 1860–1867.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  13. Cella, D. F., et al. (2010). The Patient-Reported Outcomes Measurement Information System (PROMIS) developed and tested in its first wave of adult self-reported health outcome item banks: 2005–2008. Journal of Clinical Epidemiology, 63, 1179–1194.

    Article  PubMed  PubMed Central  Google Scholar 

  14. Cella, D. F., et al. (2007). The Patient-Reported Outcomes Measurement Information System (PROMIS): progress of an NIH Roadmap cooperative group during its first two years. Medical Care, 45(5 Suppl 1), S3–S11.

    Google Scholar 

  15. Carlozzi, N. E., et al.(2016). The development of a new computer adaptive test to evaluate chorea in Huntington disease: HDQLIFE Chorea. Quality of Life Research, 25(20), 2429–2439.

    Google Scholar 

  16. Carlozzi, N. E., et al. (2016). HDQLIFE: the development of two new computer adaptive tests for use in Huntington disease, Speech Difficulties, and Swallowing Difficulties. Quality of Life Research, 25(10), 2417–2427.

    Google Scholar 

  17. Carlozzi, N. E., et al.(2016). New measures to capture end of life concerns in Huntington disease: Meaning and Purpose and Concern with Death and Dying from HDQLIFE (a patient-reported outcomes measurement system). Quality of Life Research, 25(10), 2403–2415.

    Google Scholar 

  18. Tabrizi, S. J., et al. (2011). Biological and clinical changes in premanifest and early stage Huntington’s disease in the TRACK-HD study: the 12-month longitudinal analysis. Lancet Neurology, 10(1), 31–42.

    Article  PubMed  Google Scholar 

  19. Tabrizi, S. J., et al. (2009). Biological and clinical manifestations of Huntington’s disease in the longitudinal TRACK-HD study: cross-sectional analysis of baseline data. Lancet Neurology, 8(9), 791–801.

    Article  PubMed  PubMed Central  Google Scholar 

  20. Paulsen, J. S., et al. (2014). Clinical and biomarker changes in premanifest Huntington disease show trial feasibility: A decade of the PREDICT-HD study. Frontiers in Aging Neuroscience, 6, 78.

    Article  PubMed  PubMed Central  Google Scholar 

  21. Paulsen, J. S., et al. (2001). Clinical markers of early disease in persons near onset of Huntington’s disease. Neurology, 57(4), 658–662.

    Article  CAS  PubMed  Google Scholar 

  22. Peavy, G. M., et al. (2010). Cognitive and functional decline in Huntington’s disease: dementia criteria revisited. Movement Disorders, 25(9), 1163–1169.

    Article  PubMed  PubMed Central  Google Scholar 

  23. Beglinger, L. J., et al. (2010). Cognitive change in patients with Huntington disease on the repeatable battery for the assessment of neuropsychological status. J Clin Exp Neuropsychol, 32(6), 573–578.

    Article  PubMed  Google Scholar 

  24. Stout, J. C., et al. (2012). Evaluation of longitudinal 12 and 24 month cognitive outcomes in premanifest and early Huntington’s disease. Journal of Neurology Neurosurgery Psychiatry, 12(7), 687–694.

    Article  Google Scholar 

  25. Williams, J. K., et al. (2015). Everyday cognition in prodromal Huntington disease. Neuropsychology, 29(2), 255–267.

    Article  PubMed  Google Scholar 

  26. Ward, J., et al. (2006). A four-year prospective study of cognitive functioning in Huntington’s disease. Journal of the International Neuropsychological Society, 12(4), 445–454.

    Article  PubMed  Google Scholar 

  27. Roos, R. A. (2010). Huntington’s disease: a clinical review. Orphanet Journal of Rare Diseases, 5(1), 40.

    Article  PubMed  PubMed Central  Google Scholar 

  28. Tabrizi, S. J., et al. (2012). Potential endpoints for clinical trials in premanifest and early Huntington’s disease in the TRACK-HD study: analysis of 24 month observational data. Lancet Neurology, 11(1), 42–53.

    Article  PubMed  Google Scholar 

  29. Ho, A. K., et al. (2003). Profile of cognitive progression in early Huntington’s disease. Neurology, 61(12), 1702–1706.

    Article  CAS  PubMed  Google Scholar 

  30. Kirkwood, S. C., et al. (2001). Progression of symptoms in the early and middle stages of Huntington disease. Archives of Neurology, 58(2), 273–278.

    Article  CAS  PubMed  Google Scholar 

  31. Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20(1), 10.

    Article  Google Scholar 

  32. Paulsen, J. S., et al. (2008). Detection of Huntington’s disease decades before diagnosis: the Predict-HD study. Journal of Neurology, Neurosurgery & Psychiatry, 79(8), 874–880.

    Article  CAS  Google Scholar 

  33. Hanauer, D. A., et al. (2015). Supporting information retrieval from electronic health records: A report of University of Michigan’s nine-year experience in developing and using the Electronic Medical Record Search Engine (EMERSE). Journal of Biomedical Informatics, 55, 290–300.

    Article  PubMed  PubMed Central  Google Scholar 

  34. Jackson, W., & TA, N. (1998). Effective serial measurement of cognitive orientation in rehabilitation: the Orientation Log. Archives of Physical Medicine and Rehabilitation, 79(6), 718–720.

    Article  CAS  PubMed  Google Scholar 

  35. Huntington Study Group, (1996). Unified Huntington’s disease rating scale: Reliability and consistency. Movement Disorders, 11(2): 136–142.

    Article  Google Scholar 

  36. Shoulson, I., & Fahn, S. (1979). Huntington disease clinical care and evaluation. Neurology, 29(1), 1–1.

    Article  CAS  PubMed  Google Scholar 

  37. Stroop, J. R. (1992). Studies of interference in serial verbal reactions (Reprinted from Journal Experimental-Psychology, Vol 18, Pg 643–662, 1935). Journal of Experimental Psychology: General, 121(1), 15.

    Article  Google Scholar 

  38. Marder, K., et al. (2000). Rate of functional decline in Huntington’s disease. Neurology, 54(2), 452–452.

    Article  CAS  PubMed  Google Scholar 

  39. Smith, A. (1995). Symbol Digit Modalities Test: Manual. Los Angeles: Western Psychological Services.

    Google Scholar 

  40. Crane, P. K., et al. (2006). Differential item functioning analysis with ordinal logistic regression techniques. DIFdetect and difwithpar. Medical Care, 44(11 Suppl 3), S115-S123.

    Google Scholar 

  41. Choi, S. W., Gibbons, L. E., & Crane, P. K. (2011). Lordif: An R package for detecting differential item functioning using iterative hybrid ordinal logistic regression/item response theory and monte carlo simulations. Journal of Statistical Software, 39(8), 1–30.

    Article  PubMed  PubMed Central  Google Scholar 

  42. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd edition). New York: Academic Press.

    Google Scholar 

  43. DeVellis, R. (2017) Scale development: Theory and applications. In L. Bickman & D. J. Rog (Eds.) Applied social research methods series, 4th edn. Los Angeles: Sage.

    Google Scholar 

  44. Heemskerk, A. W., & Roos, R. A. (2011). Dysphagia in Huntington’s disease: A review. Dysphagia, 26(1), 62–66.

    Article  PubMed  Google Scholar 

  45. Kagel, M. C., & Leopold, N. A. (1992). Dysphagia in Huntington’s disease: A 16-year retrospective. Dysphagia, 7(2), 106–114.

    Article  CAS  PubMed  Google Scholar 

  46. Reilmann, R., Leavitt, B. R., & Ross, C. A. (2014). Diagnostic criteria for Huntington’s disease based on natural history. Movement Disorders, 29(11), 1335–1341.

    Article  PubMed  Google Scholar 

  47. Ross, C. A., et al. (2014). Huntington disease: natural history, biomarkers and prospects for therapeutics. Nature Reviews. Neurology, 10, 204–216.

    Article  CAS  PubMed  Google Scholar 

  48. Dorsey, E. R., et al. (2013). Natural history of Huntington disease. JAMA Neurol, 70(12), 1520–1530.

    PubMed  Google Scholar 

  49. Pagan, F., Torres-Yaghi, Y., & Altshuler, M. (2017). The diagnosis and natural history of Huntington disease. Handbook of Clinical Neurology, 144, 63–67.

    Article  PubMed  Google Scholar 

  50. McCusker, E., & Loy, C. T. (2014). The many facets of unawareness in Huntington disease. Tremor Other Hyperkinet Mov (N Y), 4, 257.

    Google Scholar 

  51. Deckel, A. W., & Morrison, D. (1996). Evidence of a neurologically based “denial of illness” in patients with Huntington’s disease. Archives of Clinical Neuropsychology, 11(4), 295–302.

    Article  CAS  PubMed  Google Scholar 

  52. Vitale, C., et al. (2001). Unawareness of dyskinesias in Parkinson’s and Huntington’s diseases. Neurological Sciences, 22(1), 105–106.

    Article  CAS  PubMed  Google Scholar 

  53. Chatterjee, A., et al. (2005). A comparison of self-report and caregiver assessment of depression, apathy, and irritability in Huntington’s disease. The Journal of Neuropsychiatry and Clinical Neurosciences, 17(3), 378–383.

    Article  PubMed  Google Scholar 

  54. Duff, K., et al. (2010). “Frontal” behaviors before the diagnosis of Huntington’s disease and their relationship to markers of disease progression: evidence of early lack of awareness. The Journal of Neuropsychiatry and Clinical Neurosciences, 22(2), 196–207.

    Article  PubMed  PubMed Central  Google Scholar 

  55. Pringsheim, T., et al. (2012). The incidence and prevalence of Huntington’s disease: A systematic review and meta-analysis. Movement Disorders, 27(9), 1083–1091.

    Article  PubMed  Google Scholar 

  56. Folstein, S. E..(1989). Huntington’s disease: A disorder of families. Baltimore :Johns Hopkins University Press.

    Google Scholar 

  57. Hayden, M. R., MacGregor, J. M., & Beighton, P. H. (1980). The prevalence of Huntington’s chorea in South Africa. South African Medical Journal, 58, 193–196.

    CAS  PubMed  Google Scholar 

  58. Narabayashi, H. (1973). Huntington’s chorea in Japan: Review of the literature. Advances in neurology, 1, 253–259.

    Google Scholar 

Download references

Acknowledgements

We thank the University of Iowa, the Investigators and Coordinators of this study, the study participants, the National Research Roster for Huntington Disease Patients and Families, the Huntington Study Group, and the Huntington’s Disease Society of America. We acknowledge the assistance of Jeffrey D. Long, Hans J. Johnson, Jeremy H. Bockholt, Roland Zschiegner, and Jane S. Paulsen. We also acknowledge Roger Albin, Kelvin Chou, and Henry Paulsen for the assistance with participant recruitment. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH. HDQLIFE Site Investigators and Coordinators: Noelle Carlozzi, Praveen Dayalu, Stephen Schilling, Amy Austin, Matthew Canter, Siera Goodnight, Jennifer Miner, Nicholas Migliore (University of Michigan, Ann Arbor, MI); Jane S. Paulsen, Nancy Downing, Isabella DeSoriano, Courtney Shadrick, Amanda Miller (University of Iowa, Iowa City, IA); Kimberly Quaid, Melissa Wesson (Indiana University, Indianapolis, IN); Christopher Ross, Gregory Churchill, Mary Jane Ong (Johns Hopkins University, Baltimore, MD); Susan Perlman, Brian Clemente, Aaron Fisher, Gloria Obialisi, Michael Rosco (University of California Los Angeles, Los Angeles, CA); Michael McCormack, Humberto Marin, Allison Dicke; Judy Rokeach (Rutgers University, Piscataway, NJ); Joel Perlmutter, Stacey Barton, Shineeka Smith (Washington University, St. Louis, MO); Martha Nance, Pat Ede (Struthers Parkinson’s Center); Stephen Rao, Anwar Ahmed, Michael Lengen, Lyla Mourany, Christine Reece, (Cleveland Clinic Foundation, Cleveland, OH); Michael Geschwind, Joseph Winer (University of California – San Francisco, San Francisco, CA); and David Cella, Richard Gershon, Elizabeth Hahn, Jin-Shei Lai (Northwestern University, Chicago, IL).

Funding

Work on this manuscript was supported by the National Institutes of Health (NIH), National Institute of Neurological Disorders and Stroke (R01NS077946), and the National Center for Advancing Translational Sciences (UL1TR000433). In addition, a portion of this study sample was collected in conjunction with the Predict-HD study. The Predict-HD study was supported by the NIH, National Institute of Neurological Disorders and Stroke (R01NS040068), the NIH, Center for Inherited Disease Research (provided supported for sample phenotyping), and the CHDI Foundation (award to the University of Iowa).

Author information

Authors and Affiliations

  1. Department of Physical Medicine and Rehabilitation, University of Michigan, Ann Arbor, MI, USA

    N. E. Carlozzi & A. L. Kratz

  2. Institute for Social Research, University of Michigan, Ann Arbor, MI, USA

    S. Schilling

  3. Department of Psychiatry, Carver College of Medicine, The University of Iowa, Iowa City, IA, USA

    J. S. Paulsen

  4. Department of Neurology, Carver College of Medicine, The University of Iowa, Iowa City, IA, USA

    J. S. Paulsen

  5. Department of Psychology, The University of Iowa, Iowa City, IA, USA

    J. S. Paulsen

  6. Beth Israel Deaconess Medical Center, Boston, MA, USA

    S. Frank

  7. School of Psychological Sciences, Monash University, Clayton, VIC, Australia

    J. C. Stout

  8. Department of Physical Medicine & Rehabilitation, University of Michigan, North Campus Research Complex, 2800 Plymouth Road, Building NCRC B14, Room G216, Ann Arbor, MI, 48109-2800, USA

    N. E. Carlozzi

Authors
  1. N. E. Carlozzi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. S. Schilling
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. A. L. Kratz
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. J. S. Paulsen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. S. Frank
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. J. C. Stout
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to N. E. Carlozzi.

Ethics declarations

Conflict of interest

The authors have no conflicts of interest to report.

Ethical approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards.

Informed consent

Informed consent was obtained from all individual participants included in the study.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Carlozzi, N.E., Schilling, S., Kratz, A.L. et al. Understanding patient-reported outcome measures in Huntington disease: at what point is cognitive impairment related to poor measurement reliability?. Qual Life Res 27, 2541–2555 (2018). https://doi.org/10.1007/s11136-018-1912-6

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11136-018-1912-6

Keywords

Search

Navigation