Skip to main content
SpringerLink
Log in

Identifying clinically meaningful severity categories for PROMIS pediatric measures of anxiety, mobility, fatigue, and depressive symptoms in juvenile idiopathic arthritis and childhood-onset systemic lupus erythematosus

  • Published:
Quality of Life Research Aims and scope Submit manuscript

Abstract

Purpose

A key limitation to widespread adoption of patient-reported outcome (PRO) measures is the lack of interpretability of scores. We aim to identify clinical severity thresholds to distinguish categories of no problems, mild, moderate, and severe along the PROMIS® Pediatric T-score metric for measures of anxiety, mobility, fatigue, and depressive symptoms for use in populations with juvenile idiopathic arthritis (JIA) and childhood-onset systemic lupus erythematosus (cSLE).

Methods

We used a modified standard setting methodology from educational testing to identify clinical severity thresholds (clinical cut scores). Using item response theory-based parameters from PROMIS item banks, we developed a series of clinical vignettes that represented different severity or ability levels along the PROMIS Pediatric T-score metric. In stakeholder workshops, participants worked individually and together to reach consensus on clinical cut scores. Median cut-score placements were taken when consensus was not reached. Focus groups were recorded and qualitative analysis was conducted to identify decision-making processes.

Results

Nine adolescents (age 13–17 years) with JIA (33% female) and their caregivers, five adolescents (age 14–16 years) with cSLE (100% female) and their caregivers, and 12 pediatric rheumatologists (75% female) participated in bookmarking workshops. Placement of thresholds for bookmarks was highly similar across stakeholder groups (differences from 0 to 5 points on the PROMIS t-score metric) for all but one bookmark placement.

Conclusion

This study resulted in clinical thresholds for severity categories for PROMIS Pediatric measures of anxiety, mobility, fatigue, and depressive symptoms, providing greater interpretability of scores in JIA and cSLE populations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Helmick, C. G., Felson, D. T., Lawrence, R. C., Gabriel, S., Hirsch, R., Kwoh, C. K., et al. (2008). Estimates of the prevalence of arthritis and other rheumatic conditions in the United States. Part I. Arthritis and Rheumatism, 58(1), 15–25. https://doi.org/10.1002/art.23177.

    Article  PubMed  Google Scholar 

  2. Sacks, J. J., Helmick, C. G., Luo, Y. H., Ilowite, N. T., & Bowyer, S. (2007). Prevalence of and annual ambulatory health care visits for pediatric arthritis and other rheumatologic conditions in the United States in 2001–2004. Arthritis and Rheumatism, 57(8), 1439–1445. https://doi.org/10.1002/art.23087.

    Article  PubMed  Google Scholar 

  3. Wallace, C. A., Huang, B., Bandeira, M., Ravelli, A., & Giannini, E. H. (2005). Patterns of clinical remission in select categories of juvenile idiopathic arthritis. Arthritis and Rheumatism, 52(11), 3554–3562. https://doi.org/10.1002/art.21389.

    Article  PubMed  Google Scholar 

  4. Ringold, S., Seidel, K. D., Koepsell, T. D., & Wallace, C. A. (2009). Inactive disease in polyarticular juvenile idiopathic arthritis: Current patterns and associations. Rheumatology (Oxford), 48(8), 972–977. https://doi.org/10.1093/rheumatology/kep144.

    Article  Google Scholar 

  5. Magni-Manzoni, S., Pistorio, A., Labo, E., Viola, S., Garcia-Munitis, P., Panigada, S., et al. (2008). A longitudinal analysis of physical functional disability over the course of juvenile idiopathic arthritis. Annals of the Rheumatic Diseases, 67(8), 1159–1164. https://doi.org/10.1136/ard.2007.078121.

    Article  CAS  PubMed  Google Scholar 

  6. Gutierrez-Suarez, R., Pistorio, A., Cespedes Cruz, A., Norambuena, X., Flato, B., Rumba, I., et al. (2007). Health-related quality of life of patients with juvenile idiopathic arthritis coming from 3 different geographic areas. The PRINTO multinational quality of life cohort study. Rheumatology (Oxford), 46(2), 314–320. https://doi.org/10.1093/rheumatology/kel218.

    Article  CAS  Google Scholar 

  7. Seid, M., Opipari, L., Huang, B., Brunner, H. I., & Lovell, D. J. (2009). Disease control and health-related quality of life in juvenile idiopathic arthritis. Arthritis and Rheumatism, 61(3), 393–399. https://doi.org/10.1002/art.24477.

    Article  PubMed  PubMed Central  Google Scholar 

  8. Haverman, L., Grootenhuis, M. A., van den Berg, J. M., van Veenendaal, M., Dolman, K. M., Swart, J. F., et al. (2012). Predictors of health-related quality of life in children and adolescents with juvenile idiopathic arthritis: Results from a web-based survey. Arthritis Care Research (Hoboken). https://doi.org/10.1002/acr.21609.

    Article  Google Scholar 

  9. Kamphuis, S., & Silverman, E. D. (2010). Prevalence and burden of pediatric-onset systemic lupus erythematosus. Nature Reviews Rheumatology, 6(9), 538–546. https://doi.org/10.1038/nrrheum.2010.121.

    Article  CAS  PubMed  Google Scholar 

  10. Tucker, L. B., Menon, S., Schaller, J. G., & Isenberg, D. A. (1995). Adult- and childhood-onset systemic lupus erythematosus: A comparison of onset, clinical features, serology, and outcome. British Journal of Rheumatology, 34(9), 866–872.

    Article  CAS  PubMed  Google Scholar 

  11. Reeve, B. B., Wyrwich, K. W., Wu, A. W., Velikova, G., Terwee, C. B., Snyder, C. F., et al. (2013). ISOQOL recommends minimum standards for patient-reported outcome measures used in patient-centered outcomes and comparative effectiveness research. Quality of Life Research, 22(8), 1889–1905. https://doi.org/10.1007/s11136-012-0344-y.

    Article  PubMed  Google Scholar 

  12. Reeve, B. B., Hays, R. D., Bjorner, J. B., Cook, K. F., Crane, P. K., Teresi, J. A., et al. (2007). Psychometric evaluation and calibration of health-related quality of life item banks: Plans for the patient-reported outcomes measurement information system (PROMIS). Medical Care, 45(5 Suppl 1), S22–31. https://doi.org/10.1097/01.mlr.0000250483.85507.04.

    Article  PubMed  Google Scholar 

  13. Cella, D., Riley, W., Stone, A., Rothrock, N., Reeve, B., Yount, S., et al. (2010). The patient-reported outcomes measurement information system (PROMIS) developed and tested its first wave of adult self-reported health outcome item banks: 2005–2008. Journal of Clinical Epidemiology, 63(11), 1179–1194. https://doi.org/10.1016/j.jclinepi.2010.04.011.

    Article  PubMed  PubMed Central  Google Scholar 

  14. DeWalt, D. A., Gross, H. E., Gipson, D. S., Selewski, D. T., DeWitt, E. M., Dampier, C. D., et al. (2015). PROMIS((R)) pediatric self-report scales distinguish subgroups of children within and across six common pediatric chronic health conditions. Quality of Life Research, 24(9), 2195–2208. https://doi.org/10.1007/s11136-015-0953-3.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Reeve, B. B., Edwards, L. J., Jaeger, B. C., Hinds, P. S., Dampier, C., Gipson, D. S., et al. (2018). Assessing responsiveness over time of the PROMIS((R)) pediatric symptom and function measures in cancer, nephrotic syndrome, and sickle cell disease. Quality of Life Research, 27(1), 249–257. https://doi.org/10.1007/s11136-017-1697-z.

    Article  PubMed  Google Scholar 

  16. Quinn, H., Thissen, D., Liu, Y., Magnus, B., Lai, J. S., Amtmann, D., et al. (2014). Using item response theory to enrich and expand the PROMIS(R) pediatric self report banks. Health Quality Life Outcomes, 12, 160. https://doi.org/10.1186/s12955-014-0160-x.

    Article  Google Scholar 

  17. Thissen, D., Liu, Y., Magnus, B., Quinn, H., Gipson, D. S., Dampier, C., et al. (2016). Estimating minimally important difference (MID) in PROMIS pediatric measures using the scale-judgment method. Quality of Life Research, 25(1), 13–23. https://doi.org/10.1007/s11136-015-1058-8.

    Article  PubMed  Google Scholar 

  18. Snyder, C., Smith, K., Holzner, B., Rivera, Y. M., Bantug, E., Brundage, M., et al. (2019). Making a picture worth a thousand numbers: Recommendations for graphically displaying patient-reported outcomes data. Quality of Life Research, 28, 345–356. https://doi.org/10.1007/s11136-018-2020-3.

    Article  PubMed  Google Scholar 

  19. Brundage, M. D., Smith, K. C., Little, E. A., Bantug, E. T., Snyder, C. F., & Board, P. R. O. D. P. S. A. (2015). Communicating patient-reported outcome scores using graphic formats: Results from a mixed-methods evaluation. Quality of Life Research, 24(10), 2457–2472. https://doi.org/10.1007/s11136-015-0974-y.

    Article  PubMed  PubMed Central  Google Scholar 

  20. Cook, K. F., Cella, D., & Reeve, B. B. (2019). PRO-bookmarking to estimate clinical thresholds for patient-reported symptoms and function. Medicine Care, 57(Suppl 5), S13–S17. https://doi.org/10.1097/MLR.0000000000001087.

    Article  Google Scholar 

  21. Cook, K. F., Victorson, D. E., Cella, D., Schalet, B. D., & Miller, C. (2014). Creating meaningful cut-scores for Neuro-QOL measures of fatigue, physical functioning and sleep disturbance using standard setting with patients and providers. Quality Life Research. https://doi.org/10.1007/s11136-014-0790-9.

    Article  Google Scholar 

  22. Cella, D., Choi, S., Garcia, S., Cook, K. F., Rosenbloom, S., Lai, J. S., et al. (2014). Setting standards for severity of common symptoms in oncology using the PROMIS item banks and expert judgment. Quality of Life Research. https://doi.org/10.1007/s11136-014-0732-6.

    Article  PubMed  PubMed Central  Google Scholar 

  23. Morgan, E. M., Mara, C. A., Huang, B., Barnett, K., Carle, A. C., Farrell, J. E., et al. (2017). Establishing clinical meaning and defining important differences for patient-reported outcomes measurement information system (PROMIS((R))) measures in juvenile idiopathic arthritis using standard setting with patients, parents, and providers. Quality of Life Research, 26(3), 565–586. https://doi.org/10.1007/s11136-016-1468-2.

    Article  PubMed  Google Scholar 

  24. Buckendahl, C. W., Smith, R. W., Impara, J. C., & Plake, B. S. (2002). A comparison of angoff and bookmark standard setting methods. Journal of Educational Measurement, 39(3), 253–263.

    Article  Google Scholar 

  25. Cizek, G. J., & Bunch, M. B. (2006). The Bookmark Method. In G. Cizek & M. B. Bunch (Eds.), Standard setting: A guide to establishing and evaluating performance standards on tests. Cleveland: SAGE Publications Inc.

    Google Scholar 

  26. Beebe, J. (2001). Rapid assessment process: An introduction. Walnut Creek: AltaMira Press.

    Google Scholar 

  27. Teens Know What They Want from Online News. (2013). Do You? Newspaper Association of America Foundation and the Media Management Center at Northwestern University.

  28. Team, R. C. (2013). R: A language and environment for statistical computing. R Foundation for Statistical Computing. Vienna, Austria

  29. Irwin, D. E., Stucky, B., Langer, M. M., Thissen, D., Dewitt, E. M., Lai, J. S., et al. (2010). An item response analysis of the pediatric PROMIS anxiety and depressive symptoms scales. Quality of Life Research, 19(4), 595–607. https://doi.org/10.1007/s11136-010-9619-3.

    Article  PubMed  PubMed Central  Google Scholar 

  30. Varni, J. W., Thissen, D., Stucky, B. D., Liu, Y., Magnus, B., Quinn, H., et al. (2014). PROMIS(R) parent proxy report scales for children ages 5–7 years: An item response theory analysis of differential item functioning across age groups. Quality of Life Research, 23(1), 349–361. https://doi.org/10.1007/s11136-013-0439-0.

    Article  PubMed  Google Scholar 

  31. Irwin, D. E., Stucky, B. D., Thissen, D., Dewitt, E. M., Lai, J. S., Yeatts, K., et al. (2010). Sampling plan and patient characteristics of the PROMIS pediatrics large-scale survey. Quality of Life Research, 19(4), 585–594. https://doi.org/10.1007/s11136-010-9618-4.

    Article  PubMed  PubMed Central  Google Scholar 

  32. DeWitt, E. M., Stucky, B. D., Thissen, D., Irwin, D. E., Langer, M., Varni, J. W., et al. (2011). Construction of the eight-item patient-reported outcomes measurement information system pediatric physical function scales: Built using item response theory. Journal of Clinical Epidemiology, 64(7), 794–804. https://doi.org/10.1016/j.jclinepi.2010.10.012.

    Article  PubMed  PubMed Central  Google Scholar 

  33. Lai, J. S., Stucky, B. D., Thissen, D., Varni, J. W., DeWitt, E. M., Irwin, D. E., et al. (2013). Development and psychometric properties of the PROMIS((R)) pediatric fatigue item banks. Quality of Life Research, 22(9), 2417–2427. https://doi.org/10.1007/s11136-013-0357-1.

    Article  PubMed  Google Scholar 

  34. Messick, S. (1998). Test validity: A matter of consequence. Social Indicators Research, 45(1–3), 35–44.

    Article  Google Scholar 

  35. Guyatt, G. H., Feeny, D. H., & Patrick, D. L. (1993). Measuring health-related quality of life. Annals of Internal Medicine, 118(8), 622–629. https://doi.org/10.7326/0003-4819-118-8-199304150-00009.

    Article  CAS  PubMed  Google Scholar 

  36. Spertus, J. (2014). Barriers to the use of patient-reported outcomes in clinical care. Circular Cardiovascular Quality Outcomes, 7(1), 2–4. https://doi.org/10.1161/circoutcomes.113.000829.

    Article  Google Scholar 

  37. Snyder, C. F., Smith, K. C., Bantug, E. T., Tolbert, E. E., Blackford, A. L., & Brundage, M. D. (2017). What do these scores mean? Presenting patient-reported outcomes data to patients and clinicians to improve interpretability. Cancer, 123(10), 1848–1859. https://doi.org/10.1002/cncr.30530.

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

This study was conducted as part of the Duke PEPR Center, which is funded by the National Institutes of Health through the following grant administered by the National Institute of Arthritis and Musculoskeletal and Skin Diseases: U19AR069519. The research reported is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. We would like to acknowledge the adolescents, parents, and clinicians who participated in this study. We are grateful for their contributions. We would also like to acknowledge the Childhood and Rheumatology Research Alliance (CARRA) and the Arthritis Foundation for their help setting up bookmarking workshops at their annual meetings and conferences.

Author information

Authors and Affiliations

  1. Department of Population Health Sciences, Duke University School of Medicine, Durham, NC, USA

    C. M. Mann, N. Lucas, A. Hernandez & B. B. Reeve

  2. Department of Pediatrics and Duke Clinical Research Institute, Duke Health, Durham, NC, USA

    L. E. Schanberg & B. B. Reeve

  3. Lineberger Comprehensive Cancer Center, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA

    M. Wang

  4. Division of Pediatric Rheumatology, Department of Pediatrics, University of California, San Francisco, San Francisco, CA, USA

    E. von Scheven

  5. Division of Pediatric Rheumatology, Department of Pediatrics, Seattle Children’s Hospital, Seattle, WA, USA

    S. Ringold

Authors
  1. C. M. Mann
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. L. E. Schanberg
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. M. Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. E. von Scheven
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. N. Lucas
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. A. Hernandez
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. S. Ringold
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. B. B. Reeve
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors substantially contributed to the study conception, design, implementation or analysis. Bryce Reeve, Courtney Mann, Mian Wang, Emily von Scheven, and Laura Schanberg were responsible for conceptualization and study design. Study implementation, including material preparation, participant recruitment, and data collection were performed by Courtney Mann, Mian Wang, Nicole Lucas, Alexy Hernandez, Laura Schanberg, Emily von Scheven, Sarah Ringold, and Bryce Reeve. Data analysis was conducted by Courtney Mann, Nicole Lucas, and Alexy Hernandez and results were interpreted by the full research team. The first draft of the manuscript was written by Courtney Mann and all authors commented on subsequent drafts of the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to C. M. Mann.

Ethics declarations

Conflict of interest

Laura Schanberg, MD, Sarah Ringold, MD, and Emily von Scheven work for the Childhood Arthritis and Rheumatology Research Alliance (CARRA). Dr. Schanberg is a former Board Chair and currently sits on the Registry and Research Oversight Committee along with Dr. Sarah Ringold. Dr. Emily von Scheven will be the next Board Chair for CARRA. All other authors declare no conflict of interest.

Ethical approval

All procedures performed in studies involving human participants were conducted in accordance with ethical standards of the institutional review board (Duke Health IRB, Pro00091284 and UNC IRB #15-2442 and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed consent

Informed consent was obtained from all individual participants included in the study. For participants under the age of 18, caregiver consent was obtained in addition to child assent.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary file1 (PDF 400 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mann, C.M., Schanberg, L.E., Wang, M. et al. Identifying clinically meaningful severity categories for PROMIS pediatric measures of anxiety, mobility, fatigue, and depressive symptoms in juvenile idiopathic arthritis and childhood-onset systemic lupus erythematosus. Qual Life Res 29, 2573–2584 (2020). https://doi.org/10.1007/s11136-020-02513-6

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11136-020-02513-6

Keywords

Search

Navigation