A key limitation to widespread adoption of patient-reported outcome (PRO) measures is the lack of interpretability of scores. We aim to identify clinical severity thresholds to distinguish categories of no problems, mild, moderate, and severe along the PROMIS® Pediatric T-score metric for measures of anxiety, mobility, fatigue, and depressive symptoms for use in populations with juvenile idiopathic arthritis (JIA) and childhood-onset systemic lupus erythematosus (cSLE).
We used a modified standard setting methodology from educational testing to identify clinical severity thresholds (clinical cut scores). Using item response theory-based parameters from PROMIS item banks, we developed a series of clinical vignettes that represented different severity or ability levels along the PROMIS Pediatric T-score metric. In stakeholder workshops, participants worked individually and together to reach consensus on clinical cut scores. Median cut-score placements were taken when consensus was not reached. Focus groups were recorded and qualitative analysis was conducted to identify decision-making processes.
Nine adolescents (age 13–17 years) with JIA (33% female) and their caregivers, five adolescents (age 14–16 years) with cSLE (100% female) and their caregivers, and 12 pediatric rheumatologists (75% female) participated in bookmarking workshops. Placement of thresholds for bookmarks was highly similar across stakeholder groups (differences from 0 to 5 points on the PROMIS t-score metric) for all but one bookmark placement.
This study resulted in clinical thresholds for severity categories for PROMIS Pediatric measures of anxiety, mobility, fatigue, and depressive symptoms, providing greater interpretability of scores in JIA and cSLE populations.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Tax calculation will be finalised during checkout.
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
Tax calculation will be finalised during checkout.
Helmick, C. G., Felson, D. T., Lawrence, R. C., Gabriel, S., Hirsch, R., Kwoh, C. K., et al. (2008). Estimates of the prevalence of arthritis and other rheumatic conditions in the United States. Part I. Arthritis and Rheumatism, 58(1), 15–25. https://doi.org/10.1002/art.23177.
Sacks, J. J., Helmick, C. G., Luo, Y. H., Ilowite, N. T., & Bowyer, S. (2007). Prevalence of and annual ambulatory health care visits for pediatric arthritis and other rheumatologic conditions in the United States in 2001–2004. Arthritis and Rheumatism, 57(8), 1439–1445. https://doi.org/10.1002/art.23087.
Wallace, C. A., Huang, B., Bandeira, M., Ravelli, A., & Giannini, E. H. (2005). Patterns of clinical remission in select categories of juvenile idiopathic arthritis. Arthritis and Rheumatism, 52(11), 3554–3562. https://doi.org/10.1002/art.21389.
Ringold, S., Seidel, K. D., Koepsell, T. D., & Wallace, C. A. (2009). Inactive disease in polyarticular juvenile idiopathic arthritis: Current patterns and associations. Rheumatology (Oxford), 48(8), 972–977. https://doi.org/10.1093/rheumatology/kep144.
Magni-Manzoni, S., Pistorio, A., Labo, E., Viola, S., Garcia-Munitis, P., Panigada, S., et al. (2008). A longitudinal analysis of physical functional disability over the course of juvenile idiopathic arthritis. Annals of the Rheumatic Diseases, 67(8), 1159–1164. https://doi.org/10.1136/ard.2007.078121.
Gutierrez-Suarez, R., Pistorio, A., Cespedes Cruz, A., Norambuena, X., Flato, B., Rumba, I., et al. (2007). Health-related quality of life of patients with juvenile idiopathic arthritis coming from 3 different geographic areas. The PRINTO multinational quality of life cohort study. Rheumatology (Oxford), 46(2), 314–320. https://doi.org/10.1093/rheumatology/kel218.
Seid, M., Opipari, L., Huang, B., Brunner, H. I., & Lovell, D. J. (2009). Disease control and health-related quality of life in juvenile idiopathic arthritis. Arthritis and Rheumatism, 61(3), 393–399. https://doi.org/10.1002/art.24477.
Haverman, L., Grootenhuis, M. A., van den Berg, J. M., van Veenendaal, M., Dolman, K. M., Swart, J. F., et al. (2012). Predictors of health-related quality of life in children and adolescents with juvenile idiopathic arthritis: Results from a web-based survey. Arthritis Care Research (Hoboken). https://doi.org/10.1002/acr.21609.
Kamphuis, S., & Silverman, E. D. (2010). Prevalence and burden of pediatric-onset systemic lupus erythematosus. Nature Reviews Rheumatology, 6(9), 538–546. https://doi.org/10.1038/nrrheum.2010.121.
Tucker, L. B., Menon, S., Schaller, J. G., & Isenberg, D. A. (1995). Adult- and childhood-onset systemic lupus erythematosus: A comparison of onset, clinical features, serology, and outcome. British Journal of Rheumatology, 34(9), 866–872.
Reeve, B. B., Wyrwich, K. W., Wu, A. W., Velikova, G., Terwee, C. B., Snyder, C. F., et al. (2013). ISOQOL recommends minimum standards for patient-reported outcome measures used in patient-centered outcomes and comparative effectiveness research. Quality of Life Research, 22(8), 1889–1905. https://doi.org/10.1007/s11136-012-0344-y.
Reeve, B. B., Hays, R. D., Bjorner, J. B., Cook, K. F., Crane, P. K., Teresi, J. A., et al. (2007). Psychometric evaluation and calibration of health-related quality of life item banks: Plans for the patient-reported outcomes measurement information system (PROMIS). Medical Care, 45(5 Suppl 1), S22–31. https://doi.org/10.1097/01.mlr.0000250483.85507.04.
Cella, D., Riley, W., Stone, A., Rothrock, N., Reeve, B., Yount, S., et al. (2010). The patient-reported outcomes measurement information system (PROMIS) developed and tested its first wave of adult self-reported health outcome item banks: 2005–2008. Journal of Clinical Epidemiology, 63(11), 1179–1194. https://doi.org/10.1016/j.jclinepi.2010.04.011.
DeWalt, D. A., Gross, H. E., Gipson, D. S., Selewski, D. T., DeWitt, E. M., Dampier, C. D., et al. (2015). PROMIS((R)) pediatric self-report scales distinguish subgroups of children within and across six common pediatric chronic health conditions. Quality of Life Research, 24(9), 2195–2208. https://doi.org/10.1007/s11136-015-0953-3.
Reeve, B. B., Edwards, L. J., Jaeger, B. C., Hinds, P. S., Dampier, C., Gipson, D. S., et al. (2018). Assessing responsiveness over time of the PROMIS((R)) pediatric symptom and function measures in cancer, nephrotic syndrome, and sickle cell disease. Quality of Life Research, 27(1), 249–257. https://doi.org/10.1007/s11136-017-1697-z.
Quinn, H., Thissen, D., Liu, Y., Magnus, B., Lai, J. S., Amtmann, D., et al. (2014). Using item response theory to enrich and expand the PROMIS(R) pediatric self report banks. Health Quality Life Outcomes, 12, 160. https://doi.org/10.1186/s12955-014-0160-x.
Thissen, D., Liu, Y., Magnus, B., Quinn, H., Gipson, D. S., Dampier, C., et al. (2016). Estimating minimally important difference (MID) in PROMIS pediatric measures using the scale-judgment method. Quality of Life Research, 25(1), 13–23. https://doi.org/10.1007/s11136-015-1058-8.
Snyder, C., Smith, K., Holzner, B., Rivera, Y. M., Bantug, E., Brundage, M., et al. (2019). Making a picture worth a thousand numbers: Recommendations for graphically displaying patient-reported outcomes data. Quality of Life Research, 28, 345–356. https://doi.org/10.1007/s11136-018-2020-3.
Brundage, M. D., Smith, K. C., Little, E. A., Bantug, E. T., Snyder, C. F., & Board, P. R. O. D. P. S. A. (2015). Communicating patient-reported outcome scores using graphic formats: Results from a mixed-methods evaluation. Quality of Life Research, 24(10), 2457–2472. https://doi.org/10.1007/s11136-015-0974-y.
Cook, K. F., Cella, D., & Reeve, B. B. (2019). PRO-bookmarking to estimate clinical thresholds for patient-reported symptoms and function. Medicine Care, 57(Suppl 5), S13–S17. https://doi.org/10.1097/MLR.0000000000001087.
Cook, K. F., Victorson, D. E., Cella, D., Schalet, B. D., & Miller, C. (2014). Creating meaningful cut-scores for Neuro-QOL measures of fatigue, physical functioning and sleep disturbance using standard setting with patients and providers. Quality Life Research. https://doi.org/10.1007/s11136-014-0790-9.
Cella, D., Choi, S., Garcia, S., Cook, K. F., Rosenbloom, S., Lai, J. S., et al. (2014). Setting standards for severity of common symptoms in oncology using the PROMIS item banks and expert judgment. Quality of Life Research. https://doi.org/10.1007/s11136-014-0732-6.
Morgan, E. M., Mara, C. A., Huang, B., Barnett, K., Carle, A. C., Farrell, J. E., et al. (2017). Establishing clinical meaning and defining important differences for patient-reported outcomes measurement information system (PROMIS((R))) measures in juvenile idiopathic arthritis using standard setting with patients, parents, and providers. Quality of Life Research, 26(3), 565–586. https://doi.org/10.1007/s11136-016-1468-2.
Buckendahl, C. W., Smith, R. W., Impara, J. C., & Plake, B. S. (2002). A comparison of angoff and bookmark standard setting methods. Journal of Educational Measurement, 39(3), 253–263.
Cizek, G. J., & Bunch, M. B. (2006). The Bookmark Method. In G. Cizek & M. B. Bunch (Eds.), Standard setting: A guide to establishing and evaluating performance standards on tests. Cleveland: SAGE Publications Inc.
Beebe, J. (2001). Rapid assessment process: An introduction. Walnut Creek: AltaMira Press.
Teens Know What They Want from Online News. (2013). Do You? Newspaper Association of America Foundation and the Media Management Center at Northwestern University.
Team, R. C. (2013). R: A language and environment for statistical computing. R Foundation for Statistical Computing. Vienna, Austria
Irwin, D. E., Stucky, B., Langer, M. M., Thissen, D., Dewitt, E. M., Lai, J. S., et al. (2010). An item response analysis of the pediatric PROMIS anxiety and depressive symptoms scales. Quality of Life Research, 19(4), 595–607. https://doi.org/10.1007/s11136-010-9619-3.
Varni, J. W., Thissen, D., Stucky, B. D., Liu, Y., Magnus, B., Quinn, H., et al. (2014). PROMIS(R) parent proxy report scales for children ages 5–7 years: An item response theory analysis of differential item functioning across age groups. Quality of Life Research, 23(1), 349–361. https://doi.org/10.1007/s11136-013-0439-0.
Irwin, D. E., Stucky, B. D., Thissen, D., Dewitt, E. M., Lai, J. S., Yeatts, K., et al. (2010). Sampling plan and patient characteristics of the PROMIS pediatrics large-scale survey. Quality of Life Research, 19(4), 585–594. https://doi.org/10.1007/s11136-010-9618-4.
DeWitt, E. M., Stucky, B. D., Thissen, D., Irwin, D. E., Langer, M., Varni, J. W., et al. (2011). Construction of the eight-item patient-reported outcomes measurement information system pediatric physical function scales: Built using item response theory. Journal of Clinical Epidemiology, 64(7), 794–804. https://doi.org/10.1016/j.jclinepi.2010.10.012.
Lai, J. S., Stucky, B. D., Thissen, D., Varni, J. W., DeWitt, E. M., Irwin, D. E., et al. (2013). Development and psychometric properties of the PROMIS((R)) pediatric fatigue item banks. Quality of Life Research, 22(9), 2417–2427. https://doi.org/10.1007/s11136-013-0357-1.
Messick, S. (1998). Test validity: A matter of consequence. Social Indicators Research, 45(1–3), 35–44.
Guyatt, G. H., Feeny, D. H., & Patrick, D. L. (1993). Measuring health-related quality of life. Annals of Internal Medicine, 118(8), 622–629. https://doi.org/10.7326/0003-4819-118-8-199304150-00009.
Spertus, J. (2014). Barriers to the use of patient-reported outcomes in clinical care. Circular Cardiovascular Quality Outcomes, 7(1), 2–4. https://doi.org/10.1161/circoutcomes.113.000829.
Snyder, C. F., Smith, K. C., Bantug, E. T., Tolbert, E. E., Blackford, A. L., & Brundage, M. D. (2017). What do these scores mean? Presenting patient-reported outcomes data to patients and clinicians to improve interpretability. Cancer, 123(10), 1848–1859. https://doi.org/10.1002/cncr.30530.
This study was conducted as part of the Duke PEPR Center, which is funded by the National Institutes of Health through the following grant administered by the National Institute of Arthritis and Musculoskeletal and Skin Diseases: U19AR069519. The research reported is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. We would like to acknowledge the adolescents, parents, and clinicians who participated in this study. We are grateful for their contributions. We would also like to acknowledge the Childhood and Rheumatology Research Alliance (CARRA) and the Arthritis Foundation for their help setting up bookmarking workshops at their annual meetings and conferences.
Conflict of interest
Laura Schanberg, MD, Sarah Ringold, MD, and Emily von Scheven work for the Childhood Arthritis and Rheumatology Research Alliance (CARRA). Dr. Schanberg is a former Board Chair and currently sits on the Registry and Research Oversight Committee along with Dr. Sarah Ringold. Dr. Emily von Scheven will be the next Board Chair for CARRA. All other authors declare no conflict of interest.
All procedures performed in studies involving human participants were conducted in accordance with ethical standards of the institutional review board (Duke Health IRB, Pro00091284 and UNC IRB #15-2442 and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.
Informed consent was obtained from all individual participants included in the study. For participants under the age of 18, caregiver consent was obtained in addition to child assent.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Electronic supplementary material
Below is the link to the electronic supplementary material.
Supplementary file1 (PDF 400 kb)
About this article
Cite this article
Mann, C.M., Schanberg, L.E., Wang, M. et al. Identifying clinically meaningful severity categories for PROMIS pediatric measures of anxiety, mobility, fatigue, and depressive symptoms in juvenile idiopathic arthritis and childhood-onset systemic lupus erythematosus. Qual Life Res 29, 2573–2584 (2020). https://doi.org/10.1007/s11136-020-02513-6
- Patient-reported outcomes
- Juvenile idiopathic arthritis
- Systematic lupus erythematosus
- Reference values