Investigating the patient acceptable symptom state cut-offs: longitudinal data from a community cohort using the shoulder pain and disability index
- 50 Downloads
This prospective study aimed to determine the patient acceptable symptom state (PASS) cut-off for the patient reported outcome measure shoulder pain and disability index (SPADI), and evaluate predictors of PASS achievement following standard shoulder care. Patients with shoulder pain, referred for shoulder ultrasound were recruited from a community cohort. Patients completed both SPADI (scored 0–130) and a question on symptom state and followed-up at 6 months. PASS was calculated from Rasch-transformed scores using 2 methods: the 75th percentile of the cumulative response curve and the receiver operating characteristic curve (ROC). Logistic regression was used to identify factors associated with PASS. 304 participants (169 females, mean age 57.2 years) were included. At 6 months, 193 (63%) reported PASS. The association between SPADI at 6 months and PASS depended on baseline SPADI (interaction p = 0.036). Those with higher baseline scores had higher 6 months PASS cut-offs. Using the 75th percentile method, the 6 months total SPADI cut-off was 49.2 in those starting in the highest tertile at baseline compared to 39.4 in the lowest tertile: 46.4 vs. 36.7 for pain, 46.8 vs. 25.1 for disability. The ROC method yielded similar results. We have shown for the first time that the PASS cut-off for SPADI is dependent on baseline severity scores. Understanding the SPADI PASS threshold is important for clinical research to allow standardised reporting of shoulder intervention success at the patient level.
KeywordsShoulder Pain Outcome assessment (Health care)
Shoulder pain is common and often poorly treated . Patient-reported outcome measures (PROMs)  are used to assess symptomatic and functional change in shoulder disorders. For example, the shoulder pain and disability index (SPADI) is commonly used in clinical trials, comprising five pain and eight disability questions with severity recorded on ordinal scales . It is reliable, responsive and has good construct validity. While analysis of ordinal PROM scores may provide discriminatory ability between treatments at group level, this may not reflect therapeutic success for the individual. The patient acceptable symptom state (PASS) cut-off is the value on a PROM beyond which the patient feels good enough to continue in that state [4, 5]. Establishing PROM cut-offs for PASS would indicate therapeutic success at the individual level and provide a tool for standardising responder rates in clinical trials.
There are limited data for PASS values for shoulder outcomes  with existing studies assessing the cross-sectional relationship between PASS and PROMs, including the American shoulder and elbow surgeons (ASES) score, the simple shoulder test (SST), and the Visual Analog Scale (VAS) pain score [6, 7], with only one assessing SPADI in a shoulder pain cohort awaiting surgery .
There are no prospective longitudinal studies investigating SPADI PASS cut-offs. We aimed to determine PASS cut-offs using two different methods and assess whether these values were influenced by time or other covariates.
Materials and methods
Patients care with shoulder symptoms are initially managed in the community, but if they continue to have problems, they are referred for a diagnostic ultrasound scan. Patients attending a hospital, outpatients’ clinic, for their first shoulder scan were recruited from 2016 to 2017. Inclusion criteria were: age > 18 years, ability to provide informed consent, first ever ultrasound scan of shoulder (determined by clinical records). Exclusion criteria were: inflammatory arthritis; previous fracture/dislocation/surgery of the affected shoulder; steroid injection or physiotherapy for target shoulder within prior 6 weeks (to exclude effects of treatment, which are short term , on baseline symptoms); or complex regional pain syndrome. To avoid recruitment bias, quota sampling was used in four categories: gender (male/female) and age (younger/older split at the median); target proportions of these categories were determined from a previous study : males aged < 54 (25%), males aged ≥ 54 (22%), females aged < 54 (25%), females aged ≥ 54 (28%).
Patients completed standardised questionnaires at baseline and 6 months, assessing age (years), gender, body mass index (BMI), duration of symptoms, pain status (yes/no), other painful body joints, hospital anxiety and depression scale (HADS)  and SPADI. Treatments received (physiotherapy, corticosteroid injections and/or surgery) prior to initial scan were recorded at baseline and treatments received during the study were collected at 6 months. Ultrasound findings were collected to understand the aetiology of shoulder pain.
To determine PASS, patients were asked “in the next few months, if you were to remain as you were during the last 48 h, would this be acceptable or unacceptable to you?” This is the wording recommended by an Outcome Measures in Rheumatology (OMERACT) special interest group  and dichotomises each participant into PASS positive (PASSpos) or negative (PASSneg).
SPADI scores were transformed to interval scaling by assessing fit to the Rasch measurement model in RUMM 2030 software [13, 14].This process formally tests that the data meet minimum requirements for measurement then allows transformation from ordinal to interval linear scaling [15, 16], facilitating parametric analysis . Comparisons were made between included and excluded patients using Student’s t, Mann–Whitney U or Chi-square tests according to data type and distribution; visual inspection of approximate fit to the normal distribution was performed for continuous variables. PASS cut-offs were calculated using the 75th percentile of the cumulative percentage curve of patients who were PASSpos  and the point on the receiver operating characteristic (ROC) curve, which maximised the Youden index [18, 19]. The diagnostic accuracy of each method was reported. We simulated some data to investigate potential differences between the 75th percentile and ROC methods (see Online Resources 3 for more details).
The association between PASSpos/PASSneg and SPADI was assessed using binary logistic regression. An interaction was added between baseline and 6 month SPADI to evaluate if 6 month PASS cut-off depended on baseline score. Interactions with other baseline covariates were also explored; main effects and interactions with 6 month SPADI scores were added for age, gender, baseline painful joint count and baseline HADS scores. If an interaction was not statistically significant, both the interaction and the main effect were removed. To illustrate the influence of baseline score on PASS at 6 months, cut-offs were calculated separately within tertiles of baseline SPADI. SPADI reliability was evaluated using standard error of measurement (SEM) and smallest detectable change (SDC) calculated using Cronbach’s alpha.
SPADI showed acceptable fit to the Rasch model (Online Resource 1). To create a unidimensional total score combining the pain and disability subscales, 7% of the unique variance was lost in a bi-factor equivalent solution. There was no evidence of substantive differential item functioning by age, sex, shoulder side or visit, the latter indicating that the measurement scale remained invariant over time.
Baseline characteristics of those included in or excluded from longitudinal analysis (total n = 500)
Data at both time points
n = 304
Missing data at BL or FU
n = 196
Female, N (%)
chi-sq = 4.95
Age, years, mean ± SD
57.2 ± 13.5
47.9 ± 14.3
t = 7.33
BMI, kg/m2, mean ± SD
27.9 ± 5.2, n = 291
27.1 ± 5.0, n = 170
t = 1.59
Pain duration, months, median (IQR)
6 (3–10), n = 225
5 (3–10), n = 136
z = 1.43
Number of painful joints, median (IQR)
z = 0.33
Injection received, N (%)
chi-sq = 0.22
HADS anxiety, median (IQR)
z = − 3.32
HADS depression, median (IQR)
z = − 3.97
Possible depression (HADS depression 8–10), N (%)
Probable depression (HADS depression > 10), N (%)
Bilateral shoulder pain, yes, N (%)
chi-sq = 0.01
PASS, positive, N (%)
chi-sq = 0.87
SPADI total, mean ± SD
50.9 ± 9.6
52.3 ± 9.9, n = 192
t = − 1.55
SPADI pain, mean ± SD
49.0 ± 13.5
50.6 ± 13.4, n = 192
t = − 1.32
SPADI disability, mean ± SD
45.2 ± 14.0
46.4 ± 15.8, n = 189
t = − 0.95
Unadjusted PASS cut-offs for SPADI (n = 304)
75th percentile method
AUC (95% CI)
0.77 (0.70, 0.83)
Total 6 months
0.87 (0.83, 0.91)
0.79 (0.72, 0.85)
Pain 6 months
0.88 (0.84, 0.92)
0.74 (0.67, 0.80)
Disability 6 months
0.85 (0.80, 0.89)
This is the first study to evaluate the PASS threshold for SPADI longitudinally in a community-based cohort. We found that PASS cut-offs varied between the ROC and 75th percentile methods. PASS cut-offs did not change substantively over time. However, the PASS cut-off on the total SPADI score at 6 months was dependent on the baseline score; this was primarily driven by the disability subscale, rather than the pain subscale.
Our SPADI PASS cut-offs differed from the only other study assessing PASS and SPADI, although that cross-sectional cohort recruited post-surgical patients . Our cut-offs also varied according to the methods used; the optimum methods for calculating PASS are unknown and may depend on the purpose . The 75th percentile method may be inappropriate for determining PASS changes longitudinally, because the scores within those reporting PASS may decrease, affecting the location of the 75th percentile. PASS provides useful information on therapeutic success at the individual level, but does not reflect the desire to improve: patients may define their state as acceptable, but want to feel better. Given the negative and positive predictive values from our study (Table 2), the SPADI PASS cut-off may be more useful as a descriptive outcome for trials rather than a treatment target.
Age and gender did not modify the association between 6 month SPADI and 6 month PASS after adjusting for baseline score; however, these tests may have lacked power as testing for these interactions would require higher number of participants. Previous studies using different PROMs found age and gender affected PASS cut-off, but did not test for interactions or adjust for other covariates or baseline score [20, 21].
Disease adaptation may alter interpretation of a questionnaire, thus changing a patient’s individual PASS threshold over time. This response shift can only be assessed if the measurement properties of the questionnaire itself are time invariant, which we confirmed for SPADI. Patients with worse symptoms at baseline reported higher PASS cut-offs at 6 months. Existing research, using different scales, found that those with worse initial activity limitation in acute rotator cuff syndrome were shown to have higher PASS cut-offs at follow-up, possibly due to adaptation, whilst their pain cut-offs did not differ by baseline score, consistent with our findings . Another study reported data consistent with this trend in those undergoing hip replacement . Studies have found PASS cut-offs for other scales to be stable over time [20, 24], whereas another found thresholds declined . PASS cut-off was reported to be stable after 6 months from knee replacement . This variation in findings may be due to treatment expectation or PASS thresholds may be disease or treatment specific. Another source of variation might be the use of the 75th percentile method to identify changes in PASS cut-off over time, which, as we have shown in Online Resource 3, can give variable results depending on the level of change in symptoms, even if there is actually no change in ‘true’ PASS cut-off.
We did not adjust our analysis for treatments received between baseline and follow-up, because we needed to account for the total effect of baseline SPADI on the association between SPADI and PASS at 6 months. The level of baseline symptoms would potentially influence treatments received post-baseline; therefore, including such treatments in the model may distort this association.
This paper has several strengths. While previous studies used different wordings and time anchors for PASS [7, 10] we used the timeframe of “next few months” recommended by OMERACT . Standardisation of the wording is important for comparison across studies and variations may influence results . SPADI underwent Rasch transformation enabling inferential testing by converting ordinal scores to interval scaling and our measures showed very good reliability. Other studies did not transform their data; therefore, they may have underestimated or overestimated the extent of cut-off dependence on initial values due to the nonlinearity of estimates obtained from these PROMs.
There are limitations to this study. The 6 months response rate was 67%. However, this is better than a published overall rate for 1607 surveys of 48.3% . Our non-responders were on average younger with more anxiety and depression, consistent with published predictors of response to surveys . However, the baseline SPADI and PASS were similar between responders and non-responders. Pain duration, BMI, injections received and sex were also similar between the two groups. This study involved self-completion of questionnaires, which may introduce recall bias.
This study has shown for the first time that the PASS cut-off for SPADI varies depending on calculation method and symptom severity at baseline. Understanding the PASS threshold is important for clinical research as the relationship between SPADI and objective disease parameters is ill-defined. Further work in different centres is required to substantiate these findings.
Availability of data and materials
The data that support the findings of this study are available from the corresponding author upon reasonable request.
This article presents independent research funded by the National Institute for Health Research (NIHR) Leeds Biomedical Research Centre (BRC) and by a NIHR Doctoral Research Fellowship (GT; DRF-2016-09-159). The views expressed are those of the authors and not necessarily those of the NIHR or the Department of Health and Social Care. This study was also part-funded through the Arthritis Research UK Leeds Experimental Osteoarthritis Treatment Centre (20083).
All authors contributed to the study conception and design. Material preparation, data collection and analysis were performed by GT, BD and EMAH. The first draft of the manuscript was written by GT and BD and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.
Compliance with ethical standards
Conflict of interest
The authors declare they have no conflicts of interest.
Ethics approval was obtained from North East-Newcastle and North Tyneside 2 Research Ethics Committee (ethics approval protocol number: 16/NE/0108) and informed consent was obtained from all participants for publication.
Human and animal rights
All human studies have been approved by the appropriate ethics committee and have therefore been performed in accordance with the ethical standards laid down in the 1964 Declaration of Helsinki and its later amendments.
- 13.Rasch G (1980) Probabilistic models for some intelligence and attainment test. Expanded edition with foreword and afterword by B.D. Wright. The University of Chicago Press, ChicagoGoogle Scholar
- 14.Andrich D, Sheridan B, Luo G (2015) RUMM2030: An MS Windows computer program for the analysis of data according to Rasch unidimensional models for measurement. RUMM Laboratory, PerthGoogle Scholar
- 15.Rasch G (1960) Studies in mathematical psychology: I. Probabilistic models for some intelligence and attainment tests. Nielsen & Lydiche, Oxford, EnglandGoogle Scholar
- 21.Maksymowych WP, Gooch K, Dougados M, Wong RL, Chen N, Kupper H et al (2010) Thresholds of patient-reported outcomes that define the patient acceptable symptom state in ankylosing spondylitis vary over time and by treatment and patient characteristics. Arthritis Care Res 62:826–834CrossRefGoogle Scholar
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.