Advertisement

BMC Psychiatry

, 19:17 | Cite as

Patient-reported experience and quality of care for people with schizophrenia

  • L. AimolaEmail author
  • J. Gordon-Brown
  • A. Etherington
  • K. Zalewska
  • S. Cooper
  • M. J. Crawford
Open Access
Research article
Part of the following topical collections:
  1. Social psychiatry, therapy and provision of mental health care

Abstract

Background

Evidence is mounting that patient-reported experience can provide a valuable indicator of the quality of healthcare services. However, little is known about the relationship between the experiences of people with severe mental illness and the quality of care they receive. We conducted a study to examine the relationship between patient-reported experience and the quality of care provided to people with schizophrenia.

Methods

We calculated a composite global rating of quality of care for people with schizophrenia using data from an audit of 64 mental health providers. We then examined associations between these ratings and mean patient satisfaction and patient-rated outcome using data from a survey of 5608 schizophrenic patients treated in these services.

Results

Global rating of quality of care was positively correlated with patient-rated outcome (r = 0.33; p = 0.01) but not with patient satisfaction (r = 0.21, p = 0.10). Patient-rated outcome was also positively correlated with patient involvement (r = 0.26, p = 0.04) and the quality of prescribing practice (r = 0.31, p = 0.02). High patient satisfaction scores were significantly associated with the extent of use of care plans within each organisation (r = 0.27, p = 0.03).

Conclusions

Among people with schizophrenia, patient-rated outcome provides a better guide to the quality of care than patient-rated satisfaction. Greater use of patient-reported outcome measures should be made when assessing the quality of care provided to people with psychosis.

Keywords

Psychosis Patient satisfaction Patient outcome measures Quality improvement 

Background

Improving the quality of care that patients receive is a key objective for healthcare providers across the world. As well as safety and clinical effectiveness, patients experience of care is a key component of most definitions of quality of healthcare [1].

In recent years evidence has begun to emerge that patient-reported experience can offer a reliable indicator of the quality of care that services provide [2, 3, 4]. A systematic review summarising evidence from 55 studies, showed consistent positive associations between patient-reported experience and clinical effectiveness [2]. However, most studies included in this systematic review focused on patients using primary care and general hospital services. Only two studies included data from patients using mental health services and one study specifically excluded patients with a psychiatric diagnosis. It has been argued that assessment of patient experience of mental health services is challenging because people may not have sufficient information about what such services are able to provide [5], and because severe mental illness may impair a person’s insight [6].

Small scale studies have examined the relationship between patient-reported experience and other measures of quality of care. For instance, Shipley and colleagues [7] investigated the relationship between the quality of a community mental health service and patient-rated satisfaction with the care they received. Based on data from 113 randomly selected patients referred to this service over a two-month period, the authors found that patient-reported experience of care provided a more accurate indicator of quality of care than standard quality indicators such as waiting times. Although the results of this study support the use of patient experience as a measure of service quality in mental health settings, the generalisability of the results of study are limited by the small sample size and single site.

The second round of the National Audit of Schizophrenia conducted between 2013 and 2014 assessed the quality of care provided to patients with schizophrenia across all the Mental Health Trusts in England and Health Boards in Wales. As the audit provided data on a wide range of outcomes of care for patients with schizophrenia around the county, we set out to use data from this audit to examine whether patient-reported experience provides an indication of the services’ overall quality of care in a nationwide cohort of patients with severe mental health illness.

Methods

The National Audit of Schizophrenia provided an in-depth examination of service performance against 16 standards of care derived from national guidelines for the treatment of people with schizophrenia in England [8]. These standards concern the quality of physical health monitoring, prescribing practice, use of evidence-based psychological treatments and patient involvement in care planning. Our study comprised a secondary analysis of cross-section data from an audit of clinical records of the quality of care provided to people with psychosis and the results of a patient survey that was conducted in parallel with the audit.

Data collection

A total of 64 NHS Mental Health Trusts in England and Health Boards in Wales (referred to collectively as ‘Trusts’ in the remainder of this report) participated in the audit. Data were collected via an audit of practice tool and a contemporaneous patient survey. The audit of practice tool was developed specifically to measure standards of care and the relevant data were obtained from the patients’ case records (see Appendix for a list of the standards). The tool was completed by consultants, junior doctors and other allied health professionals within the services. The patient survey was developed to assess patient-reported experience of care. An expert patient reference group helped refine draft versions of both the audit of practice tool and the patient survey prior to the start of the audit.

Trusts were asked to complete the audit of practice tool for a random sample of 100 patients. In addition, Trusts were required to distribute the patient survey to 200 randomly selected patients including the 100 patients whose notes were examined using the audit of practice tool. The data collection was conducted between August 2013 and November 2013.

Patient sample

Patients were eligible for inclusion in the audit if they were aged above 18, with a current ICD-10 diagnosis of schizophrenia or schizoaffective disorders. To be eligible for the audit patients had to have received treatment from community mental healthcare setting within the Trust for at least 1 year.

A composite measure of global trust performance

For each Trust, we calculated the proportion of patients for whom each individual standard was met and the mean percentage of compliance for each standard relative to the total national sample. We then calculated a global trust rating score based on the mean percentage of compliance of each Trust across all standards assessed with the audit of practice tool. We also calculated five sub-scores from standards relating to:
  1. 1).

    The quality of physical health monitoring the patient received (e.g. assessments and interventions delivered).

     
  2. 2).

    The quality of patient involvement (e.g. whether there was shared decision making process in the selection of medication).

     
  3. 3).

    Compliance with recommended prescribing practice (e.g. extent of polypharmacy and prescribing above BNF recommended maxima).

     
  4. 4).

    Access and uptake of evidence-based psychological treatments.

     
  5. 5).

    Provision of a care plan jointly developed with the patients to meet their needs.

     

Patient-reported experience measures

The patient-reported experience measures we used included a measure of patient satisfaction with care and a patient-rated outcome measure. The question on patient satisfaction with care asked: “Taking everything into consideration, are you pleased with the care you have received from the service so far?” Responders had to indicate their answer by using a four-points Likert scale from 1 (not satisfied at all) to 4 (very satisfied). The question on patient-rated outcome asked: “To what extent have services helped you to achieve good mental health in the last year?” and patients responded using a four-point Likert scale ranging from 1 (made me worse) to 4 (helped a lot). We calculated the mean trust scores for these two questions and converted them into a 100-point scale.

Statistical analysis

We carried out Pearson product-moment correlations to investigate associations between patient satisfaction and patient-rated outcome scores with the global trust performance and subgroups of care scores [9].

Ethics approval and consent to participate

The National Audit of Schizophrenia (NAS) was commissioned by the Healthcare Quality Improvement Partnership (HQIP) as part of the National Clinical Audit Programme (NCA). As part of NCA, all Mental Health Trusts in England and Wales were expected to take part in NAS. Consent was not required from the patients to publish the audit data in the audit’s main report or for research purposes and the data were collected by the audit team in such a manner that the patients could not be directly identified.

Results

The audit team received 5733 returns from the audit of practice, of which 5608 were used in the analysis after data cleaning with a mean of 88 (SD = 17) returns per Trust. In total, 3379 patients submitted a survey form with a mean of 53 (SD = 19) per Trust and a response rate of 26.4%. Table 1 shows a breakdown of the demographic characteristics of the patients included in the audit of practice according to whether they did or did not respond to the survey. The demographic data on the responders and non-responders to the patient survey refer to the only two Trusts that were able to collect this information. The responders and not-responders group did not differ with regard to gender, ethnicity, and time since diagnosis.
Table 1

Demographic characteristics of the patients included in the audit and of those who responded and did not respond to the survey

Demographic characteristics

Patients included in the audit (n = 5608)

Responders to the survey (n = 61)a

Non responders (n = 240)a

Difference in proportions (95% CI)

p value

Age mean (Standard Deviation)

46 (13)

51 (11)

49 (12)

−2.0 (−5.3 to 1.4)

0.249

Gender n (%)

 Male

3655 (65.2%)

45 (73.8%)

153 (63.8%)

10.0

0.174

 Female

1949 (34.8%)

16 (26.2%)

87 (36.3%)

(−3.5 to 21.3)

 Not stated

4 (< 0.1%)

0 (0.0%)

0 (0.0%)

 

Ethnicity n (%)

 White

4400 (78.5%)

48 (78.7%)

159 (66.3%)

12.4 (−0.7 to 22.9)

0.500

 Asian or Asian British

446 (8.0%)

3 (4.9%)

21 (8.8%)

−3.8 (− 9.2 to 5.2)

 Black or Black British

454 (8.1%)

8 (13.1%)

41 (17.1%)

−4.0 (−12.2 to7.5)

 Chinese or other

108 (1.9%)

1 (1.6%)

6 (2.5%)

−0.9 (−6.4 to 4.0)

 Mixed

116 (2.1%)

0 (0.0%)

5 (2.1%)

−2.1 (−4.8 to 4.0)

 Not stated

84 (1.5%)

1 (1.6%)

8 (3.3%)

−1.69 (−5.6 to 5.1)

Time since diagnosis

 Between 1 and 2 years

226 (4.0%)

0 (0.0%)

4 (1.7%)

−1.7 (−4.4 to 4.2)

0.183

 From 2 to 4 years

495 (8.8%)

3 (4.9%)

22 (9.2%)

−4.3 (−9.6 to 4.9)

 From 4 to 10 years

1353 (24.1%)

12 (19.7%)

67 (27.9%)

−8.2 (4.5 to −18.3)

 More than 10 years

3534 (63.0%)

46 (75.4%)

147 (61.3%)

14.1 (0.7 to 25.2)

Care team n (%)

 Assertive Outreach

689 (12.2%)

3 (4.9%)

17 (7.1%)

−2.2 (−6.8 to 7.3)

0.085

 Community Team

4035 (72.0%)

57 (93.4%)

194 (80.8%)

12.6 (2.4 to 19.4)

 Crisis Resolution

13 (0.2%)

0 (0.0%)

0 (0.0%)

 Early Intervention

239 (4.3%)

0 (0.0%)

4 (1.7%)

−1.7 (−4.3 to 4.2)

 Other

632 (11.3%)

1 (1.6%)

25 (10.4%)

−8.8 (−13.5 to 1.0)

a Data in these two columns refer only to the two trusts which collected information on responders and non-responders

Association between patient-reported experience measures and overall trust rating

There was a positive significant correlation between the patient satisfaction and the patient-rated outcome measure (rs = 0.582, p < 0.01). In Table 2, we present (i) correlations between patient-rated outcome and quality of care data from the audit, and (ii) correlations between patient-rated satisfaction and quality of care data from the audit. While patient-rated outcome scores were significantly associated with the global trust rating, patient satisfaction scores were not. Figure 1 shows the scatterplot of the relationship between global trust rating and patient-rated outcome about their mental health.
Table 2

Correlations between the global trust rating and its subgroups with patient satisfaction and patient experience

Composite measure

Correlation coefficient (p value)

Patient satisfaction

Patient-rated outcome

Global trust rating

0.209 (0.098)

0.329** (0.008)

Physical health monitoring

0.169 (0.182)

0.199 (0.114)

Patient involvement

0.149 (0.241)

0.263* (0.036)

Pharmacotherapy

0.106 (0.404)

0.310* (0.013)

Psychotherapy

−0.024 (0.850)

−0.002 (0.990)

Care planning

0.274* (0.029)

0.026 (0.838)

**p < 0.01

*p < 0.05

Fig. 1

Scatterplot showing a positive linear correlation between global trust rating scores and patient-rated outcome scores among

The results relating to the five subgroups of patient care we examined are also presented in Table 2. Only the provision of a care plan was significantly correlated with high patient satisfaction scores. In contrast, patient-rated outcome scores were positively correlated with patient involvement and with the services’ compliance with recommended prescribing practice. No significant associations were observed between the patient-reported experience measures and the other subgroups of patient care.

Discussion

Secondary analysis of data from this national audit of care for people with schizophrenia found that patient-rated outcome was positively associated with an independent rating of the global quality of care provided by these services. In contrast, we did not find an association between global quality of care and the patients’ satisfaction with the service they received. High scores on patient-rated outcome were also correlated with high patient involvement in the decisions about their care and with high compliance of the services with the recommended prescribing practice. In contrast, high levels of patient satisfaction were associated only with the provision of a care plan tailored to the patients’ needs. Nonetheless the strength of these associations are not strong and caution needs to be exercised in interpreting the results of this analysis.

While the previous work of Doyle and colleagues [2] demonstrated consistent positive associations between patient-reported experience and clinical effectiveness, our study demonstrates that these associations are also true of users of mental health services. The idea that patient-reported experience of care could be a valuable indicator of quality of care in mental health services has been previously tested only in small studies with patients from a single service provider [7]. Our study builds up on this work and counters previously reported concerns about the ability of people with mental disorder to provide valid feedback about the quality of care they receive [10].

The main strength of this study is that we were able to examine the relationship between patient experience and quality of care across a large number of providers of mental health services throughout England and Wales. However, the study also has a number of limitations which need to be considered when interpreting the results. Firstly, the audit used two different samples for collecting case note data and for the patient survey, which did not allow us to determine whether individual patient ratings related to the quality of care provided to them personally by their Trust. The use of aggregate data from different samples of patients within each Trust may have underestimated the strength of the associations we might have seen, had both sets of data come from the same patient sample.

A second limitation of the study was that the measures of patient-rate experience and the global trust rating we used have not been validated. The global rating of trust quality of care was developed from standards for the treatment and management of people with schizophrenia published by the National Institute for Health and Clinical Excellence following a systematic review of published literature and extensive consultation with patients and providers [8]. However, the construct validity of the patient-rate experience questions and the global trust rating we used have not been tested. A further limitation of the patient-rate experience questions was that they were based on a single item.

While we found a moderate degree of association between patient satisfaction and patient-reported outcome in our analysis, only the latter was correlated with the independent rating of global quality of care provided Trusts. This suggests that patient-rated outcomes might be a better indicator of the quality of care that a service provides than patient-rated satisfaction with care. We also found that the two measures of patient-rated experience were associated with different elements of care delivered by participating Trusts. The more the services promoted the patients’ involvement in decisions about care and higher levels of compliance with recommended prescribing practice were associated with patients being more likely to state that services were improving their mental health. On the other hand, services that had higher compliance with recommendations that patients should have a care plan that was jointly developed to meet their needs had higher ratings on patient satisfaction with care.

These results support previous research in other healthcare settings which indicate that patients rate their experience and outcomes of care according to different criteria and highlight the importance of both in assessing the overall quality of care that healthcare services provide [11, 12].

In recent years concerns have been expressed about the burden that providers of healthcare services face in assuring the quality of care they deliver [13, 14]. The results of this study, together with other reports linking patient experience with service quality and safety [15, 16] suggest that greater use of patient surveys could be made in assuring the quality of healthcare and reducing the burden of data collection on service providers and assisting efforts to improve the quality of care that patients receive [17].

Conclusion

In summary our findings highlight that, feedback from patients with severe mental disorders can provide important information about the quality of care they receive. Therefore, greater use of patient-reported experience measures should be made when assessing the quality of care provided to people with psychosis. Future work should focus on exploring the best way to combine patient views with audit data to obtain a comprehensive picture of the quality of care that services provide.

Notes

Acknowledgments

We would like to thank staff in participating Trusts and Health Boards for their hard work and engagement in submitting data for the second round of the NAS. Also, we would like to thank the members of the NAS Advisory Group for their recommendations and the Healthcare Quality Improvement Partnership (HQIP) for their support. Thanks to the expert patient reference group for their ideas and views on how to engage patients with the audit and for helping with the development of the patient questionnaire. Thanks also to the patients who took part in the survey for their contribution.

Funding

Not applicable.

Availability of data and materials

The datasets used and analysed during the current study are available from the corresponding author on reasonable request.

Disclaimer

The National Audit of Schizophrenia (NAS) is managed by the Royal College of Psychiatrists’ College Centre for Quality Improvement. It is commissioned by the Healthcare Quality Improvement Partnership as part of the National Clinical Audit and Patient Outcomes Programme. The views expressed in this publication are those of the authors and not necessarily those of the NHS, the National Institute for Health Research or the Department of Health.

Authors’ contributions

KZ provided project management support to deliver the audit (recruitment, data collection, report writing up). SC and MC advised on the aspects of care to include in the audit tool, the design of the study and the interpretation of the results. EA advised on the content of the patient survey. LA and GJ conducted the data analysis and drafted the manuscript. All authors read and approved the final manuscript.

Ethics approval and consent to participate

The National Audit of Schizophrenia (NAS) was commissioned by the Healthcare Quality Improvement Partnership (HQIP) as part of the National Clinical Audit Programme (NCA). As part of NCA, all Mental Health Trusts in England and Wales were expected to take part in NAS. Consent was not required from the patients to publish the audit data in the audit’s main report or for research purposes and the data were collected by the audit team in such a manner that the patients could not be directly identified.

Consent for publication

Not applicable.

Competing interests

The authors report no competing interest. The authors alone are responsible for the content and writing of the article.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

References

  1. 1.
    Department of Health. High quality care for all: NHS next stage review final report. London. 2008.Google Scholar
  2. 2.
    Doyle C, Lennox L, Bell D. A systematic review of evidence on the links between patient experience and clinical safety and effectiveness. BMJ Open. 2013;3(1).Google Scholar
  3. 3.
    Isaac T, Zaslavsky AM, Cleary PD, Landon BE. The relationship between patients’ perception of care and measures of hospital quality and safety. Health Serv Res. 2010;45(4):1024–40.CrossRefGoogle Scholar
  4. 4.
    Druss BG, Rosenheck RA, Stolar M. Patient Satisfaction and Administrative Measures as Indicators of the Quality of Mental Health Care. 1999; 50(8): 1053–1058.Google Scholar
  5. 5.
    Williams B, Wilkinson G. Patient satisfaction in mental health care: evaluating an evaluative method. Br J Psychiatry. 1995;166:559–62.CrossRefGoogle Scholar
  6. 6.
    Jones K. Book review. Sociological Review. 1962;8:343–4.CrossRefGoogle Scholar
  7. 7.
    Shipley K, Hilborn B, Hansell A, Tyrer J, Tyrer P. Patient satisfaction: a valid index of quality of care in a psychiatric service. Acta Psychiatr Scand. 2000;101(4):330–3.CrossRefGoogle Scholar
  8. 8.
    The National Institute for Health and Care Excellence (NICE). Schizophrenia: Core interventions in the treatment and management of schizophrenia in primary and secondary care. London: Clinical Guideline 82; 2009.Google Scholar
  9. 9.
    Altman DG. Practical statistics for medical research. Chapman and Hall/CRC, Boca Raton, London, New York, Washington, 1991, DC.Google Scholar
  10. 10.
    Crawford MJ, Kessel AS. Not listening to patients-the use and misuse of patient satisfaction studies. Int J Soc Psychiatry. 1999;45(1):1–6.CrossRefGoogle Scholar
  11. 11.
    Black N, Varaganum M, Hutchings A. Relationship between patient reported experience (PREMs) and patient reported outcomes (PROMs) in elective surgery. BMJ Qual Saf. 2014;23(7):534–42.CrossRefGoogle Scholar
  12. 12.
    Fremont AM, Cleary PD, Hargreaves JL, Rowe RM, Jacobson NB, Ayanian JZ. Patient-centred processes of care and long-term outcomes of myocardial infarction. J Gen Intern Med. 2001;16:800–8.CrossRefGoogle Scholar
  13. 13.
    Molloy A, Martin S, Gardner T, Leatherman SA. Clear road ahead: creating a coherent quality strategy for the English NHS. London: The Health Foundation; 2016.Google Scholar
  14. 14.
    Erickson SM, Rockwern B, Koltov M, McLean RM. Putting patients first by reducing administrative tasks in health care: a position paper of the American College of Physicians. Ann Intern Med. 2017;166:659–61.CrossRefGoogle Scholar
  15. 15.
    Kemp KA, Santana MJ, Southern DA, McCormack B, Quan H. Association of inpatient hospital experience with patient safety indicators: a cross-sectional, Canadian study. BMJ Open. 2016;6(7):e011242.CrossRefGoogle Scholar
  16. 16.
    Griffiths A, Leaver MP. Wisdom of patients: predicting the quality of care using aggregated patient feedback. BMJ Qual Saf. 2017;0:1–9.Google Scholar
  17. 17.
    Coulter A, Locock L, Ziebland S, Calabrese J. Collecting data on patient experience is not enough: they must be used. BMJ. 2014;348:g2225.CrossRefGoogle Scholar

Copyright information

© The Author(s). 2019

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Authors and Affiliations

  • L. Aimola
    • 1
    Email author
  • J. Gordon-Brown
    • 1
  • A. Etherington
    • 2
  • K. Zalewska
    • 2
  • S. Cooper
    • 2
  • M. J. Crawford
    • 1
    • 2
  1. 1.The Centre for Psychiatry, Imperial College London, Hammersmith CampusLondonUK
  2. 2.The Royal College of Psychiatrists’ College Centre for Quality ImprovementLondonUK

Personalised recommendations