Background

The Institute of Medicine has defined high quality health care as the provision of appropriate services in a technically competent manner, and includes good communication, shared decision-making and is consistent with patient values and preferences [1]. Optimizing the structure (e.g., hospital resources, number of staff), processes (e.g., interactions between health care providers and patients, use of effective therapies) and outcomes (e.g., survival, quality of life) of health care services are crucial to achieving high quality care [1]. In 2001, the IOM published “Crossing the Quality Chasm” a broad framework which recommended improvements to the following six areas of healthcare in order to achieve high quality care: safety; effectiveness; timeliness; efficiency; equity; and patient-centredness [1]. Within the area of patient-centredness, the IOM also endorsed Gerteis et al’s six dimensions of patient-centred care [2] which state that care must be: 1) respectful to patients’ values, preferences, and expressed needs; 2) coordinated and integrated; 3) provide information, communication, and education; 4) ensure physical comfort; 5) provide emotional support—relieving fear and anxiety; and 6) involve family and friends [1]. The IOM’s recognition of patient-centredness as an indicator of quality acknowledges the adoption of a whole-person orientation to healthcare that goes beyond solely focusing on treatment of the disease.

A variety of sources have been used to assess the quality of care that patients receive including administrative databases, cancer registries, medical records, patient self-reported measures, physician surveys, and pharmacy and laboratory data [3]. However unlike other aspects of quality, such as efficiency, patient self-report is arguably the only way to assess constructs that relate to patient-centredness. For instance, the severity of cancer pain and levels of fatigue experienced by a patient can only reliably be assessed by the patient themselves, and self-report is widely recognised as the gold standard for such assessments [4]. The value of obtaining patient self-report data is further demonstrated by research reporting that patients’ perceptions of quality of health care have been associated with important medical and psychological outcomes, including quality of life [5–8], anxiety and depression [6–9]. Patients’ perceptions of quality of care have also been associated with factors that directly affect the effectiveness and efficiency of health care such as the under-utilisation of treatments [10–12] and mistrust of the medical system [13, 14].

Patient-reported outcome measures (PROMs) that have been designed to assess the quality of patient-centred care include measures of: 1) satisfaction with care; and 2) experiences of care. Satisfaction with care measures investigate the extent to which an individual’s health care experiences met his/her expectations [15]. However, a range of factors unrelated to the actual health care that was delivered, such as differences among patients’ expectation levels, can cause variability in satisfaction ratings, which reduce their reliability for widespread and ongoing monitoring of attempts to improve patient-centred care [15]. In contrast, experiences of care measures ask patients to indicate what actually happened during the process of care delivery, and so are less influenced by subjective patient expectations and provide more detailed information to health care providers and systems about where quality improvements are needed [16, 17]. However, in order to accurately reflect the quality of care received and identify variations in patients’ experiences, PROMs should meet recommended psychometric criteria for reliability (internal consistency, test re-test reliability), and validity (face, content, construct validity) [18–24].

There are few existing reviews that have assessed the psychometric properties of measures developed to identify patients’ experiences of care across a range of settings and diseases [25–28]. Only one of these reviews evaluated the psychometric properties of quality of care measures designed specifically for cancer patients, but focused on satisfaction measures [27]. Further, this review [27] did not investigate the degree to which these quality of care measures assessed the six IOM-endorsed dimensions of patient-centred care [1].

This systematic review identified:

  1. 1)

    the degree to which PROMs developed to assess the quality of patient-centred cancer care since the publication of the IOM’s “Crossing the Quality Chasm” report in 2001 have addressed the IOM’s six endorsed dimensions of patient-centred care [1]; and

  2. 2)

    the psychometric properties of these measures.

Methods

Search strategy and selection criteria

The electronic databases Medline, PsycINFO, Current Contents, Embase, CINAHL and Scopus were searched to retrieve published studies outlining the development of PROMs designed to assess the quality of patient-centred cancer care. Given the IOM’s Crossing the Quality Chasm report was published in 2001 [1], databases were searched between January 2001 and December 2011 inclusive. The following combinations of keywords were used: (patient-centred or patient-centered or quality of care or satisfaction or experience*) AND (questionnaire* or survey* or instrument* or measure* or scale* or tool*) AND (cancer* or neoplasm* or oncol*). The use of an * in the keywords allows words that contain that term to be captured in the literature search. For example the keyword measure* will identify articles that contain variations of that word such as measure, measures, measurement and measurements. The reference lists of retrieved articles were also checked to identify any additional relevant publications.

The inclusion criteria for this systematic review were studies that:

  1. (i)

    reported the development and psychometric properties (reliability and validity) of new PROMs designed to assess the quality of patient-centred cancer care, or reported the validation of an existing measure for use with a new population (e.g. patient-centred care measure translated for use with a Spanish cancer patient population). Given the IOM’s recommendations were published in 2001 [1], studies describing the validation of an existing measure were eligible only if the original PROM was developed from 2001 onwards.

  2. (ii)

    described PROMs specifically developed for use with adult cancer patient populations (i.e., aged 18 years or older); and

  3. (iii)

    were published in an English language peer-reviewed journal.

Publications were excluded if they:

  1. (i)

    were reviews, editorials, commentaries or protocol papers;

  2. (ii)

    reported qualitative research or used a Delphi consensus process;

  3. (iii)

    reported data from medical records, administrative databases or cancer registries (i.e., patients were not surveyed);

  4. (iv)

    focussed on cancer screening only;

  5. (v)

    predominately surveyed cancer patients under 18 years of age;

  6. (vi)

    assessed the views of health professionals such as oncologists, nurses, and general practitioners;

  7. (vii)

    examined the perceptions of relatives and/or caregivers;

  8. (viii)

    included only cancer patients with advanced cancer or those receiving end of life care; These patients were excluded because the outcome measures and care delivered to patients with advanced cancer can be unique, reflecting the specific goals of advanced disease and/or end-of-life care [29].

  9. (ix)

    reported only patient ratings of quality of care and/or patient characteristics associated with quality of care – i.e. did not develop a measure with the aim of testing its psychometric properties; and

  10. (x)

    validation of an existing measure that was not eligible for the review (e.g. the original PROM was developed prior to 2001). PROMs developed prior to 2001 were excluded because it would have been unreasonable to assess the degree to which such PROMs addressed the IOM’s dimensions of patient-centred care given the IOM recommendations were published in 2001 [1].

Study and sample characteristics

The study and sample characteristics extracted from eligible publications included: the name of the measure; country of development; patient recruitment setting (e.g. hospital, cancer registry); patient eligibility criteria; sample size; consent rate; participants’ socio-demographic characteristics (e.g. mean age, gender, level of education, employment status); and participants’ disease and treatment characteristics (e.g. cancer type, cancer stage and/or time since diagnosis, treatments received).

Items and subscales of measures

Information extracted about the characteristics of each measure included: the type of measure (i.e. satisfaction versus experiences); number of items; the type of response scale, and the names and number of subscales. Two coders (FT & SKR) independently examined each of the PROMs’ items to determine whether or not the PROM contained content that related to any of the IOM’s six patient-centred dimensions and how many of the six IOM-endorsed dimensions of patient-centred care were covered [1]. At least one item in the PROM needed to examine issues related to a particular IOM patient-centred care dimension (as defined below) for that area to be categorised as addressed. A conservative approach was taken when deciding whether or not a measure covered a particular dimension. For example, if a measure included an item that examined whether a patient was provided with information on long-term side effects, the measure was categorised as meeting the information and communication dimension, but not the physical comfort dimension. The physical comfort dimension was classified as present only if items assessed the provision of pain relief or the management of physical symptoms. The criteria used to classify each patient-centred care dimension, which are based on the definitions outlined in the IOM’s “Crossing the Quality Chasm” report [1], are described below. Only one aspect of the dimension was needed for the PROM to be classed as covering that patient-centred care dimension.

1) Respect for patients’ values, preferences, and expressed needs

PROMs were classified as covering this dimension if they assessed: a) whether care responded to the patient’s cultural and other values, preferences and needs; b) whether patients were given the opportunity to express their views; c) whether patients were treated with respect during care; and/or d) whether patients were informed and involved in decision making according to their preferences [1].

2) Coordinated and integrated care

PROMs were rated as containing this dimension if they asked: a) whether patient care was coordinated and integrated; b) whether there was timely transfer of up-to-date patient information between healthcare professionals; and/or c) whether patient transitions from one healthcare setting to another went smoothly [1].

3) Provide information, communication, and education

PROMs met the criteria for this dimension if they examined whether health care professionals: a) communicated with patients in a way they could understand; and/or b) provided accurate information regarding care including diagnosis, prognosis, treatment options, follow-up care and support services, according to the patient’s preferred level of information provision [1].

4) Physical comfort

PROMs were classified as covering this dimension if they asked patients whether health care professionals: a) promptly provided pain relief; and/or b) attended to the patient’s physical symptoms and needs [1].

5) Emotional support

PROMs were categorised as meeting this dimension if they assessed whether healthcare professionals: a) addressed the patients’ emotional and spiritual concerns, such as anxiety, which could be experienced for a variety of reasons including uncertainty about their disease, concerns about the financial impact of treatment, or worrying about the impact of the illness on their family [1].

6) Involvement of family and friends

PROMs were considered to have met this dimension if they assessed whether: a) family and friends were involved in the patient’s decision making and care according to the patient’s preferences; and/or b) whether care was responsive to the concerns of family and friends and recognised their needs [1].

Two coders (FT & SKR) also independently examined which PROMs covered all aspects within each of the IOM dimensions. For instance in terms of the physical comfort dimension, PROMs that included items that addressed both of the following criteria were identified: a) promptly provided pain relief; and b) attended to the patient’s physical symptoms and needs.

Psychometric properties of measures

The psychometric properties of each measure were assessed against the same criteria used by Clinton-McHarg and colleagues in their review of instruments designed to measure the psychosocial health of adolescent and young adult cancer survivors [30]. The psychometric criteria are described below.

Internal consistency

A measure was coded as having acceptable internal consistency if correlations for the total scale and each subscale were calculated [19] and a Cronbach’s alpha >0.70 (continuous or dichotomous scales) or Kuder-Richardson 20 (KR-20) >0.70 was reported for the total scale and each sub-scale [18, 19].

Test-retest reliability

Measures were recorded as having adequate test-retest reliability if the instrument had been administered twice to the same sample and: 1) the second administration occurred within 2-14 days of the first administration [20]; and 2) correlations for the total scale, subscales and items were calculated [21] and the agreement between scores achieved a Cohen’s kappa co-efficient (κ) > 0.60 (nominal or ordinal scales) [19] or Pearson correlation coefficient (r) > 0.70 (interval scales) [18, 19] or intraclass correlation coefficient (ICC) >0.70 (interval scales) [18, 19].

Face validity

Measures were considered to have face validity if both those who administered it, and those who completed it, agreed it appeared to measure what it was designed to measure [22].

Content validity

A measure was reported to have adequate content validity if the following processes were described: 1) how the items were developed or selected [18, 19]; 2) how and by whom the content was assessed [18, 19]; and 3) if modifications to the content were needed that the revisions addressed the issues identified [18, 19].

Construct validity

Each measure was assessed as having adequate construct validity if any of the following tests were performed: 1) comparison with other existing measures [19] resulting in Pearson correlation coefficients of (r) >0.40 (convergent validity) or (r) < 0.30 (divergent validity) [23]; 2) comparison of scores on the measure differ significantly between groups with known differences (discriminative validity) [18]; or 3) factor analysis [19] with Eigenvalues set at > 1 [24].

Cross-cultural adaptation

A measure was considered to have adequate cross-cultural adaptation if a conceptually and linguistically equivalent version of the original form confirmed the reliability and validity reflected in the original measure [18].

Coding process

Two authors (FT & SKR) independently assessed all potentially relevant publications to determine whether they met eligibility for inclusion in the review. There was 84% agreement between the two coders’ ratings. Where discrepancies emerged, inconsistent ratings were discussed between the coders until consensus was reached. Both coders also independently extracted information for the Tables from included publications to ensure accuracy. The coders then compared the information extracted and discussed any inconsistencies until agreement was reached.

Results

Study eligibility

A total of 671 publications were identified from the electronic database searches and publication reference lists. Of these, 161 publications were reviews, editorials, commentaries or protocol papers, 40 reported qualitative research and 16 used a Delphi consensus process and were excluded. A further 108 papers reported data from medical records, administrative databases or cancer registries and 53 focussed on cancer screening only and were removed. Of the 293 remaining publications, 48 assessed the views of health professionals such as oncologists, nurses, and general practitioners, 44 focussed on the perceptions of relatives or caregivers, one related to cancer patients aged under 18 years, and 37 focused on an advanced cancer population and/or those receiving end-of-life care and were excluded. Of the remaining 163 publications that surveyed adult cancer patients, 121 examined the prevalence of features of care and/or characteristics associated with patient experiences and 14 validated an existing measure that was not eligible for the review (e.g. the original PROM was developed prior to 2001). One paper that reported the development of the EORTC OUT-PATSAT35 was published in French and therefore excluded [31]. This left 27 papers that reported the development of an instrument and its psychometric properties with an adult cancer patient population, or reported the psychometric properties of a re-validated measure for use with a new population. In these papers, 21 unique PROMs were described (see Figure 1).

Figure 1
figure 1

Flowchart of methods used to identify relevant publications.

Full size image

Setting and Sample Characteristics

Table 1 provides a detailed description of the setting and sample characteristics of the eligible studies [32–55]. Six studies were conducted in the USA [32, 35, 42–44, 47], five in The Netherlands [37, 39, 40, 50, 52], three in England [41, 49, 54], two in France [53, 55], and one in Australia [33], Canada [34], Europe and Asia [38], Germany [46] and Japan [48]. Seventeen studies recruited cancer patients from hospitals or treatment centres [33, 34, 38–44, 46–50, 53–55], whereas only one study recruited patients via a population-based cancer registry [32]. The sample sizes in each study ranged from 82 to 2659 cancer patients and the consent rates varied from 43% to 85%. Thirteen studies included more than one cancer type [32–35, 38, 41, 43, 44, 46–48, 52, 55].

Table 1 Sample characteristics of studies that have developed PROMs assessing quality of patient-centred cancer care
Full size table

Patient-centred care instruments

The names of the PROMs included in the review are shown in Tables 1, 2, 3, 4 and 5. As shown in Table 2, 15 measures examined patients’ experiences of care [32, 33, 37, 39–44, 48–50, 52, 53, 55] while 6 measured satisfaction [34, 35, 38, 46, 47, 54]. The number of items for each measure ranged from 15 to 152, and the number of subscales ranged from 1 to 15. The type of response scales varied across the different instruments. The number of IOM-endorsed patient-centred care dimensions [1] that were included in each measure were as follows: two measures included one dimension [35, 54], two measures had two dimensions [42, 46], seven measures had three dimensions [34, 39, 41, 47, 48, 50, 55], five measures had four dimensions [32, 33, 37, 49, 53], and four measures had five dimensions [38, 43, 44, 52]. Only one measure, the Indicators (Non-small Cell Lung Cancer) measure, covered all six dimensions of patient-centred care [40]. Table 3 summarises the PROMs that addressed each of the IOM-endorsed patient-centred care dimensions.

Table 2 Measurement features of PROMs and included IOM-endorsed patient-centred care dimensions
Full size table
Table 3 IOM patient-centred care dimensions captured by PROMs
Full size table
Table 4 Psychometric properties of PROMs assessing quality of patient-centred cancer care
Full size table
Table 5 PROMs demonstrating adequate psychometric properties based on recommended criteria
Full size table

Figure 2 illustrates the frequency with which the six IOM-endorsed patient-centred dimensions were included across the 21 measures. “Information, communication and education” was the dimension most commonly included (19 measures). In contrast, only five measures assessed the “involvement and wellbeing of family and friends”. Thirteen measures addressed all the IOM criteria for the emotional support dimension [32–34, 37–41, 43, 48, 52, 53, 55], 8 measures for information, communication and education [32, 37, 47, 48, 50, 52, 54, 55] and one measure for physical comfort [44]. None of the measures addressed all the IOM criteria within the dimensions of respect for patient values, preferences and needs; coordinated and integrated care; and involvement and wellbeing of family and friends.

Figure 2
figure 2

Frequency of IOM-endorsed patient-centred care dimensions across 21 measures.

Full size image

Psychometric properties of instruments

A description of the psychometric properties for each PROM is reported in Table 4.

Internal consistency

Seven of the 21 measures met the criteria considered adequate for internal consistency by reporting a Cronbach’s alpha >0.70 for both the total scale and each sub-scale [33, 42, 43, 46–48, 55]. Of the 13 studies that reported Cronbach’s alpha only for the PROMs’ subscales, six of these measures showed all subscales had a Cronbach’s alpha >0.70 [34, 35, 44, 52–54].

Test-retest reliability

None of the five measures that examined test-retest reliability [33, 35, 38, 49, 53] met recommended adequacy criteria of a second administration within 2-14 days of the first administration [20] and an adequate agreement between the two administrations on scores for the total scale, subscales and items [18, 19].

Face/content validity

Fifteen measures met the criteria considered adequate for face validity and content validity [33, 35, 37–41, 43, 44, 48–50, 52, 53],[55].

Construct validity

Eighteen measures met the criteria for adequate construct validity [32, 33, 35, 37, 38, 41–44, 46–50, 52–55]. Sixteen measures conducted factor analyses [32, 33, 35, 37, 41–44, 46–50, 52, 53],[55] (although only seven reported eigenvalues) [33, 41, 44, 47, 48, 50, 53], nine measures examined convergent validity (r >0.40) or divergent validity (r < 0.30) with existing instruments [35, 38, 42, 46, 47, 49, 52–54] and six measures demonstrated significant differences on scores between known groups [35, 38, 41, 42, 53, 55].

Cross-cultural adaptation

Three measures were re-validated with non-English speaking populations. The EORTC IN-PATSAT32 was validated with Sri Lankan cancer patients [56]; the Modified version of the Perceived Involvement in Care Scale (M-PICS) was validated with Lithuanian cancer patients [57]; and the Oncology Patients’ Perceptions of the Quality Nursing Care Scale (OPPQNCS) was validated with Turkish cancer patients [58].

Table 5 summarises which PROMs met the psychometric criteria considered adequate, as described above.

Psychometric properties of PROMs containing all six IOM patient-centred care dimensions

The Indicators (Non-small Cell Lung Cancer) measure [40] was the only PROM that contained items covering all six IOM dimensions of patient-centred care. This measure met the criteria considered adequate for face/content validity, but not for any other psychometric criteria evaluated in this review.

Discussion

This is the first review to identify how many of the six IOM-endorsed dimensions of patient-centred care [1] are covered in existing PROMs assessing the quality of cancer care. Our findings demonstrate that since the publication of the IOM’s Crossing the Quality Chasm report in 2001 [1], only one of 21 patient-centred cancer care instruments, the Indicators (Non-small Cell Lung Cancer) measure, included questions relating to the six IOM dimensions of patient-centred care [40]. However this measure only met the criteria considered acceptable for face/content validity. Further psychometric testing of the Indicators (Non-small Cell Lung Cancer) measure is required before more definitive conclusions can be drawn about its reliability and validity.

Across measures, the most commonly included patient-centred care dimensions were “information, communication and education” (19 of 21 measures) followed by “respectful to patients’ values, preferences, and expressed needs” (16 of 21 measures). In contrast, only seven measures examined patient’s perceptions of “physical comfort” and five assessed the “involvement and wellbeing of family and friends.” Possible explanations for the lesser focus on issues related to family and friends could include: 1) researchers/health professionals perceiving issues related to information and communication as the most important features of patient-centredness; 2) that the patients and survey developers involved in item selection only wished to focus on specific aspects of care; and 3) issues related to family and friends are considered a less crucial feature of cancer care. Furthermore, the measures may not have adequately captured the IOM’s six dimensions of patient-centred care because they were not developed for that purpose. For example, a measure’s objective may have been to focus solely or primarily on physical comfort, rather than to address the IOM’s six dimensions of patient-centred care. Nevertheless, the lack of PROMs that included all six IOM dimensions of patient-centred care [1] limits the potential of these existing measures to capture the whole-person orientation of health care and is likely to result in an incomplete representation of the quality of care provided to cancer patients.

Improvements to the reliability of existing patient-centred care PROMs and better reporting of their internal consistency, are needed. Only seven of the 21 measures met the criteria considered adequate for internal consistency by reporting a Cronbach’s alpha >0.70 for the total scale and each sub-scale [33, 42, 43, 46–48, 55]. A further six measures showed that all subscales had a Cronbach’s alpha >0.70 [34, 35, 44, 52–54], but failed to report the internal consistency for the total scale. However, interpretation of internal consistency findings should always consider that when a subscale has a large number of items, Cronbach’s alpha can be artificially high [59, 60]. Test-retest validity was very rarely considered during the development of PROMs assessing patient-centred cancer care. Although four of the five measures that examined test-retest reliability administered a second survey within 2-14 days [33, 35, 38, 53], none of the measures demonstrated acceptable agreement between scores for the total scale, subscales and items across the two administrations [18, 19]. However possible explanations for the lack of adequate test-retest reliability among PROMs assessing patient-centred cancer care may include that: 1) patients’ experiences of care, particularly for those receiving active treatment, actually changed between the initial and second administration of the measure; and 2) completing the initial measure altered patients’ expectations of patient-centred care and as a result patients rated their care differently during the second administration of the measure. Nonetheless, future research that develops PROMs of patient-centred cancer care, or validates existing measures should examine test-retest reliability, with the aim of achieving high item-to-item agreement. Item-to-item agreement is necessary [21], as high agreement between overall subscale scores can be obtained even when corresponding items within the subscale are answered differently across the two administrations.

In terms of the validity of the PROMs developed to assess patient-centred care, most of the measures met the criteria considered adequate for face/content validity (15 of 21 measures) and construct validity (18 of 21 measures). Factor analysis was the most common strategy adopted to measure construct validity (16 measures), however, few studies indicated whether eigenvalues >1 [24] were achieved [33, 41, 44, 47, 48, 50, 53]. Eigenvalues are used to determine the number of subscales within the measure by applying the eigenvalues >1 rule which produces psychometrically reliable and psychologically meaningful results [24]. Thus improvements to reporting whether eigenvalues were >1 appears necessary for PROMs that examine patient-centred cancer care.

The context in which these PROMs assessed patient-centred cancer care should be considered. Most measures were developed with cancer patients recruited from hospitals or treatment centres [33, 34, 38–44, 46–50, 53–55]. Only one measure was developed with patients recruited via a population-based cancer registry [32], despite benefits of such recruitment including the ability to sample a representative group of patients at different stages of the disease and with varied experiences of cancer care [61]. Although measuring the quality of patient-centred cancer care during initial treatment and hospital visits is crucial, undertaking such assessments with cancer survivors who no longer visit the hospital regularly is also important. For instance, women diagnosed with breast cancer have reported that the quality and duration of their follow-up consultations with clinicians had declined compared to the quality and duration of their initial treatment experiences [62].

The limitations of this review include that studies available in a non-English language peer-reviewed journal and the grey literature were excluded which could have led to some bias in the findings. Furthermore, the survey developers’ reasons for constructing the PROM should be considered. It is possible that the PROM’s objective may have been to focus on specific features of patient-centred care rather than to include items that covered the IOM’s six dimensions of patient-centred care. This may explain why most PROMs did not adequately address the IOM’s six dimensions of patient-centred care. Additionally, insufficient or unavailable reporting of the 21 PROMs’ psychometric properties may have influenced the ratings regarding the adequacy of the measure’s psychometric properties. We did not contact the authors of each PROM to enquire if additional unpublished psychometric information was available for that measure.

Conclusions

Quality improvements to the health care system can be guided by PROMs assessing the quality of patient-centred cancer care. The Indicators (Non-small Cell Lung Cancer) measure [40] was the only identified PROM that included questions relating to the six IOM endorsed dimensions of patient-centred care [1], however psychometric inadequacies and/or incomplete reporting indicates that further psychometric testing of this measure is required. Using more than one measure or further developing existing measures to include all six patient-centred care dimensions could improve the assessment and the delivery of patient-centred care. Additionally, given the lack of psychometrically rigorous PROMs developed to assess patient-centred cancer care that capture the six IOM dimensions, the construction of new comprehensive measures whose psychometric properties are adequate may also be warranted.