Talking about risk in the context of genomic tests (TARGET): development and evaluation of an educational program for clinicians
Gene expression profiling (GEP) test scores calculate risks of recurrence and likely benefit of adjuvant chemotherapy in ER-positive, HER2-negative, early-stage breast cancer. As health literacy and numeracy skills in the general population are poor, healthcare professionals (HCPs) require a wide repertoire of communication skills to explain clearly risk of recurrence scores (RSs) and uncertainty. We developed and evaluated an educational program for HCPs discussing GEP test results and adjuvant treatment.
Eight-hour workshops contained elements aimed at improving knowledge, communication skills and self-awareness; these included the science underpinning GEP tests, an interactive risk psychology lecture, exercises and facilitated group discussions regarding seven filmed scenarios involving discussions about high, intermediate and low RSs. Attendees were recorded explaining RSs with patient simulators pre and post workshop. Researchers, blinded to time point, analysed recordings using a study-specific scoring system. Primary objective outcomes were improvements post workshop in HCPs’ competence and confidence when communicating 17 pre-specified key information areas. We estimated odds ratios (OR) using conditional logistic regression to compare pre- and post-workshop scores.
65 HCPs attended. Objective analyses revealed significant positive shifts post workshop which included explaining GEP tests (OR 2.98; 95% CI 1.38–6.42; P = .001), recurrence RSs (OR 3.99; 95% CI 1.72–9.25; P < .001), benefits of chemotherapy (OR 3.99; 95% CI 1.82–8.75; P < .001; and harms OR 2.31; 95% CI 1.37–3.92; P < .001) using jargon free language (OR 5.29; 95% CI 2.27–12.35; P < .001). Patient simulator assessments also showed significant improvements as did HCPs’ self-assessments and ratings of their self-confidence when discussing different GEP tests with diverse patient types (P < .001).
These short, intensive, interactive TARGET workshops significantly improved HCPs’ communication about GEP results in ways likely to promote more informed decision-making by patients about chemotherapy.
KeywordsRisk of recurrence Genomic test results Breast cancer Chemotherapy Communication
There have been exciting advances in diagnostics, surgical and radiotherapy techniques and systemic therapies for women with early-stage breast cancer (EBC). Better understanding about the molecular biology of cancer has permitted more targeted treatment, and thus many patients have genuine prospects of cure or living longer. Most patients with cancer want collaborative roles in treatment decision-making , but discussions about the logic and rationale behind treatment recommendations have become increasingly complex and demand excellent communication skills. Decision-making about adjuvant chemotherapy in particular requires careful balancing between likely absolute benefits in terms of preventing recurrence versus the unpleasant side effects and inconvenience of treatment. Explaining risks and benefits to patients is never easy and health literacy and numeracy skills are frequently poor in the general populations of the US, Canada, Australia and EU [2, 3]. Perceptions about health risk information are strongly related to numeracy so women with inadequate numeracy skills are more likely to overestimate their chances of dying from breast cancer and the absolute risk reduction of breast cancer screening [4, 5]. Many patients also believe that in the context of life-threatening disease, more treatment is better than less.
Various decision aids such as Adjuvant! On-line were designed to help clinicians determine and discuss treatment recommendations. Studies show, however, that patients’ numeracy profoundly affects their ability to interpret numerical estimates of treatment efficacy . In the UK, NHS Predict V2.1 is a prognostication and treatment benefit tool providing survival estimates 5 and 10 years following surgery with and without adjuvant therapy (hormone therapy, chemotherapy and trastuzumab) . Clinicians can show patients relevant information with text, graphs, charts or in icon arrays. Clinico-pathologic features of the tumour are good prognostic indicators for most patients, but for some, the additional assessment of the risk of recurrence, not just survival estimates, is also pertinent for decision-making. Linking such decision-aid information to other factors that might determine treatment recommendations can be confusing for patients.
Several gene expression profiling (GEP) tests including EndoPredict® (EPclin score), Oncotype DX® Breast Recurrence Score and Prosigna® are approved to guide adjuvant chemotherapy decisions for patients with oestrogen receptor (ER)-positive, human epidermal growth factor receptor 2 (HER2)-negative and lymph node (LN)-negative breast cancer. Discussing the classifications of high, intermediate or low risk are somewhat illusory as all GEP recurrence RSs are on a continuum with their cut-offs based on clinical trial data. In one survey, a third of women receiving risk of recurrence test results did not fully understand these discussions . Health literacy may affect patients’ retention of RS test results, their capacity for processing information, understanding and decision-making . Results from a study examining patients’ understanding and preferences for six different Oncotype DX risk score formats of increasing complexity showed high error rates irrespective of health literacy or numeracy .
Even when RSs suggest high or low risk of recurrence, interpretation of these is subject to various unconscious biases. The personality characteristics of patients and doctors, especially their tolerance of uncertainty, may contribute to indecision or seemingly irrational choices . Clinicians less tolerant of uncertainty may, when encountering ambiguous clinical situations, order more diagnostic tests or extra treatments than colleagues who are more accepting . Furthermore, they may be less likely to discuss uncertainty openly with patients or engage in shared decision-making [12, 13]. Anxious patients with lower tolerances of ambiguity are often uncomfortable forgoing all possible treatments including chemotherapy even if benefits are modest or negligible.
Consequently health care professionals (HCPs) require personal numeracy expertise and wide repertoires of communication skills when explaining concepts to patients craving certainty about their prognoses and treatment outcomes.
As clinicians admit to communication difficulties when discussing GEP test scores, we developed an 8-h educational program—Talking About Risk in the context of GEnomic Tests (TARGET). The intervention’s central aim was to assist HCPs when communicating OncotypeDX® and EndoPredict® results thus helping their patients to receive clearer information about putative adjuvant chemotherapy benefits permitting more educated shared decision-making.
Workshop development and contents
Research shows that improving both the competence and the self-confidence of HCPs are necessary prerequisites for the effectiveness of educational initiatives and transfer of skills into a clinic setting. Programs must include areas that enhance skills development (SD), knowledge acquisition (KA) and personal awareness (PA) [14, 15, 16].
We developed training materials following review of the risk literature and discussions with key clinicians and scientists. We mapped out the difficulties encountered explaining high, intermediate and low RSs together with the added challenges faced when communicating with patients with diverse personality and socio-educational characteristics. Other difficulties included developing understandable explanations that survival following adjuvant treatments is dependent on the risk of distant recurrence of breast cancer, how to transform percentages into different graphical representations, translating risks of recurrence into simple but non-patronising language, and the pros and cons of using analogies to describe risk.
TARGET contents and educational aims
An interactive session with a lecture about the psychology of risk and uncertainty and group exercises on basic numeracy, tolerance of uncertainty and frequency descriptors of numerical concepts
An interview with a basic scientist about the principles underpinning GEP tests
2 scenarios in which 2 oncologists discuss a high and a low risk, EndoPredict® test results with patients
3 scenarios in which different oncologists and a specialist nurse discuss low, intermediate and high OncotypeDX® RSs
2 further OncotypeDX® scenarios filmed following the publication of the TAILORx trial which changed some of the cut-offs and classifications of risk
Workshops lasted approximately 8 h and were facilitator-led encouraging group discussion between participants about the specific communication skills displayed by oncologists in different scenarios, and practical exercises on different means of conveying risk information needed for decision-making and handling varied patient reactions. Workshops concluded with attendees generating lists of optimal and necessary points that must be covered when discussing GEP test results.
After designing, piloting and refining the program, we conducted an evaluation of its efficacy and acceptability with HCPs actively engaged in discussing RSs.
Participants based in the UK were recruited through SHORE-C website adverts and flyers distributed at breast cancer meetings following presentations showcasing filmed materials. Workshops were accredited 9 Continuing Professional Development (CPD) points from the Royal College of Physicians. Brighton & Sussex Medical School Regional Ethics Committee approved and sponsored the study (ref: ER/RMLS21/1) funded by the Breast Cancer Research Foundation.
On the morning prior to and the afternoon following the workshop, participants were recorded explaining RSs with simulated patients. Their case studies were based on real clinical situations, and included consultations with low-risk patients, who nevertheless wished to have chemotherapy, and higher risk patients averse to chemotherapy. Before recording, participants read their assessment case study, the GEP test result, and any associated reports, printed handouts or additional information which they normally used in their own clinical practice, such as Predict outputs. They could access computer notes and displays if required. All simulated patients were experienced in improvisation and well briefed about breast cancer, its symptoms and reasons for the consultation. To enhance authenticity, different actors and scenarios were used pre and post workshop.
Participants gave written consent to all assessments prior to the first recording.
Assessments of recorded interviews
There were three separate assessments of all digitally recorded interviews: (1) objective assessment by trained researchers, (2) self-assessment by attendees and (3) patient simulator assessment.
An independent data manager assigned random numbers to each digital interview which researchers assessed using a study-specific checklist, blinded to assessment time point. Researchers’ coding involved checking the presence of 17 key communication and information areas, together with the HCP’s competence (‘did not do this’, ‘not very well’, ‘reasonably well’, ‘very well’). Areas covered issues such as explaining the patient’s risk of recurrence with and without treatment, implications for survival, making appropriate use of print outs/graphs to aid understanding about RSs and other general communication behaviours including structuring and summarising information.
Following each recorded interview, HCPs self-rated their performances on all 17 areas, and how satisfied and confident they felt with their interviews and chemotherapy decisions.
Post interview, simulated patients also scored their HCP’s communication skills on the same 17 items, indicated the decision made and provided general comments for feedback.
Instruments similar to those used in previous assessments of our educational interventions [17, 18] were adapted for attendees to self-rate their confidence pre and post workshop when discussing 9 general aspects of RSs on a scale from 0 (none) to 10 (very confident).
Self-awareness, numeracy and communication exercises
The Intolerance of Uncertainty Scale (IUS) measures responses to uncertainty, ambiguous situations and the future . Intolerance of uncertainty is the tendency to consider possibilities of a negative event as unacceptable and threatening irrespective of the probability of its occurrence. Individuals with high intolerances attempt to make seemingly less risky choices, so the scale has utility when examining HCPs’ decision-making behaviours.
Basic numeracy-related skills, in particular the facility to transform probabilities into proportions, proportions into percentages and vice versa, are vital when helping patients to understand risk. Participants completed a basic numeracy assessment (based on Schwartz) probing their numeracy abilities .
Attendees also completed a short exercise examining frequencies and verbal descriptors. They imagined a fictitious drug for indigestion and indicated how many people out of 100 might get various side effects described as ‘fairly common’, ‘often’, ‘unlikely’ and ‘very rare’.
At the workshop conclusion, attendees rated the quality of the educational materials, specific aspects of the content, and whether or not they would recommend the program to colleagues.
Our a priori hypotheses were that post workshop (a) attendees’ communication when discussing GEP test results would improve, namely that their competence would be measurably better, and (b) that they would feel more confident conducting these interviews, that is, their self-efficacy/self-confidence would be enhanced.
As the primary objective outcome was the analysis of participants’ digitally recorded interviews with simulated patients, pre and post workshop, each of the two researchers coding interviews performed rate–rerate reliability checks on their own assessments and inter-coder reliability checks for 10% of each other’s interviews. Both rate–rerate and inter-coder reliability were examined using Kappa coefficients. A conditional logistic regression model was used to compare pre- and post-workshop scores for confidence levels and for the presence and competent handling of each key communication item, using a numeric score for responses 0 (did not do this), 1 (not very well), 2 (reasonably well), 3 (very well). This model estimates, for each individual, the probability that a score is observed post workshop rather than before it. This probability is expressed in terms of the difference between pre- and post-scores. The parameter of interest in the model is the odds ratio (OR). The larger the OR, the more likely it is that higher scores are observed post workshop. No distributional assumptions are required for the scores. Thus, this approach provides a robust method for before-and-after comparisons. The key data used by the estimation procedure are numbers of attendees with different scores at the two time points. Large positive ORs occur when changes fall towards larger score values rather than smaller values. Each item reflects a distinctive communication area, the interpretation of which is of interest on its own. We do not make an overall communication recommendation based on amalgamation of all items; neither do we control the overall error rate. Consequently, corrections for multiple testing are not necessary. The results are exploratory and all inferential statements are valid marginally; that is, they are to be interpreted individually for each item [21, 22].
We also analysed the levels of agreement between (a) coder and HCP, (b) coder and actor and (c) actor and HCP on their assessment of competence for every information area post workshop. For each of the communication areas, firstly the rating was dichotomised as (0 = ‘did not do this’ or ‘not very well’ and 1=‘reasonably well’ or ‘very well’), and secondly a binary variable was formulated to indicate agreement (1 = Yes, 0 = No) between each pair of raters for each recording. We used a logistic regression model for this variable to estimate an overall odds of agreement in each information area considered. To yield chance-corrected measures of agreement, models incorporated corrections for chance agreement in the linear predictor following Lipsitz et al. . The total score for Intolerance of Uncertainty Scale was analysed using a linear regression model with attendees’ levels of confidence on each area of information as explanatory variables.
Between March and July 2018, seven workshops were attended by 65 breast cancer specialists (38 women; 27 men), of whom 32 were oncologists, 24 surgeons and 9 nurses.
Analyses of digital recordings
The inter-rater (two coders) and rate–rerate reliability showed good agreement (k = .9123; SE = .0712; k = .8101, SE = .1517 , respectively). A descriptive analysis of pre- and post-workshop scores (supplementary Tables D–G) show the trends of improvement in most communication areas and HCP self-reported confidence overall by speciality.
High levels of agreement (concordance) were observed between coders, HCPs and simulated patients for ratings of “reasonably well” and “very well” responses across the 17 items. There were too few disagreements in these data for some items to make any further analyses of agreement worthwhile.
Attendees expressed significantly higher levels of confidence with their discussions following post-workshop interviews (OR 3.27; 95% CI 1.53–6.99; P = .0022).
Intolerance of Uncertainty Scale (IUS)
Participants’ mean IUS score was in the normal range but 12/65 (18%) had high scores, 7 of whom were extremely intolerant of uncertainty (Supplementary table H). Higher intolerance of uncertainty was associated with lower self-rated confidence when discussing prognosis with patients with EBC and when discussing intermediate RSs (beta = − 1.67; 95% CI − 2.89 to − .45; P = .009; beta = − 1.76; 95% CI − 3.15 to − 0.37, P = .016). Higher intolerance of uncertainty scores was also associated with higher levels of self-rated confidence when discussing high RSs generally and when communicating with patients of lower socio-educational background (beta = 1.81; 95% CI 0.16–3.45; P = .035; beta = 1.95; 95% CI 0.18–3.72, P = .035.)
Results from the basic numeracy exercise
N = 65 (%)
A person taking Drug A has a 1% chance of an allergic reaction. If 1000 people take the drug, how many will have a reaction? (10)
A person taking Drug B has a 1 in a 1000 chance of an allergic reaction. What % of people taking the drug will have a reaction? (0.1%)
The chance of getting a serious viral infection is 0.0005. How many of 10,000 exposed people might get the infection? (5)
Imagine I flip a fair coin 1000 times. How many times will the coin land heads up? (500)
Frequency and verbal descriptors
Participants attributed wide numerical ranges to the different verbal descriptors: The term ‘fairly common’ elicited a range from 10/100 to 90/100; likewise ‘often’ produced estimates ranged from 1 to 90, ‘unlikely’ 0.1 to 30 and ‘very rare’ 0.01 to 10.
Feedback at workshop completion revealed that most attendees found TARGET useful (9.6/10), informative 9.6/10) and enjoyable (9.7/10) and 100% would ‘definitely’ recommend the program to their colleagues.
This intervention was designed specifically to assist HCPs when discussing GEP test scores with patients and implications these have for adjuvant chemotherapy recommendations. There are many generic aspects of the program which are suitable for any HCP tasked with explaining risk to patients.
TARGET was based on successful evidence-based educational packages that contain elements enhancing knowledge acquisition, skills development and self-awareness [17, 18]. Prior research shows that interventions which change both competence and self-confidence/efficacy often transfer successfully into a clinical setting and are enduring [14, 15]. Objective analyses of attendees’ pre- and post-workshop interviews revealed positive behavioural changes in the style and content of discussions and an increase in HCPs’ reported self-confidence.
An important feature of TARGET was enhancement of attendees’ personal awareness about their own numeracy skills and attitudes to uncertainty. Conversations with patients about the rationale and logic for different management policies requires discussion about the potential risks and benefits of adjuvant treatments aimed at reducing recurrence, and the impact that recurrence has on survival. During such discussions many different numbers are provide in dissimilar formats. Numeracy skills of general populations in the US and UK are often poor, and many get confused by numbers with different denominators and experience difficulties with percentages . Studies show that even with decision aids that include graphs, bar charts and icon arrays, some patients have difficulty identifying their own personal risks . It is really important therefore that HCPs communicating risk information can transform numbers into formats appropriate for individual patients in order to truly permit shared/collaborative decision-making. Some TARGET attendees, in common with many highly educated people, had personal numeracy difficulties , in particular problems converting percentages into proportions, proportions into percentages and calculating expected frequencies from probabilities.
Using verbal descriptors seems a reasonable way to circumvent numeracy difficulties. Attendees’ responses to the verbal descriptors for common side effect exercise demonstrated widely differing ranges. This revealed the pitfalls if, when discussing risk, numbers are avoided altogether and communicators rely instead on verbal descriptors or frequencies. There is no way of knowing if either their colleagues’ or patients’ interpretations of phrases such as ‘fairly common’ would match their own. The European drug regulatory agency (EMA) has definitions for numbers that should be associated with words relating to side effect frequencies and the Royal College of Anaesthetists has produced pictorial charts assisting understanding about risk, e.g. ‘rare’ is described as 1:10,000, with a sketch and description of ‘someone in a small town’ .
Ambiguity about management options can ‘leak’ through in obvious and nuanced ways during interviews with patients. Interestingly, those attendees with a high intolerance of uncertainty had lower levels of confidence when discussing intermediate risk of recurrence scores with patients. In clinic settings, these HCPs would be more likely to prescribe chemotherapy especially when faced with an anxious patient, also intolerant of uncertainty, and who had an intermediate or borderline risk score.
One limitation of this work is that unless communication skills programs are made a mandatory part of training, then HCPs who attend workshops maybe a self-selected group, although we saw that the baseline skills of participants ranged widely. TARGET workshops lasted 8 h, split over 2 days but could be adapted for shorter modular-based delivery, lasting 2 h per module. It is unknown whether or not this would produce the same improvements. To assist with wider dissemination, we have produced a facilitator handbook with time-coded transcripts of scenarios enabling facilitators to stop at key points and engage groups in exercises or discussions. There are suggested prompts and comments about the issues illustrated and for less experienced facilitators, examples of structuring teaching sessions. Facilitator training programs are currently being held.
Although TARGET focussed on OncotypeDX® and EndoPredict® RSs, other available GEP tests share similar communication challenges. Discussing risk and managing the understandable anxiety and uncertainty of women about their need for adjuvant chemotherapy and ultimate decision-making demands a wide repertoire of communication skills that TARGET workshops appeared to enrich.
We wish to thank all the HCPs who permitted us to film their unscripted scenarios for the TARGET course materials, the attendees on workshops and the patient simulators.
This study was funded by the Breast Cancer Research Foundation (BCRF17-050).
Compliance with ethical standards
Conflict of interest
LJF has received research grants from Bristol Myers Squibb, and from Novartis. LJF has received speaker honorarium from Amgen, TEVA, Voluntis, Oncolia, Pfizer, Eisai and Genomic Health and had a consultancy role with Puma Biotechnology, Roche, Eisai, Voluntis, AstraZeneca and Takeda. VAJ has received research grants from Aventis Pharma trading as Sanofi. All other authors have no conflicts of interest to declare.
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards (ref: ER/RMLS21/1). The datasets during and/or analysed during the current study are available from the corresponding author on reasonable request
- 2.Department for Business Innovation and Skills. The 2011 skills for life survey: a survey of literacy, numeracy and ICT levels in England. BIS Research paper number 81. London: Department for Business Innovation and Skills, 2012. https://www.gov.uk/government/publications/2011-skills-for-life-survey accessed Feb 25th 2019
- 7.NHS Predict V2.1 www.predict.nhs.uk/index.html; Accessed Feb 25, 2019
- 16.Jenkins V, Fallowfield L (2002) Can communication skills training alter physicians’ beliefs and behaviour in clinics? JCO 20(3):765–769Google Scholar
- 19.Carleton RN, Norton MAP, Asmundson GJG (2007) Fearing the unknown: a short version of the Intolerance of Uncertainty Scale. J Anxiety Disord 21(105):107–117Google Scholar
- 26.Ghazal S, Cokely ET, Garcia-Retamero R (2014) Predicting biases in very highly educated samples: numeracy and metacognition. Judgm Decis Mak 9(1):15–34Google Scholar
- 27.Royal College of Anesthetists. https://www.rcoa.ac.uk/patientinfo/risk-and-probability. Accessed Mar 12, 2019
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.