Serious illness care Programme UK: assessing the ‘face validity’, applicability and relevance of the serious illness conversation guide for use within the UK health care setting
- 95 Downloads
When doctors have honest conversations with patients about their illness and involve them in decisions about their care, patients express greater satisfaction with care and lowered anxiety and depression. The Serious Illness Care Programme (the Programme), originally developed in the United States (U.S), promotes meaningful, realistic and focused conversations about patient’s wishes, fears and worries for the future with their illness. The Serious Illness Conversation Guide (the guide) provides a framework to structure these conversations. The aim of this paper is to present findings from a study to examine the ‘face validity’, acceptability and relevance of the Guide for use within the United Kingdom (UK) health care setting.
Nominal Group Technique with clinician ‘expert groups’ to review the Serious Illness Conversation Guide: 14 ‘experts’ in Oncology, Palliative Care and Communication Skills;
Cognitive Interviews with 6 patient and public representatives, using the ‘think aloud technique’; to explore the cognitive processes involved in answering the questions in the guide, including appropriateness of language, question wording and format
Final stakeholder review and consensus.
Nominal Group Technique
Unanimous agreement the conversation guide could provide a useful support to clinicians. Amendments are required but should be informed directly from the cognitive interviews. Training highlighted as key to underpin the use of the guide.
The ‘holistic’ attention to the person as a whole was valued rather than a narrow focus on their disease. Some concern was raised regarding the ‘formality’ of some wording however and suggestions for amendments were made.
Final stakeholder review
Stakeholders agreed amendments to 5/13 prompts and unanimously agreed the UK guide should be implemented as a part of the pilot implementation of the Serious Illness Care Programme UK.
Use of the guide has the potential to benefit patients, facilitating a ‘person-centred’ approach to these important conversations, and providing a framework to promote shared decision making and care planning. Further research is ongoing, to understand the impact of these conversations on patients, families and clinicians and on concordance of care delivery with expressed patient wishes.
KeywordsSerious illness Communication Intervention Person Centred Care planning
Nominal Group Technique
National Health Service
- the guide
The Serious Illness Conversation Guide
- the Programme
The Serious Illness Care Programme
The need for improved clinician communication and a more ‘patient centred’ approach to care delivery is consistently highlighted in government policy within the United Kingdom (UK) [1, 2, 3, 4] and research literature [5, 6, 7]. Clinicians frequently fail to involve seriously ill patients in discussions and decisions regarding their current and future care, resulting in limited opportunity to engage in conversations about hopes, fears and concerns with their clinical team [1, 2, 8]. Poor clinician-led communication can engender or exacerbate psychological distress for patients with serious illness and their families [9, 10, 11]; indeed ‘staff attitudes’, ‘behaviour‘and ‘poor communication’ account for the majority of complaints about the National Health Service (NHS) [12, 13]. A recent report from the Royal College of Physicians (RCP) ‘Talking About Dying’ highlighted significant barriers to initiating conversations about prognosis and treatment choices, including: clinician confidence and personal fears about having these conversations; a culture of targets and organisational pressures within the NHS; and a belief that a culture exists among the public to avoid discussing death .
The Serious Illness Care Programme (“the programme”) is a multi-component, systems-based intervention designed to improve the care of all persons with serious illness . Serious illness is defined as: “a health condition that carries a high risk of mortality and negatively impacts a person’s daily function or quality of life, excessively burdens their caregivers, or both” . Developed by Ariadne Labs in the United States (U.S.) the Programme promotes meaningful, realistic and focussed conversations about the patients fears, worries and wishes for the future with their illness and consists of three elements: 1) clinical tools, including a specially developed Serious Illness Conversation Guide (‘the guide’) using ‘patient tested language’, to ensure these important issues are addressed consistently and sensitively; 2) expert training and coaching for members of the multidisciplinary team in how to have these conversations; 3) systems innovations and Electronic Medical Records (EMR) modifications, whereby organisations create clear processes to identify patients and document details derived from the conversation in order to share it with other health professionals. The shared information then informs ongoing care and treatment planning and promotes ongoing conversations .
Preliminary data from a large cluster randomised controlled trial among cancer patients at the Dana Farber Cancer Institute in Boston, showed implementation of the Serious Illness Care Programme leads to “more, earlier and better conversations” by oncologists; lowered anxiety and depression in patients; and high levels of patient satisfaction and positive behaviour change resulting from the conversation [16, 17]. A recently published study in primary care patients also suggests that the Serious Illness Care Programme promotes more and better conversations, as well as the need for further research .
In response to continued governmental challenges to improve communication and care for patients with serious illness [19, 20], NHS England provided funding in 2016/17 for a service improvement project to pilot implementation of the Programme at three clinical sites; two primary care, one cancer centre. In preparation for the pilot, a three stage research project was undertaken to explore the ‘face validity’, applicability and relevance of the clinical tool, the Serious Illness Conversation Guide, to explore whether adaptations were required for the UK before its use in the pilot. This paper describes the process and outcomes of this study.
A multi-stage approach was taken in order to engage a wide range of stakeholders; healthcare professionals and members of a patient and public involvement (PPI) group from within one cancer centre in the North West. The research was structured into three stages:
A variation of a small-group discussion to reach consensus. The Nominal Group Technique (NGT) provides a structured format to collect and synthesise multiple views from different groups or individuals and commonly involves 4 stages: silent generation of ideas, round robin, clarification of ideas and voting. Three ‘expert groups’ made up the participant groups and the following structure, based on the 4 stages, was used to guide the discussion and achieve consensus:
Generation of ideas: each individual group reviewed the guide and considered the following discussion points:
Discussion Point 1: What do you think about the overall utility of incorporating the Serious Illness Care Guide into clinical practice? Consider:
Timing of discussions;
Incorporating this guide into conversations during clinical consultations;
Planning for future care.
Discussion Point 2: For each ‘prompt’, consider the following:
What do you think of how the questions are worded?
Are there any phrases/concepts within any of these questions that may be problematic?
Is there anything currently missing from this list of prompts, that you feel would be important to discuss/consider at this time?
Round robin: each group feedback the ‘ideas’ generated from the individual group discussion.
Clarification: Points raised by individual groups (concerns/problems identified or suggestions for amendment) were then opened up for discussion in the wider group.
Voting: Consensus was gained for each discussion point and suggestion for amendment. Consensus opinion was recorded on flip charts, written up following the meeting, and circulated to all members for accuracy and confirmation of content.
Sample: participants for the NGT were recruited from three healthcare professional groups with expertise in Oncology, Palliative Care and Communication Skills. Clinicians were sent an information sheet to explain the purpose of the meeting and invite them to take part. Consent to participate was gained following clinician ‘sign up’ and subsequent attendance at the meeting.
Stage 2 - cognitive interviewing
In depth one-to-one cognitive interviews were undertaken with lay representatives, as a proxy for the patient or ‘user’ perspective, to explore the cognitive processes involved in answering the questions in the guide, including appropriateness of language, question wording and format [23, 24].
Sample: Participants were recruited from patient and public representative groups from within the cancer centre, with experience of living with or caring for a loved one with cancer. Specifically, the researcher attended a Patient Council meeting within the cancer centre, where the purpose of the research was introduced. During the meeting, permission was given for the researcher to send a participant information sheet (PIS) to members of the Patient Council and other lay representative groups located in the cancer centre. Following this, and giving at least 24 h for consideration of the information, the researcher made contact with individual members to ask if they wished to take part. If they agreed to take part, then a time and place convenient to the participant was arranged.
Due to the in-depth nature of the interviews that were undertaken, the ‘richness’ of the data generated (information power), and the specific focus of the ‘phenomena’ (the guide), it was felt that the sample of 6 lay representatives would meet the objectives of the study .
Five interviews were undertaken within the participants own home, and one interview was undertaken in a quiet room within the cancer centre identified by the lay participant. The Cognitive Interview itself was guided by the ‘think aloud’ process , which encouraged participants to articulate their thoughts (or ‘think aloud’) as they read through the guide. The interviews did not use a specific topic guide; participants were given the Serious Illness Conversation Guide (the guide) and asked to read each prompt aloud and talk through their understanding, including how they might formulate an answer. This process also enabled words or concepts that participants were unsure of to be highlighted and discussed. Each interview was audio recorded and then transcribed verbatim to aid analysis. Thematic Analysis was undertaken, in order to generate key themes from the data, using the “substance of the interview” to interpret “meanings and perceptions created and shared during a conversation” .
Written, informed consent to participate was gained from all participants, via a consent form. Ethical approval for this element was received from Wales Research Ethics Committee 5 (Bangor), reference number 16/WA/0062.
Stage 3 – final stakeholder review and consensus of the serious illness care guide
Multiple stakeholders reviewed results from stage 1 and 2 before achieving final agreement on suggested amendments to the guide. This meeting was integrated into the scheduled Steering Group Meeting held as part of the service improvement project, funded by NHS England, to adapt the Serious Illness Care Programme for use in the UK. Consent to participate was gained through attendance at the meeting. All stakeholders were sent written information on the purpose of the meeting, which was to gain consensus on the final amendments to the Serious Illness Conversation Guide. Stakeholders at this meeting included members from: Patient Council (lay members); NHS England (policy makers); Ariadne Labs (research and health professional members); Marie Curie Palliative Care Institute Liverpool (research and health professional members) and Clatterbridge Cancer Centre (research and health professional members). All members in attendance at the meeting participated in this element.
Stage 1 - Nominal Group Technique (NGT)
5 Consultant Medical Oncologists
4 palliative care experts: 3 Consultants in Palliative Medicine and 2 Specialist Registrars in Palliative Medicine
4 communication skills experts: 3 Senior Lecturers in communication skills (all RGN) and 1 Consultant in Palliative Medicine with a lead role in communication skills training for undergraduate medical education.
Results: Agreed discussion points from group discussion:
Acceptability, applicability and relevance of the Serious Illness Conversation Programme:
Serious Illness Conversation Guide: Oncologists valued this, viewing it as a structure to initiate, engage and support in serious illness conversations. They indicated that they would derive confidence in initiating these conversations, knowing the language in the guide was ‘patient tested’. Views from the communication skills and palliative care ‘expert’ groups were that the guide was potentially reductionist, due to the ‘scripted’ format, however they acknowledged this may be due to their increased clinical experience in handling these types of conversations.
Education and training: The importance of a robust programme of education and training, incorporated into the implementation of the programme, was highlighted. All groups emphasised the need for ongoing training and support throughout the implementation period, for example, by employing coaching and mentoring to help clinicians overcome initial anxieties and address challenges in the change to current practice.
Systems innovations: All ‘expert’ groups agreed that adherence to the systems innovations (infrastructure and monitoring) as part of the overall Programme was essential to underpin use of the guide and support clinicians to have conversations with their patients.
Wording of the Serious Illness Conversation Guide: discussion/suggestion for amendment
All members of the Nominal Group agreed it would be appropriate to leave suggestions for wording changes until after analysis of the results from the cognitive interviews with PPI representatives. All members felt that the perspectives of this group, as a proxy for the patient ‘user perspective’, would ensure that any suggestions for amendment would be rooted in the ‘patient voice’.
The group unanimously agreed that, pending modifications informed by cognitive interviews, the guide would effectively support clinicians in initiating high quality serious illness conversations.
The final consensus from the NGT was that amendments to the guide should not be put forward by this group. Instead, suggestions for wording amendments should be underpinned by the ‘rich’ data that would be generated by the Cognitive Interviews (stage 2) with lay representatives.
Stage 2 - cognitive interviews
Promoting a partnership approach to care planning conversations
Importance of flexibility of approach and ability to incorporate the prompts as part of a ‘natural’ conversation
‘Opening the door’ – Importance of ongoing conversations throughout the patient journey
Education and training essential as part of the implementation of the programme
‘Formality’ of some words/phrases a concern: goals, priorities and wishes, abilities, critical
1. Promoting a partnership approach to care planning conversations
“...it starts a conversation...it’s an aide memoir…you can use the guide and...you’ll go off script undoubtedly, because conversations will take a different turn, and you can’t anticipate, but it allows you to make sure you...start here and you finish here, and by the time you’ve finished, you have covered everything...you know...” (P1, male)
“I mean, “how much information of what is likely to be ahead” ... everything, because as far as I’m concerned it’s a partnership, you’re not abdicating responsibility to somebody to do whatever they want to you, you want to know what they’re going to do. You’re part of the decision making process...” (P2, female)
“It’s seeing the person as an individual and that they’ve got a life.” (P2, female)
2. Importance of flexibility of approach and ability to incorporate the prompts as part of a ‘natural’ conversation
“...I think sometimes using these words just makes it sound like a formal interview...you need...to be able to put the patient into a situation where they feel comfortable to be able to open up with their concerns...and saying to them, what are your “goals”, or what are your “fears and worries”, or what “strength” just makes it sound as though you’re sitting across from the desk, from somebody in an office, having an interview” (P1, male)
The quote above reinforces the perspective that the guide should not be seen as a ‘script’ from which deviation is not possible. Clinicians should be able to adapt or amend their conversation as necessary, and it is important that the patient’s wishes are respected:
“...they need to acknowledge [what’s important]…when [the patients]…say it as well... [regarding a care plan] one of the things for me, one of the key things was about using some natural stuff and so I told them, they knew, but then every time it was mentioned it was like eyes rolling”. (P2, female)
“...what if patients start talking about it in a different order...I’m not going to sit there and watch somebody tick a box, talking to me about I’m dying...” (P5, female)
3. ‘Opening the door’ – importance of ongoing conversations throughout the patient journey
“If you’re going to have this sort of conversation, that’s only the beginning...you’ve then got to be prepared to follow up what the patient says, haven’t you, and be able to...it’s a starter isn’t it...” (P2, female)
“They need to know before this point, how that person wants that information, or, if they want that information... erm...or if they want a friend to have, you know, their loved one to have that information or what that...but that needs to be done” (P2, female)
4. Education and training essential as part of the implementation of the programme
“...but the most important thing is the clinician, because there are some who can never do that in a million years because it’s not in their nature...it [use of the guide] could make it [conversation] considerably worse...it would be a very uncomfortable experience” (P2, female)
“...and they [patient] need to be able to have the facility to be able to ask questions and one of the difficulties I found with some clinicians, not all of them...is that they can be erm... difficult...less than communicative when it comes to patients wanting to question them... and that’s the training down to the doctors.” (P1, male)
5. ‘Formality’ of words or phrases is a concern
“Goals...would they see it as a goal? I don’t know whether they’d see it as a goal...it’s more about love isn’t it, love for the family...it doesn’t have a feeling of wanting to see my first grandchild, that’s not a goal.” (P5, female)
“I don’t like the certain phrases which makes it sound far too stilted and formal...so you know, if you can soften the language...so that it makes them, makes somebody feel more comfortable, then...I would have no problem at all” (P3, female)
“...I think the last sentence is a bit naff...cos you’re not actually in it together...if somebody said that to me I’d go “pardon”...you’re not, I mean yes put some comforting words there but...we’re in it together is hardly comforting, when you know it’s not true.” (P5, female)
Stage 3 – final stakeholder review and consensus of the serious illness conversation guide UK
Results from stages 1 and 2 were presented to the group, alongside suggested wording amendments to the guide which were identified from the Cognitive Interviews. All thirteen individual ‘prompts’ on the guide were reviewed, using facilitated discussion. Figure 1 illustrates the main discussion points and confirmation of the final amendments to the guide, following consensus agreement. Five prompts were subsequently amended. All stakeholders at the meeting agreed this would be the final version of the UK guide. Stakeholders also unanimously agreed that the UK guide could now be implemented as a part of the pilot implementation of the Serious Illness Care Programme UK.
Results from this study suggest that the adapted Serious Illness Conversation Guide has the potential to support clinicians to have conversations with seriously ill patients regarding their current and future care, and use of the guide has the potential to promote shared decision making and identification of what matters most to patients. Further, the guide provides a good foundation from which to improve the way important conversations about future care planning are undertaken, documented and used to improve care planning and decision making. This study also highlighted useful information about the way the guide might be used in practice, as well as how it is implemented as part of the Serious Illness Care Programme UK.
Through the cognitive interviews, participants identified that the guide could benefit patients by facilitating a more ‘person-centred’ approach to these important conversations, although some cited their own negative experiences of clinician communication as a reason for their positive reaction to the guide and the programme. Good communication and patient satisfaction with care is often dependent on the existing relationship between patient and clinician; with trust, honesty and empathy being the qualities most appreciated by patients . Studies have shown that communication skills training which employs the use of communication ‘tools’ can improve clinicians ‘responsiveness’ to patient’s concerns and engender more ‘empathic’ encounters with patients following training [9, 29]. While participants uniformly valued the use of the guide, the structured approach, although also seen as a benefit, was identified as a potential weakness, particularly if used/adopted by clinicians that had not undergone training and followed the guide by rote. Findings from stage 1 and 2 both highlight the guide should not act as a ‘script’ that cannot be deviated from or adapted as necessary, according to individual patient needs and wishes. For example, during the cognitive interviews, participants commented that the clinician should be able to ‘adapt’ the language as part of a natural conversation, rather than reading verbatim as if in an ‘interview’ situation. Education and training must, therefore, be a key part of the implementation of the UK programme, and in supporting the sustainable use of the guide within any organisation.
It has been argued that engaging in honest discussion about difficult information may empower patients with regards to decision-making as well as enhance rather than diminish hope [30, 31]. This sentiment was reflected in findings from this study, where participants described the guide as a useful ‘conversation piece’, which would allow the patient to think about, and talk through, what is most important to them in regards to their current and future care. This was an important element for participants and they also identified that the guide, with its focus on exploring the patients’ ‘holistic’ needs, treats patients as individuals with ‘a life’ rather than exclusively focussing on their ‘disease’. Delaying important discussions about the future when prognosis is poor can subsequently preclude the identification and achievement of a patient’s preferences for end-of-life care , which also highlights the value that a tool such as the guide could bring to the improvement of patient care.
Although the majority of prompts were viewed positively, discussions from stage 1 and 2 highlighted concerns with some of the language within the guide. For example, concern was felt regarding the more ‘formal’ language within the guide, in particular the use of the word ‘goals’. The consensus was that this word may not translate particularly well, as the word ‘goal’ is often associated with an organisational context rather than within a sensitive conversation with a patient. Given the overall feedback received as part of this project, prompt 5 was then amended to remove ‘goal’. While other comments from stage 1 and 2 identified other minor concerns with ‘formal’ words and phrases such as ‘what gives you strength’ and ‘fears and worries’, these terms were not identified as problematic and were therefore not amended. However, given the concerns participants had, it is imperative that clinicians are able to integrate the prompts into a ‘natural’ conversation. It is recommended that general training in communication skills for clinicians, as well as in use of the tool, is essential for the implementation and ongoing use of the programme. Without this, the potential for challenges and variation of quality in the delivery of the serious illness conversation may occur.
The final amendment was to the concluding prompt ‘we’re in this together’; unanimous agreement from ‘expert’ groups and lay representatives, this prompt would not translate to the UK context and should be removed from the guide. This prompt received the strongest negative reaction, particularly from the lay representatives in this study. Concerns were raised that this could appear ‘superficial’ and ‘disingenuous’ on the part of the clinician, potentially undermining the ‘person centred’ approach to this conversation and the relationship that the clinician may have built up with the patient. Following the final stakeholder review, it was felt that the conversation guide did need an additional prompt to reaffirm a commitment to the ongoing care and support of the patient, and alternative wording was agreed: ‘I am here to make sure you get the best care possible’.
Use of the guide to support conversations between clinicians and patients with serious illness has the potential to improve ‘person-centred’ care, and facilitate improved care planning through shared decision making, to ensure greater concordance between documented wishes and patient outcomes. Although the guide, and the programme as a whole, was viewed as a positive intervention, further robust research and evaluation on its effect in clinical practice was recommended to develop the evidence base prior to any local or national ‘roll out’. It was suggested that research should also include in-depth ‘user experience’ of the guide in practice. Specifically qualitative and quantitative methods should be used to unpack patient and family perception and understanding of engaging in serious illness conversations, as well as clinician perception of using the guide to support their conversations. Following this study pilot implementation of the Programme was undertaken in 3 NHS sites, two primary care sites and a cancer centre. In line with the findings from this study, further research is ongoing. A feasibility study is currently being undertaken to explore use of the Programme within the cancer centre, and results from this study will inform any further potential roll out of the Programme.
The authors would like to thank all those who participated in this study, especially the lay representatives who gave their time and valuable insight, despite the sensitive nature of the subject matter. The authors would also like to thank colleagues from Ariadne Labs in Boston, for their expertise and support throughout this project and beyond. This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.
TM and SM were involved in the process of designing the study. TM and SM facilitated the Nominal Group discussion. TM conducted the interviews. TM and SM participated in the data analysis process. TM drafted the manuscript. All authors (TM, SM, AC, AR, MM, JS, FM, SB, JE, and PK) provided critical review and comments on drafts of the manuscript and approved the final version.
The Palliative Care Institute Liverpool, University of Liverpool funded the research activities for this project. This study was undertaken independently, alongside the service improvement project funded by NHS England, and NHS England did not contribute to the design of the study, the collection, analysis, and interpretation of data, or in writing this manuscript.
Ethics approval and consent to participate
Ethical approval was sought for the ‘cognitive interview’ element of this study, undertaken with members of the patient and public involvement groups. All participants were provided with written information about the project and were encouraged to ask any questions. Written, informed consent was gained from each participant prior to interview. Ethical approval for this element of the study was received from Wales Research Ethics Committee 5 (Bangor) reference number: 16/WA/0062. All data collected was stored in line with the University of Liverpool data storage policy (http://www.liv.ac.uk/csd/regulations/informationsecuritypolicy.pdf), and handled in confidence in line with the Caldicott principles. Ethical review via the Health Research Authority (HRA) was not required for the service improvement elements of the project; nominal group meeting and stakeholder review and consensus meeting with healthcare professionals. An information sheet about the project was sent, inviting them to participate, and consent to participate was obtained through ‘sign up’ to attend both the Nominal Group Technique and the Final Stakeholder Review and Consensus meeting.
Consent for publication
The authors declare they have no competing interest.
- 1.Francis, R. Report of the mid Staffordshire NHS Foundation trust public inquiry: executive summary. 2013 [cited 2018 30th August 2018]; Available from: https://www.gov.uk/government/publications/report-of-the-mid-staffordshire-nhs-foundation-trust-public-inquiry.Google Scholar
- 2.Neuberger, J., et al. More care less pathway; a review of the Liverpool care pathway. 2013 24th August 2018]; Available from: https://www.gov.uk/government/publications/review-of-liverpool-care-pathway-for-dying-patients.
- 3.The National Health Service (Revision of NHS Constitution Guiding Principles) Regulations 2015. 2015 24th August 2018]; Available from: http://www.legislation.gov.uk/uksi/2015/1426/introduction/made.
- 4.Bailey, S.J. and K. Cogle. Talking about dying: how to begin honest conversations about what lies ahead. 2018 23rd October 2018]; Available from: https://www.rcplondon.ac.uk/projects/outputs/talking-about-dying-how-begin-honest-conversations-about-what-lies-ahead.
- 6.Moore PM, et al. Communication skills training for healthcare professionals working with people who have cancer. Cochrane Database Syst Rev. (2013, 3):p. Cd003751.Google Scholar
- 8.National Survey of Bereaved People (VOICES): England, 2015. 2016 [cited 2018 30th August 2018]; Available from: https://www.ons.gov.uk/peoplepopulationandcommunity/healthandsocialcare/healthcaresystem/bulletins/nationalsurveyofbereavedpeoplevoices/england2015.
- 12.Complaints about acute trusts 2014-15, Parliamentary and Health Service Ombudsman. 2015; Available from: https://www.ombudsman.org.uk/publications/complaints-about-acute-trusts-2014-15.
- 13.Data on written complaints in the NHS, 2016–2017. 2017 24th August 2018]; Available from: https://digital.nhs.uk/data-and-information/publications/statistical/data-on-written-complaints-in-the-nhs/data-on-written-complaints-in-the-nhs-2016-17.
- 14.Bernacki R, et al. Development of the serious illness care program: a randomised controlled trial of a palliative care communication intervention. BMJ Open. 2015;5(10).Google Scholar
- 15.Moving Towards Universal Access to High-Quality Serious Illness Care. 2017 13th November 2018]; Available from: https://palliativeinpractice.org/palliative-pulse/palliative-pulse-september-2017/moving-towards-universal-access-high-quality-serious-illness-care/.
- 16.Bernacki, R., et al., Delivering more, earlier, and better goals-of-care conversations to seriously ill oncology patients. J Clin Oncol, 2015. 33(29_suppl): p. 39–39.Google Scholar
- 17.Paladino, J., et al., Effect of conversations about values and goals on anxiety in patients. J Clin Oncol, 2015. 33(29_suppl): p. 9–9.Google Scholar
- 19.Capcity, care planning and advance care planning in life limiting illness. 2014 [cited 4th June 2019; Available from: https://www.england.nhs.uk/improvement-hub/wp-content/uploads/sites/44/2017/11/ACP_Booklet_2014.pdf.
- 20.Our Commitment to you for end of life care: The Government Response to the Review of Choice in End of Life Care. 2016 [cited 2018 30th August 2018]; Available from: https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/536326/choice-response.pdf.
- 22.Delbecq, A.L., A.H. Van De Ven, and D.H. Gustafson, Group techniques for program planning: a guide to nominal group and delphi processes. Management applications series. 1975: Glenview (Ill.) : Scott, Foresman, 1975.Google Scholar
- 23.Presser S, et al. Methods for Testing and Evaluating Survey Questions. The Public Opinion Quarterly. 2004;(1):109.Google Scholar
- 29.Back AL, et al. Communication about cancer near the end of life. Cancer. 2008:113.Google Scholar
- 30.Smith, T.J., et al., Giving honest information to patients with advanced cancer maintains hope. Oncology (Williston Park), 2010. 24(6): p. 521–5.Google Scholar
- 32.Mack JW, Cronin A, Keating NL. Associations between end-of-life discussion characteristics and care received near death: a prospective cohort study. American Society of Clinical Oncology: United States p. 2012:4387.Google Scholar
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.