The Patient - Patient-Centered Outcomes Research

, Volume 12, Issue 5, pp 513–526 | Cite as

Patient Preferences in the Medical Product Life Cycle: What do Stakeholders Think? Semi-Structured Qualitative Interviews in Europe and the USA

  • Rosanne JanssensEmail author
  • Selena Russo
  • Eline van Overbeeke
  • Chiara Whichello
  • Sarah Harding
  • Jürgen Kübler
  • Juhaeri Juhaeri
  • Karin Schölin Bywall
  • Alina Comanescu
  • Axel Hueber
  • Matthias Englbrecht
  • Nikoletta Nikolenko
  • Gabriella Pravettoni
  • Steven Simoens
  • Hilde Stevens
  • Richard Hermann
  • Bennett Levitan
  • Irina Cleemput
  • Esther de Bekker-Grob
  • Jorien Veldwijk
  • Isabelle Huys
Open Access
Original Research Article



Patient preferences (PP), which are investigated in PP studies using qualitative or quantitative methods, are a growing area of interest to the following stakeholders involved in the medical product lifecycle: academics, health technology assessment bodies, payers, industry, patients, physicians, and regulators. However, the use of PP in decisions along the medical product lifecycle remains limited. As the adoption of PP heavily relies on these stakeholders, knowledge of their perceptions of PP is critical.


This study aimed to characterize stakeholders’ attitudes, needs, and concerns with respect to PP in decision making along the medical product lifecycle.


Semi-structured interviews (n = 143) were conducted with academics (n = 24), health technology assessment/payer representatives (n = 24), industry representatives (n = 24), patients, caregivers and patient representatives (n = 24), physicians (n = 24), and regulators (n = 23) from seven European countries and the USA. Interviews were conducted between April and August 2017. The framework method was used to organize the data and identify themes and key findings in each interviewed stakeholder group.


Interviewees reported being unfamiliar (43%), moderately familiar (42%), or very familiar (15%) with preference methods and studies. Interviewees across stakeholder groups generally supported the idea of using PP in the medical product lifecycle but expressed mixed opinions about the feasibility and impact of using PP in decision making. Interviewees from all stakeholder groups stressed the importance of increasing stakeholders’ understanding of the concept of PP and preference methods and ensuring patients’ understanding of the questions asked in PP studies. Key concerns and needs in each interviewed stakeholder group were as follows: (1) academics: investigating the validity, reliability, reproducibility, and generalizability of preference methods; (2) health technology assessment/payer representatives: developing quality criteria for evaluating PP studies and gaining insights into how to weigh them in reimbursement/payer decision making; (3) industry representatives: obtaining guidance on PP studies and recognition on the importance of PP from decision makers; (4) patients, caregivers, and patient representatives: providing an incentive and adequate information towards patients when participating in PP studies; (5) physicians: avoiding bias as a result of commercial agendas in PP studies and clarifying how to deal with subjective and emotional elements when measuring PP; and (6) regulators: avoiding the misuse of PP study results to overrule the traditional efficacy and safety criteria used for marketing authorization and obtaining robust PP study results.


Despite the interest all interviewed stakeholder groups reported in PP, the effective use of PP in decision making across the medical product lifecycle is currently hampered by a lack of standardization and consensus on how to both measure and use PP.

Key Points for Decision Makers

Despite increased attention towards patient preferences (PP), the use of PP in medical product decision making remains limited and unstructured.

In this qualitative research, 143 individual interviews were conducted to characterize stakeholders’ attitudes, needs, and concerns towards PP in decision making along the medical product lifecycle.

To increase the use of PP, efforts are needed on three levels: (1) the cultural and educational level, via increasing acceptance and understanding among stakeholders about PP; (2) the methodological level, via increasing understanding of quality criteria of PP studies; and (3) the procedural level, via increasing understanding on how to integrate PP in current medical product decision making.

1 Introduction

The role of patient preferences (PP) is being increasingly explored in several decision-making contexts throughout the life cycle of drugs and medical devices (i.e., the medical product life cycle [MPLC]) [1, 2, 3]. Different definitions, conceptualizations, and categorizations for PP exist. The US Food and Drug Administration (FDA) adopted the following definition for ‘patient preference information’: “qualitative or quantitative assessments of the relative desirability or acceptability to patients of specified alternatives or choices among outcomes or other attributes that differ among alternative health interventions” [3]. Others have defined PP and methods to investigate them in a narrower manner by solely referring to ‘utilities’ and quantitative ‘elicitation’ methods, respectively [4, 5, 6, 7, 8, 9]. In this study, the terms ‘PP’ and ‘preference methods’ are used to refer to both qualitative and quantitative PP and preference methods, according to the FDA definition [3].

Decision-making stakeholders interested in PP include regulatory agencies responsible for the marketing authorization of medical products, such as the European Medicines Agency (EMA) and the FDA, and health technology assessment (HTA) bodies and reimbursement agencies responsible for HTA and the reimbursement of medical products [2, 10, 11, 12]. In the regulatory context, PP have been argued to provide insights into what benefit-risk trade-offs patients accept [3, 5, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23]. On the HTA/reimbursement level, PP could provide information about patients’ preferred medical products and clinical outcomes [6, 24, 25, 26, 27, 28, 29]. Outside the regulatory and reimbursement context, industry stakeholders are exploring how PP can inform priority setting, clinical trial design and analysis, and post-marketing risk assessments [3, 7, 15, 19, 21, 30].

Reflecting these interests, decision-making bodies have started assessing the value of preference methods; the EMA investigated patient benefit-risk trade-offs for treatment outcomes [31], and the FDA quantified PP for the treatment attributes of weight-loss devices. The FDA issued two guidance documents about PP: recommendations for collecting PP in the context of medical devices and draft guidance for collecting patient experience data, including PP, in the context of drugs [3, 17, 32, 33, 34, 35]. Examples of efforts at the reimbursement level include the conduct of PP studies by the Institute for Quality and Efficiency in Health Care1 and the first scientific advice on a PP study design from the National Institute for Health and Care Excellence2 [29, 36, 37].

Despite demonstrated interest, PP studies are currently not submitted and evaluated systematically during marketing authorization [38]. Similarly, the use of PP in HTA has been described to be in its infancy, and few HTA organizations have guidelines on how PP should be measured [35, 39]. In addition, no evidence-based guidance on the design and conduct of PP studies is available for industry. As the adoption of PP depends on the stakeholders involved in the MPLC (academics, HTA bodies, payers, industry, patients, physicians, and regulators), knowledge of their perceptions of PP is a critical step towards increasing the use of PP. Moreover, as these stakeholders’ decisions depend on each other, implementing PP in the MPLC would impact each stakeholder involved. This study therefore aimed to characterize stakeholders’ perceptions of PP and, more specifically, their attitudes, needs, and concerns related to PP in decision making along the MPLC.

2 Materials and Methods

2.1 General Design

Because this study aimed to gain deeper insights into stakeholders’ attitudes, needs, and concerns, a qualitative study design was chosen. In view of the geographical dispersion, diversity, and number of interviewees within each stakeholder group, interviews were chosen as the data collection method [40]. Interviews were conducted as part of PREFER3 between April and August 2017.

2.2 Interviewee Selection

Interviewees were recruited from six different stakeholder groups to account for the diversity of stakeholders involved in the MPLC. Interviewees were recruited from eight countries to obtain insights from stakeholders familiar with different healthcare systems. Patients, caregivers, physicians, and patient organization representatives were recruited from four disease areas4 to increase the heterogeneity of the sample in terms of disease types. To further ensure a heterogeneous sample, inclusion criteria and quota were set (Electronic Supplementary Material [ESM] 1). An initial sample was established through suggestions from PREFER consortium members. From these suggestions, 22 persons met the inclusion criteria and were invited to participate via e-mail. Twelve persons (55%) accepted; the others did not answer, did not consider themselves knowledgeable enough, or did not have time. Subsequent interviewees were suggested by interviewed persons (snowballing). Recruitment was slowest in Romania and Italy and in the HTA and regulatory stakeholder group.5 No relationship between interviewers and interviewees was established prior to invitation.

2.3 Interview Guide

The interview guide was developed based upon the research aims, 16 exploratory interviews, and a literature review (Table 1, ESM 2). Exploratory interviews were conducted with HTA/payer representatives (n = 4), regulators (n = 2), industry representatives (n = 3), patient organization representatives (n = 4), physicians (n = 2), and one academic (n = 1). The literature review identified the opportunities and challenges associated with PP (Janssens et al., submitted, 2019). One version of the interview guide was developed for HTA/payer representatives, regulators, industry representatives, and academics and one version for patients, patient organization representatives, caregivers, and physicians. The latter version was shorter, used simpler terminology, included more comprehensive information parts, and did not address the more methodological topics (Table 1). Both versions were reviewed and pilot tested with five interviewees of the targeted stakeholder groups.
Table 1

Topics addressed in the interview

1. Definition for PP

 (a) Personal definition for PP

 (b) FDA definitiona

2. Familiarity and experience with preference methods and PP studies

3. Importance and role for PP in the MPLC

 (a) Reasons why PP should (not) be used

 (b) Stages and decisions in the MPLC PP where PP should (not) play a role

4. Needs for implementing PP in decisions and aspects that are lacking now

 (a) Approach for conducting PP studiesb

 (b) Type of sample when conducting PP studies

 (c) Methodological requirements when conducting PP studiesb

 (d) Quality criteria for evaluating PP studiesb

5. Concerns related to PP

 (a) Heterogeneity of the patient sampleb

6. Impact or change when using PP

For the complete interview guide, see Electronic Supplementary Material 2

FDA US Food and Drug Administration, MPLC medical product life cycle, PP patient preferences

aA simplified version of the FDA definition was presented to patients, patient organization representatives, caregivers, and physicians

bTopics not discussed with patients, patient organization representatives, caregivers, and physicians

2.4 Data Collection

An interview protocol was developed and discussed with eight interviewers (RJ, SR, EvO, CW, KSB, AC, ME, RD)6 as a form of training for conducting the interviews and to minimize variability across interviews (ESM 3). A glossary was sent to patients, patient organization representatives, caregivers, and physicians prior to the interview (ESM 4). At the start of the semi-structured7 interview, interviewers presented themselves and explained the aim of the PREFER project and interview. Interviews with patients, patient organization representatives, caregivers, and physicians were conducted in their native language. Other interviews were conducted in English unless preferred otherwise by the interviewee. Interviews took approximately 1 hour and were conducted via telephone or face to face, the latter at a location preferred by the interviewee. In addition to the interviewer and interviewee, no one was present during the interview. Interviews were audio-recorded. One interview was conducted per interviewee.8

2.5 Data Analysis

Interviews were analyzed thematically by two researchers (RJ, SR). The framework method was chosen for analysis, as it enables multiple researchers to independently analyse one large dataset [41, 42]. Table 2 explains the implementation of the seven stages of the framework method: transcription, familiarization, coding, developing an analytical framework, applying the analytical framework, charting, and interpreting. Open answers to the question asking about interviewees’ familiarity with PP were categorized into “not familiar”, “moderately familiar”, or “very familiar” and were analyzed descriptively in Excel.
Table 2

Stages of the framework method

1. Transcription

The audio-recordings were transcribed verbatim. Transcripts were not returned to interviewees for comments and/or corrections, except upon explicit request by the interviewee. Interviews were transcribed in the original language, and if necessary, translated to English

2. Familiarization

RJ and SR thoroughly read and re-read each transcript and listened back to the audio-recorded interviews whenever a certain part of the transcript was unclear. The margins of the transcripts were used to write down analytical notes, thoughts, or impressions (e.g., when interviewees expressed exceptionally strong or contrasting views to other interviewees)

3. Coding

Based upon pre-defined sets of interests to the research, the research questions and questions in the interview guide, RJ and SR developed a pre-defined coding list and a brief definition for each of the codes (Electronic Supplementary Material 5, coding list 1). RJ and SR independently coded the first available 6 transcripts from 6 interviewees belonging to the 6 different stakeholder groups, using the predefined codes, to assess whether all relevant topics could be assigned a code. If a certain topic was not covered with the pre-defined coding list, a new code was assigned (open coding)

4. Developing a working analytical framework

RJ and SR discussed the codes they assigned to each passage. They discussed why they coded it, i.e., why they perceived it as meaningful to answer the research questions. After the discussion, RJ and SR agreed on an adapted set of codes. RJ and SR then independently coded 6 more transcripts, using the adapted coding list, taking care to note any new codes or impressions that did not fit the existing set. RJ and SR then met again and evaluated the coding list to incorporate new codes or rename codes. At this point RJ and SR also decided whether certain codes were related and if so, whether they could be grouped

5. Applying the analytical framework

The list of codes was uploaded in NVivo (11th edition, QSR International). The transcripts were divided equally among RJ and SR. Using NVivo, SR and RJ went through each transcript and highlighted passages of text and selected and attached an appropriate code from the coding list (coding). During coding, SR and RJ regularly discussed to refine the coding list until no changes were necessary (i.e., when coding saturation was reached and the final coding list was established). The coding list consisted of codes and sub-codes, each with a brief explanatory description of their meaning and examples of what ideas or elements could be summarized under that code (Electronic Supplementary Material 5, coding list 2)

6. Charting the data into the framework matrix

Excel was used for charting (summarizing) the data. RJ and SR exported the coded text per code from the final list of codes, from NVivo to Excel. In Excel, a separate tab sheet was created per ‘overarching’ code (Electronic Supplementary Material 6). Each tab sheet comprised one row per interviewee and one column per sub-code. To allow for within- and across-stakeholder group comparisons, the interviewees from the same stakeholder group were placed next to each other. SR and RJ each charted half of the transcripts. To retain links to the transcripts, verbatim text was indicated by underlining it. RJ and SR held regular meetings to compare their charting approaches and to ensure consistency in their approaches

7. Mapping and interpretation

The development of themes was done both deductively (i.e., influenced by the research questions) and inductively (i.e., influenced by new codes generated inductively from the data). The framework matrix in Excel was analyzed qualitatively so that relevant statements could emerge even if only mentioned a few times by interviewees. In a stepwise manner, RJ and SR: (1) independently wrote down their interpretations for each overarching code and stakeholder group and (2) convened to discuss their interpretations per overarching code and stakeholder group (investigators’ triangulation). During these discussions, RJ and SR reached a consensus about stakeholder groups’ priorities regarding PP and subsequently derived the statements in Fig. 1. RJ and SR also discussed potential themes and reached a consensus about these themes. Whenever the data were rich enough, the interpretations generated in this stage went beyond the description of a particular interviewee to the explanation of potential reasons or beliefs of multiple interviewees. Participants were not asked to provide feedback on the results

Steps followed for the analysis of the interviews

3 Results

The results section is structured according to the themes identified during analysis. Key findings per stakeholder group are visualized in Fig. 1.
Fig. 1

Stakeholders’ perceptions about patient preferences (PP). Left: interviewees’ self-reported familiarity with PP studies and methods and how they defined PP. Right: stakeholders’ key needs and concerns related to PP in decision making along the medical product life cycle. The patient group includes patients, caregivers, and patient organization representatives

3.1 Interviewees’ Characteristics

One hundred and forty-three persons participated. Interviewees reported being unfamiliar (43%), moderately familiar (42%), or very familiar (15%) with preference methods and studies (Table 3). Self-reported familiarity was highest in academics, among whom 50% reported being very familiar, and it was the lowest in patients and physicians, among whom 86% and 75%, respectively, reported being unfamiliar with PP (ESM 7).
Table 3

Interviewees’ characteristics


Interviewees (n = 143)






















 The Netherlands






Stakeholder group




 HTA/payer representativeb



 Industry representativec






 Patient organization representatived












Self-reported familiarity with PPf

 Not familiar



 Moderately familiar



 Very familiar



For demographics per stakeholder group, country, and disease area, see Electronic Supplementary Material 7

HTA health technology assessment, PP patient preferences

aPersons with anticipated knowledge on patient involvement or preference methods and working in an academic or research institution

bPersons formally involved in HTA or reimbursement

cPersons working in or providing consultancy to drug or medical device companies

dDiagnosed with, associated with, or providing care to patients in the following disease areas: lung cancer, rheumatoid arthritis, myotonic dystrophy type 1, cardiovascular diseases

ePersons with a formal role in national or European marketing authorization

3.2 Definition of Patient Preferences (PP) and Awareness

3.2.1 Defining ‘PP’

When interviewees were asked about their personal definition for PP, interviewees from all stakeholder groups underlined the challenge of providing such a definition. Whereas academics more often used a focused definition (e.g., trade-offs between treatment outcomes), interviewees from the other stakeholder groups most frequently adopted a broad definition (e.g., patients’ perspectives) (Fig. 1). An academic and regulator explained how the lack of an agreed-upon definition causes confusion: “Well, I think it is a term that is being used in a number of different ways, and I think it would be beneficial if a definition can be agreed upon, just to reduce confusion” (RE_US_6).9

3.2.2 Awareness and Acceptance

Interviewees from all stakeholders mentioned a need for higher understanding of the concept of PP and preference methods. In this context, some academics and regulators called for training opportunities for decision makers and researchers. Several patient organization representatives, academics, and industry representatives mentioned that more acceptance of the concept of PP is needed: “A kind of general alignment of what patient preferences are, why they are important and why they need to be included in the drug development pathway” (AC_IT_24).

3.3 Attitudes Towards PP, Their Value and Impact on Decision Making

3.3.1 Interviewees’ Attitudes Towards PP

The majority of interviewees across stakeholder groups were positive towards using PP in the MPLC, as patients are final healthcare users, have disease experience, and the use of PP could improve decision making (see Sect. 3.3.3). Whereas industry interviewees seemed most positive towards PP, physicians most frequently were hesitant towards using PP or against using PP. Interviewees who doubted the value and use of PP made the following arguments:
  • Patients lack scientific knowledge (AC, IN, PA, CA, PH, RE; see Sect. 3.4.2);

  • Patients have unrealistic preferences and are not objective enough (PH);

  • Preference methods are of low quality (PY, RE, AC, see Sect. 3.4.5);

  • PP do not fit in current decision-making processes (PH, AC, PY, see Sect. 3.5.1);

  • Using PP would not alter the decision (PH, AC, see Sect. 3.3.2); and

  • In the context of reimbursement, societal preferences rather than patient preferences should be used (PY, see Sect. 3.4.3) (Fig. 1).

3.3.2 Value of PP is Context Dependent

Interviewees from all stakeholder groups, particularly regulators, underlined that the value of PP in decision making depends on factors such as the type of medical product and disease area (Fig. 1). Furthermore, some academics and regulators stated that PP studies only need to be conducted when they are relevant for decision making.

3.3.3 Impact of PP on Decision Making

There were diverging views regarding the impact of PP on decisions. Interviewees from different stakeholder groups (AC, PY, PA, IN) underlined that it would take a long time before PP would become part of the regular decision-making system because of the complexity of doing so (see Sect. 3.5.1).

Regarding development, interviewees from different stakeholder groups described the impact as follows:
  • Better industry investments (PA, PY, RE);

  • Enhanced patient recruitment into clinical trials (PA);

  • Increased development of medical products that meet patient needs (AC, PY, IN, PH, RE, PO);

  • Easier or faster market access (RE, IN); and

  • Higher acceptance of the drug among patients (IN, PH, PO, RE, AC).

Regarding the impact on marketing authorization and HTA/reimbursement, some HTA/payers, industry representatives, and regulators predicted the following:
  • A higher external acceptance of decisions (IN, RE);

  • A positive change in the decision makers’ approach to benefit-risk assessments (RE): “A dramatic change in the concern of regulators […], it would become normal when discussing the benefits and risks” (RE_UK_11);

  • A price increase (PY): “If medical products have attributes that patients value, then presumably manufacturers will charge more for those products” (PY_SE_14);

  • Higher quality decisions (RE, PY, IN) by increasing certainty for decision makers when there are uncertainties regarding the clinical evidence; and

  • More decisions are made (PY): “Now some decisions […] just can’t be made simply because they don’t have the proper data” (PY_DE_8).

Regarding the impact of PP on decision outcomes, different opinions were expressed. With respect to marketing authorization, several interviewees (IN, AC, PY, RE) foresaw a marginal effect: “If you already approve 90% of all treatments, then you cannot really make that much of an impact” (AC_NL_12). One regulator explained that specifically in cases in which there is a clear benefit-risk balance, the decision outcome would not necessarily change if PP were considered. On the HTA/reimbursement level, some HTA/payers and industry representatives stated that the use of PP would lead to different decisions being made “due to the system we have here currently (…) it says that only patient relevant endpoints are considered. However, patients were never asked what patient-relevant end-points are” (IN_DE_10).

3.4 Concerns and Needs Regarding Measuring PP for Decision Making

3.4.1 Guiding Framework for Measuring PP

Several academics and industry representatives expressed concerns about a lack of consensus on how to measure PP and, more specifically, how to select the following: (1) a sample (see Sect. 3.4.3); (2) a preference method; (3) attributes and levels; and (4) the stage in the MPLC to conduct a preference study. Several of them elaborated on how more standardization, guidance, and best practices would demonstrate the manner in which PP studies should be undertaken. Some of them argued that standardization could be achieved through regulatory guidance, a change in legislation, or a clearer regulatory framework: “Some rules, clearly defined, telling you in what process you involve the patients, how representative they should be (…) and so on” (AC_RO_22). Some industry interviewees explained that a regulatory framework would provide them with more certainty to invest in a preference study: “Every study needs investment when the clinical development of a drug is a costly process, where we have to make choices” (IN_IT_18). Some academics underlined, however, the difficulty of formalizing preference research and argued that developing strict guidance would stop preference research: “It is very risky to have to sit down and write rules about how to do things involving research” (AC_US_12). Similarly, there were mixed views on the level of detail this guidance should have; whereas one academic highlighted that existing guidelines “are probably too basic” (AC_UK_21), another academic and industry representative elaborated that this guidance should be flexible so that “according to the specific context, you are free to choose which one (method) is most suitable” (AC_IT_21). Several academics and HTA/payer interviewees particularly described how a checklist of quality criteria would be useful, and some academics mentioned that more research on how to rate the quality of PP studies and their outcomes would help in the construction of such a checklist.

3.4.2 Patient Knowledge and Education

Interviewees from all stakeholder groups described the current level of patient knowledge as a concern for the measurement and use of PP in decision making. One patient attributed the lack of knowledge to the lack of information given to patients about the preference study. In addition to knowledge about the preference study, interviewees from all stakeholder groups also referred to patients’ educational level and knowledge about the MPLC and about the medical product. To increase patient understanding, some patients advocated for the inclusion of an educational component in PP studies. Interviewees from all stakeholder groups were concerned about the complexity of questions in PP studies. Some regulators and academics underlined how a lack of understanding among participants decreases the reliability of the preference study results and advocated for more research verifying whether patients actually understood the questions.

3.4.3 Sample

Interviewees from all stakeholder groups, except patients and physicians, raised a need for clarifying what would be the “ideal” sample in terms of the size, heterogeneity, and type of participants that would ensure the usefulness of the study results for decision making: “Should it be European, Nordic, American, Asian? (…) I mean, there are a number of questions like these that you have to sit down and decide” (IN_SE_17). Some academics and HTA/payer representatives specifically raised the question of using societal vs. patient preferences to guide reimbursement decisions: “I think I personally would have a preference for doing public so - I feel like preferences are important, it’s just whose preferences that I’m worried about” (UK_AC_21). The concept of ‘heterogeneity’, both of the patient sample and of the measured preferences, triggered diverging views across academics, regulators, HTA/reimbursement, and industry representatives. Some of them considered heterogeneity to be a desirable element. Others were concerned about preference heterogeneity being present but not being measured or reported transparently. Finally, some considered heterogeneity to be both negative and positive: “It’s true; we have to do a better job, we have to explain heterogeneity, I think it will be positive in the future because that could actually lead to a more personalised approach to treatments” (AC_DE_11).

3.4.4 Factors Influencing PP Study Results

Several academics, physicians, and regulators described factors that may influence PP: sociocultural status, emotions, time, media, psychological factors, and disease status. They reasoned that the impact of these factors on PP study results is often unknown to researchers. One regulator was concerned about how the sole act of measuring PP might impact preferences of participants: “If I obtain the patient’s preference, the problem is understanding what I have really obtained (…) I have also directed this preference involuntary” (RE_IT_11). Interviewees across all stakeholder groups were apprehensive about methodological choices in the design of PP studies (e.g., question framing, selecting certain patient groups), leading to biased results, and several mentioned the risk that pharmaceutical companies would want to use these results to “persuade the regulator of something that is actually quite potentially dangerous” (RE_US_6).

3.4.5 Scientifically Robust Preference Methods

Interviewees from all stakeholder groups except patients, caregivers, and patient organization representatives indicated a current lack of methodological understanding related to the available methods for measuring PP. Some of them considered the preference research field to be “not a very well-developed area of research” (UK_PY_15). More specifically, interviewees across these stakeholder groups expressed doubts about the validity, reliability, reproducibility, and generalizability of preference methods and results. Some of these interviewees reasoned that not knowing how preference methods perform on these criteria lowers their usefulness for decision making: “We know little about the reliability, the reproducibility, etc. (…) It (preference research) has little weight on policy decision making because of those reasons” (AC_FR_11).

Several academics, regulators, and HTA/payer and industry representatives stressed the need for more research that validates different preference methods. However, only a few suggested specific study designs to better characterize preference methods: “Different types of methods in the same patient population and this type of information I think we need” (RE_NL_6). Furthermore, while methods need to be “scientifically credible” (IN_DE_9), some industry and academic interviewees raised a need for simpler methods because (1) “many of these questions come up on a very time critical path” and (2) complex methods are “cognitively burdensome” (AC_IT_21).

3.4.6 Multi-Stakeholder Approach

Some patient organization representatives and regulators underlined how relevant stakeholder groups (e.g., sponsors, patients, caregivers, regulatory agencies) need to be involved in the design and conduct of PP studies to increase their value for decision making. For example, collaboration between researchers and regulators was described as useful because scientists have preference research expertise and regulators can “determine whether PP information would be useful to answer certain regulatory decision questions” (US_RE_5).

3.5 Concerns and Needs Regarding the Use of PP in Decision Making

3.5.1 Integrating PP in Decision Making

Interviewees from all stakeholder groups except patients, caregivers, and patient organization representatives questioned the feasibility and lack of a clear strategy for including PP in current development, marketing authorization, and reimbursement decisions. Several HTA/payers, industry representatives, and academics questioned how PP should be weighted in decisions. These stakeholder groups but also particularly regulators underlined that PP should be considered together with other evidence. Similarly, regulators and physicians stated that PP should not completely guide the decision, as they perceived other “traditional” criteria (e.g., efficacy, safety) to be more important. To be useful for marketing authorization, one US regulator underlined that regulators need to be able to link the preference study results to the clinical trial outcomes of the medical product under review. Similarly, one HTA/payer representative raised the need for a “consistent approach of measuring patient preferences in different indications so that they can be very directly translatable to decision making” (HT_NL_8).

3.5.2 Framework for Using PP

There were diverging views regarding what specifically is needed to integrate PP into decision making. While several academics, regulators, payers, and patient organization interviewees felt that PP should be integrated based on a framework that allows for the transparent use of PP in decision making, another academic argued to “just use this as a source of information, and they will come up with decisions” (AC_DE_11).

3.5.3 Handling Preference Study Results

Several regulators, academics, and physicians emphasized that the submission of PP studies to decision makers needs to be transparent. More specifically, they argued how all participant variables as well as all the preference study results need to be reported to avoid the selective reporting of results. One industry representative raised the question of dealing with unexpected results: “But what if (…) it doesn’t really fit in our advantage; how will we deal with it?” (IN_NL_10). Some patients and patient organization representatives raised the issue of confidentiality of patient information: “Getting access to your name and identity, and that whole thing, or someone sending a bulk email” (PA_SE_1). One caregiver mentioned the need to disclose preference study outcomes to patients.

3.5.4 Misuse of PP in Decision Making and Creating Disparities

Several regulators, academics, and HTA/payer representatives expressed concerns about the risk of misusing PP in decision making “to overcome other important aspects of the technology” (AC_IT_24) or “to make more profit” (PY_IT_13). Two HTA/payer representatives further proposed that the risk of misusing PP to force decision makers towards a certain decision outcome should be mitigated when embarking on preference research: “I am concerned that there will be (…) a huge amount of so-called preference studies that will be used to say ‘Oh, you say, the patient said that so you have to act’” (PY_IT_13). In the context of marketing authorization, some regulators described the risk that pharmaceutical companies could misuse PP to “circumvent the normal regulatory process” (RE_UK_9). Two HTA/payer representatives were concerned about disparities generated by PP being central for regulatory and HTA decision making as more influential advocacy groups may have a more central political role: “It favours that group of patients for the sole reason that they prefer it that way and they’ve been able to influence because of they have a very strong patient organization” (PY_SE_14).

4 Discussion

This study provides insights into stakeholders’ attitudes, needs, and concerns related to PP in decision making along the MPLC. Interviewees from all stakeholder groups stressed the importance of increasing stakeholders’ understanding of the concept of PP and preference methods and ensuring patients’ understanding of the questions asked in PP studies. Key concerns and needs per stakeholder group included the following (Fig. 1):
  • Academics: researching the validity, reliability, reproducibility, and generalizability of preference methods;

  • HTA/payer representatives: developing quality criteria for evaluating PP studies and gaining insights into how to weigh PP study results in decision making;

  • Industry representatives: obtaining guidance on the design and conduct of PP studies and recognition on the importance of PP from decision makers;

  • Patients, caregivers, and patient representatives: providing incentive and adequate information towards patients when participating in PP studies;

  • Physicians: avoiding bias due to commercial agendas in PP studies and clarifying how to address subjective and emotional elements when measuring PP; and

  • Regulators: avoiding the misuse of PP study results to overrule the traditional efficacy and safety criteria for marketing authorization and obtaining robust PP study results.

This study underpins the importance of educating stakeholders about the concept of PP and preference methods; 43% of interviewees reported not being familiar with PP, and interviewees from all stakeholder groups raised the need for higher awareness among stakeholders (decision makers, clinicians, academics, and patients) on PP studies. Examples of efforts to increase familiarity among stakeholders are accumulating, e.g., webinars by the FDA and congresses about health preference research [43, 44]. This study also points towards a critical need for a framework to measure PP more structurally. Such efforts are also ongoing, e.g., the issuance of draft guidance by the FDA for collecting and submitting patient experience data (including PP) [33].

Interviewees raised several methodological questions (e.g., about the sample and the validity of PP study results), some of which are touched upon in regulatory guidance [3] already available from before the initiation of this study. This finding could imply that interviewees were unaware of this guidance and/or that more research on these topics is needed because the existing guidance is not sufficiently detailed. The former implication might be explained by a lack of guidance in the European context; thus far, there is no guidance from the EMA on how to measure PP for regulatory purposes. The second implication has also been raised by Johnson et al. [45], stating that available FDA guidance “provides little help for readers in assessing the relevance of the listed methods for FDA regulatory reviews”. Regarding method selection, the present study reveals tension related to the desired level of robustness of preference methods; while interviewees from several stakeholder groups found it crucial that preference methods are scientifically robust, some interviewees argued that “simpler” methods are necessary to reduce the cognitive burden of the method and to reduce the cost and time needed to conduct the preference study. The FDA guidance [3] states that “newer methods may also be acceptable” but does not explicitly mention what their quality criteria would be.

Both on the HTA/reimbursement and marketing authorization level, some interviewees argued that using PP could improve the quality of decision making by increasing certainty for decision makers when they need to evaluate uncertain clinical evidence. Examples that reflect the interest of HTA bodies are the Institute for Quality and Efficiency in Health Care PP studies and the report by the Belgian Health Care Knowledge Centre10 on how to involve patients and the public in reimbursement and resource allocation decisions [2, 29, 36]. However, no formal guidance has been issued by HTA bodies or reimbursement decision makers, which might explain the uncertainty expressed by interviewees on how to conduct PP studies for HTA and reimbursement.

As suggested by some regulators in this study and reflected in a recent example in which the National Institute for Health and Care Excellence provided scientific advice [37], gathering input from decision makers on specific PP study designs may be an important step forward. Based on their experience, the National Institute for Health and Care Excellence recommends that companies seek their advice on PP study designs to allow for the PP study results to be used in clinical development programs for new treatments. Such advice may be both valuable from the company’s perspective to ensure that PP study results will be useful for decision makers as well as from the decision makers’ perspective to ensure that potential methodological issues are addressed prior to conducting the PP study.

The main strength of this study is its inclusiveness; interviewees were equally recruited from six different stakeholder groups and eight countries, resulting in a heterogeneous and large sample. However, the magnitude of this study can also be viewed as a limitation because it precluded gaining in-depth insights into the motivations and background of each interviewee. We captured the country, stakeholder group, and familiarity of interviewees but did not differentiate results according to other interviewee characteristics. For example, their opinions may also be shaped by the type of preference method they were most familiar with.

It is important to interpret the results in light of interviewees’ knowledge of the topic; interviewees reported being unfamiliar (43%), moderately familiar (42%), or very familiar (15%) with preference methods and studies. Self-reported familiarity was highest among academics and lowest among patients and physicians. We tried to account for a lower familiarity among these latter stakeholder groups by developing a simpler version of the interview guide for these stakeholder groups. Nonetheless, interviewees’ opinions were most likely influenced by their familiarity regarding the topic. For example, the stakeholder groups least familiar with PP studies may also be the individuals least positive about using PP study results in decisions. Second, interviewees were recruited upon suggestion by PREFER members and by interviewed persons. This might have resulted in a biased sample because these persons may be more positive about PP. The opinions presented in this paper should therefore not be interpreted as representative of a larger population than the interviewed sample; there may very well be persons within each stakeholder group with views other than those presented in the paper.

Additionally, four interviewees per stakeholder group from each country do not enable comparison across all possible permutations of relevant interviewees’ features (country, stakeholder group, disease area). There were eight interviewers with likely different interviewing styles. However, having eight interviewers was necessary to conduct interviews in some interviewees’ native language. We tried to minimize differences in interviewing styles by using an interviewing protocol and organizing an explanatory protocol meeting. All but one interviewer (AC) were part of the PREFER project. Although interviewers had no personal interest in the study outcome, their knowledge and assumptions about the research topic may have shaped the direction of the interviews. A final limitation relates to terminology; after interviewees’ personal definitions of PP were elicited, a definition of PP was provided at the beginning of the interview. Nevertheless, interviewees might have referred to their own pre-existing definitions during the interview.

5 Conclusions

Despite interest from interviewees within each of the stakeholder groups, an effective use of PP is hampered by a lack of standardization and consensus on how to measure PP and use PP in decision making. To advance usage of PP, efforts are needed on three levels that mirror stakeholders’ needs and concerns: (1) the educational and cultural level, via increased awareness and understanding among stakeholders about concept of PP, preference methods and the potential value of using PP in decision making; (2) the methodological level, via increased understanding of quality criteria of PP studies; and (3) the procedural level, via increased understanding on how to integrate PP into decision making.


  1. 1.

    Scientific institute advising reimbursement decisions in Germany.

  2. 2.

    Consultancy service providing scientific advice to industry for National Institute for Health and Care Excellence evaluations and supporting discussions with payers and commissioners to enable market access in the UK.

  3. 3.

    A 5-year project that received funding from the Innovative Medicines Initiative 2 Joint Undertaking. PREFER aims to establish recommendations to guide industry, regulatory authorities, and HTA/reimbursement bodies on how and when to include PP.

  4. 4.

    Lung cancer, rheumatoid arthritis, myotonic dystrophy type 1, and cardiovascular disease were selected to include diseases varying in prevalence and chronicity.

  5. 5.

    The median interview date occurred at the latest time point in Italy and Romania. The final interviews took place with HTA body and regulatory interviewees.

  6. 6.

    RJ, EvO, CW, and KSB are PhD researchers with backgrounds in biomedical sciences (RJ, EvO), global health (CW), and public health (KSB). SR is a post-doctoral researcher with a background in health and cognitive psychology and trained in qualitative research methods. AC is the founder of Community Health Association Romania. ME is a data scientist with a background in psychology. RD is a senior director of benefit-risk and epidemiology at Janssen Research & Development.

  7. 7.

    The interview guide included fixed questions, obligatory to address in the interview, and questions to be asked if interviewees were familiar with preference methods and studies. Per question, additional questions could be asked for further explanation, for confirmation or for more in-depth answers from interviewees.

  8. 8.

    Repeat interviews were not performed.

  9. 9.

    Quotations are coded as follows: abbreviation of stakeholder group, country, and ID number. Abbreviations for the stakeholder groups are as follows: PA patients, PO patient organization representatives, CA caregivers, IN industry representatives, RE regulators, PY HTA/payer representatives, PH physicians, AC academics.

  10. 10.

    Scientific institute advising reimbursement decisions in Belgium.



The authors thank the interviewees for their valuable input and time. The authors are also thankful for all the help they received from everyone within and outside the PREFER project for making this study possible, from their efforts in applying for ethics approval through to their valuable suggestions for interviewees. A special thank you to Rachael DiSantostefano for helping with the US interviews.

Author contributions

All authors contributed to the protocol design of this study and/or the acquisition of the data. RJ, SR, EvO, CW, KSB, AC, ME, and RD conducted the interviews. RJ and SR performed the analysis and wrote the initial draft and IH and JV were involved in the further refinement of the main text and figures. All authors reviewed the study materials and findings. All authors read and approved the final version before submission.

Compliance with Ethical Standards


This study is part of the PREFER project. PREFER is a 5-year project that has received funding from the Innovative Medicines Initiative 2 Joint Undertaking under Grant agreement no. 115966. This joint undertaking receives support from the European Union’s Horizon 2020 research and innovation programme and the European Federation of Pharmaceutical Industries and Associations. The funding source did not have a role in the study design, collection, analysis, and interpretation of the data, in the writing of the article, and in the decision to submit it for publication.

Conflict of interest

Jürgen Kübler’s work was funded by CSL Behring. Bennett Levitan is an employee of Janssen Research and Development, LLC. He is a stockholder in Johnson & Johnson, Baxter International, Inc., Pharmaceutical Holders Trust, and Zimmer Holdings, Inc. He also owns stock in a variety of companies that at times include other pharmaceutical and healthcare-related companies. Richard Hermann is an employee and stockholder at AstraZeneca Pharmaceuticals, LP. Juhaeri Juhaeri is an employee of Sanofi and a holder of stock options and restricted shares of Sanofi. He also owns accounts in various investment companies that may include other biopharmaceutical and healthcare-related companies. Sarah Harding is an employee of Takeda International, UK Branch.

Ethics Approval

Ethics approval was obtained in all countries where interviewees were recruited (registration number, country): Medical Ethics Committee of UZ KU Leuven/Research (S59790, Belgium), Commission Nationale de l’Informatique et des Libertés (2036344, CNIL, France), Ethik-Kommission der Friedrich-Alexander Universität (92_17 B, Germany), Comitato Etico Instituto Europeo di Oncologia (R587/17-IEO 609, Italy), Medisch Ethische Toetsings Commissie Erasmus Medical Centre (WT/ss/METC306661, the Netherlands), Comisia de Bioetica a Medicamentului si a Dispozitivelor Medicale [CNBMDM] (5 SNI, Romania), Regionala Etikprövningsnämnden Uppsala (EPN] (2017/001/1, Sweden), Newcastle University Ethics Committee (11307/2016, UK), and Western Institutional Review Board [WIRB] (1-1010535-1, USA).

Consent to participate

All interviewees provided written informed consent to participate in the interview and for using the coded data of their interview for publication in scientific journals.

Data availability

The data are not publicly available because they contain information that could compromise interviewees’ privacy and consent.

Supplementary material

40271_2019_367_MOESM1_ESM.docx (27 kb)
Supplementary material 1 (DOCX 27 kb)
40271_2019_367_MOESM2_ESM.docx (34 kb)
Supplementary material 2 (DOCX 33 kb)
40271_2019_367_MOESM3_ESM.docx (17 kb)
Supplementary material 3 (DOCX 16 kb)
40271_2019_367_MOESM4_ESM.docx (29 kb)
Supplementary material 4 (DOCX 28 kb)
40271_2019_367_MOESM5_ESM.docx (30 kb)
Supplementary material 5 (DOCX 30 kb)
40271_2019_367_MOESM6_ESM.xlsx (40 kb)
Supplementary material 6 (XLSX 39 kb)
40271_2019_367_MOESM7_ESM.docx (29 kb)
Supplementary material 7 (DOCX 28 kb)
40271_2019_367_MOESM8_ESM.pdf (511 kb)
Supplementary material 8 (PDF 511 kb)


  1. 1.
    Marsh K, Caro JJ, Hamed A, Zaiser E. Amplifying each patient’s voice: a systematic review of multi-criteria decision analyses involving patients. Appl Health Econ Health Policy. 2017;15(2):155–62.Google Scholar
  2. 2.
    Christiaens W, Kohn L, Léonard C, Denis A, Daue F, Cleemput I. Models for citizen and patient involvement in health care policy. Part I: exploration of their feasibility and acceptability. Health Services Research (HSR). Brussels: Belgian Health Care Knowledge Centre (KCE); 2012: report no.: KCE reports 195C, D/2012/10.273/105.Google Scholar
  3. 3.
    U.S. Department of Health and Human Services, Food and Drug Administration, Center for Devices and Radiological Health and Center for Biologics Evaluation and Research. Patient preference information: voluntary submission, review in premarket approval applications, humanitarian device exemption applications, and de novo requests, and inclusion in decision summaries and device labeling: guidance for industry, Food and Drug Administration staff, and other stakeholders. 2016.
  4. 4.
    Carson RT, Louviere JJ. A Common nomenclature for stated preference elicitation approaches. Environ Resource Econ. 2011;49:539–59.Google Scholar
  5. 5.
    Weernink MGM, Janus SIM, van Til JA, Raisch DW, van Manen JG, Ijzerman MJ. A systematic review to identify the use of preference elicitation methods in healthcare decision making. Pharm Med. 2014;28(4):175–85.Google Scholar
  6. 6.
    Gutknecht M, Schaarschmidt ML, Herrlein O, Augustin M. A systematic review on methods used to evaluate patient preferences in psoriasis treatments. J Eur Acad Dermatol Venereol. 2016;30(9):1454–64.Google Scholar
  7. 7.
    van Til JA, Ijzerman MJ. Why should regulators consider using patient preferences in benefit-risk assessment? Pharmacoeconomics. 2014;32(1):1–4.Google Scholar
  8. 8.
    Mühlbacher AC, Juhnke C, Beyer AR, Garner S. Patient-focused benefit-risk analysis to inform regulatory decisions: the European Union perspective. Value Health. 2016;19(6):734–40.Google Scholar
  9. 9.
    Dirksen CD. The use of research evidence on patient preferences in health care decision-making: issues, controversies and moving forward. Expert Rev Pharmacoecon Outcomes Res. 2014;14(6):785–94.Google Scholar
  10. 10.
    Turley M, Wang S, Meng D, Kanter MH, Garrido T. An information model for automated assessment of concordance between advance care preferences and care delivered near the end of life. J Am Med Inform Assoc. 2016;23(e1):e118–24.Google Scholar
  11. 11.
    European Medicines Agency, Stakeholders and Communication Division. The patient’s voice in the evaluation of medicines. Report no. EMA/607864/2013.
  12. 12.
    European Medicines Agency. Regulatory and methodological standards to improve benefit-risk evaluation of medicines 2014. Accessed 4 June 2019.
  13. 13.
    Peay HL, Hollin I, Fischer R, Bridges JF. A community-engaged approach to quantifying caregiver preferences for the benefits and risks of emerging therapies for Duchenne muscular dystrophy. Clin Ther. 2014;36(5):624–37.Google Scholar
  14. 14.
    Smith MY, Hammad TA, Metcalf M, Levitan B, Noel R, Wolka AM, et al. Patient engagement at a tipping point: the need for cultural change across patient, sponsor, and regulator stakeholders: insights from the DIA conference, “Patient Engagement in Benefit Risk Assessment Throughout the Life Cycle of Medical Products”. Ther Innov Regul Sci. 2016;50(5):546–53.Google Scholar
  15. 15.
    Bridges JFP, Paly VF, Barker E, Kervitsky D. Identifying the benefits and risks of emerging treatments for idiopathic pulmonary fibrosis: a qualitative study. Patient. 2014;8(1):85–92.Google Scholar
  16. 16.
    Egbrink MO, Ijzerman M. The value of quantitative patient preferences in regulatory benefit-risk assessment. J Mark Access Health Policy. 2014;2:22761.Google Scholar
  17. 17.
    Ho MP, Gonzalez JM, Lerner HP, Neuland CY, Whang JM, McMurry-Heath M, et al. Incorporating patient-preference evidence into regulatory decision making. Surg Endosc. 2015;29(10):2984–93.Google Scholar
  18. 18.
    Ho M, Saha A, McCleary KK, Levitan B, Christopher S, Zandlo K, et al. A framework for incorporating patient preferences regarding benefits and risks into regulatory assessment of medical technologies. Value Health. 2016;19(6):746–50.Google Scholar
  19. 19.
    Medical Device Innovation Consortium. Patient centered benefit-risk project report: a framework for incorporating information on patient preferences regarding benefit and risk into regulatory assessments of new medical technology. Arlington: Medical Device Innovation Consortium; 2015.
  20. 20.
    Morel T, Ayme S, Cassiman D, Simoens S, Morgan M, Vandebroek M. Quantifying benefit-risk preferences for new medicines in rare disease patients and caregivers. Orphanet J Rare Dis. 2016;11(1):70.Google Scholar
  21. 21.
    Selig WKD. Key considerations for developing and integrating patient perspectives in drug development: examination of the duchenne case study. Biotechnology Innovation Organization and Parent Project Muscular Dystrophy; 2016.
  22. 22.
    Mühlbacher AC, Bridges JF, Bethge S, Dintsios CM, Schwalm A, Gerber-Grote A, et al. Preferences for antiviral therapy of chronic hepatitis C: a discrete choice experiment. Eur J Health Econ. 2017;18(2):155–65.Google Scholar
  23. 23.
    Hughes D, Waddingham EAJ, Mt-Isa S, Goginsky A, Chan E, Downey G, et al. Recommendations for the methodology and visualisation techniques to be used in the assessment of benefit and risk of medicines. Innovative Medicines Initiative, Pharmacoepidemiological Research on Outcomes of Therapeutics by a European ConsorTium (PROTECT), Benefit-Risk Group; 2013.
  24. 24.
    Avila M, Becerra V, Guedea F, Suarez JF, Fernandez P, Macias V, et al. Estimating preferences for treatments in patients with localized prostate cancer. Int J Radiat Oncol Biol Phys. 2015;91(2):277–87.Google Scholar
  25. 25.
    Martin-Fernandez J, Polentinos-Castro E, del Cura-Gonzalez MI, Ariza-Cardiel G, Abraira V, Gil-LaCruz AI, et al. Willingness to pay for a quality-adjusted life year: an evaluation of attitudes towards risk and preferences. BMC Health Serv Res. 2014;14:287.Google Scholar
  26. 26.
    Mott DJ, Najafzadeh M. Whose preferences should be elicited for use in health-care decision-making? A case study using anticoagulant therapy. Expert Rev Pharmacoecon Outcomes Res. 2016;16(1):33–9.Google Scholar
  27. 27.
    Mühlbacher AC. Patient-centric HTA: different strokes for different folks. Expert Rev Pharmacoecon Outcomes Res. 2015;15(4):591–7.Google Scholar
  28. 28.
    Danner M, Hummel JM, Volz F, Van Manen JG, Wiegard B, Dintsios CM, et al. Integrating patients’ views into health technology assessment: analytic hierarchy process (AHP) as a method to elicit patient preferences. Int J Technol Assess Health Care. 2011;27(4):369–75.Google Scholar
  29. 29.
    Hummel MJM, Volz F, Van Manen JG, Danner M, Dintsios CM, Ijzerman MJ, et al. Using the analytic hierarchy process to elicit patient preferences: prioritizing multiple outcome measures of antidepressant drug treatment. Patient. 2012;5(4):22–37.Google Scholar
  30. 30.
    Chaudhuri SE, Ho MP, Irony T, Sheldon M, Lo AW. Patient-centered clinical trials. Drug Discov Today. 2018;23(2):395–401.Google Scholar
  31. 31.
    European Medicines Agency. Values and preferences for treatment outcomes: the Macbeth approach. Accessed 4 June 2019.
  32. 32.
    US FDA. PDUFA VI: PDUFA reauthorization performance goals and procedures fiscal years 2018 through 2022. 2016. Accessed 4 June 2019.
  33. 33.
    U.S. Department of Health and Human Services Food and Drug Administration, Center for Biologics Evaluation and Research (CBER), editor. Patient-focused drug development: collecting comprehensive and representative input: guidance for industry, Food and Drug Administration staff, and other stakeholders: draft guidance; 2018.
  34. 34.
    U.S. Food and Drug Administration. FDA’s external resources or information related to patients’ experience. Accessed 14 Sept 2018.
  35. 35.
    Mott DJ. Incorporating quantitative patient preference data into healthcare decision making processes: is HTA falling behind? Patient. 2018;11(3):249–52.Google Scholar
  36. 36.
    Choice-based conjoint analysis: pilot project to identify, weight, and prioritize multiple attributes in the indication “hepatitis C”. Institute for Quality and Efficiency in Health Care (IQWiG); 2014: report no. GA10-03.Google Scholar
  37. 37.
    NICE. NICE provides first scientific advice on patient preference study design [updated 15 February 2019]. Accessed 4 June 2019.
  38. 38.
    Postmus D, Mavris M, Hillege HL, Salmonson T, Ryll B, Plate A, et al. Incorporating patient preferences into drug development and regulatory decision making: results from a quantitative pilot study with cancer patients, carers, and regulators. Clin Pharmacol Ther. 2016;99(5):548–54.Google Scholar
  39. 39.
    Mühlbacher AC, Kaczynski A. Making good decisions in healthcare with multi-criteria decision analysis: the use, current research and future development of MCDA. Appl Health Econ Health Policy. 2016;14(1):29–40.Google Scholar
  40. 40.
    Ritchie J, Lewis J. Qualitative research practice: a guide for social science students and researchers. Thousand Oaks: SAGE Publications; 2003.Google Scholar
  41. 41.
    Gale N, Heath G, Cameron E, Rashid S, Redwood S. Using the framework method for the analysis of qualitative data in multi-disciplinary health research. BMC Med Res Methodol. 2013;13(1):117.Google Scholar
  42. 42.
    Lacey A, Luff D. Qualitative research analysis: the NIHR RDS for the East Midlands/Yorkshire & the Humber; 2007.
  43. 43.
    US Food and Drug Administration. Webinar: final guidance on patient preference information. 2016. Accessed 4 June 2019.
  44. 44.
    International Academy of Health Preference Research. Accessed 14 Sept 2018.
  45. 45.
    Johnson FR, Beusterien K, Ozdemir S, Wilson L. Giving patients a meaningful voice in United States regulatory decision making: the role for health preference research. Patient. 2017;10(4):523–6.Google Scholar

Copyright information

© The Author(s) 2019

Open AccessThis article is distributed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License (, which permits any noncommercial use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Authors and Affiliations

  • Rosanne Janssens
    • 1
    Email author
  • Selena Russo
    • 2
    • 3
    • 4
  • Eline van Overbeeke
    • 1
  • Chiara Whichello
    • 5
  • Sarah Harding
    • 6
  • Jürgen Kübler
    • 7
  • Juhaeri Juhaeri
    • 8
  • Karin Schölin Bywall
    • 9
  • Alina Comanescu
    • 10
  • Axel Hueber
    • 11
  • Matthias Englbrecht
    • 11
  • Nikoletta Nikolenko
    • 12
  • Gabriella Pravettoni
    • 2
  • Steven Simoens
    • 1
  • Hilde Stevens
    • 13
  • Richard Hermann
    • 14
  • Bennett Levitan
    • 15
  • Irina Cleemput
    • 16
  • Esther de Bekker-Grob
    • 5
  • Jorien Veldwijk
    • 5
  • Isabelle Huys
    • 1
  1. 1.Department of Pharmaceutical and Pharmacological SciencesKU LeuvenLeuvenBelgium
  2. 2.Applied Research Division for Cognitive and Psychological ScienceIEO European Institute of OncologyMilanItaly
  3. 3.Faculty of Medicine, School of Women’s and Children’s HealthUniversity of New South WalesKensingtonAustralia
  4. 4.Behavioural Sciences Unit, Kids Cancer CentreSydney Children’s HospitalRandwickAustralia
  5. 5.Erasmus School of Health Policy & Management (ESHPM) and Erasmus Choice Modelling Centre (ECMC)Erasmus University RotterdamRotterdamThe Netherlands
  6. 6.Takeda InternationalLondonUK
  7. 7.QSciConMarburgGermany
  8. 8.SanofiBridgewaterUSA
  9. 9.Centre for Research Ethics & BioethicsUppsala UniversityUppsalaSweden
  10. 10.Community Health Association RomaniaBucharestRomania
  11. 11.Department of Internal Medicine 3, Rheumatology and ImmunologyFriedrich-Alexander-Universität Erlangen-Nürnberg and Universitätsklinikum ErlangenErlangenGermany
  12. 12.John Walton Muscular Dystrophy Research Centre, Institute of Genetic MedicineNewcastle UniversityNewcastle Upon TyneUK
  13. 13.Institute for Interdisciplinary Innovation in Healthcare (I3h)Université libre de BruxellesBrusselsBelgium
  14. 14.AstraZenecaGaithersburgUSA
  15. 15.Global R&D EpidemiologyJanssen Research & DevelopmentTitusvilleUSA
  16. 16.Belgian Health Care Knowledge Centre (KCE)BrusselsBelgium

Personalised recommendations