Decision-Making in Drug Development: Application of a Clinical Utility Index^SM

Abstract

The Clinical Utility Index^SM (CUI^SM) is a practical and useful approach for informing drug development decisions where multiple aspects of a drug’s clinical profile must be optimally balanced. In its most common application, a CUI is a model of population-level preferences across and within the various efficacy, safety, and tolerability attributes of a product profile for a given indication. By combining the multiple dimensions of a product profile into a single unit of utility, the CUI can easily be linked to pharmacometric simulations of clinical endpoints, thus extending the reach of clinical trial modeling and simulations further into the decision-making process. Although the use of CUI (and related multiattribute methods) is relatively new to the field of drug development, its application is steadily growing as project teams discover the benefits of integrating “utility” models into the overall pharmacometric toolbox.

Appendix: History and Theory

Decision theory traces its roots back to the classic text by John Von Neumann and Oskar Morgenstern, Games and Economic Behavior (Von Neumann and Morgenstern 1944). Although the concept of utility can be traced back to the even earlier philosophers, this text gave the first axiomatic foundation for the concept of utility. Many authors elaborated on the work of Von Neumann and Morgenstern, but the classic text of Keeney and Raiffa (1976) provided one of the most detailed explorations of this area. Edwards (1961) was an early proponent of Bayesian methods in psychological research. He and his colleagues investigated additive utility functions in great detail. In general, they found that this approximation works extremely well in practice, even though it may be criticized on theoretical grounds for lack of preferential independence.

Francis Anscombe was one of the first to point out the problems with traditional Neyman–Pearson statistics in the performance of clinical trials and advocated the Bayesian sequential approach (Anscombe 1963, 1990). Although he did not specifically identify utility functions, he advocated a “decision function” approach or a “net profitability” as a result of “economic valuation.” This approach is very similar to the “west coast school of decision analysis,” which uses a utility function to reflect the risk aversion of the decision-maker.

This approach lay dormant and essentially ignored by the pharmaceutical industry until recently. A series of papers by Berry and various colleagues has proposed the use of sequential trials and the use of financial evaluations (Rosner and Berry 1995; Berry 2006).

1.1 Notes on Multiplicative Functions

Multiattribute additive utility has been widely used and demonstrated to be robust. An attribute is “preferentially independent” from all other attributes when changes in the rank ordering of preferences of other attributes does not change the preference order of the attribute. Since the change in utility with respect to any individual attribute (indicated as a derivative) does not depend upon any other attribute, multiattribute additive utility is “preferentially independent.” Additive independence is slightly stronger than preferential independence. As an example (Fig. 5.13), consider two attributes: (1) vehicle color and (2) vehicle type. Further, let us restrict ourselves to two colors: red and black. The vehicle types are: (1) sports car and (2) SUV. If the decision-maker (DM) is shown the following two lotteries (note that “lottery” here refers to simple “coin-flip” games of chance), and if the DM is indifferent between the two lotteries, then color is additively independent of car type and the multiattribute additive utility form is appropriate.

Fig. 5.13

Illustration of preferential independence

If these conditions do not hold, multiplicative forms of the utility function may be appropriate. There is one last condition, called “utility independence,” which implies a specific multiplicative form. Suppose we have two characteristics: (1) vehicle type (sports car and SUV once again) and (2) vehicle color (red, green and black in rank order), and suppose a probability p can be found such that the following two indifference conditions (Fig. 5.14) hold: the DM is indifferent to a green sports car and a 50–50 lottery for a red or black sports car and the DM is also indifferent between the same color choices of red or black with SUV replacing sports car (with the same probability). In this case, the DM is said to be “utility independent” and the form of the utility function is appropriate (Fig. 5.15).

Fig. 5.14

Illustration of utility independence

Fig. 5.15

Mathematical form of a multiplicative utility function

Once again, it is important to note that simpler forms of the utility function are often pretty reliable, in terms of choosing the right decision, even though the precise conditions needed to employ a particular form of the utility function are not fully met.

1.2 Basic Elicitation Steps

There are generally five parts to an elicitation of a univariate utility function (Keeney and Raiffa 1976):

• 1. Structure and scope relevant outcomes: Frame the decision problem which includes understanding the most important outcomes and attributes, developing the appropriate scales for measuring each attribute, exploring and explaining the assessment process and tools.

• 2. Identify relevant characteristics: Are certain attributes bounded, continuous, binary, categorical, monotonic, and preferentially independent? These attributes provide important information about the form of the utility index to be assessed.

• 3. Quantify specific values along the decision-makers utility index: Usually a series of comparisons among gambles is used to provide this insight.

• 4. Select the functional form of the CUI: The elicitor must make a judgment about the specific family of mathematical functions that will be used to encode the information obtained previously. The multiple attributes can be combined in an additive or multiplicative fashion.

• 5. Check for consistency: During this phase, certain lotteries not previously assessed are performed and compared to the preferences implied by the functional form fitted to previous assessments.

These parts are not necessarily performed in sequence. During the course of the assessment, it may be necessary to revisit previous parts. This serves to refine the judgments of the decision-maker or decision-making body. Assessments made in a group tend to be more difficult to perform but more robust in their application and acceptance. Committees should, if possible, be allowed to share information with each other and come to a joint belief. If a common view cannot be assessed, individual assessments can be made. There are mathematical procedures for combining several assessments, although sensitivity analysis is more useful to focus upon. Basically, if the decisions are identical regardless of the CUI chosen, we do not care about the differences. If the clinical recommendations would be different based on differing CUIs, then we may need to revisit this with the relevant decision makers.

1.3 Link to Conjoint Analysis

In a pharmaceutical company, it is typical that there is someone with fiduciary responsibility for the decision making. However, the decisions need to be made with two major groups in mind: (1) regulatory agencies and (2) prescribing physicians. This means that the CUI should reflect the judgments of these groups. There are group assessment methods that can be applied to a sample from either or both populations. These techniques are generally known as conjoint analysis (Elrod et al. 1992). Preference measurement comprises three interrelated components: (1) the problem that the study is ultimately intended to address; (2) the design of the preference measurement task and the data collection approach; (3) the specification and estimation of a preference model. Most of the group CUIs made by techniques such as conjoint analysis are limited in structure and less flexible than those assessed in the manner described earlier. However, they have one great advantage; they are direct measurements from a large sample of the population such as prescribing physicians who are ultimately the means by which prescription pharmaceuticals reach the target population. Hence, pharmaceutical development decisions can be focused to maximize the match between the preferences of the prescribing physicians and the target product profile.

Conjoint analysis requires research participants to make a series of tradeoffs. Analysis of these tradeoffs will reveal the relative importance of component attributes. The data is collected from survey respondents in a number of different ways. Traditionally it is administered as a ranking exercise and sometimes as a rating exercise (where the respondent awards each tradeoff scenario a score indicating appeal). In recent years it has become common practice to present the trade-offs as a choice exercise (where the respondent simply chooses the most preferred alternative from a selection of competing alternatives – common when simulating consumer choices) or as a constant sum allocation exercise (common in pharmaceutical market research, where physicians indicate likely shares of prescribing, and each alternative in the tradeoff is the description of a real or hypothetical therapy). Analysis is traditionally carried out with some form of multiple regressions, but recently, the use of hierarchical Bayesian analysis has become widespread, enabling robust statistical models of individual respondent decision behavior to be developed.