Skip to main content
SpringerLink
Log in

Exploring Uncertainty in Cost-Effectiveness Analysis

  • Briefing Paper
  • Published:
PharmacoEconomics Aims and scope Submit manuscript

Abstract

This paper describes the key principles of why an assessment of uncertainty and its consequences are critical for the types of decisions that a body such as the UK National Institute for Health and Clinical Excellence (NICE) has to make. In doing so, it poses the question of whether formal methods may be useful to NICE and its advisory committees in making such assessments. Broadly, these include the following: (i) should probabilistic sensitivity analysis continue to be recommended as a means to characterize parameter uncertainty; (ii) which methods should be used to represent other sources of uncertainty; (iii) when can computationally expensive models be justified and is computation expense a sufficient justification for failing to express uncertainty; (iv) which summary measures of uncertainty should be used to present the results to decision makers; and (v) should formal methods be recommended to inform the assessment of the need for evidence and the consequences of an uncertain decision for the UK NHS?

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Table I
Fig. 2
Fig. 3
Table II
Table III
Fig. 4

Similar content being viewed by others

References

  1. Culyer AJ, McCabe C, Briggs A, et al. Searching for a threshold, not setting one: the role of the National Institute for Health and Clinical Excellence. J Health Serv Res Pol 2007; 12: 56–58

    Article  Google Scholar 

  2. McCabe C, Claxton K, Culyer AJ. The NICE cost-effectiveness threshold: what it is and what that means. Pharmacoeconomics 2008; 26 (9): 733–744

    Article  PubMed  Google Scholar 

  3. Phelps CE, Mushlin Al. On the (near) equivalence of cost-effectiveness and cost-benefit analyses. Int J Technol Assess Health Care 1991; 7 (1): 12–21

    Article  PubMed  CAS  Google Scholar 

  4. Stinnett A, Mullahy J. Net health benefits: a new framework for the analysis of uncertainty in cost-effectiveness analysis. Med Decis Making 1998; 18: S68–S80

    Article  PubMed  CAS  Google Scholar 

  5. Sculpher MJ. Subgroups and heterogeneity in cost-effectiveness analysis. Pharmacoeconomics 2008; 26 (9): 799–806

    Article  PubMed  Google Scholar 

  6. Claxton K. The irrelevance of inference: a decision making approach to the stochastic evaluation of health care technologies. J Health Econ 1999; 18: 342–364

    Article  Google Scholar 

  7. Briggs A, Sculpher M. An introduction to Markov modelling for economic evaluation. Pharmacoeconomics 1998; 13: 397–409

    Article  PubMed  CAS  Google Scholar 

  8. Briggs A, Claxton K, Sculpher MJ. Decision analytic modelling for the evaluation of health technologies. Oxford: Oxford University Press, 2006

    Google Scholar 

  9. Eckermann S, Willan A. Expected value of information and decision making in HTA. Health Econ 2007; 16: 195–209

    Article  PubMed  Google Scholar 

  10. Palmer S, Smith PC. Incorporating option values into the economic evaluation of health care technologies. J Health Econ 2000; 19: 755–766

    Article  PubMed  CAS  Google Scholar 

  11. Griffin S, Claxton K, Palmer S, et al. Dangerous omissions: the consequences of ignoring decision uncertainty. Med Dec Making 2007; 27 (1): E8

    Google Scholar 

  12. Chalkidou K, Hoy A, Littlejohns P. Making a decision to wait for more evidence: when the National Institute for Health and Clinical Excellence recommends a technology only in the context of research. J R Soc Med 2007; 100 (10): 453–460

    Article  PubMed  Google Scholar 

  13. National Institute for Health and Clinical Excellence. NICE guide to the methods of health technology appraisal. London: NICE, 2004

    Google Scholar 

  14. Cooksey D. A review of UK health research funding. London: Stationery Office, 2006

    Google Scholar 

  15. Office of Fair Trading. The pharmaceutical price regulation scheme: an OFT market study. London: Office of Fair Trading, 2007

    Google Scholar 

  16. Claxton K, Briggs A, Buxton MJ, et al. Value-based pricing for NHS drugs: an opportunity not to be missed? BMJ 2008; 336: 251–254

    Article  PubMed  Google Scholar 

  17. Claxton K, Sculpher MJ, McCabe C, et al. Probabilistic sensitivity analysis for NICE technology assessment: not an optional extra. Health Econ 2005; 14: 339–347

    Article  PubMed  Google Scholar 

  18. Thompson K, Graham J. Going beyond the single number: using probabilistic risk assessment to improve risk management. Human Ecologic Risk Assess 1996; 2: 1008–1034

    Article  Google Scholar 

  19. O’Hagan A, Luce BR. A primer on Bayesian statistics in health economics and outcomes research. Bethesda (MD): Bayesian initiative in health economic and outcomes research, MEDTAP International, 2003

    Google Scholar 

  20. O’Hagan A, Buck CE, Daneshkhah A, et al. Uncertain judgements: eliciting expert probabilities. Chichester: John Wiley and Sons, 2006

    Book  Google Scholar 

  21. Sutton A, Ades AE, Cooper N, et al., on behalf of the NICE Decision Support Unit. Use of indirect and mixed treatment comparisons for technology assessment. Pharmacoeconomics 2008; 26 (9): 753–767

    Article  PubMed  Google Scholar 

  22. Ades AE, Sutton AJ. Multiparameter evidence synthesis in epidemiology and medical decision-making: current approaches. JRSS (A) 2005; 16 (1): 5–35

    Google Scholar 

  23. Ades AE, Claxton K, Sculpher MJ. Evidence synthesis, parameter correlation, and probabilistic sensitivity analysis. Health Econ 2006; 15: 373–381

    Article  PubMed  CAS  Google Scholar 

  24. Bojke L, Claxton K, Palmer S, et al. Defining and characterising structural uncertainty in decision analytic models [CHE research paper 9]. York: Centre for Health Economics, University of York, 2006

    Google Scholar 

  25. McCabe C, Dixon S. Testing the validity of cost-effectiveness models. Pharmacoeconomics 2000; 17 (5): 501–513

    Article  PubMed  CAS  Google Scholar 

  26. Draper D. Assessment and propagation of model uncertainty. J Royal Stat Soc 1995; 57: 45–97

    Google Scholar 

  27. Chatfield C. Model uncertainty, data mining and statistical inference (with discussion). J Royal Stat Soc 1995; 158: 419–466

    Article  Google Scholar 

  28. Bojke L, Claxton K, Bravo Vergel Y. Using expert elicitation to resolve issues of structural uncertainty in decision analytic models. Med Dec Making 2007; 27 (1): E8

    Google Scholar 

  29. White IR, Pocock SJ, Wang D. Eliciting and using expert opinions about dropout bias in randomised controlled trials. Clin Trials 2007; 4 (2): 125–139

    Article  PubMed  Google Scholar 

  30. Leal J, Wordsworth S, Legood R, et al. Eliciting expert opinion for economic models. Value Health 2007; 10 (3): 195–203

    Article  PubMed  Google Scholar 

  31. Griffin S, Claxton K, Hawkins N, et al. Probabilistic analysis and computationally expensive models: necessary and required? Value Health 2006; 9: 244–252

    Article  PubMed  Google Scholar 

  32. Edmunds WJ, Medley GF, Nokes DJ. Evaluating the cost-effectiveness of vaccination programmes: a dynamic perspective. Stat Med 1999; 18 (23): 3263–3282

    Article  PubMed  CAS  Google Scholar 

  33. O’Hagan A, Stevenson M, Madan J. Monte Carlo probabilistic sensitivity analysis for patient level simulation models: efficient estimation of mean and variance using ANOVA. Health Econ 2007; 16 (10): 1009–1023

    Article  PubMed  Google Scholar 

  34. Stevenson M, Oakley J, Chilcott J. Gaussian process modelling in conjunction with individual patient simulation modelling: a case study describing the calculation of cost-effectiveness ratios for the treatment of established osteoporosis. Med Dec Making 2004: 24: 89–100

    Article  CAS  Google Scholar 

  35. Ades AE, Lu G, Claxton K. Expected value of sample information in medical decision modelling. Med Dec Making 2004; 24: 207–227

    Article  CAS  Google Scholar 

  36. Van Hout B A, Al MJ, Gordon GS, et al. Costs, effects and c/e-ratios alongside a clinical trial. Health Econ 1994; 3: 309–319

    Article  Google Scholar 

  37. Fenwick E, Claxton K, Sculpher MJ. Representing uncertainty: the role of cost-effectiveness acceptability curves. Health Econ 2001; 10: 779–789

    Article  PubMed  CAS  Google Scholar 

  38. Fenwick E, O’Brien B, Briggs AH. Cost-effectiveness acceptability curves: facts, fallacies and frequently asked questions. Health Econ 2004; 13: 405–415

    Article  PubMed  Google Scholar 

  39. Colbourn T, Asseburg C, Bojke L, et al. Preventive strategies for group B streptococcal and other bacterial infections in early infancy: cost effectiveness and value of information analyses. BMJ 2007; 335: 655–662

    Article  PubMed  Google Scholar 

  40. Pratt J, Raiffa H, Schlaifer R. Statistical decision theory. Cambridge (MA): MIT Press, 1995

    Google Scholar 

  41. Yokota F, Thompson KM. Value of information literature analysis: a review of applications in health risk management. Med Dec Making 2004; 24: 287–298

    Article  Google Scholar 

  42. Poynard T, Munteanu M, Ratziu V, et al. Truth survival in clinical research: an evidence-based requiem? Ann Intern Med 2002; 136: 888–895

    PubMed  Google Scholar 

  43. Philips Z, Claxton K, Palmer P. The half-life of truth: what are appropriate time horizons for research decisions? Med Dec Making 2008; 28: 287–299

    Article  Google Scholar 

  44. Brennan A, Kharroubi S, O’Hagan A, et al. Calculating partial expected value of perfect information via Monte Carlo sampling algorithms. Med Dec Making 2007; 27: 448–470

    Article  Google Scholar 

  45. Felli JC, Hazen GB. Sensitivity analysis and the expected value of perfect information. Med Dec Making 1998; 18: 95–109

    Article  CAS  Google Scholar 

  46. Griffin S, Claxton K. Interpreting the expected value of perfect information about parameters. Med Dec Making 2007; 27 (1): E8

    Google Scholar 

  47. Brennan A, Kharroubi S. Expected value of sample information for Weibull survival data. Health Econ 2007: 16: 1205–1225

    Article  PubMed  Google Scholar 

  48. Conti S, Claxton K. Dimensions of design space: a decision theoretic approach to optimal research design [CHE research paper 38]. York: Centre for Health Economics, University of York, 2008

    Google Scholar 

Download references

Acknowledgements

This paper was initially prepared as a briefing paper for NICE as part of the process of updating the Institute’s 2004 Guide to the Methods of Technology Appraisal. The work was funded by NICE through its Decision Support Unit (DSU), which is based at the universities of Sheffield, Leicester, York, Leeds and at the London School of Hygiene and Tropical Medicine.

The author has no conflicts of interest that are directly related to the contents of this article.

The author thanks members of the DSU who commented on the briefing document that forms the basis of this paper as well as Iain Chalmers, Alex Sutton, Alan Brennan, Louise Longworth and Carole Longson, who provided helpful comments on earlier drafts of this paper. All errors and omissions are the responsibility of the author.

Author information

Authors and Affiliations

  1. Centre for Health Economics, Department of Economics and NICE Decision Support Unit, University of York, Heslington, York, YO10 5DD, UK

    Karl Claxton

Authors
  1. Karl Claxton
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Karl Claxton.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Claxton, K. Exploring Uncertainty in Cost-Effectiveness Analysis. Pharmacoeconomics 26, 781–798 (2008). https://doi.org/10.2165/00019053-200826090-00008

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.2165/00019053-200826090-00008

Keywords

Search

Navigation