The National Institute for Health and Care Excellence (NICE) is only one of several organisations internationally that uses economic evaluation as part of decision making regarding funding and pricing of new medical technologies. However, it can be argued that NICE has developed a more prominent international profile than most in their use of economics. After 20 years of operation, it is timely to assess the extent of NICE’s achievements, including the economic evaluation methods it has used and its willingness to adapt these as new evaluative approaches emerge and when NICE faces particular policy challenges. This paper considers some of the important policy and contextual developments in the UK over the last 20 years and how these may have shaped NICE’s approach to economic evaluation. It then assesses key areas of NICE methods, including perspective, defining benefits, modelling and uncertainty. The paper concludes that NICE has provided important support for the development of new methods, in particular through its role in identifying priorities for methods research funding and its sponsorship of the NICE Decision Support Unit. However, potentially important developments in methods in a number of important areas have yet to be formally included in NICE’s methods guidance and this should be addressed in the Institute’s 2019/2020 methods review.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Tax calculation will be finalised during checkout.
Commonwealth Department of Health Housing and Community Services. Guidelines for the pharmaceutical industry on preparation of submissions to the Pharmaceutical Benefits Advisory Committee. Canberra: Australian Government Publishing Service; 1992.
Ministry of Health. Ontario guidelines for economic analysis of pharmaceutical products. Toronto: Ministry of Health; 1994.
Longworth L, Bojke L, Sculpher MJ, Tosh JC. Bridging the gap between methods research and the needs of policy makers: a review of the research priorities of the National Institute for Health and Clinical Excellence. Int J Technol Assess Health Care. 2011;27:1–8.
National Institute for Health and Care Excellence (NICE). Methods for the development of NICE public health guidance (third edition). London: NICE; 2018.
National Institute for Health and Care Excellence (NICE). Medical technologies evaluation programme methods guide. London: NICE; 2017.
National Institute of Clinical Excellence. Appraisal of new and existing technologies: interim guidance for manufacturers and sponsors. London: National Institute of Clinical Excellence; 1999.
Treasury HM. The green book. Central government guidance on appraisal and evaluation. London: HM Treasury; 2018.
National Institute Health and Care Excellence (NICE). Review of methods for the Health Technology Evaluation programme. Presented in papers for the Public Board Meeting and Annual General Meeting 17 July. London: NICE; 2019. https://www.nice.org.uk/Media/Default/Get-involved/Meetings-In-Public/Public-board-meetings/agenda-and-papers-july-19.pdf. Accessed 11 Nov 19.
National Institute for Clinical Excellence (NICE). Guide to the technology appraisal process. NICE: 2001.
National Institute for Clinical Excellence (NICE). Guide to the methods of technology appraisal. London: NICE; 2004.
Gold MR, Siegel JE, Russell LB, Weinstein MC. Cost-effectiveness in health and medicine. New York: Oxford University Press; 1996.
National Institute for Health and Clinical Excellence (NICE). Guide to the methods of technology appraisal. London: NICE; 2008.
National Institute for Health and Clinical Excellence (NICE). Updated guide to the methods of technology appraisal. London: NICE; 2013.
National Institute for Health and Care Excellence (NICE). NICE Citizens’ Council. London: NICE; 2019. https://www.nice.org.uk/get-involved/citizens-council. Accessed 26 Jul 2019.
National Institute for Clinical Excellence (NICE). Social value judgements, second edition. London: NICE; 2008. https://www.nice.org.uk/Media/Default/About/what-we-do/Research-and-development/Social-Value-Judgements-principles-for-the-development-of-NICE-guidance.pdf. Accessed 26 Jul 2019.
Kennedy I. Appraising the value of innovation and other benefits: a short study for NICE. London: National Institute for Health and Clinical Excellence; 2009.
Department of Health. A new value-based approach to the pricing of branded medicines—a consultation. London: Department of Health; 2010.
National Institute for Health and Care Excellence (NICE). Centre for Health Technology Evaluation, value based assessment of health technologies. London: NICE; 2014.
Department of Health and Social Care (DHSC). The. Voluntary scheme for branded medicines pricing and access—chapters and glossary. London: DHSC; 2019. p. 2018.
Claxton K, Martin S, Soares M, Rice N, Spackman E, Hinde S, et al. Methods for the estimation of the NICE cost effectiveness threshold. Health Technol Assess. 2015;19(14):503.
Department of Health and Social Care (DHSC). Impact assessment: 2018 statutory scheme—branded medicines pricing. London: DHSC; 2018. https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/761064/impact-assessment-2018-statutory-scheme-branded-medicines-pricing.pdf. Accessed 11 Nov 2019.
Woods B, Sideris E, Palmer S, Latimer N, Soares M. NICE Decision Support Unit Technical Support Document 19: partitioned survival analysis for decision modelling in health care: a critical review. Sheffield: NICE Decision Support Unit; 2017.
Faria R, Hernandez M, Manca A, Wailoo A. NICE Decision Support Unit Technical Support Document 17: the use of observational data to inform estimates of treatment effectiveness in technology appraisal: methods for comparative individual patient data. Sheffield: NICE Decision Support Unit; 2015.
Latimer NR. NICE Decision Support Unit Technical Support Document 14: survival analysis for economic evaluations alongside clinical trials: extrapolation with patient-level data. Sheffield: NICE Decision Support Unit; 2013.
Longworth L, Yang Y, Young T, Mulhern B, Hernández Alava M, Mukuria C, et al. Use of generic and condition-specific measures of Health-Related Quality of Life in NICE decision making. Health Technol Assess. 2014;18(9):1–224.
Claxton K, Palmer SJ, Longworth L, Bojke L, Griffiths D, McKenna C, et al. Informing a decision framework for when NICE should recommend the use of health technologies only in the context of an appropriately designed programme of evidence development. Health Technol Assess. 2012;16(46):1–342.
National Institute for Health and Care Excellence (NICE). Our projects and partners. London: NICE; 2019. https://www.nice.org.uk/about/what-we-do/our-research-work/our-projects-and-partners. Accessed 11 Nov 2019.
National Institute for Health and Care Excellence (NICE). Guide to the processes of technology appraisal. London: NICE; 2014. nice.org.uk/process/pmg19.
National Institute for Health and Clinical Excellence (NICE). Guide to the single technology appraisal process. London: NICE; 2006.
National Institute for Health and Care Excellence (NICE). Fast track appraisal: addendum to the guide to the processes of technology appraisal. London: NICE; 2018.
National Institute for Health and Care Excellence (NICE). PMG9 addendum—final amendments to the NICE technology appraisal methods guide to support the new Cancer Drugs Fund arrangements. London: NICE; 2018.
National Institute for Health and Care Excellence (NICE). Annual report and accounts 2016/17. London: NICE; 2017.
National Institute for Health and Care Excellence (NICE). Charging for technology appraisals and highly specialised technologies. London: NICE; 2019. https://www.nice.org.uk/about/what-we-do/our-programmes/nice-guidance/nice-technology-appraisal-guidance/charging. Accessed 13 Aug 2019.
Drummond MF, Sculpher MJ, Claxton K, Torrance GW, Stoddart GL. Methods for the economic evaluation of health care programmes. 4th ed. Oxford: Oxford University Press; 2015.
Brazier J, Rowen D. NICE Decision Support Unit Technical Support Document 11: alternatives to EQ-5D for generating health state utility values. Sheffield: NICE Decision Support Unit; 2011.
National Institute for Health and Care Excellence (NICE). Position statement on use of the EQ-5D-5L valuation set for England (updated November 2018). London: NICE: 2018. https://www.nice.org.uk/about/what-we-do/our-programmes/nice-guidance/technology-appraisal-guidance/eq-5d-5l. Accessed 13 Aug 2019.
Culyer AJ. NICE’s use of cost effectiveness as an exemplar of a deliberative process. Health Econ Policy Law. 2006;1:299–318.
Scottish Medicines Consortium. SMC modifiers used in appraising new medicines. Glasgow: Scottish Medicines Consortium; 2012.
National Institute for Health and Clinical Excellence (NICE). Appraising end of life medicines. London: NICE; 2009. https://www.nice.org.uk/guidance/gid-tag387/documents/appraising-life-extending-end-of-life-treatments-paper2. Accessed 13 Aug 2019.
National Institute for Health and Care Excellence (NICE). NICE highly specialised technologies guidance. London: NICE; 2019. https://www.nice.org.uk/About/What-we-do/Our-Programmes/NICE-guidance/NICE-highly-specialised-technologies-guidance. Accessed 13 Aug 2019.
McCabe C, Claxton K, Tsuchiya A. Orphan drugs and the NHS: should we value rarity? BMJ. 2005;331:1016–9.
Rowen D, Brazier J, Mukuria C, Keetharuth A, Risa Hole A, Tsuchiya A, et al. Eliciting societal preferences for weighting QALYs for burden of illness and end of life. Med Decis Making. 2016;36:210–22.
Linley WG, Hughes DA. Societal views on NICE, cancer drugs fund and value-based pricing criteria for prioritising medicines: a cross-sectoral survey of 4118 adults in Great Britain. Health Econ. 2013;22:948–64.
Shah KK. Does society place special value on end of life treatments? In: Round J, editor. Care at end of life: an economic perspective. Switzerland: Springer; 2016.
Paulden M, O’Mohony JF, Culyer AJ, McCabe C. Some inconsistencies in NICE’s consideration of social values. Pharmacoeconomics. 2014;32:1043–53.
Gov.uk. Health and Social Care Act 2012 (c. 7). 2012. http://www.legislation.gov.uk/ukpga/2012/7. Accessed 12 Aug 2019.
Asaria M, Griffin S, Cookson R, Whyte S, Tappenden P. Distributional cost-effectiveness analysis of health care programmes—a methodological case study of the UK Bowel Cancer Screening Programme. Health Econ. 2015;24:742–54.
National Institute for Health and Care Excellence (NICE). NICE clinical guideline CG 132, caesarian section. London: NICE; 2011.
Pennington B, Wong R. Modelling carer health-related quality of life in NICE technology appraisals and highly specialised technologies—report by the Decision Support Unit. Sheffield: School of Health and Related Research, University of Sheffield; 2019.
Jonsson B. Ten arguments for a societal perspective in economic evaluation of medical interventions. Eur J Health Econ. 2009;10:357–9.
Neumann PJ, Sanders GD, Russell LB, Siegel JE, Ganiats TG. Cost-effectiveness in health and medicine. Oxford: Oxford University Press; 2016.
Walker S, Griffin S, Asaria M, Tsuchiya A, Sculpher M. Striving for a societal perspective: a framework for economic evaluations when costs and effects fall on multiple sectors and decision makers. Appl Health Econ Health Policy. 2019;17(5):577–90.
van Baal P, Melzer D, Brouwer W. Future costs, fixed healthcare budgets, and the decision rules of cost-effectiveness analysis. Health Econ. 2016;25:237–48.
Morton A, Amanda AA, Bell D, Briggs A, Brouwer W, Claxton K, et al. Unrelated future costs and unrelated future benefits: reflections on NICE guide to the methods of technology appraisal. Health Econ. 2016;25:933–8.
McPherson N. Review of quality assurance of Government analytical models: final report. London: HM Treasury; 2013.
NICE Decision Support Unit. Quality assurance—models. Sheffield: NICE Decision Support Unit; 2015. http://nicedsu.org.uk/methods-development/quality-assurance-models/. Accessed 12 Aug 2019.
Vemer P, Corro Ramos I, van Voorn G, Al MJ, Feenstra TL. AdViSHE: a validation-assessment tool of health-economic models for decision makers and model user. Pharmacoeconomics. 2016;34:349–61.
Dias S, Ades AE, Welton NJ, Jansen JP, Sutton AJ. Network meta-analysis for decision making. New Jersey: Wiley; 2018.
Phillippo DM, Ades AE, Dias S, Palmer S, Abrams KR, Welton NJ. Methods for population-adjusted indirect comparisons in health technology appraisal. Med Decis Making. 2018;38:200–11.
Owen RK, Cooper NJ, Quinn TJ, Lees R, Sutton AJ. Network meta-analysis of diagnostic test accuracy studies identifies and ranks the optimal diagnostic tests and thresholds for health care policy and decision-making. J Clin Epidemiol. 2018;99:64–74.
Ades AE, Caldwell DM, Reken S, et al. NICE Decison Support Unit Technical Support Document 7: evidence synthesis of treatment efficacy I. Sheffield: University of Sheffield, NICE Decision Support Unit; 2012.
Dias S, Sutton AJ, Welton N, Ades AE. Embedding evidence synthesis in probabilistic cost-effectiveness analysis: software choices. Sheffield: University of Sheffield, NICE Decision Support Unit; 2011.
Dias S, Welton N, Sutton A, Ades AE. NICE Decision Support Unit Technical Support Document 5: evidence synthesis in the baseline natural history model. Sheffield: University of Sheffield, NICE Decision Support Unit; 2011.
Fleetwood K, Glanville J, McCool R, Wood H, Wilson K, Marshall C, et al. A Review of the use of network meta-analysis In NICE Single Technology Appraisals. Value Health. 2016;19:A348.
Bagust A, Beale S. Survival analysis and extrapolation modeling of time-to-event clinical trial data for economic evaluation: an alternative approach. Med Decis Making. 2014;34:343–51.
Latimer NR. Response to “Survival analysis and extrapolation modeling of time-to-event clinical trial data for economic evaluation: an alternative approach” by Bagust and Beale. Med Decis Making. 2014;34:279–82.
Jackson CH, Stevens J, Ren S, Latimer N, Bojke L, Manca A, et al. Extrapolating survival from randomized trials using external data: a review of methods. Med Decis Making. 2017;37:377–90.
Du X, Li M, Zhu Z, Wang J, Hou L, Li J, et al. Comparison of the flexible parametric survival model and Cox model in estimating Markov transition probabilities using real-world data. PLoS One. 2018;13(8):e0200807.
Bansal A, Sullivan PD, Lin VW, Purdum AG, Navale L, Cheng P, et al. Estimating long-term survival for patients with relapsed or refractory large B-Cell lymphoma treated with chimeric antigen receptor therapy: a comparison of standard and mixture cure models. Med Decis Making. 2019;39:294–8.
Briggs AH, Weinstein MC, Fenwick EAL, Karnon J, Sculpher MJ, Paltiel AD. Model parameter estimation and uncertainty: a report of the ISPOR-SMDM Modelling Good Research Practices Task Force Working Group-6. Med Decis Making. 2012;32:722–32.
Claxton K, Sculpher M, McCabe C, Briggs A, Akehurst R, Buxton M, et al. Probabilistic sensitivity analysis for NICE technology assessment: not an optional extra. Health Econ. 2005;14:339–47.
Briggs AH, Wonderling DE, Mooney CZ. Pulling cost-effectiveness analysis up by its bootstraps: a non-parametric approach to confidence interval estimation. Health Econ. 1997;1997(6):327–40.
Jackson CH, Bojke L, Thompson SG, Claxton K, Sharples LD. A framework for addressing structural uncertainty in decision models. Med Decis Making. 2011;31:662–74.
O’Hagan A, Buck CE, Daneshkhah A, Eiser JR, Garthwaite PH, Jenkinson DJ, et al. Uncertain judgements: eliciting experts’ probabilities. Chichester: Wiley; 2006.
Medical Research Council. Successfully funded proposals—developing a reference protocol for expert elicitation in health care decision making. London: Medical Research Council; 2019. https://mrc.ukri.org/funding/how-we-fund-research/opportunities/methodology-for-eliciting-expert-opinion/successfully-funded-proposals/. Accessed 16 Aug 2019.
Van Hout BA, Al MJ, Gordon GS, Rutten FFH. Costs, effects and c/e-ratios alongside a clinical trial. Health Econ. 1994;1994(3):309–19.
The authors are grateful for the comments and suggestions made by three anonymous reviewers, and the suggestions from Professor Michael Drummond; however, the views expressed and any errors are solely the responsibility of the authors.
No explicit funding was used for work detailed in this paper. The Centre for Health Economics, University of York benefits from funding from various sources, including the UK NIHR.
Conflicts of interest
The Centre for Health Economics, University of York receives funding from the NIHR for a programme of technology assessment for NICE, and is also part of the NICE DSU. Mark Sculpher and Stephen Palmer are, or have been, members of various NICE advisory committees.
About this article
Cite this article
Sculpher, M., Palmer, S. After 20 Years of Using Economic Evaluation, Should NICE be Considered a Methods Innovator?. PharmacoEconomics 38, 247–257 (2020). https://doi.org/10.1007/s40273-019-00882-6