Skip to main content
SpringerLink
Log in

After 20 Years of Using Economic Evaluation, Should NICE be Considered a Methods Innovator?

  • Current Opinion
  • Published:
PharmacoEconomics Aims and scope Submit manuscript

A Commentary to this article was published on 24 April 2020

A Commentary to this article was published on 04 February 2020

Abstract

The National Institute for Health and Care Excellence (NICE) is only one of several organisations internationally that uses economic evaluation as part of decision making regarding funding and pricing of new medical technologies. However, it can be argued that NICE has developed a more prominent international profile than most in their use of economics. After 20 years of operation, it is timely to assess the extent of NICE’s achievements, including the economic evaluation methods it has used and its willingness to adapt these as new evaluative approaches emerge and when NICE faces particular policy challenges. This paper considers some of the important policy and contextual developments in the UK over the last 20 years and how these may have shaped NICE’s approach to economic evaluation. It then assesses key areas of NICE methods, including perspective, defining benefits, modelling and uncertainty. The paper concludes that NICE has provided important support for the development of new methods, in particular through its role in identifying priorities for methods research funding and its sponsorship of the NICE Decision Support Unit. However, potentially important developments in methods in a number of important areas have yet to be formally included in NICE’s methods guidance and this should be addressed in the Institute’s 2019/2020 methods review.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Commonwealth Department of Health Housing and Community Services. Guidelines for the pharmaceutical industry on preparation of submissions to the Pharmaceutical Benefits Advisory Committee. Canberra: Australian Government Publishing Service; 1992.

    Google Scholar 

  2. Ministry of Health. Ontario guidelines for economic analysis of pharmaceutical products. Toronto: Ministry of Health; 1994.

    Google Scholar 

  3. Longworth L, Bojke L, Sculpher MJ, Tosh JC. Bridging the gap between methods research and the needs of policy makers: a review of the research priorities of the National Institute for Health and Clinical Excellence. Int J Technol Assess Health Care. 2011;27:1–8.

    Google Scholar 

  4. National Institute for Health and Care Excellence (NICE). Methods for the development of NICE public health guidance (third edition). London: NICE; 2018.

  5. National Institute for Health and Care Excellence (NICE). Medical technologies evaluation programme methods guide. London: NICE; 2017.

  6. National Institute of Clinical Excellence. Appraisal of new and existing technologies: interim guidance for manufacturers and sponsors. London: National Institute of Clinical Excellence; 1999.

    Google Scholar 

  7. Treasury HM. The green book. Central government guidance on appraisal and evaluation. London: HM Treasury; 2018.

    Google Scholar 

  8. National Institute Health and Care Excellence (NICE). Review of methods for the Health Technology Evaluation programme. Presented in papers for the Public Board Meeting and Annual General Meeting 17 July. London: NICE; 2019. https://www.nice.org.uk/Media/Default/Get-involved/Meetings-In-Public/Public-board-meetings/agenda-and-papers-july-19.pdf. Accessed 11 Nov 19.

  9. National Institute for Clinical Excellence (NICE). Guide to the technology appraisal process. NICE: 2001.

  10. National Institute for Clinical Excellence (NICE). Guide to the methods of technology appraisal. London: NICE; 2004.

    Google Scholar 

  11. Gold MR, Siegel JE, Russell LB, Weinstein MC. Cost-effectiveness in health and medicine. New York: Oxford University Press; 1996.

    Google Scholar 

  12. National Institute for Health and Clinical Excellence (NICE). Guide to the methods of technology appraisal. London: NICE; 2008.

    Google Scholar 

  13. National Institute for Health and Clinical Excellence (NICE). Updated guide to the methods of technology appraisal. London: NICE; 2013.

    Google Scholar 

  14. National Institute for Health and Care Excellence (NICE). NICE Citizens’ Council. London: NICE; 2019. https://www.nice.org.uk/get-involved/citizens-council. Accessed 26 Jul 2019.

  15. National Institute for Clinical Excellence (NICE). Social value judgements, second edition. London: NICE; 2008. https://www.nice.org.uk/Media/Default/About/what-we-do/Research-and-development/Social-Value-Judgements-principles-for-the-development-of-NICE-guidance.pdf. Accessed 26 Jul 2019.

  16. Kennedy I. Appraising the value of innovation and other benefits: a short study for NICE. London: National Institute for Health and Clinical Excellence; 2009.

    Google Scholar 

  17. Department of Health. A new value-based approach to the pricing of branded medicines—a consultation. London: Department of Health; 2010.

    Google Scholar 

  18. National Institute for Health and Care Excellence (NICE). Centre for Health Technology Evaluation, value based assessment of health technologies. London: NICE; 2014.

    Google Scholar 

  19. Department of Health and Social Care (DHSC). The. Voluntary scheme for branded medicines pricing and access—chapters and glossary. London: DHSC; 2019. p. 2018.

    Google Scholar 

  20. Claxton K, Martin S, Soares M, Rice N, Spackman E, Hinde S, et al. Methods for the estimation of the NICE cost effectiveness threshold. Health Technol Assess. 2015;19(14):503.

    Google Scholar 

  21. Department of Health and Social Care (DHSC). Impact assessment: 2018 statutory scheme—branded medicines pricing. London: DHSC; 2018. https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/761064/impact-assessment-2018-statutory-scheme-branded-medicines-pricing.pdf. Accessed 11 Nov 2019.

  22. Woods B, Sideris E, Palmer S, Latimer N, Soares M. NICE Decision Support Unit Technical Support Document 19: partitioned survival analysis for decision modelling in health care: a critical review. Sheffield: NICE Decision Support Unit; 2017.

    Google Scholar 

  23. Faria R, Hernandez M, Manca A, Wailoo A. NICE Decision Support Unit Technical Support Document 17: the use of observational data to inform estimates of treatment effectiveness in technology appraisal: methods for comparative individual patient data. Sheffield: NICE Decision Support Unit; 2015.

    Google Scholar 

  24. Latimer NR. NICE Decision Support Unit Technical Support Document 14: survival analysis for economic evaluations alongside clinical trials: extrapolation with patient-level data. Sheffield: NICE Decision Support Unit; 2013.

    Google Scholar 

  25. Longworth L, Yang Y, Young T, Mulhern B, Hernández Alava M, Mukuria C, et al. Use of generic and condition-specific measures of Health-Related Quality of Life in NICE decision making. Health Technol Assess. 2014;18(9):1–224.

    PubMed  PubMed Central  Google Scholar 

  26. Claxton K, Palmer SJ, Longworth L, Bojke L, Griffiths D, McKenna C, et al. Informing a decision framework for when NICE should recommend the use of health technologies only in the context of an appropriately designed programme of evidence development. Health Technol Assess. 2012;16(46):1–342.

    PubMed  CAS  Google Scholar 

  27. National Institute for Health and Care Excellence (NICE). Our projects and partners. London: NICE; 2019. https://www.nice.org.uk/about/what-we-do/our-research-work/our-projects-and-partners. Accessed 11 Nov 2019.

  28. National Institute for Health and Care Excellence (NICE). Guide to the processes of technology appraisal. London: NICE; 2014. nice.org.uk/process/pmg19.

  29. National Institute for Health and Clinical Excellence (NICE). Guide to the single technology appraisal process. London: NICE; 2006.

    Google Scholar 

  30. National Institute for Health and Care Excellence (NICE). Fast track appraisal: addendum to the guide to the processes of technology appraisal. London: NICE; 2018.

  31. National Institute for Health and Care Excellence (NICE). PMG9 addendum—final amendments to the NICE technology appraisal methods guide to support the new Cancer Drugs Fund arrangements. London: NICE; 2018.

  32. National Institute for Health and Care Excellence (NICE). Annual report and accounts 2016/17. London: NICE; 2017.

    Google Scholar 

  33. National Institute for Health and Care Excellence (NICE). Charging for technology appraisals and highly specialised technologies. London: NICE; 2019. https://www.nice.org.uk/about/what-we-do/our-programmes/nice-guidance/nice-technology-appraisal-guidance/charging. Accessed 13 Aug 2019.

  34. Drummond MF, Sculpher MJ, Claxton K, Torrance GW, Stoddart GL. Methods for the economic evaluation of health care programmes. 4th ed. Oxford: Oxford University Press; 2015.

    Google Scholar 

  35. Brazier J, Rowen D. NICE Decision Support Unit Technical Support Document 11: alternatives to EQ-5D for generating health state utility values. Sheffield: NICE Decision Support Unit; 2011.

    Google Scholar 

  36. National Institute for Health and Care Excellence (NICE). Position statement on use of the EQ-5D-5L valuation set for England (updated November 2018). London: NICE: 2018. https://www.nice.org.uk/about/what-we-do/our-programmes/nice-guidance/technology-appraisal-guidance/eq-5d-5l. Accessed 13 Aug 2019.

  37. Culyer AJ. NICE’s use of cost effectiveness as an exemplar of a deliberative process. Health Econ Policy Law. 2006;1:299–318.

    PubMed  Google Scholar 

  38. Scottish Medicines Consortium. SMC modifiers used in appraising new medicines. Glasgow: Scottish Medicines Consortium; 2012.

    Google Scholar 

  39. National Institute for Health and Clinical Excellence (NICE). Appraising end of life medicines. London: NICE; 2009. https://www.nice.org.uk/guidance/gid-tag387/documents/appraising-life-extending-end-of-life-treatments-paper2. Accessed 13 Aug 2019.

  40. National Institute for Health and Care Excellence (NICE). NICE highly specialised technologies guidance. London: NICE; 2019. https://www.nice.org.uk/About/What-we-do/Our-Programmes/NICE-guidance/NICE-highly-specialised-technologies-guidance. Accessed 13 Aug 2019.

  41. McCabe C, Claxton K, Tsuchiya A. Orphan drugs and the NHS: should we value rarity? BMJ. 2005;331:1016–9.

    PubMed  PubMed Central  Google Scholar 

  42. Rowen D, Brazier J, Mukuria C, Keetharuth A, Risa Hole A, Tsuchiya A, et al. Eliciting societal preferences for weighting QALYs for burden of illness and end of life. Med Decis Making. 2016;36:210–22.

    PubMed  Google Scholar 

  43. Linley WG, Hughes DA. Societal views on NICE, cancer drugs fund and value-based pricing criteria for prioritising medicines: a cross-sectoral survey of 4118 adults in Great Britain. Health Econ. 2013;22:948–64.

    PubMed  Google Scholar 

  44. Shah KK. Does society place special value on end of life treatments? In: Round J, editor. Care at end of life: an economic perspective. Switzerland: Springer; 2016.

    Google Scholar 

  45. Paulden M, O’Mohony JF, Culyer AJ, McCabe C. Some inconsistencies in NICE’s consideration of social values. Pharmacoeconomics. 2014;32:1043–53.

    PubMed  Google Scholar 

  46. Gov.uk. Health and Social Care Act 2012 (c. 7). 2012. http://www.legislation.gov.uk/ukpga/2012/7. Accessed 12 Aug 2019.

  47. Asaria M, Griffin S, Cookson R, Whyte S, Tappenden P. Distributional cost-effectiveness analysis of health care programmes—a methodological case study of the UK Bowel Cancer Screening Programme. Health Econ. 2015;24:742–54.

    PubMed  Google Scholar 

  48. National Institute for Health and Care Excellence (NICE). NICE clinical guideline CG 132, caesarian section. London: NICE; 2011.

    Google Scholar 

  49. Pennington B, Wong R. Modelling carer health-related quality of life in NICE technology appraisals and highly specialised technologies—report by the Decision Support Unit. Sheffield: School of Health and Related Research, University of Sheffield; 2019.

    Google Scholar 

  50. Jonsson B. Ten arguments for a societal perspective in economic evaluation of medical interventions. Eur J Health Econ. 2009;10:357–9.

    PubMed  Google Scholar 

  51. Neumann PJ, Sanders GD, Russell LB, Siegel JE, Ganiats TG. Cost-effectiveness in health and medicine. Oxford: Oxford University Press; 2016.

    Google Scholar 

  52. Walker S, Griffin S, Asaria M, Tsuchiya A, Sculpher M. Striving for a societal perspective: a framework for economic evaluations when costs and effects fall on multiple sectors and decision makers. Appl Health Econ Health Policy. 2019;17(5):577–90.

    PubMed  PubMed Central  Google Scholar 

  53. van Baal P, Melzer D, Brouwer W. Future costs, fixed healthcare budgets, and the decision rules of cost-effectiveness analysis. Health Econ. 2016;25:237–48.

    PubMed  Google Scholar 

  54. Morton A, Amanda AA, Bell D, Briggs A, Brouwer W, Claxton K, et al. Unrelated future costs and unrelated future benefits: reflections on NICE guide to the methods of technology appraisal. Health Econ. 2016;25:933–8.

    PubMed  Google Scholar 

  55. McPherson N. Review of quality assurance of Government analytical models: final report. London: HM Treasury; 2013.

  56. NICE Decision Support Unit. Quality assurance—models. Sheffield: NICE Decision Support Unit; 2015. http://nicedsu.org.uk/methods-development/quality-assurance-models/. Accessed 12 Aug 2019.

  57. Vemer P, Corro Ramos I, van Voorn G, Al MJ, Feenstra TL. AdViSHE: a validation-assessment tool of health-economic models for decision makers and model user. Pharmacoeconomics. 2016;34:349–61.

    PubMed  CAS  Google Scholar 

  58. Dias S, Ades AE, Welton NJ, Jansen JP, Sutton AJ. Network meta-analysis for decision making. New Jersey: Wiley; 2018.

    Google Scholar 

  59. Phillippo DM, Ades AE, Dias S, Palmer S, Abrams KR, Welton NJ. Methods for population-adjusted indirect comparisons in health technology appraisal. Med Decis Making. 2018;38:200–11.

    PubMed  Google Scholar 

  60. Owen RK, Cooper NJ, Quinn TJ, Lees R, Sutton AJ. Network meta-analysis of diagnostic test accuracy studies identifies and ranks the optimal diagnostic tests and thresholds for health care policy and decision-making. J Clin Epidemiol. 2018;99:64–74.

    PubMed  Google Scholar 

  61. Ades AE, Caldwell DM, Reken S, et al. NICE Decison Support Unit Technical Support Document 7: evidence synthesis of treatment efficacy I. Sheffield: University of Sheffield, NICE Decision Support Unit; 2012.

    Google Scholar 

  62. Dias S, Sutton AJ, Welton N, Ades AE. Embedding evidence synthesis in probabilistic cost-effectiveness analysis: software choices. Sheffield: University of Sheffield, NICE Decision Support Unit; 2011.

    Google Scholar 

  63. Dias S, Welton N, Sutton A, Ades AE. NICE Decision Support Unit Technical Support Document 5: evidence synthesis in the baseline natural history model. Sheffield: University of Sheffield, NICE Decision Support Unit; 2011.

    Google Scholar 

  64. Fleetwood K, Glanville J, McCool R, Wood H, Wilson K, Marshall C, et al. A Review of the use of network meta-analysis In NICE Single Technology Appraisals. Value Health. 2016;19:A348.

    Google Scholar 

  65. Bagust A, Beale S. Survival analysis and extrapolation modeling of time-to-event clinical trial data for economic evaluation: an alternative approach. Med Decis Making. 2014;34:343–51.

    PubMed  Google Scholar 

  66. Latimer NR. Response to “Survival analysis and extrapolation modeling of time-to-event clinical trial data for economic evaluation: an alternative approach” by Bagust and Beale. Med Decis Making. 2014;34:279–82.

    PubMed  Google Scholar 

  67. Jackson CH, Stevens J, Ren S, Latimer N, Bojke L, Manca A, et al. Extrapolating survival from randomized trials using external data: a review of methods. Med Decis Making. 2017;37:377–90.

    PubMed  Google Scholar 

  68. Du X, Li M, Zhu Z, Wang J, Hou L, Li J, et al. Comparison of the flexible parametric survival model and Cox model in estimating Markov transition probabilities using real-world data. PLoS One. 2018;13(8):e0200807.

    PubMed  PubMed Central  Google Scholar 

  69. Bansal A, Sullivan PD, Lin VW, Purdum AG, Navale L, Cheng P, et al. Estimating long-term survival for patients with relapsed or refractory large B-Cell lymphoma treated with chimeric antigen receptor therapy: a comparison of standard and mixture cure models. Med Decis Making. 2019;39:294–8.

    PubMed  Google Scholar 

  70. Briggs AH, Weinstein MC, Fenwick EAL, Karnon J, Sculpher MJ, Paltiel AD. Model parameter estimation and uncertainty: a report of the ISPOR-SMDM Modelling Good Research Practices Task Force Working Group-6. Med Decis Making. 2012;32:722–32.

    PubMed  Google Scholar 

  71. Claxton K, Sculpher M, McCabe C, Briggs A, Akehurst R, Buxton M, et al. Probabilistic sensitivity analysis for NICE technology assessment: not an optional extra. Health Econ. 2005;14:339–47.

    PubMed  Google Scholar 

  72. Briggs AH, Wonderling DE, Mooney CZ. Pulling cost-effectiveness analysis up by its bootstraps: a non-parametric approach to confidence interval estimation. Health Econ. 1997;1997(6):327–40.

    Google Scholar 

  73. Jackson CH, Bojke L, Thompson SG, Claxton K, Sharples LD. A framework for addressing structural uncertainty in decision models. Med Decis Making. 2011;31:662–74.

    PubMed  Google Scholar 

  74. O’Hagan A, Buck CE, Daneshkhah A, Eiser JR, Garthwaite PH, Jenkinson DJ, et al. Uncertain judgements: eliciting experts’ probabilities. Chichester: Wiley; 2006.

    Google Scholar 

  75. Medical Research Council. Successfully funded proposals—developing a reference protocol for expert elicitation in health care decision making. London: Medical Research Council; 2019. https://mrc.ukri.org/funding/how-we-fund-research/opportunities/methodology-for-eliciting-expert-opinion/successfully-funded-proposals/. Accessed 16 Aug 2019.

  76. Van Hout BA, Al MJ, Gordon GS, Rutten FFH. Costs, effects and c/e-ratios alongside a clinical trial. Health Econ. 1994;1994(3):309–19.

    Google Scholar 

Download references

Acknowledgements

The authors are grateful for the comments and suggestions made by three anonymous reviewers, and the suggestions from Professor Michael Drummond; however, the views expressed and any errors are solely the responsibility of the authors.

Author information

Authors and Affiliations

  1. Centre for Health Economics, University of York, York, YO10 5DD, UK

    Mark Sculpher & Stephen Palmer

Authors
  1. Mark Sculpher
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Stephen Palmer
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

MS took the lead on drafting the manuscript and incorporating revisions following the referees’ comments. SP drafted sections, edited throughout and contributed to decisions regarding revisions following the referees’ comments.

Corresponding author

Correspondence to Mark Sculpher.

Ethics declarations

Funding

No explicit funding was used for work detailed in this paper. The Centre for Health Economics, University of York benefits from funding from various sources, including the UK NIHR.

Conflicts of interest

The Centre for Health Economics, University of York receives funding from the NIHR for a programme of technology assessment for NICE, and is also part of the NICE DSU. Mark Sculpher and Stephen Palmer are, or have been, members of various NICE advisory committees.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sculpher, M., Palmer, S. After 20 Years of Using Economic Evaluation, Should NICE be Considered a Methods Innovator?. PharmacoEconomics 38, 247–257 (2020). https://doi.org/10.1007/s40273-019-00882-6

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40273-019-00882-6

Search

Navigation