Replicating Health Economic Models: Firm Foundations or a House of Cards?
Health economic evaluation is a framework for the comparative analysis of the incremental health gains and costs associated with competing decision alternatives. The process of developing health economic models is usually complex, financially expensive and time-consuming. For these reasons, model development is sometimes based on previous model-based analyses; this endeavour is usually referred to as model replication. Such model replication activity may involve the comprehensive reproduction of an existing model or ‘borrowing’ all or part of a previously developed model structure. Generally speaking, the replication of an existing model may require substantially less effort than developing a new de novo model by bypassing, or undertaking in only a perfunctory manner, certain aspects of model development such as the development of a complete conceptual model and/or comprehensive literature searching for model parameters. A further motivation for model replication may be to draw on the credibility or prestige of previous analyses that have been published and/or used to inform decision making. The acceptability and appropriateness of replicating models depends on the decision-making context: there exists a trade-off between the ‘savings’ afforded by model replication and the potential ‘costs’ associated with reduced model credibility due to the omission of certain stages of model development. This paper provides an overview of the different levels of, and motivations for, replicating health economic models, and discusses the advantages, disadvantages and caveats associated with this type of modelling activity. Irrespective of whether replicated models should be considered appropriate or not, complete replicability is generally accepted as a desirable property of health economic models, as reflected in critical appraisal checklists and good practice guidelines. To this end, the feasibility of comprehensive model replication is explored empirically across a small number of recent case studies. Recommendations are put forward for improving reporting standards to enhance comprehensive model replicability.
Compliance with Ethical Standards
IB, PT and JY did not receive any funding to support the drafting of this manuscript and have no conflicts of interest to declare. PT attempted to replicate four of the five models included in the pilot study, IB replicated the remaining model, and JY replicated one of the models. All authors were involved in writing the manuscript.
- 1.Briggs A, Claxton K, Sculpher M. Decision modelling for health economic evaluation. 1st ed. New York: Oxford University Press; 2006.Google Scholar
- 2.Popper K. The logic of scientific discovery. 1st ed. London: Routledge; 1992.Google Scholar
- 3.Weinstein M, O’Brien B, Hornberger J, Jackson J, Johannesson M, McCabe M, et al. Principles of good practice for decision analytic modeling in health-care evaluation: report of the ISPOR task force on good research practices—modeling studies. Value Health. 2003;6(1):9–17.CrossRefPubMedGoogle Scholar
- 4.Husereau D, Drummond M, Petrou S, Carswell C, Moher D, Greenberg D. Consolidated health economic evaluation reporting standards (CHEERS)—explanation and elaboration: a report of the ISPOR health economic evaluation publication guidelines good reporting practices task force. Value Health. 2013;16(2):231–50.CrossRefPubMedGoogle Scholar
- 5.Eddy D. Technology assessment: the role of mathematical modeling assessing medical technology. 1st ed. Washington, DC: National Academy Press; 1985.Google Scholar
- 8.McManus E. How reproducible are economic decision-analytic models? A systematic review and replication study—study outline. 2016. Available at: http://www.clahrcprojects.co.uk/impact/publications/how-reproducible-are-economic-decision-analytic-models-systematic-review-and.
- 13.AstraZeneca Ltd. Ticagrelor for the treatment of acute coronary syndromes: manufacturer’s submission to the National Institute for Health and Care Excellence. 2011. Available at: https://www.nice.org.uk/guidance/TA236/documents/acute-coronary-syndromes-ticagrelor-astrazeneca4.
- 14.The Medicines Company. Bivalirudin (Angiox®) for the treatment of ST-segment elevation myocardial infarction: manufacturer’s submission to the National Institute for Health and Care Excellence. 2011. Available at: https://www.nice.org.uk/guidance/TA230/documents/myocardial-infarction-persistent-stsegment-elevation-bivalirudin-the-medicines-company2.
- 15.Chilcott J, McCabe C, Tappenden P, O’Hagan A, Cooper N, Abrams K, et al. Modelling the cost effectiveness of interferon beta and glatiramer acetate in the management of multiple sclerosis. Commentary: evaluating disease modifying treatments in multiple sclerosis. Br Med J. 2003;326(7388):522.CrossRefGoogle Scholar
- 17.Biogen Idec Ltd. Natalizumab (Tysabri®) for the treatment of adults with highly active relapsing remitting multiple sclerosis: manufacturer’s submission to the National Institute for Health and Care Excellence. 2007. Available at: https://www.nice.org.uk/guidance/TA127/documents/multiple-sclerosis-natalizumab-manufacturer-submissions-biogen-idec-uk-and-elan-pharma-international-ltd-joint-development-agreement-confidential-information-removed2.
- 18.Gani R, Giovannoni G, Bates D, Kemball B, Hughes S, Kerrigan J. Cost-effectiveness analyses of natalizumab (Tysabri) compared with other disease-modifying therapies for people with highly active relapsing-remitting multiple sclerosis in the UK. Pharmacoeconomics. 2008;26(7):617–27.CrossRefPubMedGoogle Scholar
- 19.Genzyme. Teriflunomide for the treatment of relapsing remitting multiple sclerosis in adults: manufacturer’s submission to the National Institute for Health and Care Excellence. 2013. Available at: https://www.nice.org.uk/guidance/TA303/documents/multiple-sclerosis-relapsing-teriflunomide-evaluation-report4.
- 20.Novartis Pharmaceuticals UK Ltd. Fingolimod for the treatment of relapsing remitting multiple sclerosis in adults: manufacturer’s submission to the National Institute for Health and Care Excellence. 2011. Available at: https://www.nice.org.uk/guidance/TA254/documents/manufacturer-submission-from-novartis2.
- 21.Biogen Idec Ltd. Dimethyl fumarate for the treatment of adult patients with relapsing remitting multiple sclerosis: manufacturer’s submission to the National Institute for Health and Care Excellence. 2013. Available at: https://www.nice.org.uk/guidance/TA320/documents/multiple-sclerosis-relapsingremitting-dimethyl-fumarate-evaluation-report2.
- 22.Woolacott N, Hawkins N, Mason A, Kainth A, Khadjesari Z, Bravo Vergel Y, et al. Efalizumab and etanercept for the treatment of psoriasis: assessment report submitted to the National Institute for Health and Care Excellence. 2007. Available at: https://www.nice.org.uk/guidance/TA103/documents/psoriasis-efalizumab-and-etanercept-assessment-report2.
- 23.Schering-Plough Ltd. Remicade® in the treatment of psoriasis in England and Wales: manufacturer’s submission to the National Institute for Health and Care Excellence. 2007. Available at: https://www.nice.org.uk/guidance/TA134/documents/psoriasis-infliximab-scheringplough-ltd2.
- 24.Abbott Laboratories Ltd. Adalimumab (Humira®) for the treatment of moderate to severe plaque psoriasis: manufacturer’s submission to the National Institute for Health and Care Excellence. 2007. Available at: https://www.nice.org.uk/guidance/TA146/documents/psoriasis-adalimumab-appraisal-consultation-manufacturer-submission-abbott-laboratories-ltd2.
- 25.Janssen-Cilag Ltd. Ustekinumab (Stelara) for the treatment of moderate to severe plaque psoriasis in England and Wales: manufacturer’s submission to the National Institute for Health and Care Excellence. 2009. Available at: https://www.nice.org.uk/guidance/TA180/documents/psoriasis-ustekinumab-manufacturer-submission-janssencilag2.
- 26.Eggington S, Tappenden P, Pandor A, Paisley S, Saunders M, Seymour M, et al. The clinical and cost-effectiveness of oxaliplatin and capecitabine for the adjuvant treatment of colon cancer: systematic review and economic evaluation. Br J Cancer. 2006;95(9):1195–201.CrossRefPubMedCentralPubMedGoogle Scholar
- 28.Kearns B, Rafia R, Leaviss J, Preston L, Brazier J, Palmer S, et al. The cost-effectiveness of changes to the care pathway used to identify depression and provide treatment amongst people with diabetes in England: a model-based economic evaluation. BMC Health Serv Res. 2017;17(1):78.CrossRefPubMedCentralPubMedGoogle Scholar
- 32.Hayes A, Leal J, Gray A, Holman R, Clarke P, et al. UKPDS outcomes model 2: a new version of a model to simulate lifetime health outcomes of patients with type 2 diabetes mellitus using data from the 30 year United Kingdom Prospective Diabetes Study: UKPDS 82. Diabetologia. 2013;56(9):1925–33.CrossRefPubMedGoogle Scholar
- 33.National Institute for Health and Care Excellence. Bosutinib for previously treated chronic myeloid leukaemia. 2013. Available at: https://www.nice.org.uk/guidance/ta299.
- 34.National Institute for Health and Care Excellence. Dasatinib, high-dose imatinib and nilotinib for the treatment of imatinib-resistant chronic myeloid leukaemia (CML) (part review of NICE technology appraisal guidance 70), and dasatinib and nilotinib for people with CML for whom treatment with imatinib has failed because of intolerance. 2012. Available at: https://www.nice.org.uk/guidance/ta299.
- 35.Stevenson M, Archer R, Tosh J, Simpson E, Everson-Hock E, Stevens J, et al. Adalimumab, etanercept, infliximab, certolizumab pegol, golimumab, tocilizumab and abatacept for the treatment of rheumatoid arthritis not previously treated with disease-modifying antirheumatic drugs and after the failure of conventional disease-modifying antirheumatic drugs only: systematic review and economic evaluation. Health Technol Assess. 2016;20(35):1–610.CrossRefGoogle Scholar
- 36.National Institute for Health and Care Excellence. Adalimumab, etanercept, infliximab, rituximab and abatacept for the treatment of rheumatoid arthritis after the failure of a TNF inhibitor. 2013. Available at: https://www.nice.org.uk/guidance/ta195.
- 37.National Institute for Health and Care Excellence. Adalimumab, etanercept and infliximab for the treatment of rheumatoid arthritis. 2013. Available at: https://www.nice.org.uk/guidance/ta130.
- 41.Tappenden P, Ren S, Archer R, Harvey R, Martyn St-James M, Basarir H, et al. A model-based economic evaluation of biologic and non-biologic options for the treatment of adults with moderately-to-severely active ulcerative colitis after the failure of conventional therapy. Pharmacoeconomics. 2016;34(10):1023–38.CrossRefPubMedGoogle Scholar
- 43.Oddershede L, Walker S, Stöhr W, Dunn D, Arenas-Pinto A, Paton NI, et al. Cost effectiveness of protease inhibitor monotherapy versus standard triple therapy in the long-term management of HIV patients: analysis using evidence from the PIVOT trial. Pharmacoeconomics. 2016;34(8):795–804.CrossRefPubMedGoogle Scholar