A Flexible Open-Source Decision Model for Value Assessment of Biologic Treatment for Rheumatoid Arthritis
The nature of model-based cost-effectiveness analysis can lead to disputes in the scientific community. We propose an iterative and collaborative approach to model development by presenting a flexible open-source simulation model for rheumatoid arthritis (RA), accessible to both technical and non-technical end-users.
The RA model is a discrete-time individual patient simulation with 6-month cycles. Model input parameters were estimated based on currently available evidence and treatment effects were obtained with Bayesian network meta-analysis techniques. The model contains 384 possible model structures informed by previously published models. The model consists of the following components: (i) modifiable R and C++ source code available in a GitHub repository; (ii) an R package to run the model for custom analyses; (iii) detailed model documentation; (iv) a web-based user interface for full control over the model without the need to be well-versed in the programming languages; and (v) a general audience web-application allowing those who are not experts in modeling or health economics to interact with the model and contribute to value assessment discussions.
A primary function of the initial version of RA model is to help understand and quantify the impact of parameter uncertainty (with probabilistic sensitivity analysis), structural uncertainty (with multiple competing model structures), the decision framework (cost-effectiveness analysis or multi-criteria decision analysis), and perspective (healthcare or limited societal) on estimates of value.
In order for a decision model to remain relevant over time it needs to evolve along with its supporting body of clinical evidence and scientific insight. Multiple clinical and methodological experts can modify or contribute to the RA model at any time due to its open-source nature.
DI and JPJ designed the study, developed the model, and wrote the manuscript. JRC provided clinical input on model design and contributed to the writing of the manuscript; JS and DNL provided economic input on model design and contributed to writing of the manuscript.
Compliance with Ethical Standards
This research was funded through the Innovation and Value Initiative (IVI), a multi-stakeholder research initiative.
Conflicts of interest
Devin Incerti, Jason Shafrin, and Jeroen Jansen are salaried employees of Precision Medicine Group. Darius Lakdawalla and Jeroen Jansen are shareholders of Precision Medicine Group, the parent company of Precision Health Economics (PHE), and Darius Lakdawalla is also a paid consultant to PHE. Jeffrey Curtis is a paid consultant to IVI. At the time of the current study, IVI was part of PHE and partly funded by different pharmaceutical companies.
Source code and data for the model are available at: https://github.com/InnovationValueInitiative/IVI-RA. A webpage with links to all components of the model (R package, tutorial, supplemental documentation, and web-interfaces) can be found at: https://innovationvalueinitiative.github.io/IVI-RA.
- 1.Dunlop WC, Mason N, Kenworthy J, Akehurst RL. Benefits, challenges and potential strategies of open source health economic models. Pharmacoeconomics. 2017;35(1):125–8.Google Scholar
- 2.Schramm W, Sailer F, Pobiruchin M, Weiss C. PROSIT open source disease models for diabetes mellitus. Stud Health Technol Inform. 2016;226:115–8.Google Scholar
- 3.Cohen JT, Neumann PJ, Wong JB. A call for open-source cost-effectiveness analysis. Ann Intern Med. 2018;168(7):529.Google Scholar
- 4.Sanders GD, Neumann PJ, Basu A, Brock DW, Feeny D, Krahn M, et al. Recommendations for conduct, methodological practices, and reporting of cost-effectiveness analyses: second panel on cost-effectiveness in health and medicine. JAMA. 2016;316(10):1093–103.Google Scholar
- 5.Jackson CH, Sharples LD, Thompson SG. Structural and parameter uncertainty in Bayesian cost-effectiveness models. J Roy Stat Soc Ser C (Appl Stat). 2010;59(2):233–53.Google Scholar
- 6.Incerti D, Jansen JP. A description of the IVI-RA model. 2017. https://innovationvalueinitiative.github.io/IVI-RA/model-description/model-description.pdf.
- 7.Helmick CG, Felson DT, Lawrence RC, Gabriel S, Hirsch R, Kwoh CK, et al. Estimates of the prevalence of arthritis and other rheumatic conditions in the United States. Part I. Arthritis Rheum. 2008;58(1):15–25.Google Scholar
- 8.da Rocha Castelar Pinheiro G, Khandker R, Sato R, Rose A, Piercy J. Impact of rheumatoid arthritis on quality of life, work productivity and resource utilisation: an observational, cross-sectional study in Brazil. Clin Exp Rheumatol. 2013;31(3):334–40.Google Scholar
- 9.Birnbaum H, Pike C, Kaufman R, Marynchenko M, Kidolezi Y, Cifaldi M. Societal cost of rheumatoid arthritis patients in the US. Curr Med Res Opin. 2010;26(1):77–90.Google Scholar
- 10.Lundkvist J, Kastäng F, Kobelt G. The burden of rheumatoid arthritis and access to treatment: health burden and costs. Eur J Health Econ. 2008;8(2):49–60.Google Scholar
- 11.Institute for Clinical and Economic Review. Targeted immune modulators for rheumatoid arthritis: effectiveness & value. Institute for Clinical and Economic Review; 2017.Google Scholar
- 12.Madan J, Ades AE, Welton NJ. An overview of models used in economic analyses of biologic therapies for arthritis—from current diversity to future consensus. Rheumatology. 2011;50 Suppl 4:iv10–iv8.Google Scholar
- 13.Brennan A, Bansback N, Reynolds A, Conway P. Modelling the cost-effectiveness of etanercept in adults with rheumatoid arthritis in the UK. Rheumatology (Oxford). 2004;43(1):62–72.Google Scholar
- 14.Tosh J, Brennan A, Wailoo A, Bansback N. The Sheffield rheumatoid arthritis health economic model. Rheumatology. 2011;50(Suppl 4):iv26–iv31.Google Scholar
- 15.Wailoo AJ, Bansback N, Brennan A, Michaud K, Nixon RM, Wolfe F. Biologic drugs for rheumatoid arthritis in the Medicare program: a cost-effectiveness analysis. Arthritis Rheum. 2008;58(4):939–46.Google Scholar
- 16.Carlson JJ, Ogale S, Dejonckheere F, Sullivan SD. Economic evaluation of tocilizumab monotherapy compared to adalimumab monotherapy in the treatment of severe active rheumatoid arthritis. Value Health. 2015;18(2):173–9.Google Scholar
- 17.Stephens S, Botteman MF, Cifaldi MA, van Hout BA. Modelling the cost-effectiveness of combination therapy for early, rapidly progressing rheumatoid arthritis by simulating the reversible and irreversible effects of the disease. BMJ Open. 2015;5(6):e006560.Google Scholar
- 18.Athanasakis K, Tarantilis F, Tsalapati K, Konstantopoulou T, Vritzali E, Kyriopoulos J. Cost-utility analysis of tocilizumab monotherapy in first line versus standard of care for the treatment of rheumatoid arthritis in Greece. Rheumatol Int. 2015;35(9):1489–95.Google Scholar
- 19.Stevenson M, Archer R, Tosh J, Simpson E, Everson-Hock E, Stevens J, et al. Adalimumab, etanercept, infliximab, certolizumab pegol, golimumab, tocilizumab and abatacept for the treatment of rheumatoid arthritis not previously treated with disease-modifying antirheumatic drugs and after the failure of conventional disease-modifying antirheumatic drugs only: systematic review and economic evaluation. Health Technol Assess. 2016;20(35):1–610.Google Scholar
- 20.Stevenson MD, Wailoo AJ, Tosh JC, Hernandez-Alava M, Gibson LA, Stevens JW, et al. The cost-effectiveness of sequences of biological disease-modifying antirheumatic drug treatment in england for patients with rheumatoid arthritis who can tolerate methotrexate. J Rheumatol. 2017;44(7):973–80.Google Scholar
- 21.Diamantopoulos A, Finckh A, Huizinga T, Sungher D, Sawyer L, Neto D, et al. Tocilizumab in the treatment of rheumatoid arthritis: a cost-effectiveness analysis in the UK. Pharmacoeconomics. 2014;32(8):775–87.Google Scholar
- 22.Jalal H, Pechlivanoglou P, Krijkamp E, Alarid-Escudero F, Enns E, Hunink MGM. An overview of R in health decision sciences. Med Decis Making. 2017;37(7):735–46.Google Scholar
- 23.Thokala P, Devlin N, Marsh K, Baltussen R, Boysen M, Kalo Z, et al. Multiple criteria decision analysis for health care decision making—an introduction: report 1 of the ISPOR MCDA Emerging Good Practices Task Force. Value Health. 2016;19(1):1–13.Google Scholar
- 24.Briggs AH, Claxton K, Sculpher MJ. Decision modelling for health economic evaluation. Oxford: Oxford University Press; 2006.Google Scholar
- 25.Meltzer DO, Smith PC. Theoretical issues relevant to the economic evaluation of health technologies. Handb Health Econ. 2011;2:433–69.Google Scholar
- 26.Drummond MF, Sculpher MJ, Claxton K, Stoddart GL, Torrance GW. Methods for the economic evaluation of health care programmes. Oxford: Oxford University Press; 2015.Google Scholar
- 27.Lakdawalla D, Malani A, Reif J. The insurance value of medical innovation. J Public Econ. 2017;145:94–102.Google Scholar
- 28.Garrison LP, Kamal-Bahl S, Towse A. Toward a broader concept of value: identifying and defining elements for an expanded cost-effectiveness analysis. Value Health. 2017;20(2):213–6.Google Scholar
- 29.Singh JA, Saag KG, Bridges SL, Akl EA, Bannuru RR, Sullivan MC, et al. 2015 American college of rheumatology guideline for the treatment of rheumatoid arthritis. Arthritis Care Res (Hoboken). 2016;68(1):1–25.Google Scholar
- 30.Anderson J, Caplan L, Yazdany J, Robbins ML, Neogi T, Michaud K, et al. Rheumatoid arthritis disease activity measures: American College of Rheumatology recommendations for use in clinical practice. Arthritis Care Res (Hoboken). 2012;64(5):640–7.Google Scholar
- 31.Aletaha D, Ward MM, Machold KP, Nell VP, Stamm T, Smolen JS. Remission and active disease in rheumatoid arthritis: defining criteria for disease activity states. Arthritis Rheum. 2005;52(9):2625–36.Google Scholar
- 32.Prevoo ML, van’t Hof MA, Kuper HH, van Leeuwen MA, van de Putte LB, van Riel PL. Modified disease activity scores that include twenty-eight-joint counts. Development and validation in a prospective longitudinal study of patients with rheumatoid arthritis. Arthritis Rheum. 1995;38(1):44–8.Google Scholar
- 33.Smolen JS, Breedveld FC, Schiff MH, Kalden JR, Emery P, Eberl G, et al. A simplified disease activity index for rheumatoid arthritis for use in clinical practice. Rheumatology (Oxford). 2003;42(2):244–57.Google Scholar
- 34.Aletaha D, Nell VP, Stamm T, Uffmann M, Pflugbeil S, Machold K, et al. Acute phase reactants add little to composite disease activity indices for rheumatoid arthritis: validation of a clinical activity score. Arthritis Res Ther. 2005;7(4):R796–806.Google Scholar
- 35.Aletaha D, Smolen J. The Simplified Disease Activity Index (SDAI) and the Clinical Disease Activity Index (CDAI): a review of their usefulness and validity in rheumatoid arthritis. Clin Exp Rheumatol. 2005;23(5 Suppl 39):S100–8.Google Scholar
- 36.Wolfe F, Michaud K. The loss of health status in rheumatoid arthritis and the effect of biologic therapy: a longitudinal observational study. Arthritis Res Ther. 2010;12(2):R35.Google Scholar
- 37.Michaud K, Wallenstein G, Wolfe F. Treatment and nontreatment predictors of health assessment questionnaire disability progression in rheumatoid arthritis: a longitudinal study of 18,485 patients. Arthritis Care Res (Hoboken). 2011;63(3):366–72.Google Scholar
- 38.Gibson L, Alava MH, Wailoo A. Progression of disease in people with rheumatoid arthritis treated with non-biologic therapies. Sheffield: School of Health and Related Research, University of Sheffield; 2015.Google Scholar
- 39.Norton S, Fu B, Scott DL, Deighton C, Symmons DP, Wailoo AJ, et al. Health Assessment Questionnaire disability progression in early rheumatoid arthritis: systematic review and analysis of two inception cohorts. Semin Arthritis Rheum. 2014;44(2):131–44.Google Scholar
- 40.Strand V, Williams S, Miller P, Saunders K, Grant S, Kremer J. OP0064 discontinuation of biologic therapy in rheumatoid arthritis (RA): analysis from the Consortium of Rheumatology Researchers of North America (CORRONA) database. Ann Rheum Dis. 2013;72(Suppl 3):A71–2.Google Scholar
- 41.Zhang J, Shan Y, Reed G, Kremer J, Greenberg JD, Baumgartner S, et al. Thresholds in disease activity for switching biologics in rheumatoid arthritis patients: experience from a large US cohort. Arthritis Care Res. 2011;63(12):1672–9.Google Scholar
- 42.Ramiro S, Sepriano A, Chatzidionysiou K, Nam JL, Smolen JS, van der Heijde D, et al. Safety of synthetic and biological DMARDs: a systematic literature review informing the 2016 update of the EULAR recommendations for management of rheumatoid arthritis. Ann Rheum Dis. 2017;76(6):1101–36.Google Scholar
- 43.Singh JA, Wells GA, Christensen R, Tanjong Ghogomu E, Maxwell LJ, MacDonald JK, et al. Adverse effects of biologics: a network meta-analysis and Cochrane overview. London: The Cochrane Library; 2011.Google Scholar
- 44.Arias E. United States life tables, 2011. Natl Vital Stat Rep. 2015;64(11):1–63.Google Scholar
- 45.Wolfe F, Michaud K, Gefeller O, Choi HK. Predicting mortality in patients with rheumatoid arthritis. Arthritis Rheum. 2003;48(6):1530–42.Google Scholar
- 46.Michaud K, Vera-Llonch M, Oster G. Mortality risk by functional status and health-related quality of life in patients with rheumatoid arthritis. J Rheumatol. 2012;39(1):54–9.Google Scholar
- 47.Hernández Alava M, Wailoo A, Wolfe F, Michaud K. The relationship between EQ-5D, HAQ and pain in patients with rheumatoid arthritis. Rheumatology (Oxford). 2013;52(5):944–50.Google Scholar
- 48.Wailoo A, Brennan A, Bansback N, Nixon R, Wolfe F, Michaud K. Modeling the cost effectiveness of etanercept, adalimumab and anakinra compared to infliximab in the treatment of patients with rheumatoid arthritis in the Medicare program. Rockville: Agency for Healthcare Research and Quality; 2006.Google Scholar
- 49.Oppong R, Kaambwa B, Nuttall J, Hood K, Smith RD, Coast J. The impact of using different tariffs to value EQ-5D health state descriptions: an example from a study of acute cough/lower respiratory tract infections in seven countries. Eur J Health Econ. 2013;14(2):197–209.Google Scholar
- 50.Claxton K, Sculpher M, McCabe C, Briggs A, Akehurst R, Buxton M, et al. Probabilistic sensitivity analysis for NICE technology assessment: not an optional extra. Health Econ. 2005;14(4):339–47.Google Scholar
- 51.Claxton L, Jenks M, Taylor M, Wallenstein G, Mendelsohn AM, Bourret JA, et al. An Economic evaluation of tofacitinib treatment in rheumatoid arthritis: modeling the cost of treatment strategies in the united states. J Manag Care Spec Pharm. 2016;22(9):1088–102.Google Scholar
- 52.Baio G, Dawid AP. Probabilistic sensitivity analysis in health economics. Stat Methods Med Res. 2015;24(6):615–34.Google Scholar
- 53.Vemer P, Corro Ramos I, van Voorn GA, Al MJ, Feenstra TL. AdViSHE: a validation-assessment tool of health-economic models for decision makers and model users. Pharmacoeconomics. 2016;34(4):349–61.Google Scholar
- 54.Gonzalez A, Maradit Kremers H, Crowson CS, Nicola PJ, Davis JM, Therneau TM, et al. The widening mortality gap between rheumatoid arthritis patients and the general population. Arthritis Rheum. 2007;56(11):3583–7.Google Scholar
- 55.Radovits BJ, Fransen J, Al Shamma S, Eijsbouts AM, van Riel PL, Laan RF. Excess mortality emerges after 10 years in an inception cohort of early rheumatoid arthritis. Arthritis Care Res (Hoboken). 2010;62(3):362–70.Google Scholar
- 56.How is lifespan affected by RA. National Rheumatoid Arthritis Society; 2016. https://www.nras.org.uk/how-is-lifespan-affected-by-ra. Accessed July 2017.
- 57.Hoeting JA, Madigan D, Raftery AE, Volinsky CT. Bayesian model averaging: a tutorial. Stat Sci. 1999:382–401.Google Scholar
- 58.Jalal H, Dowd B, Sainfort F, Kuntz KM. Linear regression metamodeling as a tool to summarize and present simulation model results. Med Decis Making. 2013;33(7):880–90.Google Scholar
- 59.Heath A, Manolopoulou I, Baio G. Estimating the expected value of partial perfect information in health economic evaluations using integrated nested Laplace approximation. Stat Med. 2016;35(23):4264–80.Google Scholar
- 60.Curtis JR, Jain A, Askling J, Bridges SL Jr, Carmona L, Dixon W, et al. (eds). A Comparison of Patient Characteristics and Outcomes in Selected European and US Rheumatoid Arthritis Registries. Seminars in arthritis and rheumatism; 2010;40(1):2–14.e1.Google Scholar