Advertisement

PharmacoEconomics

, Volume 37, Issue 12, pp 1485–1494 | Cite as

Performance of the UKPDS Outcomes Model 2 for Predicting Death and Cardiovascular Events in Patients with Type 2 Diabetes Mellitus from a German Population-Based Cohort

  • Michael LaxyEmail author
  • Verena Maria Schöning
  • Christoph Kurz
  • Rolf Holle
  • Annette Peters
  • Christa Meisinger
  • Wolfgang Rathmann
  • Kristin Mühlenbruch
  • Katharina Kähm
Original Research Article

Abstract

Background and Objective

Accurate prediction of relevant outcomes is important for targeting therapies and to support health economic evaluations of healthcare interventions in patients with diabetes. The United Kingdom Prospective Diabetes Study (UKPDS) risk equations are some of the most frequently used risk equations. This study aims to analyze the calibration and discrimination of the updated UKPDS risk equations as implemented in the UKPDS Outcomes Model 2 (UKPDS-OM2) for predicting cardiovascular (CV) events and death in patients with type 2 diabetes mellitus (T2DM) from population-based German samples.

Methods

Analyses are based on data of 456 individuals diagnosed with T2DM who participated in two population-based studies in southern Germany (KORA (Cooperative Health Research in the Region of Augsburg)-A: 1997/1998, n = 178; KORA-S4: 1999–2001, n = 278). We compared the participants’ 10-year observed incidence of mortality, CV mortality, myocardial infarction (MI), and stroke with the predicted event rate of the UKPDS-OM2. The model’s calibration was evaluated by Greenwood–Nam–D’Agostino tests and discrimination was evaluated by C-statistics.

Results

Of the 456 participants with T2DM (mean age 65 years, mean diabetes duration 8 years, 56% male), over the 10-year follow-up time 129 died (61 due to CV events), 64 experienced an MI, and 46 a stroke. The UKPDS-OM2 significantly over-predicted mortality and CV mortality by 25% and 28%, respectively (Greenwood–Nam–D’Agostino tests: p < 0.01), but there was no significant difference between predicted and observed MI and stroke risk. The model poorly discriminated for death (C-statistic [95% confidence interval] = 0.64 [0.60–0.69]), CV death (0.64 [0.58–0.71]), and MI (0.58 [0.52–0.66]), and failed to discriminate for stroke (0.57 [0.47–0.66]).

Conclusions

The study results demonstrate acceptable calibration and poor discrimination of the UKPDS-OM2 for predicting death and CV events in this population-based German sample. Those limitations should be considered when using the UKPDS-OM2 for economic evaluations of healthcare strategies or using the risk equations for clinical decision-making.

Notes

Author Contributions

ML planned the analysis, supervised analysis of the data, and drafted the manuscript. VMS analyzed the data and was a major contributor to the draft of the manuscript. CK consulted on the data analysis and commented on the manuscript draft. RH consulted on the data analysis, was involved in the data collection, and commented on the manuscript draft. AP, CM, WR, and KM were involved in the data collection and commented on the manuscript draft. KK was a major contributor to the draft of the manuscript.

Compliance with Ethical Standards

Conflict of interest

Michael Laxy, Verena M. Schöning, Christoph Kurz, Rolf Holle, Annette Peters, Christa Meisinger, Wolfgang Rathmann, Kristin Mühlenbruch, and Katharina Kähm declare that they have no competing interests.

Funding

The KORA (Cooperative Health Research in the Region of Augsburg) research platform was initiated and financed by the Helmholtz Zentrum München (the German Research Center for Environmental Health), which is funded by the German Federal Ministry of Education and Research and by the State of Bavaria. Furthermore, KORA research was supported within the Munich Center of Health Sciences (MC Health), Ludwig-Maximilians-Universität (LMU), as part of the LMUinnovative.

Supplementary material

40273_2019_822_MOESM1_ESM.doc (64 kb)
Supplementary material 1 (DOC 64 kb)

References

  1. 1.
    Jacobs E, Hoyer A, Brinks R, Kuss O, Rathmann W. Burden of mortality attributable to diagnosed diabetes: a nationwide analysis based on claims data from 65 million people in Germany. Diabetes Care. 2017;40(12):1703–9.CrossRefGoogle Scholar
  2. 2.
    Bommer C, Sagalova V, Heesemann E, Manne-Goehler J, Atun R, Barnighausen T, et al. Global economic burden of diabetes in adults: projections from 2015 to 2030. Diabetes Care. 2018;41(5):963–70.CrossRefGoogle Scholar
  3. 3.
    Hunger M, Thorand B, Schunk M, Doring A, Menn P, Peters A, et al. Multimorbidity and health-related quality of life in the older population: results from the German KORA-age study. Health Qual Life Outcomes. 2011;9:53.CrossRefGoogle Scholar
  4. 4.
    American Diabetes Association Consensus Panel. Guidelines for computer modeling of diabetes and its complications. Diabetes Care. 2004;27(9):2262–5.CrossRefGoogle Scholar
  5. 5.
    Baena-Diez JM, Peñafiel J, Subirana I, Ramos R, Elosua R, Marín-Ibañez A, FRESCO Investigators, et al. Risk of cause-specific death in individuals with diabetes: a competing risks analysis. Diabetes Care. 2016;39(11):1987–95.CrossRefGoogle Scholar
  6. 6.
    National Institute for Health and Care Excellence. Lipid-modifying drugs; 2017. https://www.nice.org.uk/advice/ktt3/chapter/evidence-context#statins. Accessed 3 Sept 2018.
  7. 7.
    Chou R, Dana T, Blazina I, Daeges M, Bougatsos C, Grusing S, et al. Statin use for the prevention of cardiovascular disease in adults: a systematic review for the U.S. Preventive Services Task Force. Rockville: Agency for Healthcare Research and Quality (US); 2016. (Evidence Syntheses, No. 139). https://www.ncbi.nlm.nih.gov/books/NBK396415/. Accessed 3 Sept 2018.
  8. 8.
    Govan L, Wu O, Lindsay R, Briggs A. How do diabetes models measure up? A review of diabetes economic models and ADA guidelines. J Health Econ Outcomes Res. 2015;3(2):132–52.Google Scholar
  9. 9.
    UK Prospective Diabetes Study (UKPDS). VIII. Study design, progress and performance. Diabetologia. 1991;34(12):877–90.CrossRefGoogle Scholar
  10. 10.
    Hayes AJ, Leal J, Gray AM, Holman RR, Clarke PM. UKPDS outcomes model 2: a new version of a model to simulate lifetime health outcomes of patients with type 2 diabetes mellitus using data from the 30 year United Kingdom Prospective Diabetes Study: UKPDS 82. Diabetologia. 2013;56(9):1925–33.CrossRefGoogle Scholar
  11. 11.
    McEwan P, Bergenheim K, Currie CJ. External validation of the UKPDS Oucomes model equations (UKPDS 68), and the UKPDS risk engine equations (UKPDS 56 and 60) in forecasting cardiovascular outcomes in people with type 2 diabetes. Value Health. 2006;9(3):A37–8.CrossRefGoogle Scholar
  12. 12.
    Simmons RK, Coleman RL, Price HC, Holman RR, Khaw K-T, Wareham NJ, et al. Performance of the UK Prospective Diabetes Study Risk Engine and the Framingham risk equations in estimating cardiovascular disease in the EPIC-Norfolk cohort. Diabetes Care. 2009;32(4):708–13.CrossRefGoogle Scholar
  13. 13.
    Tao L, Wilson ECF, Griffin SJ, Simmons RK. Performance of the UKPDS outcomes model for prediction of myocardial infarction and stroke in the ADDITION-Europe trial cohort. Value Health. 2013;16(6):1074–80.CrossRefGoogle Scholar
  14. 14.
    van Dieren S, Peelen LM, Nöthlings U, van der Schouw YT, Rutten GE, Spijkerman AM, et al. External validation of the UK Prospective Diabetes Study (UKPDS) risk engine in patients with type 2 diabetes. Diabetologia. 2011;54(2):264–70.CrossRefGoogle Scholar
  15. 15.
    Kengne AP, Patel A, Colagiuri S, Heller S, Hamet P, Marre M, ADVANCE Collaborative Group, et al. The Framingham and UK Prospective Diabetes Study (UKPDS) risk equations do not reliably estimate the probability of cardiovascular events in a large ethnically diverse sample of patients with diabetes: the Action in Diabetes and Vascular Disease: Preterax and Diamicron-MR Controlled Evaluation (ADVANCE) study. Diabetologia. 2010;53(5):821–31.CrossRefGoogle Scholar
  16. 16.
    Diabetes Trials Unit, H.E.R.C. UKPDS Outcomes Model user manual. Version 2; 2015. https://www.dtu.ox.ac.uk/outcomesmodel/OM2Manual.pdf. Accessed 26 July 2017.
  17. 17.
    Willis M, Johansen P, Nilsson A, Asseburg C. Validation of the economic and health outcomes model of type 2 diabetes mellitus (ECHO-T2DM). Pharmacoeconomics. 2017;35(3):375–96.CrossRefGoogle Scholar
  18. 18.
    Lundqvist A, Steen Carlsson K, Johansen P, Andersson E, Willis M. Validation of the IHE Cohort Model of Type 2 Diabetes and the impact of choice of macrovascular risk equations. PLoS One. 2014;9(10):e110235.CrossRefGoogle Scholar
  19. 19.
    McEwan P, Foos V, Palmer JL, Lamotte M, Lloyd A, Grant D. Validation of the IMS CORE Diabetes Model. Value Health. 2014;17(6):714–24.CrossRefGoogle Scholar
  20. 20.
    McEwan P, Ward T, Bennett H, Bergenheim K. Validation of the UKPDS 82 risk equations within the Cardiff Diabetes Model. Cost Effect Resour Alloc. 2015;13:12.CrossRefGoogle Scholar
  21. 21.
    Palmer AJ, Mount Hood 5 Modeling Group, Clarke P, Gray A, Leal J, Lloyd A, et al. Computer modeling of diabetes and its complications: a report on the Fifth Mount Hood challenge meeting. Value Health. 2013;16(4):670–85.CrossRefGoogle Scholar
  22. 22.
    Basu S, Sussman JB, Berkowitz SA, Hayward RA, Bertoni AG, Correa A, et al. Validation of Risk Equations for Complications of Type 2 Diabetes (RECODe) using individual participant data from diverse longitudinal cohorts in the U.S. Diabetes Care. 2018;41(3):586–95.CrossRefGoogle Scholar
  23. 23.
    Holle R, Happich M, Lowel H, Wichmann HE, MONICA/KORA Study Group. KORA—a research platform for population based health research. Gesundheitswesen. 2005;67(Suppl 1):S19–25.CrossRefGoogle Scholar
  24. 24.
    Laxy M, Knoll G, Schunk M, Meisinger C, Huth C, Holle R. Quality of diabetes care in Germany improved from 2000 to 2007 to 2014, but improvements diminished since 2007. Evidence from the population-based KORA studies. PLoS One. 2016;11(10):e0164704.CrossRefGoogle Scholar
  25. 25.
    Laxy M, Mielck A, Hunger M, Schunk M, Meisinger C, Ruckert IM, et al. The association between patient-reported self-management behavior, intermediate clinical outcomes, and mortality in patients with type 2 diabetes: results from the KORA-A study. Diabetes Care. 2014;37(6):1604–12.CrossRefGoogle Scholar
  26. 26.
    Yuan Y. Multiple imputation using SAS software. J Stat Softw. 2011;45(6):1–25.CrossRefGoogle Scholar
  27. 27.
    Kuhfeld W, So Y. Creating and customizing the Kaplan–Meier survival plot in PROC LIFETEST in SAS Global Forum Statistics and Data Analysis. Paper 427. 2013. https://support.sas.com/resources/papers/proceedings13/427-2013.pdf. Accessed 24 June 2019.
  28. 28.
    Demler OV, Paynter NP, Cook NR. Tests of calibration and goodness-of-fit in the survival setting. Stat Med. 2015;34(10):1659–80.CrossRefGoogle Scholar
  29. 29.
    Altman DG, Vergouwe Y, Royston P, Moons KGM. Prognosis and prognostic research: validating a prognostic model. BMJ (Clin Res Ed). 2009;338:b605.CrossRefGoogle Scholar
  30. 30.
    Steyerberg EW, Vickers AJ, Cook NR, Gerds T, Gonen M, Obuchowski N, et al. Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology. 2010;21(1):128–38.CrossRefGoogle Scholar
  31. 31.
    Pencina MJ, D’Agostino RB. Overall C as a measure of discrimination in survival analysis: model specific population value and confidence interval estimation. Stat Med. 2004;23(13):2109–23.CrossRefGoogle Scholar
  32. 32.
    Olchanski N, Cohen JT, Neumann PJ, Wong JB, Kent DM. Understanding the value of individualized information: the impact of poor calibration or discrimination in outcome prediction models. Med Decis Making. 2017;37(7):790–801.CrossRefGoogle Scholar
  33. 33.
    Kähm K, Laxy M, Schneider U, Rogowski WH, Lhachimi SK, Holle R. Health care costs associated with incident complications in patients with type 2 diabetes in Germany. Diabetes Care. 2018;41(5):971–8.CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Michael Laxy
    • 1
    • 2
    Email author
  • Verena Maria Schöning
    • 1
    • 3
  • Christoph Kurz
    • 1
    • 2
  • Rolf Holle
    • 1
    • 2
  • Annette Peters
    • 4
  • Christa Meisinger
    • 4
  • Wolfgang Rathmann
    • 5
  • Kristin Mühlenbruch
    • 6
  • Katharina Kähm
    • 1
    • 2
  1. 1.Institute of Health Economics and Health Care Management, Helmholtz Zentrum München (GmbH)NeuherbergGermany
  2. 2.German Center for Diabetes Research, DZDNeuherberg-MunichGermany
  3. 3.Institute for Medical Information Processing, Biometrics and EpidemiologyLudwig-Maximilians-Universität MünchenMunichGermany
  4. 4.Institute of Epidemiology II, Helmholtz Zentrum München (GmbH)NeuherbergGermany
  5. 5.Institute for Biometrics and Epidemiology, German Diabetes CenterDüsseldorfGermany
  6. 6.Department of Molecular EpidemiologyGerman Institute of Human NutritionPotsdamGermany

Personalised recommendations