Use of Mortality as an Endpoint in Noninferiority Trials May Lead to Ethically Problematic Conclusions

  • Andrew M. HershEmail author
  • Robert J. Walter
  • Scott K. Abberegg
Review Paper



Noninferiority trials are becoming more common. Their design often requires investigators to “trade” a secondary benefit for efficacy. Use of mortality as an outcome of interest leads to important ethical conflicts whereby researchers must establish a minimal clinically important difference for mortality, a process which has the potential to result in problematic conclusions.


We sought to investigate the frequency of the use of mortality as an outcome in noninferiority trials, as well as to determine the average pre-specified noninferiority (“delta”) values.


We searched MEDLINE for reports of parallel-group randomized controlled noninferiority trials published in five high-impact general medical journals.

Main Outcome Measures

Data abstracted from articles including trial design parameters, results, and interpretation of results based on CONSORT recommendations.


One hundred seventy-three manuscripts reporting 196 noninferiority comparisons were included in our analysis. Of these, over a third (67 trials) used mortality either as their sole endpoint (11 trials) or as part of a composite endpoint (56 trials). Nine trials were consort A, 21 trials consort B, 19 trials consort C, 12 were consort F, 4 consort G, and 2 were consort H. Four analyses showed statistically significant more deaths in the new treatment arm, while meeting consort criteria as “inconclusive” (consort G), (Behringer et al. in Lancet. 385(9976):1418–1427, 2015; Kaul et al. in N Engl J Med. 373(18):1709–1719, 2015; Bwakura-Dangarembizi et al. in N Engl J Med. 370(1):41–53, 2014) and thirteen trials utilizing mortality as an endpoint and had an absolute increase of > 3%, and six had an absolute increase of > 5%.


The use of mortality as an outcome in noninferiority trials is not rare and scenarios where the new treatment is statistically worse, but a conclusion of noninferiority or inconclusive do occur. We highlight these issues and propose simple steps to reduce the risk of ethically dubious conclusions.


noninferiority trials medical ethics clinical trials outcomes measures 


Author Contribution

Contributions made to the following manuscript by the listed authors include conception and design: AH, RW, and SA; analysis and interpretation of data: AH and SA; and drafting and editing of the manuscript: AH, RW, and SA.

Compliance with Ethical Standards

Conflict of Interest

The authors declare that they do not have a conflict of interest.


  1. 1.
    Soonawala D, Middelburg RA, Egger M, Vandenbroucke JP, Dekkers OM. Efficacy of experimental treatments compared with standard treatments in non-inferiority trials: a meta-analysis of randomized controlled trials. Int J Epidemiol. 2010;39(6):1567–1581.CrossRefGoogle Scholar
  2. 2.
    Le Henanff A, Giraudeau B, Baron G, Ravaud P. Quality of reporting of noninferiority and equivalence randomized trials. JAMA. 2006;295(10):1147–1151.CrossRefGoogle Scholar
  3. 3.
    Flacco ME, Manzoli L, Boccia S, et al. Head-to-head randomized trials are mostly industry sponsored and almost always favor the industry sponsor. J Clin Epidemiol. 2015;68(7):811–820.CrossRefGoogle Scholar
  4. 4.
    Garattini S, Bertele’ V. Non-inferiority trials are unethical because they disregard patients' interests. Lancet. 2007;370(9602):1875–1877.CrossRefGoogle Scholar
  5. 5.
    Lesaffre E. Superiority, equivalence, and non-inferiority trials. Bull NYU Hosp Jt Dis. 2008;66(2):150–154.PubMedGoogle Scholar
  6. 6.
    Piaggio G, Elbourne DR, Pocock SJ, Evans SJ, Altman DG, Group C. Reporting of noninferiority and equivalence randomized trials: extension of the CONSORT 2010 statement. JAMA. 2012;308(24):2594–2604.CrossRefGoogle Scholar
  7. 7.
    Aberegg SK, Hersh AM, Samore MH. Empirical Consequences of Current Recommendations for the Design and Interpretation of Noninferiority Trials. J Gen Intern Med. 2018;33(1):88–96.CrossRefGoogle Scholar
  8. 8.
    Schulman S, Kearon C, Kakkar AK, et al. Extended use of dabigatran, warfarin, or placebo in venous thromboembolism. N Engl J Med. 2013;368(8):709–718.CrossRefGoogle Scholar
  9. 9.
    Motzer RJ, Hutson TE, Cella D, et al. Pazopanib versus sunitinib in metastatic renal-cell carcinoma. N Engl J Med. 2013;369(8):722–731.CrossRefGoogle Scholar
  10. 10.
    von Birgelen C, Sen H, Lam MK, et al. Third-generation zotarolimus-eluting and everolimus-eluting stents in all-comer patients requiring a percutaneous coronary intervention (DUTCH PEERS): a randomised, single-blind, multicentre, non-inferiority trial. Lancet. 2014;383(9915):413–423.CrossRefGoogle Scholar
  11. 11.
    Pilgrim T, Heg D, Roffi M, et al. Ultrathin strut biodegradable polymer sirolimus-eluting stent versus durable polymer everolimus-eluting stent for percutaneous coronary revascularisation (BIOSCIENCE): a randomised, single-blind, non-inferiority trial. Lancet. 2014;384(9960):2111–2122.CrossRefGoogle Scholar
  12. 12.
    Büller HR, Prins MH, Lensin AW, et al. Oral rivaroxaban for the treatment of symptomatic pulmonary embolism. N Engl J Med. 2012;366(14):1287–1297.CrossRefGoogle Scholar
  13. 13.
    Raungaard B, Jensen LO, Tilsted HH, et al. Zotarolimus-eluting durable-polymer-coated stent versus a biolimus-eluting biodegradable-polymer-coated stent in unselected patients undergoing percutaneous coronary intervention (SORT OUT VI): a randomised non-inferiority trial. Lancet. 2015;385(9977):1527–1535.CrossRefGoogle Scholar
  14. 14.
    Feres F, Costa RA, Abizaid A, et al. Three vs twelve months of dual antiplatelet therapy after zotarolimus-eluting stents: the OPTIMIZE randomized trial. JAMA. 2013;310(23):2510–2522.PubMedGoogle Scholar
  15. 15.
    Jacobs AK, Normand SL, Massaro JM, et al. Nonemergency PCI at hospitals with or without on-site cardiac surgery. N Engl J Med. 2013;368(16):1498–1508.CrossRefGoogle Scholar
  16. 16.
    Ellis SG, Kereiakes DJ, Metzger DC, et al. Everolimus-Eluting Bioresorbable Scaffolds for Coronary Artery Disease. N Engl J Med. 2015;373(20):1905–1915.CrossRefGoogle Scholar
  17. 17.
    Smits PC, Hofma S, Togni M, et al. Abluminal biodegradable polymer biolimus-eluting stent versus durable polymer everolimus-eluting stent (COMPARE II): a randomised, controlled, non-inferiority trial. Lancet. 2013;381(9867):651–660.CrossRefGoogle Scholar
  18. 18.
    Pritchard-Jones K, Bergeron C, de Camargo B, et al. Omission of doxorubicin from the treatment of stage II-III, intermediate-risk Wilms' tumour (SIOP WT 2001): an open-label, non-inferiority, randomised controlled trial. Lancet. 2015;386(9999):1156–1164.CrossRefGoogle Scholar
  19. 19.
    Rosenfield K, Matsumura JS, Chaturvedi S, et al. Randomized Trial of Stent versus Surgery for Asymptomatic Carotid Stenosis. N Engl J Med. 2016;374(11):1011–1020.CrossRefGoogle Scholar
  20. 20.
    Crook JM, O'Callaghan CJ, Duncan G, et al. Intermittent androgen suppression for rising PSA level after radiotherapy. N Engl J Med. 2012;367(10):895–903.CrossRefGoogle Scholar
  21. 21.
    Ardehali A, Esmailian F, Deng M, et al. Ex-vivo perfusion of donor hearts for human heart transplantation (PROCEED II): a prospective, open-label, multicentre, randomised non-inferiority trial. Lancet. 2015;385(9987):2577–2584.CrossRefGoogle Scholar
  22. 22.
    Stone GW, Sabik JF, Serruys PW, et al. Everolimus-Eluting Stents or Bypass Surgery for Left Main Coronary Artery Disease. N Engl J Med. 2016;375(23):2223–2235.CrossRefGoogle Scholar
  23. 23.
    Merle CS, Fielding K, Sow OB, et al. A four-month gatifloxacin-containing regimen for treating tuberculosis. N Engl J Med. 2014;371(17):1588–1598.CrossRefGoogle Scholar
  24. 24.
    Christiansen EH, Jensen LO, Thayssen P, et al. Biolimus-eluting biodegradable polymer-coated stent versus durable polymer-coated sirolimus-eluting stent in unselected patients receiving percutaneous coronary intervention (SORT OUT V): a randomised non-inferiority trial. Lancet. 2013;381(9867):661–669.CrossRefGoogle Scholar
  25. 25.
    Paton NI, Kityo C, Hoppe A, et al. Assessment of second-line antiretroviral regimens for HIV therapy in Africa. N Engl J Med. 2014;371(3):234–247.CrossRefGoogle Scholar
  26. 26.
    Kirchhof P, Andresen D, Bosch R, et al. Short-term versus long-term antiarrhythmic drug treatment after cardioversion of atrial fibrillation (Flec-SL): a prospective, randomised, open-label, blinded endpoint assessment trial. Lancet. 2012;380(9838):238–246.CrossRefGoogle Scholar
  27. 27.
    Johnson P, Federico M, Kirkwood A, et al. Adapted Treatment Guided by Interim PET-CT Scan in Advanced Hodgkin's Lymphoma. N Engl J Med. 2016;374(25):2419–2429.CrossRefGoogle Scholar
  28. 28.
    Radford J, Illidge T, Counsell N, et al. Results of a trial of PET-directed therapy for early-stage Hodgkin's lymphoma. N Engl J Med. 2015;372(17):1598–1607.CrossRefGoogle Scholar
  29. 29.
    Park SJ, Ahn JM, Kim YH, et al. Trial of everolimus-eluting stents or bypass surgery for coronary disease. N Engl J Med. 2015;372(13):1204–1212.CrossRefGoogle Scholar
  30. 30.
    Anderson CS, Robinson T, Lindley RI, et al. Low-Dose versus Standard-Dose Intravenous Alteplase in Acute Ischemic Stroke. N Engl J Med. 2016;374(24):2313–2323.CrossRefGoogle Scholar
  31. 31.
    Bousser MG, Amarenco P, Chamorro A, et al. Terutroban versus aspirin in patients with cerebral ischaemic events (PERFORM): a randomised, double-blind, parallel-group trial. Lancet. 2011;377(9782):2013–2022.CrossRefGoogle Scholar
  32. 32.
    Paul M, Bishara J, Yahav D, et al. Trimethoprim-sulfamethoxazole versus vancomycin for severe infections caused by meticillin resistant Staphylococcus aureus: randomised controlled trial. BMJ. 2015;350:h2219.CrossRefGoogle Scholar
  33. 33.
    Hussain M, Tangen CM, Berry DL, et al. Intermittent versus continuous androgen deprivation in prostate cancer. N Engl J Med. 2013;368(14):1314–1325.CrossRefGoogle Scholar
  34. 34.
    Behringer K, Goergen H, Hitz F, et al. Omission of dacarbazine or bleomycin, or both, from the ABVD regimen in treatment of early-stage favourable Hodgkin's lymphoma (GHSG HD13): an open-label, randomised, non-inferiority trial. Lancet. 2015;385(9976):1418–1427.CrossRefGoogle Scholar
  35. 35.
    Kaul U, Bangalore S, Seth A, et al. Paclitaxel-Eluting versus Everolimus-Eluting Coronary Stents in Diabetes. N Engl J Med. 2015;373(18):1709–1719.CrossRefGoogle Scholar
  36. 36.
    Bwakura-Dangarembizi M, Kendall L, Bakeera-Kitaka S, et al. A randomized trial of prolonged co-trimoxazole in HIV-infected children in Africa. N Engl J Med. 2014;370(1):41–53.CrossRefGoogle Scholar
  37. 37.
    Jindani A, Harrison TS, Nunn AJ, et al. High-dose rifapentine with moxifloxacin for pulmonary tuberculosis. N Engl J Med. 2014;371(17):1599–1608.CrossRefGoogle Scholar
  38. 38.
    Mulvenna P, Nankivell M, Barton R, et al. Dexamethasone and supportive care with or without whole brain radiotherapy in treating patients with non-small cell lung cancer with brain metastases unsuitable for resection or stereotactic radiotherapy (QUARTZ): results from a phase 3, non-inferiority, randomised trial. Lancet. 2016;388(10055):2004–2014.CrossRefGoogle Scholar
  39. 39.
    Aberegg S. Reporting noninferiority trials. JAMA. 2013;309(15):1584–1585.CrossRefGoogle Scholar
  40. 40.
    Gladstone B, Vach W. Analyzing noninferiority trials: it is time for advantage deficit assessment – an observational study of published noninferiority trials. Open Access J Clin Trials. 2015;7:11–21.CrossRefGoogle Scholar
  41. 41.
    Prieto L, Sacristán JA. Problems and solutions in calculating quality-adjusted life years (QALYs). Health Qual Life Outcomes. 2003;1:80.CrossRefGoogle Scholar

Copyright information

© Society of General Internal Medicine 2019

Authors and Affiliations

  • Andrew M. Hersh
    • 1
    Email author
  • Robert J. Walter
    • 1
  • Scott K. Abberegg
    • 2
  1. 1.Division of Pulmonary and Critical Care Brooke Army Medical CenterFort Sam HoustonUSA
  2. 2.Division of Pulmonary and Critical Care MedicineUniversity of UtahSalt Lake CityUSA

Personalised recommendations