, Volume 36, Issue 10, pp 1135–1141 | Cite as

Using Evidence from Randomised Controlled Trials in Economic Models: What Information is Relevant and is There a Minimum Amount of Sample Data Required to Make Decisions?

  • John W. StevensEmail author
Current Opinion


Evidence from randomised controlled trials (RCTs) is used to support regulatory approval and reimbursement decisions. I discuss how these decisions are typically made and argue that the amount of sample data and regulatory authorities’ concerns over multiplicity are irrelevant when making reimbursement decisions. Decision analytic models (DAMs) are usually necessary to meet the requirements of an economic evaluation. DAMs involve inputs relating to health benefits and resource use that represent unknown true population parameters. Evidence about parameters may come from a variety of sources, including RCTs, and uncertainty about parameters is represented by their joint posterior distribution. Any impact of multiplicity is mitigated through the prior distribution. I illustrate my perspective with three examples: the estimation of a treatment effect on a rare event; the number of RCTs available in a meta-analysis; and the estimation of population mean overall survival. I conclude by recommending that reimbursement decisions should be followed by an assessment of the value of sample information and the DAM revised structurally as necessary and to include any new sample data that may be generated.


Compliance with Ethical Standards

The opinions presented are the opinions of the author.


No funding was provided in support of this manuscript.

Conflict of interest

The author, John W. Stevens, declares no conflicts of interest.


  1. 1.
    Sculpher M, Claxton K, Drummond M, McCabe C. Whither trial-based economic evaluation for health care decision making? Health Econ. 2006;15:677–87.CrossRefGoogle Scholar
  2. 2.
    ICH harmonised tripartite guideline. Statistical principles for clinical trials 1998. ICH.E9.
  3. 3.
    Senn S. Statistical issues in drug development. Chichester: Wiley; 2008. ISBN: 9780470723586.Google Scholar
  4. 4.
    Senn S. Two cheers for P-values? J Epidemiol Biostat. 2001;6:193–204.CrossRefGoogle Scholar
  5. 5.
  6. 6.
    Claxton K, Sculpher M, McCabe C, Briggs A, Akehurst R, Buxton M, Brazier J, O’Hagan A. Probabilistic sensitivity analysis for NICE technology assessment: not an optional extra. Health Econ. 2005;14:339–47.CrossRefGoogle Scholar
  7. 7.
    Oakley JE, O’Hagan A. Probabilistic sensitivity analysis of complex models: a Bayesian approach. J R Stat Soc B. 2004;66:751–69.CrossRefGoogle Scholar
  8. 8.
    Claxton K. The irrelevance of inference: a decision-making approach to the stochastic evaluation of health care technologies. J Health Econ. 1999;18:341–64.CrossRefGoogle Scholar
  9. 9.
    O’Hagan A, Stevens JW. Inference for the cost-effectiveness acceptability curve and cost-effectiveness ratio. Pharmacoeconomics. 2000;17:339–49.CrossRefGoogle Scholar
  10. 10.
    Senn S. Controversies concerning randomisation and additivity in clinical trials. Stat Med. 2004;23:3729–53.CrossRefGoogle Scholar
  11. 11.
    Dias S, Sutton AJ, Ades AE, Welton NJ. Evidence synthesis for decision making 2: a generalised linear modelling framework for pairwise and network meta-analysis of randomised controlled trials. Med Decis Mak. 2013;33:607–17.CrossRefGoogle Scholar
  12. 12.
    Welton NJ, Sutton AJ, Cooper NJ, Abrams KR, Ades AE. Evidence synthesis for decision making in healthcare. Chichester: Wiley; 2012. ISBN: 9780470061091.CrossRefGoogle Scholar
  13. 13.
    Stevens JW, O’Hagan A. Incorporation of genuine prior information in cost-effectiveness analysis of clinical trial data. Int J Technol Assess Health Care. 2002;18:782–90.CrossRefGoogle Scholar
  14. 14.
    Kass RE, Wasserman L. The selection of prior distributions by formal rules. JAMA. 1996;91:1343–70.Google Scholar
  15. 15.
    Sweeting MJ, Sutton AJ, Lambert PC. What to add to nothing? Use and avoidance of continuity corrections in meta-analysis of sparse data. Stat Med. 2004;23:1351–75.CrossRefGoogle Scholar
  16. 16.
    Hedges LV, Vevea JL. Fixed- and random-effects models in meta-analysis. Psychol Methods. 1998;3:486–504.CrossRefGoogle Scholar
  17. 17.
    Higgins JPT, Thompson SG, Spiegelhalter DJ. A re-evaluation of random effects meta-analysis. J R Stat Soc A. 2009;172:137–59.CrossRefGoogle Scholar
  18. 18.
    Veroniki AA, Jacjson D, Viechtbauer W, Bender R, Bowden J, Knapp G, Kuss O, Higgins JPT, Langan D, Salanti G. Methods to estimate the between-study variance and its uncertainty in meta-analysis. Res Synth Methods. 2016;7:55–79.CrossRefGoogle Scholar
  19. 19.
    Gelman A. Prior distributions for variance parameters in hierarchical models. Bayesian Anal. 2006;1:515–33.CrossRefGoogle Scholar
  20. 20.
    Turner RM, Davey J, Clarke MJ, Thompson SG, Higgins JPT. Predicting the extent of heterogeneity in meta-analysis using empirical data from the cochrane database of systematic review. Int J Epidemiol. 2012;41:818–27.CrossRefGoogle Scholar
  21. 21.
    Ren S, Oakley J, Stevens J. Incorporating genuine prior information about between-study heterogeneity in random effects pairwise and network meta-analyses. Med Decis Mak. 2018;38:531–42.CrossRefGoogle Scholar
  22. 22.
    Cox C. The generalised F distribution: an umbrella for parametric survival analysis. Stat Med. 2008;27:4301–12.CrossRefGoogle Scholar
  23. 23.
    Royston P, Parmar MKB. Flexible parametric proportional-hazards and proportional-odds models for censored survival data, with application to prognostic modelling and estimation of treatment effects. Stat Med. 2002;21:2175–97.CrossRefGoogle Scholar
  24. 24.
    Lambert PC, Thompson JR. Estimating and modelling the cure fraction in population-based cancer survival analysis. Biotatistics. 2007;8:576–94.CrossRefGoogle Scholar
  25. 25.
    Latimer N. Survival analysis for economic evaluations alongside clinical trials—extrapolation with patient-level data: inconsistencies, limitations and practical guide. Med Decis Mak. 2013;33:743–54.CrossRefGoogle Scholar
  26. 26.
    Ishak KJ, Kreif N, Benedict A, Muszbek N. Overview of parametric survival analysis for health-economic applications. Pharmacoeconomics. 2013;31:663–75.CrossRefGoogle Scholar
  27. 27.
    Bagust A, Beale S. Survival analysis and extrapolation modelling of time-to-event clinical trial data for economic evaluation: an alternative approach. Med Decis Mak. 2014;34:343–51.CrossRefGoogle Scholar
  28. 28.
    Tremblay G, Haines P, Briggs A. A criterion-based approach for the systematic and transparent extrapolation of clinical trial survival data. J Health Econ Outcomes Res. 2015;2:147–60.Google Scholar
  29. 29.
    Gibson E, Koblbauer I, Begum N, Dranitsaris G, Liew D, McEwan P, Monfared AAT. Modelling the survival outcomes of immuno-oncology drugs in economic evaluations: a systematic approach to data analysis and extrapolation. Pharmacoeconomics. 2017;35:1257–70.CrossRefGoogle Scholar
  30. 30.
    Ouwens MJNM, Philips Z, Jansen JP. Network meta-analysis of parametric survival curves. Res Synth Methods. 2010;1:258–71.CrossRefGoogle Scholar
  31. 31.
    Jansen JP. Network meta-analysis of survival data with fractional polynomials. BMC Med Res Methodol. 2011;11:61. Scholar
  32. 32.
    Jansen JP, Cope S. Meta-regression models to address heterogeneity and inconsistency in network meta-analysis of survival outcomes. BMC Med Res Methodol. 2012;12;152. Scholar
  33. 33.
    Jackson C, Stevens J, Ren S, Latimer N, Bojke L, Manca A, Sharples L. Extrapolating survival from randomised trials using external data: a review of methods. Med Decis Mak. 2017;37:377–90.CrossRefGoogle Scholar
  34. 34.
    Guyot P, Ades AE, Beasley M, Lueza B, Pignon J-P, Welton NJ. Extrapolation of survival curves from cancer trials using external information. Med Decis Mak. 2017;37:353–66.CrossRefGoogle Scholar
  35. 35.
    O’Hagan A, Buck CE, Daneshkhah A, Eiser JR, Garthwaite PH, Jenkinson DJ, Oakley JE, Rakow T. Uncertain judgements. Eliciting experts’ beliefs. Chichester: Wiley; 2006. ISBN: 9780470029992.CrossRefGoogle Scholar
  36. 36.
    Bojke L, Grigore B, Jankovic D, Peters J, Soares M, Stein K. Informing reimbursement decisions using cost-effectiveness modelling: a guide to the process of generating elicited prior to capture model uncertainties. Pharmacoeconomics. 2017;35:867–77.CrossRefGoogle Scholar
  37. 37.
    O’Hagan A, Forster J. Kendall’s advanced theory of statistics. Volume 2B. Bayesian inference. 2nd ed. London: Arnold; 2004. ISBN: 0340807520.Google Scholar
  38. 38.
    Sculpher M, Claxton K. Establishing the cost-effectiveness of new pharmaceuticals under conditions of uncertainty—when is there sufficient evidence? Value Health. 2005;8:433–46.CrossRefGoogle Scholar
  39. 39.
    Strong M, Oakley JE, Brennan A, Breeze P. Estimating the expected value of sample information using the probabilistic sensitivity analysis sample: a fast, nonparametric regression-based method. Med Decis Mak. 2015;35:570–83.CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.School of Health and Related ResearchUniversity of SheffieldSheffieldUK

Personalised recommendations