Skip to main content
SpringerLink
Log in

Using Evidence from Randomised Controlled Trials in Economic Models: What Information is Relevant and is There a Minimum Amount of Sample Data Required to Make Decisions?

  • Current Opinion
  • Published:
PharmacoEconomics Aims and scope Submit manuscript

    We’re sorry, something doesn't seem to be working properly.

    Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.

Abstract

Evidence from randomised controlled trials (RCTs) is used to support regulatory approval and reimbursement decisions. I discuss how these decisions are typically made and argue that the amount of sample data and regulatory authorities’ concerns over multiplicity are irrelevant when making reimbursement decisions. Decision analytic models (DAMs) are usually necessary to meet the requirements of an economic evaluation. DAMs involve inputs relating to health benefits and resource use that represent unknown true population parameters. Evidence about parameters may come from a variety of sources, including RCTs, and uncertainty about parameters is represented by their joint posterior distribution. Any impact of multiplicity is mitigated through the prior distribution. I illustrate my perspective with three examples: the estimation of a treatment effect on a rare event; the number of RCTs available in a meta-analysis; and the estimation of population mean overall survival. I conclude by recommending that reimbursement decisions should be followed by an assessment of the value of sample information and the DAM revised structurally as necessary and to include any new sample data that may be generated.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

References

  1. Sculpher M, Claxton K, Drummond M, McCabe C. Whither trial-based economic evaluation for health care decision making? Health Econ. 2006;15:677–87.

    Article  Google Scholar 

  2. ICH harmonised tripartite guideline. Statistical principles for clinical trials 1998. ICH.E9. http://www.ifpma.org/ich1.html.

  3. Senn S. Statistical issues in drug development. Chichester: Wiley; 2008. ISBN: 9780470723586.

    Google Scholar 

  4. Senn S. Two cheers for P-values? J Epidemiol Biostat. 2001;6:193–204.

    Article  CAS  Google Scholar 

  5. Points to consider on multiplicity issues in clinical trials. http://www.ema.europa.eu/docs/en_GB/document_library/Scientific_guideline/2009/09/WC500003640.pdf.

  6. Claxton K, Sculpher M, McCabe C, Briggs A, Akehurst R, Buxton M, Brazier J, O’Hagan A. Probabilistic sensitivity analysis for NICE technology assessment: not an optional extra. Health Econ. 2005;14:339–47.

    Article  Google Scholar 

  7. Oakley JE, O’Hagan A. Probabilistic sensitivity analysis of complex models: a Bayesian approach. J R Stat Soc B. 2004;66:751–69.

    Article  Google Scholar 

  8. Claxton K. The irrelevance of inference: a decision-making approach to the stochastic evaluation of health care technologies. J Health Econ. 1999;18:341–64.

    Article  CAS  Google Scholar 

  9. O’Hagan A, Stevens JW. Inference for the cost-effectiveness acceptability curve and cost-effectiveness ratio. Pharmacoeconomics. 2000;17:339–49.

    Article  Google Scholar 

  10. Senn S. Controversies concerning randomisation and additivity in clinical trials. Stat Med. 2004;23:3729–53.

    Article  Google Scholar 

  11. Dias S, Sutton AJ, Ades AE, Welton NJ. Evidence synthesis for decision making 2: a generalised linear modelling framework for pairwise and network meta-analysis of randomised controlled trials. Med Decis Mak. 2013;33:607–17.

    Article  Google Scholar 

  12. Welton NJ, Sutton AJ, Cooper NJ, Abrams KR, Ades AE. Evidence synthesis for decision making in healthcare. Chichester: Wiley; 2012. ISBN: 9780470061091.

    Book  Google Scholar 

  13. Stevens JW, O’Hagan A. Incorporation of genuine prior information in cost-effectiveness analysis of clinical trial data. Int J Technol Assess Health Care. 2002;18:782–90.

    Article  Google Scholar 

  14. Kass RE, Wasserman L. The selection of prior distributions by formal rules. JAMA. 1996;91:1343–70.

    Google Scholar 

  15. Sweeting MJ, Sutton AJ, Lambert PC. What to add to nothing? Use and avoidance of continuity corrections in meta-analysis of sparse data. Stat Med. 2004;23:1351–75.

    Article  Google Scholar 

  16. Hedges LV, Vevea JL. Fixed- and random-effects models in meta-analysis. Psychol Methods. 1998;3:486–504.

    Article  Google Scholar 

  17. Higgins JPT, Thompson SG, Spiegelhalter DJ. A re-evaluation of random effects meta-analysis. J R Stat Soc A. 2009;172:137–59.

    Article  Google Scholar 

  18. Veroniki AA, Jacjson D, Viechtbauer W, Bender R, Bowden J, Knapp G, Kuss O, Higgins JPT, Langan D, Salanti G. Methods to estimate the between-study variance and its uncertainty in meta-analysis. Res Synth Methods. 2016;7:55–79.

    Article  Google Scholar 

  19. Gelman A. Prior distributions for variance parameters in hierarchical models. Bayesian Anal. 2006;1:515–33.

    Article  Google Scholar 

  20. Turner RM, Davey J, Clarke MJ, Thompson SG, Higgins JPT. Predicting the extent of heterogeneity in meta-analysis using empirical data from the cochrane database of systematic review. Int J Epidemiol. 2012;41:818–27.

    Article  Google Scholar 

  21. Ren S, Oakley J, Stevens J. Incorporating genuine prior information about between-study heterogeneity in random effects pairwise and network meta-analyses. Med Decis Mak. 2018;38:531–42.

    Article  Google Scholar 

  22. Cox C. The generalised F distribution: an umbrella for parametric survival analysis. Stat Med. 2008;27:4301–12.

    Article  Google Scholar 

  23. Royston P, Parmar MKB. Flexible parametric proportional-hazards and proportional-odds models for censored survival data, with application to prognostic modelling and estimation of treatment effects. Stat Med. 2002;21:2175–97.

    Article  Google Scholar 

  24. Lambert PC, Thompson JR. Estimating and modelling the cure fraction in population-based cancer survival analysis. Biotatistics. 2007;8:576–94.

    Article  Google Scholar 

  25. Latimer N. Survival analysis for economic evaluations alongside clinical trials—extrapolation with patient-level data: inconsistencies, limitations and practical guide. Med Decis Mak. 2013;33:743–54.

    Article  Google Scholar 

  26. Ishak KJ, Kreif N, Benedict A, Muszbek N. Overview of parametric survival analysis for health-economic applications. Pharmacoeconomics. 2013;31:663–75.

    Article  Google Scholar 

  27. Bagust A, Beale S. Survival analysis and extrapolation modelling of time-to-event clinical trial data for economic evaluation: an alternative approach. Med Decis Mak. 2014;34:343–51.

    Article  Google Scholar 

  28. Tremblay G, Haines P, Briggs A. A criterion-based approach for the systematic and transparent extrapolation of clinical trial survival data. J Health Econ Outcomes Res. 2015;2:147–60.

    Google Scholar 

  29. Gibson E, Koblbauer I, Begum N, Dranitsaris G, Liew D, McEwan P, Monfared AAT. Modelling the survival outcomes of immuno-oncology drugs in economic evaluations: a systematic approach to data analysis and extrapolation. Pharmacoeconomics. 2017;35:1257–70.

    Article  Google Scholar 

  30. Ouwens MJNM, Philips Z, Jansen JP. Network meta-analysis of parametric survival curves. Res Synth Methods. 2010;1:258–71.

    Article  Google Scholar 

  31. Jansen JP. Network meta-analysis of survival data with fractional polynomials. BMC Med Res Methodol. 2011;11:61. http://www.biomedcentral.com/1471-2288/11/61.

    Article  Google Scholar 

  32. Jansen JP, Cope S. Meta-regression models to address heterogeneity and inconsistency in network meta-analysis of survival outcomes. BMC Med Res Methodol. 2012;12;152. http://www.biomedcentral.com/1471-2288/12/152.

    Article  Google Scholar 

  33. Jackson C, Stevens J, Ren S, Latimer N, Bojke L, Manca A, Sharples L. Extrapolating survival from randomised trials using external data: a review of methods. Med Decis Mak. 2017;37:377–90.

    Article  Google Scholar 

  34. Guyot P, Ades AE, Beasley M, Lueza B, Pignon J-P, Welton NJ. Extrapolation of survival curves from cancer trials using external information. Med Decis Mak. 2017;37:353–66.

    Article  Google Scholar 

  35. O’Hagan A, Buck CE, Daneshkhah A, Eiser JR, Garthwaite PH, Jenkinson DJ, Oakley JE, Rakow T. Uncertain judgements. Eliciting experts’ beliefs. Chichester: Wiley; 2006. ISBN: 9780470029992.

    Book  Google Scholar 

  36. Bojke L, Grigore B, Jankovic D, Peters J, Soares M, Stein K. Informing reimbursement decisions using cost-effectiveness modelling: a guide to the process of generating elicited prior to capture model uncertainties. Pharmacoeconomics. 2017;35:867–77.

    Article  Google Scholar 

  37. O’Hagan A, Forster J. Kendall’s advanced theory of statistics. Volume 2B. Bayesian inference. 2nd ed. London: Arnold; 2004. ISBN: 0340807520.

    Google Scholar 

  38. Sculpher M, Claxton K. Establishing the cost-effectiveness of new pharmaceuticals under conditions of uncertainty—when is there sufficient evidence? Value Health. 2005;8:433–46.

    Article  Google Scholar 

  39. Strong M, Oakley JE, Brennan A, Breeze P. Estimating the expected value of sample information using the probabilistic sensitivity analysis sample: a fast, nonparametric regression-based method. Med Decis Mak. 2015;35:570–83.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Health and Related Research, University of Sheffield, Sheffield, UK

    John W. Stevens

Authors
  1. John W. Stevens
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to John W. Stevens.

Ethics declarations

The opinions presented are the opinions of the author.

Funding

No funding was provided in support of this manuscript.

Conflict of interest

The author, John W. Stevens, declares no conflicts of interest.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Stevens, J.W. Using Evidence from Randomised Controlled Trials in Economic Models: What Information is Relevant and is There a Minimum Amount of Sample Data Required to Make Decisions?. PharmacoEconomics 36, 1135–1141 (2018). https://doi.org/10.1007/s40273-018-0681-y

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40273-018-0681-y

Search

Navigation