, Volume 37, Issue 7, pp 871–877 | Cite as

The Curve of Optimal Sample Size (COSS): A Graphical Representation of the Optimal Sample Size from a Value of Information Analysis

  • Eric Jutkowitz
  • Fernando Alarid-EscuderoEmail author
  • Karen M. Kuntz
  • Hawre Jalal
Practical Application


Value of information (VOI) analysis quantifies the opportunity cost associated with decision uncertainty, and thus informs the value of collecting further information to avoid this cost. VOI can inform study design, optimal sample size selection, and research prioritization. Recent methodological advances have reduced the computational burden of conducting VOI analysis and have made it easier to evaluate the expected value of sample information, the expected net benefit of sampling, and the optimal sample size of a study design (\(n^{*}\)). The volume of VOI analyses being published is increasing, and there is now a need for VOI studies to conduct sensitivity analyses on VOI-specific parameters. In this practical application, we introduce the curve of optimal sample size (COSS), which is a graphical representation of \(n^{*}\) over a range of willingness-to-pay thresholds and VOI parameters (example data and R code are provided). In a single figure, the COSS presents summary data for decision makers to determine the sample size that optimizes research funding given their operating characteristics. The COSS also presents variation in the optimal sample size given variability or uncertainty in VOI parameters. The COSS represents an efficient and additional approach for summarizing results from a VOI analysis.


Author contributions

EJ, FAE, KMK, and HJ: study design and analysis. All authors participated in the interpretation of the data, drafting of the manuscript, critical revision of the manuscript, and approval of the final manuscript.

Compliance with Ethical Standards

Data availability statement

Data and statistical code are provided in the online appendix.


Financial support for this study was provided in part by a Doctoral Dissertation Fellowship from the Graduate School of the University of Minnesota as part of Dr. Alarid-Escudero’s doctoral program. Drs. Kuntz and Alarid-Escudero were supported by a Grant from the National Cancer Institute (U01-CA-199335) as part of the Cancer Intervention and Surveillance Modeling Network (CISNET). Dr. Jutkowitz was supported by a Grant from the National Institute on Aging (1R21AG059623-01) and a Grant from the Brown School of Public Health. The funding agencies had no role in the design of the study, interpretation of results, or writing of the manuscript. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. The funding agreement ensured the authors’ independence in designing the study, interpreting the data, and writing and publishing the report.

Conflict of interest

EJ reports no conflicts of interest. FAE reports no conflicts of interest. KMK reports no conflicts of interests. HL reports no conflicts of interest.

Supplementary material

40273_2019_770_MOESM1_ESM.docx (865 kb)
Supplementary material 1 (DOCX 864 kb)
40273_2019_770_MOESM2_ESM.r (18 kb)
Supplementary material 2 (R 19 kb)
40273_2019_770_MOESM3_ESM.r (25 kb)
Supplementary material 3 (R 25 kb)
40273_2019_770_MOESM4_ESM.csv (2.7 mb)
Supplementary material 4 (CSV 2722 kb)


  1. 1.
    Claxton K, Posnett J. An economic approach to clinical trial design and research priority-setting. Health Econ. 1996;5(6):513–24.CrossRefGoogle Scholar
  2. 2.
    Jalal H, Alarid-Escudero F. A Gaussian approximation approach for value of information analysis. Med Decis Mak. 2018;38(2):174–88.CrossRefGoogle Scholar
  3. 3.
    Menzies NA. An efficient estimator for the expected value of sample information. Med Decis Mak. 2016;36(3):308–20.CrossRefGoogle Scholar
  4. 4.
    Jalal H, Goldhaber-Fiebert JD, Kuntz KM. Computing expected value of partial sample information from probabilistic sensitivity analysis using linear regression metamodeling. Med Decis Mak. 2015;35(5):584–95.CrossRefGoogle Scholar
  5. 5.
    Baio G, Berardi A, Heath A. Bayesian cost-effectiveness analysis with the R package BCEA. London: Springer; 2017.CrossRefGoogle Scholar
  6. 6.
    Heath A, Manolopoulou I, Baio G. Efficient Monte Carlo estimation of the expected value of sample information using moment matching. Med Decis Mak. 2018;38(2):163–73.CrossRefGoogle Scholar
  7. 7.
    Strong M, Oakley JE, Brennan A, Breeze P. Estimating the expected value of sample information using the probabilistic sensitivity analysis sample: a fast, nonparametric regression-based method. Med Decis Mak. 2015;35(5):570–83.CrossRefGoogle Scholar
  8. 8.
    Jutkowitz E, Alarid-Escudero F, Choi HK, Kuntz KM, Jalal H. Prioritizing future research on allopurinol and febuxostat for the management of gout: value of information analysis. Pharmacoeconomics. 2017;35(10):1073–85.CrossRefGoogle Scholar
  9. 9.
    Tuffaha HW, Gordon LG, Scuffham PA. Value of information analysis informing adoption and research decisions in a portfolio of health care interventions. MDM Policy Pract. 2016;1(1):1–11.Google Scholar
  10. 10.
    Tuffaha HW, Reynolds H, Gordon LG, Rickard CM, Scuffham PA. Value of information analysis optimizing future trial design from a pilot study on catheter securement devices. Clin Trials. 2014;11(6):648–56.CrossRefGoogle Scholar
  11. 11.
    Kearns B, Chilcott J, Whyte S, Preston L, Sadler S. Cost-effectiveness of screening for ovarian cancer amongst postmenopausal women: a model-based economic evaluation. BMC Med. 2016;14(1):200.CrossRefGoogle Scholar
  12. 12.
    Rabideau DJ, Pei PP, Walensky RP, Zheng A, Parker RA. Implementing generalized additive models to estimate the expected value of sample information in a microsimulation model: results of three case studies. Med Decis Mak. 2018;38(2):189–99.CrossRefGoogle Scholar
  13. 13.
    Steuten L, van de Wetering G, Groothuis-Oudshoorn K, Retèl V. A systematic and critical review of the evolving methods and applications of value of information in academia and practice. Pharmacoeconomics. 2013;31(1):25–48.CrossRefGoogle Scholar
  14. 14.
    Wilson EC. A practical guide to value of information analysis. Pharmacoeconomics. 2015;33(2):105–21.CrossRefGoogle Scholar
  15. 15.
    Willan A, Kowgier M. Determining optimal sample sizes for multi-stage randomized clinical trials using value of information methods. Clin Trials. 2008;5(4):289–300.CrossRefGoogle Scholar
  16. 16.
    Willan AR. Optimal sample size determinations from an industry perspective based on the expected value of information. Clin Trials. 2008;5(6):587–94.CrossRefGoogle Scholar
  17. 17.
    Willan AR, Pinto EM. The value of information and optimal clinical trial design. Stat Med. 2005;24(12):1791–806.CrossRefGoogle Scholar
  18. 18.
    Heath A, Manolopoulou I, Baio G. A review of methods for analysis of the expected value of information. Med Decis Mak. 2017;37(7):747–58.CrossRefGoogle Scholar
  19. 19.
    Briggs A, Sculpher M, Claxton K. Decision modelling for health economic evaluation. Oxford: Oxford University Press; 2006.Google Scholar
  20. 20.
    Eckermann S, Willan AR. Expected value of information and decision making in HTA. Health Econ. 2007;16(2):195–209.CrossRefGoogle Scholar
  21. 21.
    Tuffaha HW, Gordon LG, Scuffham PA. Value of information analysis in healthcare: a review of principles and applications. J Med Econ. 2014;17(6):377–83.CrossRefGoogle Scholar
  22. 22.
    Tuffaha HW, Gordon LG, Scuffham PA. Value of information analysis in oncology: the value of evidence and evidence of value. J Oncol Pract. 2014;10(2):e55–62.CrossRefGoogle Scholar
  23. 23.
    Strong M, Oakley JE, Brennan A. Estimating multiparameter partial expected value of perfect information from a probabilistic sensitivity analysis sample: a nonparametric regression approach. Med Decis Mak. 2014;34(3):311–26.CrossRefGoogle Scholar
  24. 24.
    Madan J, Ades AE, Price M, Maitland K, Jemutai J, Revill P, Welton NJ. Strategies for efficient computation of the expected value of partial perfect information. Med Decis Mak. 2014;34(3):327–42.CrossRefGoogle Scholar
  25. 25.
    Philips Z, Claxton K, Palmer S. The half-life of truth: what are appropriate time horizons for research decisions? Med Decis Mak. 2008;28(3):287–99.CrossRefGoogle Scholar
  26. 26.
    Ades AE, Lu G, Claxton K. Expected value of sample information calculations in medical decision modeling. Med Decis Mak. 2004;24(2):207–27.CrossRefGoogle Scholar
  27. 27.
    Meltzer DO, Hoomans T, Chung JW, Basu A. Minimal modeling approaches to value of information analysis for health research. Med Decis Mak. 2011;31(6):E1–22.CrossRefGoogle Scholar
  28. 28.
    Jutkowitz E, Choi HK, Pizzi LT, Kuntz KM. Cost-effectiveness of allopurinol and febuxostat for the management of gout. Ann Intern Med. 2014;161(9):617–26.CrossRefGoogle Scholar
  29. 29.
    Eckermann S, Willan AR. Time and expected value of sample information wait for no patient. Value Health. 2008;11(3):522–6.CrossRefGoogle Scholar
  30. 30.
    Willan AR, Eckermann S. Optimal clinical trial design using value of information methods with imperfect implementation. Health Econ. 2010;19(5):549–61.Google Scholar
  31. 31.
    U.S. Department of Health and Human Services; U.S. Food & Drug Administration. Orange book: approved drug products with therapeutic equivalence evaluations. 2016 Accessed 15 Mar 2018.
  32. 32.
    Neumann PJ, Sanders GD, Russell LB, Siegel JE, Ganiats TG. Cost-effectiveness in health and medicine. 2nd ed. New York: Oxford University Press; 2017.Google Scholar
  33. 33.
    Zhu Y, Pandya BJ, Choi HK. Prevalence of gout and hyperuricemia in the US general population: the National Health and Nutrition Examination Survey 2007–2008. Arthritis Rheum. 2011;63(10):3136–41.CrossRefGoogle Scholar
  34. 34.
    Wallace KL, Riedel AA, Joseph-Ridge N, Wortmann R. Increasing prevalence of gout and hyperuricemia over 10 years among older adults in a managed care population. J Rheumatol. 2004;31(8):1582–7.Google Scholar
  35. 35.
    Arromdee E, Michet CJ, Crowson CS, O’Fallon WM, Gabriel SE. Epidemiology of gout: is the incidence rising? J Rheumatol. 2002;29(11):2403–6.Google Scholar
  36. 36.
    Mikuls TR, Farrar JT, Bilker WB, Fernandes S, Schumacher HR Jr, Saag KG. Gout epidemiology: results from the UK General Practice Research Database, 1990–1999. Ann Rheum Dis. 2005;64(2):267–72.CrossRefGoogle Scholar
  37. 37.
    Johnston SC, Rootenberg JD, Katrak S, Smith WS, Elkins JS. Effect of a US National Institutes of Health programme of clinical trials on public health and costs. Lancet. 2006;367(9519):1319–27.CrossRefGoogle Scholar
  38. 38.
    Emanuel EJ, Schnipper LE, Kamin DY, Levinson J, Lichter AS. The costs of conducting clinical research. J Clin Oncol. 2003;21(22):4145–50.CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Department of Health Services, Policy and PracticeBrown University School of Public HealthProvidenceUSA
  2. 2.Drug Policy ProgramCenter for Research and Teaching in Economics (CIDE)-CONACyTAguascalientesMexico
  3. 3.Division of Health Policy and ManagementUniversity of Minnesota School of Public HealthMinneapolisUSA
  4. 4.Division of Health Policy and Management, Graduate School of Public HealthUniversity of PittsburghPittsburghUSA

Personalised recommendations