Using the Threshold Technique to Elicit Patient Preferences: An Introduction to the Method and an Overview of Existing Empirical Applications

  • Brett HauberEmail author
  • Joshua Coulter
Review Article


Patient preference information (PPI) is a topic of interest to regulators and industry. One of many known methods for eliciting PPI is the threshold technique (TT). However, empirical studies of the TT differ from each other in many ways and no effort to date has been made to summarize them or the evidence regarding the performance of the method. We sought to describe the TT and summarize the empirical applications of the method. Forty-three studies were reviewed. Most studies estimated the minimum level of benefit required to make a treatment worthwhile, and over half estimated the maximum level of risk patients would accept to achieve a treatment benefit. The evidence demonstrates that the TT can be used to elicit multiple types of thresholds and can be used to explore preference heterogeneity and preference non-linearity. Some evidence suggests that the method may be sensitive to anchoring and shift-framing effects; however, no evidence suggests that the method is more or less sensitive to these potential biases than other stated-preference methods. The TT may be a viable method for eliciting PPI to support regulatory decision-making; however, additional understanding of the performance of this method may be needed. Future research should focus on TT performance compared with other stated-preference methods, the extent to which results predict patient choice, and the ability of the TT to inform individual treatment decisions at the point of healthcare delivery.



The authors would like to acknowledge Mo Zhou for assistance in identifying some of the papers included here, Martin Ho and Mo Zhou for input that influenced some of the material in the paper, and John Forbes for extensive editorial review.

Authors’ contributions

Brett Hauber conceived of and designed this study, conducted the literature search, reviewed and summarized abstracts and manuscripts identified in the literature search, and contributed to writing the manuscript. Joshua Coulter assisted with the literature search, reviewed and summarized the abstracts and manuscripts identified in the literature search, and contributed to writing the manuscript.

Compliance with Ethical Standards


This work was funded in part by a professional development award from RTI International.

Conflict of interest

Brett Hauber and Joshua Coulter have no conflicts of interest to declare.


  1. 1.
    Ho MP, Gonzalez JM, Lerner HP, et al. Incorporating patient preference evidence into regulatory decision making. Surg Endosc. 2015;29(10):2984–93.CrossRefPubMedGoogle Scholar
  2. 2.
    Al-Faruque F. FDA weighs patients’ risk tolerance in approving obesity device. The Pink Sheet; 2015 Jan 20. Accessed 29 Jun 2019.
  3. 3.
    US Food and Drug Administration. Patient preference information—voluntary submission, review in premarket approval applications, humanitarian device exemption applications, and de novo requests, and inclusion in decision summaries and device labeling: guidance for industry, Food and Drug Administration staff, and other stakeholders. 2016. Accessed 27 Dec 2018.
  4. 4.
    US Food and Drug Administration. Plan for issuance of patient-focused drug development guidance under 21st Century Cures Act Title III Section 3002. 2017. Accessed 29 Jun 2019.
  5. 5.
    US Food and Drug Administration. Benefit-risk assessment throughout the drug lifecycle: FDA discussion document. 2019. Accessed 29 Jun 2019.
  6. 6.
    Postmus D, Mavris M, Hillege HL, et al. Incorporating patient preferences into drug development and regulatory decision making: results from a quantitative pilot study with cancer patients, carers, and regulators. Clin Pharmacol Ther. 2016;99(5):548–54.CrossRefPubMedGoogle Scholar
  7. 7.
    Medical Device Innovation Consortium. Medical Device Innovation Consortium (MDIC) patient centered benefit-risk project report: a framework for incorporating information on patient preferences regarding benefit and risk in regulatory assessments of new medical technology. 2015. Accessed 27 Dec 2018.
  8. 8.
    Hauber AB, Fairchild AO, Johnson FR. Quantifying benefit-risk preferences for medical interventions: an overview of a growing empirical literature. Appl Health Econ Health Policy. 2013;11(4):319–29.CrossRefGoogle Scholar
  9. 9.
    Soekhai V, Whichello C, Levitan B, et al. Methods for exploring and eliciting patient preference in the medical product lifecycle: a literature review. Drug Discov Today. 2019;24(7):1324–31.CrossRefPubMedGoogle Scholar
  10. 10.
    Medical Device Innovation Consortium. A case study in how patient preference information contributed to regulatory decisions for medical devices. 2017. Accessed 27 Dec 2018.
  11. 11.
    Llewellyn-Thomas HA. Threshold technique. In: Kattan MW, editor. Encyclopedia of medical decision making. Thousand Oaks: Sage; 2009. p. 1134–7.Google Scholar
  12. 12.
    Brundage MD, Davidson JR, Mackillop WJ, Feldman-Stewart D, Groome P. Using a treatment-tradeoff method to elicit preferences for the treatment of locally advanced non-small-cell lung cancer. Med Decis Making. 1998;18(3):256–67.CrossRefPubMedGoogle Scholar
  13. 13.
    Kopec JA, Richardson CG, Llewellyn-Thomas H, Klinkhoff A, Carswell A, Chalmers A. Probabilistic threshold technique showed that patients’ preferences for specific trade-offs between pain relief and each side effect of treatment in osteoarthritis varied. J Clin Epidemiol. 2007;60(9):929–38.CrossRefPubMedGoogle Scholar
  14. 14.
    Llewellyn-Thomas HA, Arshinoff R, Bell M, Williams JI, Naylor CD. In the queue for total joint replacement: patients’ perspectives on waiting times. Ontario Hip and Knee Replacement Project Team. J Eval Clin Pract. 1998;4(1):63–74.Google Scholar
  15. 15.
    Llewellyn-Thomas H, McGreal MJ, Thiel EC, Fine S, Erlichman C. Patients’ willingness to enter clinical trials: measuring the association with perceived benefit and preference for decision participation. Soc Sci Med. 1991;32:35–42.CrossRefPubMedGoogle Scholar
  16. 16.
    Llewellyn-Thomas HA, Paterson JM, Carter JA, et al. Primary prevention drug therapy: can it meet patients’ requirements for reduced risk? Med Decis Making. 2002;22:326–39.CrossRefPubMedGoogle Scholar
  17. 17.
    Llewellyn-Thomas H, Thiel E, Paterson M, Naylor D. In the queue for coronary artery bypass grafting: patients’ perceptions of risk and ‘maximal acceptable waiting time’. J Health Serv Res Policy. 1999;4(2):65–72.CrossRefPubMedGoogle Scholar
  18. 18.
    Llewellyn-Thomas HA, Williams JI, Levy L, Naylor CD. Using a trade-off technique to assess patients’ treatment preferences for benign prostatic hyperplasia. Med Decis Making. 1996;16(3):262–82.CrossRefPubMedGoogle Scholar
  19. 19.
    Naylor CD, Llewellyn-Thomas HA. Can there be a more patient-centred approach to determining clinically important effect sizes for randomized treatment trials? J Clin Epidemiol. 1994;47(7):787–95.CrossRefPubMedGoogle Scholar
  20. 20.
    Palda VA, Llewellyn-Thomas HA, Mackenzie RG, Pritchard KI, Naylor CD. Breast cancer patients’ attitudes about rationing postlumpectomy radiation therapy: applicability of trade-off methods to policy-making. J Clin Oncol. 1997;15(10):3192–200.CrossRefPubMedGoogle Scholar
  21. 21.
    Finlayson SR, Birkmeyer JD, Tosteson AN, Nease RF Jr. Patient preferences for location of care: implications for regionalization. Med Care. 1999;37(2):204–9.CrossRefPubMedGoogle Scholar
  22. 22.
    Blinman P, McLachlan SA, Nowak AK, et al. Lung cancer clinicians’ preferences for adjuvant chemotherapy in non-small-cell lung cancer: what makes it worthwhile? Lung Cancer. 2011;72(2):213–8. Scholar
  23. 23.
    Kunneman M, Pieterse AH, Stiggelbout AM, et al. Treatment preferences and involvement in treatment decision making of patients with endometrial cancer and clinicians. Br J Cancer. 2014;111:674–9.CrossRefPubMedPubMedCentralGoogle Scholar
  24. 24.
    Bossema E, Stiggelbout A, Baas-Thijssen M, van de Velde C, Marijnen C. Patients’ preferences for low rectal cancer surgery. Eur J Surg Oncol. 2008;34(1):42–8.CrossRefPubMedGoogle Scholar
  25. 25.
    Couture J, Chan R, Bouharaoui F. Patient’s preferences for adjuvant postoperative chemoradiation therapy in rectal cancer. Dis Colon Rectum. 2005;48(11):2055–60.CrossRefPubMedGoogle Scholar
  26. 26.
    Dales RE, O’Connor A, Hebert P, et al. Intubation and mechanical ventilation for COPD: development of an instrument to elicit patient preferences. Chest. 1999;116(3):792–800.CrossRefPubMedGoogle Scholar
  27. 27.
    Devereaux PJ, Anderson DR, Gardner MJ, et al. Differences between perspectives of physicians and patients on anticoagulation in patients with atrial fibrillation: observational study. BMJ. 2001;323(7323):1218–22.CrossRefPubMedPubMedCentralGoogle Scholar
  28. 28.
    Stafinski T, Menon D, Nardelli A, et al. Incorporating patient preferences into clinical trial design: results of the Opinions of Patients on Treatment Implications of New Studies (OPTIONS) project. Am Heart J. 2015;169:122–31.CrossRefPubMedGoogle Scholar
  29. 29.
    Bayram N, van Wely M, van der Veen F, Bossuyt PM, Nieuwkerk P. Treatment preferences and trade-offs for ovulation induction in clomiphene citrate-resistant patients with polycystic ovary syndrome. Fertil Steril. 2005;84(2):420–5.CrossRefPubMedGoogle Scholar
  30. 30.
    Tervonen T, Naci H, van Valkenhoef G, et al. Applying multiple criteria decision analysis to comparative benefit-risk assessment: choosing among statins in primary prevention. Med Decis Making. 2015;35(7):859–71.CrossRefPubMedGoogle Scholar
  31. 31.
    Cameron AC, Trivedi PK. Microeconometrics using Stata. Rev. ed. College Station: Stata Press; 2010.Google Scholar
  32. 32.
    Blinman P, Hughes B, Crombie C, et al; Australasian Lung Cancer Trials Group (ALTG). Patients’ and doctors’ preferences for adjuvant chemotherapy in resected non-small-cell lung cancer: what makes it worthwhile? Eur J Cancer. 2015;51(12):1529–37.
  33. 33.
    Brundage MD, Davidson JR, Mackillop WJ. Trading treatment toxicity for survival in locally advanced non-small cell lung cancer. J Clin Oncol. 1997;15(1):330–40.CrossRefPubMedGoogle Scholar
  34. 34.
    Brundage MD, Feldman-Stewart D, Cosby R, et al. Cancer patients’ attitudes toward treatment options for advanced non-small cell lung cancer: implications for patient education and decision support. Patient Educ Couns. 2001;45(2):149–57.CrossRefPubMedGoogle Scholar
  35. 35.
    Duric VM, Stockler MR, Heritier S, et al. Patients’ preferences for adjuvant chemotherapy in early breast cancer: what makes AC and CMF worthwhile now? Ann Oncol. 2005;16(11):1786–94.CrossRefPubMedGoogle Scholar
  36. 36.
    Duric VM, Fallowfield LJ, Saunders C, Houghton J, Coates AS, Stockler MR. Patients’ preferences for adjuvant endocrine therapy in early breast cancer: what makes it worthwhile? Br J Cancer. 2005;93(12):1319–23.CrossRefPubMedPubMedCentralGoogle Scholar
  37. 37.
    Simes RJ, Coates AS. Patient preferences for adjuvant chemotherapy of early breast cancer: how much benefit is needed? J Natl Cancer Inst Monogr. 2001;30:146–52.CrossRefGoogle Scholar
  38. 38.
    Blinman P, Duric V, Nowak AK, et al. Adjuvant chemotherapy for early colon cancer: what survival benefits make it worthwhile? Eur J Cancer. 2010;46(10):1800–7. Scholar
  39. 39.
    Molinari M, De Coutere S, Krahn M, Helton S, Urbach DR. Patients’ preferences and trade-offs for the treatment of early stage hepatocellular carcinoma. J Surg Res. 2014;189:57–67.CrossRefPubMedGoogle Scholar
  40. 40.
    Wilke DR, Krahn M, Tomlinson G, Bezjak A, Rutledge R, Warde P. Sex or survival: short-term versus long-term androgen deprivation in patients with locally advanced prostate cancer treated with radiotherapy. Cancer. 2010;116:1909–17.CrossRefPubMedGoogle Scholar
  41. 41.
    Kennedy ED, Schmocker S, Victor C, et al. Do patients consider preoperative chemoradiation for primary rectal cancer worthwhile? Cancer. 2011;117(13):2853–62.CrossRefPubMedGoogle Scholar
  42. 42.
    Cuffe S, Hon H, Qiu X, et al. Cancer patients’ acceptance, understanding, and willingness-to-pay for pharmacogenomic testing. Pharmacogenet Genomics. 2014;24(7):348–55.CrossRefPubMedGoogle Scholar
  43. 43.
    Tomlinson D, Bartels U, Gammon J, et al. Chemotherapy versus supportive care alone in pediatric palliative care for cancer: comparing the preferences of parents and health care professionals. CMAJ. 2011;183(17):E1252–8.CrossRefPubMedPubMedCentralGoogle Scholar
  44. 44.
    Kennedy ED, To T, Steinhart AH, Detsky A, Llewellyn-Thomas HA, McLeod RS. Do patients consider postoperative maintenance therapy for Crohn’s disease worthwhile? Inflamm Bowel Dis. 2008;14(2):224–35.CrossRefPubMedGoogle Scholar
  45. 45.
    Bates SM, Alonso-Coello P, Tikkinen KAO, et al. Women’s values and preferences and health state valuations for thromboprophylaxis during pregnancy: a cross-sectional interview study. Thromb Res. 2016;140:22–9.CrossRefPubMedGoogle Scholar
  46. 46.
    McAlister FA, O’Connor AM, Wells G, Grover SA, Laupacis A. When should hypertension be treated? The different perspectives of Canadian family physicians and patients. CMAJ. 2000;163(4):403–8.PubMedPubMedCentralGoogle Scholar
  47. 47.
    Lloyd CB, Nietert PJ, Silvestri GA. Intensive care decision making in the seriously ill and elderly. Crit Care Med. 2004;32(3):649–54.CrossRefPubMedGoogle Scholar
  48. 48.
    Percy ME, Llewellyn-Thomas H. Assessing preferences about the DNR order: does it depend on how you ask? Med Decis Making. 1995;15(3):209–16.Google Scholar
  49. 49.
    Turner DP, Golding AN, Houle TT. Using a graphical risk tool to examine willingness-to-take migraine prophylactic medications. Pain. 2016;157(10):2226–34.CrossRefPubMedPubMedCentralGoogle Scholar
  50. 50.
    Steures P, Berkhout JC, Hompes PG, et al. Patients’ preferences in deciding between intrauterine insemination and expectant management. Hum Reprod. 2005;20(3):752–5.CrossRefPubMedGoogle Scholar
  51. 51.
    van Weert JM, van den Broek J, van der Steeg JW, et al. Patients’ preferences for intrauterine insemination or in vitro fertilization. Reprod Biomed Online. 2007;15(4):422–7.CrossRefPubMedGoogle Scholar
  52. 52.
    Alvarado MD, Conolly J, Park C, et al. Patient preferences regarding intraoperative versus external beam radiotherapy following breast-conserving surgery. Breast Cancer Res Treat. 2014;143(1):135–40. Scholar
  53. 53.
    Corica T, Joseph D, Saunders C, Bulsara M, Nowak AK. Intraoperative radiotherapy for early breast cancer: do health professionals choose convenience or risk? Radiat Oncol. 2014;25(9):33. Scholar
  54. 54.
    Sung L, Feldman BM, Schwamborn G, et al. Inpatient versus outpatient management of low-risk pediatric febrile neutropenia: measuring parents’ and healthcare professionals’ preferences. J Clin Oncol. 2004;22(19):3922–9.CrossRefPubMedGoogle Scholar
  55. 55.
    Gupta AA, Donen RM, Sung L, et al. Testicular biopsy for fertility preservation in prepubertal boys with cancer: identifying preferences for procedure and reactions to disclosure practices. J Urol. 2016;196(1):219–24. Scholar
  56. 56.
    Alonso-Coello P, Montori VM, Díaz MG, et al. Values and preferences for oral antithrombotic therapy in patients with atrial fibrillation: physician and patient perspectives. Health Expect. 2015;18(6):2318–27. Scholar
  57. 57.
    Kok M, Gravendeel L, Opmeer BC, van der Post JA, Mol BW. Expectant parents’ preferences for mode of delivery and trade-offs of outcomes for breech presentation. Patient Educ Couns. 2008;72(2):305–10.CrossRefPubMedGoogle Scholar
  58. 58.
    Crump T, Llewellyn-Thomas HA. Assessing Medicare beneficiaries’ strength-of-preference scores for health care options: how engaging does the elicitation technique need to be? Health Expect. 2011;14(Suppl 1):33–45. Scholar
  59. 59.
    Crump RT, Llewellyn-Thomas H. Characterizing the public’s preferential attitudes toward end-of-life care options: a role for the threshold technique? Health Serv Res. 2013;48(6 Pt 1):2101–24. Scholar
  60. 60.
    Soekhai V, de Bekker-Grob EW, Ellis AR, Vass CM. Discrete choice experiments in health economics: past, present and future. Pharmacoeconomics. 2019;37(2):201–26. Scholar
  61. 61.
    Clark MD, Determann D, Petrou S, Moro D, de Bekker-Grob EW. Discrete choice experiments in health economics: a review of the literature. Pharmacoeconomics. 2014;32(9):883–902.CrossRefPubMedGoogle Scholar
  62. 62.
    de Bekker-Grob EW, Ryan M, Gerard K. Discrete choice experiments in health economics: a review of the literature. Health Econ. 2012;21(2):145–72.CrossRefPubMedGoogle Scholar
  63. 63.
    Klose T. The contingent valuation method in health care. Health Policy. 1999;47:97–123.CrossRefPubMedGoogle Scholar
  64. 64.
    Keeney RL, Raiffa H. Decisions with multiple objectives: preferences and value trade-offs. Cambridge: Cambridge University Press; 1993.CrossRefGoogle Scholar
  65. 65.
    Swait J, Louviere J. The role of the scale parameter in the estimation and comparison of multinomial logit models. J Mark Res. 1993;30:305–14.CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.RTI Health SolutionsResearch Triangle ParkUSA

Personalised recommendations