, Volume 37, Issue 1, pp 29–43 | Cite as

One Method, Many Methodological Choices: A Structured Review of Discrete-Choice Experiments for Health State Valuation

  • Brendan MulhernEmail author
  • Richard Norman
  • Deborah J. Street
  • Rosalie Viney
Systematic Review



Discrete-choice experiments (DCEs) are used in the development of preference-based measure (PBM) value sets. There is considerable variation in the methodological approaches used to elicit preferences.


Our objective was to carry out a structured review of DCE methods used for health state valuation.


PubMed was searched until 31 May 2018 for published literature using DCEs for health state valuation. Search terms to describe DCEs, the process of valuation and preference-based instruments were developed. English language papers with any study population were included if they used DCEs to develop or directly inform the production of value sets for generic or condition-specific PBMs. Assessment of paper quality was guided by the recently developed Checklist for Reporting Valuation Studies. Data were extracted under six categories: general study information, choice task and study design, type of designed experiment, modelling and analysis methods, results and discussion.


The literature search identified 1132 published papers, and 63 papers were included in the review. Paper quality was generally high. The study design and choice task formats varied considerably, and a wide range of modelling methods were employed to estimate value sets.


This review of DCE methods used for developing value sets suggests some recurring limitations, areas of consensus and areas where further research is required. Methodological diversity means that the values should be seen as experimental, and users should understand the features of the value sets produced before applying them in decision making.



The authors thank Liz Chinchen for her support with the literature searching, and Elly Stolk for comments on an earlier version of the review. This work was presented at the EuroQol Group Plenary (Krakow 2015) and the International Society of Pharmacoeconomics and Outcomes Research (Washington 2016) meetings.

Author Contributions

BM carried out the literature search, extracted the data, led the data synthesis and interpretation and developed the first draft of the manuscript. RN, RV and DJS supported the data-extraction process and interpretation of the results and were involved in the development of the manuscript.

Compliance with Ethical Standards


This review was partly funded by the Australian National Health and Medical Research Council. Mr Mulhern was funded by a University of Technology Sydney President’s Scholarship.

Conflicts of interest

BM, RN, DJS and RV have no conflicts of interest.

Data availability statement

No datasets were generated or analysed during this study.

Supplementary material

40273_2018_714_MOESM1_ESM.docx (13 kb)
Supplementary material 1 (DOCX 13 kb)
40273_2018_714_MOESM2_ESM.docx (32 kb)
Supplementary material 2 (DOCX 32 kb)
40273_2018_714_MOESM3_ESM.docx (20 kb)
Supplementary material 3 (DOCX 20 kb)


  1. 1.
    Brooks R. EuroQol: the current state of play. Health Policy. 1996;37:53–72.Google Scholar
  2. 2.
    Herdman M, Gudex C, Lloyd A, Janssen MF, Kind P, Parkin D, et al. Development and preliminary testing of the new five-level version of EQ-5D (EQ-5D-5L). Qual Life Res. 2011;20(10):1727–36.Google Scholar
  3. 3.
    Brazier J, Roberts J, Deverill M. The estimation of a preference-based measure of health from the SF-36. J Health Econ. 2002;21:271–92.Google Scholar
  4. 4.
    de Bekker-Grob E, Ryan M, Gerard K. Discrete choice experiments in health economics: a review of the literature. Health Econ. 2010;21(2):145–72.Google Scholar
  5. 5.
    Clark M, Determann D, Petrou S, et al. Discrete choice experiments in health economics: a review of the literature. Pharmacoeconomics. 2014;32:883–902.Google Scholar
  6. 6.
    Bansback N, Brazier J, Tsuchiya A, Anis A. Using a discrete choice experiment to estimate societal health state utility values. J Health Econ. 2012;31(1):306–18.Google Scholar
  7. 7.
    Norman R, Cronin P, Viney R. A pilot discrete choice experiment to explore preferences for EQ-5D-5L health states. Appl Health Econ Health Policy. 2013;11(3):287–98.Google Scholar
  8. 8.
    Norman R, Viney R, Brazier J, et al. Valuing SF-6D health states using a discrete choice experiment. Med Decis Mak. 2014;34(6):773–86.Google Scholar
  9. 9.
    Mulhern B, Bansback N, Hole AR, Tsuchiya A. Using Discrete Choice Experiments with duration to model EQ-5D-5L health state preferences: testing experimental design strategies. Med Decis Mak. 2017;37(3):285–97.Google Scholar
  10. 10.
    Viney R, Norman R, Brazier J, et al. An Australian discrete choice experiment to value EQ-5D health states. Health Econ. 2013;23:729–42.Google Scholar
  11. 11.
    Jonker M, Attema A, Donkers B, Stolk E, Versteegh M. Are health state valuations from the general public biased? A test of health state reference dependency using self-assessed health and an efficient discrete choice experiment. Health Econ. 2017;26(12):1534–47.Google Scholar
  12. 12.
    Rowen D, Brazier J, van Hout B. A comparison of methods for converting DCE values onto the full health- dead QALY scale. Med Decis Making. 2015;35(3):328–40.Google Scholar
  13. 13.
    Ratcliffe J, Brazier J, Tsuchiya A, Symonds T, Brown M. Using DCE and ranking data to estimate cardinal values for health states for deriving a preference based single index from the sexual quality of life questionnaire. Health Econ. 2009;18:1261–76.Google Scholar
  14. 14.
    Street DJ, Burgess L. The construction of optimal stated choice experiments. Hoboken: Wiley; 2007.Google Scholar
  15. 15.
    Bansback N, Hole AR, Mulhern B, Tsuchiya A. Testing a discrete choice experiment including duration to value health states for large descriptive systems: addressing design and sampling issues. Soc Sci Med. 2014;114:38–48.Google Scholar
  16. 16.
    StataCorp. Stata statistical software: release 15. College Station: StataCorp LP; 2017.Google Scholar
  17. 17.
    Metrics Choice. Ngene [software for experimental design]. Sydney: Choice Metrics; 2012.Google Scholar
  18. 18.
    Craig BM, Busschbach JV. The episodic random utility model unifies time trade off and discrete choice approaches in health state valuation. Pop Health Metr. 2009;7:3.Google Scholar
  19. 19.
    Craig BM, Rand K, Bailey H, Stalmeier PFM. Quality adjusted life years without constant proportionality. Value Health. 2018. Scholar
  20. 20.
    Norman R, Mulhern B, Viney R. The impact of different DCE-based approaches when anchoring utility scores. Pharmacoeconomics. 2016;34(8):805–14.Google Scholar
  21. 21.
    Hensher DA, Rose JM, Greene WH. Applied choice analysis. 2nd ed. Cambridge: University Press; 2015.Google Scholar
  22. 22.
    Ramos Goni JM, Pinto Prades JL, Oppe M, Cabases JM, Serrano Aguiar P, Rivero Arias O. Valuation and modelling of EQ-5D-5L health states using a hybrid approach. Med Care. 2017;55(7):e51–8.Google Scholar
  23. 23.
    Cheung KL, Wijnen BFM, Hollin IL, Janssen EM, Bridges JF, Evers SMAA, Hiligsmann M. Using best-worst scaling to investigate preferences in health care. Pharmacoeconomics. 2016;34(12):1195–209.Google Scholar
  24. 24.
    Karimi M, Brazier J, Paisley S. How do individuals value health states? A qualitative investigation. Soc Sci Med. 2017;172:80–8.Google Scholar
  25. 25.
    Mulhern B, Bansback N, Brazier J, Buckingham K, Cairns J, Devlin N, Dolan P, Hole AR, Kavetsos G, Longworth L, Rowen D, Tsuchiya A. Preparatory study for the revaluation of the EQ-5D tariff: methodology report. Health Technol Assess. 2014;18:12.Google Scholar
  26. 26.
    Oppe M, Devlin NJ, van Hout B, Krabbe PFM, de Charro F. A program of methodological research to arrive at the new international EQ-5D-5L valuation protocol. Value Health. 2014;17:445–53.Google Scholar
  27. 27.
    Kim SH, Ahn J, Ock M, Shin S, Park J, Luo N, Jo MW. The EQ-5D-5L valuation study in Korea. Qual Life Res. 2016;25(7):1845–52.Google Scholar
  28. 28.
    Xie F, Pickard AS, Krabbe PF, Revicki D, Viney R, Devlin N, Feeny D. A checklist for reporting valuation studies of multi-attribute utility-based instruments (CREATE). Pharmacoeconomics. 2015;33(8):867–77.Google Scholar
  29. 29.
    Brazier J, Connell J, Papaioannou D, Mukuria C, Mulhern B, Peasgood T, Lloyd Jones M, Paisley S, O’Cathain A, Barkham M, Knapp M, Byford S, Gilbody S, Parry G. A systematic review, psychometric analysis and qualitative assessment of Generic Preference-Based Measures of Health in Mental Health Populations and the estimation of mapping functions from widely used specific measures. Health Technol Assess. 2014;18:34.Google Scholar
  30. 30.
    Longworth L, Yang Y, Young T, Mulhern B, Hernandez-Alava M, Mukuria C, Rowen D, Tosh J, Tsuchiya A, Evans P. Use of generic and condition specific measures of health related quality of life in NICE decision making: systematic review, statistical modelling and survey. Health Technol Assess. 2014;18:9.Google Scholar
  31. 31.
    Hakim Z, Pathak D. Modelling the EuroQol data: a comparison of discrete choice, conjoint and conditional preference modelling. Health Econ. 1999;8:103–16.Google Scholar
  32. 32.
    Ryan M, Netten A, Skatun D, Smith P. Using discrete choice experiments to estimate a preference based measure of outcome—an application to social care for older people. J Health Econ. 2006;25:927–44.Google Scholar
  33. 33.
    Burr J, Kilonzo M, Vale L, Ryan M. Developing a preference based glaucoma utility index using a discrete choice experiment. Optom Vis Sci. 2007;84(8):797–809.Google Scholar
  34. 34.
    Stolk E, Oppe M, Scalone L, Krabbe P. Discrete choice modelling for the quantification of health states: the case of the EQ-5D. Value Health. 2010;13(8):1005–13.Google Scholar
  35. 35.
    Hauber AB, Mohamed A, Johnson FR, Oyelowo O, Curtis B, Coon C. Estimating importance weights for the IWQOL-Lite using conjoint analysis. Qual Life Res. 2011;19:701–9.Google Scholar
  36. 36.
    Potoglu D, Burge P, Flynn T, Netten A, Malley J, Forder J, Brazier J. Best worst scaling vs. discrete choice experiments: an empirical comparison using social care data. Soc Sci Med. 2011;72:1717–27.Google Scholar
  37. 37.
    Bailey H. Results from a preliminary study to develop the quality adjustments for quality adjusted life year values for Trinidad and Tobago. West Indian Med J. 2013;62(6):543–7.Google Scholar
  38. 38.
    Pullenayegum E, Xie F. Scoring the 5-level EQ-5D: can latent utilities derived from a discrete choice model be transformed to health utilities derived from a time trade off task. Med Decis Mak. 2013;33:567–78.Google Scholar
  39. 39.
    Ramos Goni JM, Rivero-Arias O, Errea M, Stolk E, Herdman M, Cabases JM. Dealing with the health state ‘dead’ when using discrete choice experiments to obtain values for EQ-5D-5L health states. Eur J Health Econ. 2013;14(S1):S33–42.Google Scholar
  40. 40.
    Craig B, Pickard AS, Stolk E, Brazier J. US valuation of the SF-6D. Med Decis Mak. 2013;33(6):793–803.Google Scholar
  41. 41.
    Krabbe P, Devlin N, Stolk E, Shah K, Oppe M, van Hout B, Quik E, Pickard AS, Xie F. Multinational evidence of the applicability and robustness of discrete choice modelling for deriving EQ-5D-5L health state values. Med Care. 2014;52(11):935–43.Google Scholar
  42. 42.
    Xie F, Pullenayegum E, Gaebel K, Oppe M, Krabbe P. Eliciting preferences to the EQ-5D-5L health states: discrete choice experiment or multiprofile case of best worst scaling. Eur J Health Econ. 2014;15:281–8.Google Scholar
  43. 43.
    Gu Y, Norman R, Viney R. Estimating health state utility values from discrete choice experiments—a QALY space model approach. Health Econ. 2014;23(9):1098–114.Google Scholar
  44. 44.
    van Hoorn R, Donders A, Oppe M, Stalmeier P. The better than dead method: feasibility and interpretation of a valuation study. Pharmacoeconomics. 2014;32:789–99.Google Scholar
  45. 45.
    Robinson A, Spencer A, Moffatt P. A framework for estimating health state utility values within a discrete choice experiment: modelling risky choices. Med Decis Mak. 2015;35(3):341–50.Google Scholar
  46. 46.
    Hoefman R, van Exel J, Rose J, van de Wetering E, Brouwer W. A discrete choice experiment to obtain a tariff for valuing informal care situations measured with the CarerQol instrument. Med Decis Mak. 2014;34(1):84–96.Google Scholar
  47. 47.
    Craig B, Reeve B, Brown P, Cella D, Hays R, Lipscomb J, Pickard AS, Revicki D. US valuation of health outcomes measured using the PROMIS-29. Value Health. 2014;2014(7):846–53.Google Scholar
  48. 48.
    Scalone L, Stalmeier P, Milani S, Krabbe P. Values for health states with different life durations. Eur J Health Econ. 2015;16(9):917–25.Google Scholar
  49. 49.
    Gartner F, de Bekker-Grob E, Stiggelbout A, Rijnders M, Freeman L, Middeldorp J, Bloemenkamp K, de Miranda E, van den Akker-van Marle M. Calculating preference weights for the labor and delivery index: a discrete choice experiment on women’s birth experiences. Value Health. 2015;18:856–64.Google Scholar
  50. 50.
    Hole AR, Norman R, Viney R. Response patterns in health state valuation using endogenous attribute attendance and latent class analysis. Health Econ. 2016;25(2):212–24.Google Scholar
  51. 51.
    Mulhern B, Shah K, Janssen MF, Longworth L. Valuing health using Time Trade Off and Discrete Choice methods: does dimension order impact on health state values? Value Health. 2016;19(2):210–7.Google Scholar
  52. 52.
    Shiroiwa T, Ikeda S, Noto S, Igarashi A, Fukuda T, Saito S, Shimizuma K. Comparison of value set based on DCE and/or TTO data: scoring for EQ-5D-5L health states in Japan. Value Health. 2016;19:648–54.Google Scholar
  53. 53.
    Norman R, Viney R, Aaronson N, Brazier J, Cella D, Costa D, Fayers P, Kemmler G, Peacock S, Pickard AS, Rowen D, Street D, Velikova G, Young T, King M. Using a discrete choice experiment to value the QLU-C10D: feasibility and sensitivity to presentation format. Qual Life Res. 2016;25:637–49.Google Scholar
  54. 54.
    Craig B, Greiner W, Brown D, Reeve B. Valuation of child health related quality of life in the United States. Health Econ. 2016;25:768–77.Google Scholar
  55. 55.
    Craig B, Brown D, Reeve B. Valuation of child behavioural problems from the perspective of US adults. Med Decis Mak. 2016;36(2):199–209.Google Scholar
  56. 56.
    Versteegh M, Vermeulen K, Evers S, de Wit G, Prenger R, Stolk E. Dutch tariff for the five level version of EQ-5D. Value Health. 2016;19:343–52.Google Scholar
  57. 57.
    Bailey H, Stolk E, Kind P. Toward explicit prioritisation for the Caribbean: an EQ-5D value set for Trinidad and Tobago. Value Health Regional Issues. 2016;11C:60–7.Google Scholar
  58. 58.
    Norman R, Kemmler G, Viney R, Pickard AS, Gamper E, Holzner B, Nerich V, King M. Order of presentation of dimensions does not systematically bias utility weights from a discrete choice experiment. Value Health. 2016;19:1033–8.Google Scholar
  59. 59.
    Robinson A, Spencer A, Pinto-Prades J, Covey J. Exploring differences between TTO and DCE in the valuation of health states. Med Decis Mak. 2017;37(3):273–84.Google Scholar
  60. 60.
    Xie F, Pullenayegum E, Pickard AS, Ramos Goni JM, Jo MW, Igarashi A. Transforming latent utilities to health utilities: east does not meet west. Health Econ. 2017;26(12):1524–33.Google Scholar
  61. 61.
    Krucien N, Watson V, Ryan M. Is best worst scaling suitable for health state valuation? A comparison with discrete choice experiments. Health Econ. 2017;26(12):e1–16.Google Scholar
  62. 62.
    Mulhern B, Norman R, Lorgelly P, Lancsar E, Ratcliffe J, Brazier J, Viney R. Is dimension order important when valuing health states using Discrete Choice Experiments including duration? Pharmacoeconomics. 2017;35(4):439–51.Google Scholar
  63. 63.
    Goossens LMA, Rutten-van Mölken MPMH, Boland MRS, Donkers B, Jonker MF, Slok A, Salomé P,5 van Schayck OCP, in ‘t Veen JCCM, Stolk EA. ABC Index: quantifying experienced burden of COPD in a discrete choice experiment and predicting costs. BMJ Open. 2017;7:e017831.Google Scholar
  64. 64.
    Huynh E, Coast J, Rose J, Kinghorn P, Flynn T. Values for the ICECAP-supportive care measure (ICECAP-SCM) for use in economic evaluation at end of life. Soc Sci Med. 2017;189:114–28.Google Scholar
  65. 65.
    Purba FD, Hunfeld JAM, Iskandarsyah A, Fitriana TS, Sadarjoen SS, Ramos-Goni JM, Passchier J, van Busschbach JJ. The Indonesian EQ-5D-5L value set. Pharmacoeconomics. 2017;35(11):1153–65.Google Scholar
  66. 66.
    Hoefman RJ, van Exel J, Brouwer WB. Measuring care-related quality of life of caregivers for use in economic evaluations: CarerQol tariffs for Australia, Germany, Sweden, UK, and US. Pharmacoeconomics. 2017;35(4):469–78.Google Scholar
  67. 67.
    Jonker MF, Donkers B, de Bekker-Grob E, Stolk E. The effect of level overlap and color coding on attribute non-attendance in discrete choice experiments. Value Health. 2018;21(7):767–71.Google Scholar
  68. 68.
    Devlin N, Shah K, Feng Y, Mulhern BJ, van Hout B. Valuing health-related quality of life: an EQ-5D-5L value set for England. Health Econ. 2018;27(1):7–22.Google Scholar
  69. 69.
    Feng Y, Devlin N, Shah K, Mulhern BJ, van Hout B. New methods for modelling EQ-5D-5L value sets: an application to English data. Health Econ. 2018;27(1):23–38.Google Scholar
  70. 70.
    Rowen D, Mulhern B, Stevens K, Vermaire E. Estimating a Dutch value set for the paediatric preference-based CHU-9D using a discrete choice experiment with duration. Value Health. 2018. Scholar
  71. 71.
    King MT, Viney R, Pickard AS, Rowen D, Aaronson NK, Brazier JE, Cella D, Costa D, Fayers P, Kemmler G, McTaggart-Cowen H, Mercieca-Bebber R, Peacock S, Street DJ, Young TA, Norman R. Australian utility weights for the EORTC QLU-C10D, a multi-attribute utility instrument derived from the cancer-specific quality of life questionnaire, EORTC QLQ-C30. PharmacoEconomics. 2018;36(2):225–38.Google Scholar
  72. 72.
    Cole A, Shah K, Mulhern B, Feng Y, Devlin N. Valuing EQ-5D-5L health states ‘in context’ using a discrete choice experiment. Eur J Health Econ. 2018;19(4):595–605.Google Scholar
  73. 73.
    Mulhern B, Norman R, Shah K, Bansback N, Longworth L, Viney R. How should DCE with duration choice sets be presented for the valuation of health states? Med Decis Mak. 2018;38(3):306–18.Google Scholar
  74. 74.
    Purba F, Hunfeld JAM, Timman R, Iskandarsyah A, Fitriana T, Sadarjoen S, Passchier J, Busschbach JJV. Test-retest reliability of EQ-5D-5L valuation techniques: the composite time trade off and discrete choice experiments. Value Health. 2018. Scholar
  75. 75.
    Gamper EM, Holzner B, King MT, Norman R, Viney R, Nerich V, Kemmler G. Test-retest reliability of discrete choice experiment for valuations of QLU-C10D health states. Value Health. 2018;21(8):958–66.Google Scholar
  76. 76.
    Ramos-Goñi JM, Craig B, Oppe M, Ramallo-Fariña Y, Pinto-Prades JL, Luo N, Rivero-Arias O. Handling data quality issues to estimate the Spanish EQ-5D-5L value set using a hybrid interval regression approach. Value Health. 2018;21(5):596–604.Google Scholar
  77. 77.
    Craig B, Rand K. Choice defines QALYs: a US valuation of the EQ-5D-5L. Med Care. 2018;56(6):529–36.Google Scholar
  78. 78.
    Jakubczyk M, Craig B, Barra M, Groothuis-Oudshoorn C, Hartman J, Huynh E, Ramos-Goñi JM, Stolk E, Rand K. Choice defines value: a predictive modeling competition in health preference research. Value Health. 2018;21(2):229–38.Google Scholar
  79. 79.
    Craig BM, Rand K, Bailey H, Stalmeier P. Quality-adjusted life-years without constant proportionality. Value Health. 2018. Scholar
  80. 80.
    Jonker M, Donkers B, de Bekker-Grob E, Stolk E. Advocating a paradigm shift in health-state valuations: the estimation of time-preference corrected QALY tariffs. Value Health. 2018;21(8):993–1001.Google Scholar
  81. 81.
    Feng Y, Hole AR, Karimi M, Tsuchiya A, van Hout B. An exploration of the non-iterative time trade-off method to value health states. Health Econ. 2018;27(8):1247–63.Google Scholar
  82. 82.
    Hole AR. CLOGITHET: Stata module to estimate heteroscedastic conditional logit models. Statistical Software Components S456737; 2006.Google Scholar
  83. 83.
    Hole AR. Small-sample properties of tests for heteroscedasticity in the conditional logit model. Econ Bull. 2006;3(18):1–14.Google Scholar
  84. 84.
    Swait J, Louviere J. The role of the scale parameter in the estimation and comparison of multinomial logit models. J Mark Res. 1993;30(3):305–14.Google Scholar
  85. 85.
    Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6(7):e1000097.Google Scholar
  86. 86.
    Mulhern B, Longworth L, Brazier JE, Rowen D, Bansback N, Devlin N, Tsuchiya A. Health state valuation questions: head to head comparison of online and CAPI. Value Health. 2013;16(1):104–13.Google Scholar
  87. 87.
    Street DJ, Mulhern B, Norman R, Viney R. Using simulations to compare DCE designs that could be used to value EQ-5D. Barcelona: EuroQol Plenary; 2017.Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.Centre for Health Economics Research and EvaluationUniversity of TechnologySydneyAustralia
  2. 2.School of Public HealthCurtin UniversityPerthAustralia

Personalised recommendations