Applied Health Economics and Health Policy

, Volume 17, Issue 4, pp 533–543 | Cite as

Applying an Implementation Framework to the Use of Evidence from Economic Evaluations in Making Healthcare Decisions

  • Gregory MerloEmail author
  • Katie Page
  • Pauline Zardo
  • Nicholas Graves
Original Research Article


Background and Objective

There is a need for the application of theory in understanding the use of evidence from economic evaluations in healthcare decision making. The purpose of this study is to review the published literature on the use of evidence from economic evaluations for healthcare decision making and to map the findings to the Consolidated Framework for Implementation Research (CFIR).


A systematic search strategy was used to identify studies investigating the factors that determine the use of evidence from economic evaluation in healthcare decision making. Barriers and facilitators identified in the included studies were mapped across the five CFIR domains, with the “intervention” referring to the use of economic evaluations in decision making. Gaps, inconsistencies and emergent relations were identified through the mapping process.


Fifty-three studies met eligibility criteria and were included in the review. The CFIR constructs associated with the Intervention Characteristics and those associated with the knowledge and beliefs of users of economic evaluations were widely cited in the identified barriers and facilitators. Other constructs from the CFIR had not been reported in the literature, such as ‘organisational networks’ and ‘individual stage of change’. Most of the stages in the implementation process as described by the CFIR were reflected in the identified barriers and facilitators.


By categorising barriers and facilitators into domains, the CFIR provides a systematic approach to assess how these factors interact. Literature gaps in the literature regarding the use of economic evaluation in healthcare decision making were identified, specifically issues regarding organisational networks and the role of feedback.


Through mapping findings from studies of the use of evidence from economic evaluations in healthcare decision making, we present an implementation framework based on the CFIR for understanding the use of economic evaluations into practice.


Authors’ Contributions

All authors developed the concept and research design. GM conducted the literature review and data extraction and prepared the manuscript. KP and NG reviewed the manuscript and added content.

Compliance with Ethical Standards


This research was funded through a scholarship grant from the Centre of Research Excellence for Reducing Hospital Acquired Infection. The funding agreement ensured the authors’ independence in designing the study, interpreting the data, and writing and publishing the report.

Conflict of Interest

Gregory Merlo, Katie Page, Pauline Zardo and Nicholas Graves have no conflicts of interest that are directly relevant to the content of this article.

Ethics Approval

Not applicable.

Consent for Publication

Not applicable.

Data Availability

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Supplementary material

40258_2019_477_MOESM1_ESM.docx (18 kb)
Supplementary material 1 (DOCX 18 kb)


  1. 1.
    Elshaug AG, McWilliams JM, Landon BE. The value of low-value lists. JAMA. 2013;309(8):775–6.Google Scholar
  2. 2.
    Melfi CA, Drake BG, Tierney WM. The role of public opinion in drug resource allocation decisions. Pharmacoeconomics. 1996;9(2):106–12.Google Scholar
  3. 3.
    Traulsen JM, Almarsdóttir AB. Pharmaceutical policy and the lay public. Pharm World Sci. 2005;27(4):273–7.Google Scholar
  4. 4.
    Banks G. Evidence-based policy making: what is it? How do we get it?. Canberra (ACT): Productivity Commission; 2009.Google Scholar
  5. 5.
    Haines IE. Over 150 potentially low-value health care practices: an Australian study. Med J Aust. 2013;198(2):84–5.Google Scholar
  6. 6.
    Rosenberg W, Donald A. Evidence based medicine: an approach to clinical problem-solving. BMJ. 1995;310(6987):1122.Google Scholar
  7. 7.
    Merlo G, Page K, Ratcliffe J, Halton K, Graves N. Bridging the gap: exploring the barriers to using economic evidence in healthcare decision making and strategies for improving uptake. Appl Health Econ Health Policy. 2015;13(3):303–9.Google Scholar
  8. 8.
    Kislov R. Engaging with theory: from theoretically informed to theoretically informative improvement research. BMJ Qual Saf. 2019;28(3):177–9.Google Scholar
  9. 9.
    Davidoff F, Dixon-Woods M, Leviton L, Michie S. Demystifying theory and its use in improvement. BMJ Qual Saf. 2015;24(3):228–38.Google Scholar
  10. 10.
    Foy R, Ovretveit J, Shekelle PG, Pronovost PJ, Taylor SL, Dy S, et al. The role of theory in research to develop and evaluate the implementation of patient safety practices. BMJ Qual Saf. 2011;20(5):453–9.Google Scholar
  11. 11.
    Davies P, Walker AE, Grimshaw JM. A systematic review of the use of theory in the design of guideline dissemination and implementation strategies and interpretation of the results of rigorous evaluations. Implement Sci. 2010;5(1):14.Google Scholar
  12. 12.
    Colquhoun HL, Brehaut JC, Sales A, Ivers N, Grimshaw J, Michie S, et al. A systematic review of the use of theory in randomized controlled trials of audit and feedback. Implement Sci. 2013;8(1):66.Google Scholar
  13. 13.
    Williams I, Bryan S. Understanding the limited impact of economic evaluation in health care resource allocation: a conceptual framework. Health Policy. 2007;80(1):135–43. Scholar
  14. 14.
    Bryan S, Williams I, McIver S. Seeing the nice side of cost-effectiveness analysis: a qualitative investigation of the use of CEA in nice technology appraisals. Health Econ. 2007;16(2):179–93. Scholar
  15. 15.
    Williams I, Bryan S, McIver S. How should cost-effectiveness analysis be used in health technology coverage decisions? Evidence from the National Institute for Health and Clinical Excellence approach. J Health Serv Res Policy. 2007;12(2):73–9. Scholar
  16. 16.
    Williams I, McIver S, Moore D, Bryan S. The use of economic evaluations in NHS decision-making: a review and empirical investigation. Health Technol Assess. 2008;12(7):iii–63.Google Scholar
  17. 17.
    Dixon-Woods M. Using framework-based synthesis for conducting reviews of qualitative studies. BMC Med. 2011;9(1):39.Google Scholar
  18. 18.
    Kirk MA, Kelley C, Yankey N, Birken SA, Abadie B, Damschroder L. A systematic review of the use of the consolidated framework for implementation research. Implement Sci. 2016;11(1):1.Google Scholar
  19. 19.
    Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q. 2004;82(4):581–629.Google Scholar
  20. 20.
    Kitson A. From research to practice: one organisational model for promoting research based practice. EDTNA ERCA J. 1996;23(4):39–45.Google Scholar
  21. 21.
    Graham ID, Logan J. Innovations in knowledge transfer and continuity of care. Can J Nurs Res. 2004;36(2):89–103.Google Scholar
  22. 22.
    Damschroder LJ, Lowery JC. Evaluation of a large-scale weight management program using the consolidated framework for implementation research (CFIR). Implementat Sci. 2013;8(1):51.Google Scholar
  23. 23.
    English M, Nzinga J, Mbindyo P, Ayieko P, Irimu G, Mbaabu L. Explaining the effects of a multifaceted intervention to improve inpatient care in rural Kenyan hospitals-interpretation based on retrospective examination of data from participant observation, quantitative and qualitative studies. Implement Sci. 2011;6(1):124.Google Scholar
  24. 24.
    Connell LA, McMahon NE, Watkins CL, Eng JJ. Therapists’ use of the graded repetitive arm supplementary program (GRASP) intervention: a practice implementation survey study. Phys Ther. 2014;94(5):632.Google Scholar
  25. 25.
    Hoffmann C, Graf von der Schulenburg JM. The influence of economic evaluation studies on decision making: a European survey. Health Policy. 2000;52(3):179–92.Google Scholar
  26. 26.
    Anell A, Svarvar P. Pharmacoeconomics and clinical practice guidelines: a survey of attitudes in Swedish formulary committees. Pharmacoeconomics. 2000;17(2):175–85.Google Scholar
  27. 27.
    Chaikledkaew U, Lertpitakpong C, Teerawattananon Y, Thavorncharoensap M, Tangcharoensathien V. The current capacity and future development of economic evaluation for policy decision-making: a survey among researchers and decision-makers in Thailand. Value Health. 2009;12(Suppl. 3):S31–5. Scholar
  28. 28.
    Sloan FA, Whetten-Goldstein K, Wilson A. Hospital pharmacy decisions, cost containment, and the use of cost-effectiveness analysis. Soc Sci Med. 1997;45(4):523–33. Scholar
  29. 29.
    West R, Borden EK, Coller JP, Rawson NSB, Tonks RS. “Cost-effectiveness” estimates result in flawed decision-making in listing drugs for reimbursement. Can J Public Health. 2002;93(6):421–5.Google Scholar
  30. 30.
    Zwart-Van Rijkom JEF, Leufkens HGM, Busschbach JJV, Broekmans AW, Rutten FFH. Differences in attitudes, knowledge and use of economic evaluations in decision-making in the Netherlands: the Dutch results from the EUROMET project. Pharmacoeconomics. 2000;18(2):149–60.Google Scholar
  31. 31.
    Tarride JE, McCarron CE, Lim M, Bowen JM, Blackhouse G, Hopkins R, et al. Economic evaluations conducted by Canadian health technology assessment agencies: where do we stand? Int J Technol Assess Health Care. 2008;24(4):437–44. Scholar
  32. 32.
    Baghbanian A, Hughes I, Khavarpour FA. Resource allocation and economic evaluation in Australia’s healthcare system. Aust Health Rev. 2011;35(3):278–83. Scholar
  33. 33.
    Haslé-Pham E, Arnould B, Späth HM, Follet A, Duru G, Marquis P. Role of clinical, patient-reported outcome and medico-economic studies in the public hospital drug formulary decision-making process: results of a European survey. Health Policy. 2005;71(2):205–12. Scholar
  34. 34.
    Jansson S, Anell A. The impact of decentralised drug-budgets in Sweden: a survey of physicians’ attitudes towards costs and cost-effectiveness. Health Policy. 2006;76(3):299–311. Scholar
  35. 35.
    Fattore G, Torbica A. Economic evaluation in health care: the point of view of informed physicians. Value Health. 2006;9(3):157–67. Scholar
  36. 36.
    Walley T, Baron S, Cooke J, Drummond M. Economic evaluations of drug therapy: attitudes of primary care prescribing advisors in Great Brittain. Health Policy. 1997;41(1):61–72.Google Scholar
  37. 37.
    Bloom BS. Use of formal benefit/cost evaluations in health system decision making. Am J Manag Care. 2004;10(5):329–35.Google Scholar
  38. 38.
    Drummond M, Cooke J, Walley T. Economic evaluation under managed competition: evidence from the U.K. Soc Sci Med. 1997;45(4):583–95. Scholar
  39. 39.
    Kulsomboon V, Palumbo FB, Mullins CD. Criteria to request pharmacoeconomic data and data sources for hospital formulary decisions. Drug Inf J. 2001;35(1):231–40.Google Scholar
  40. 40.
    Odedina FT, Sullivan J, Nash R, Clemmons CD. Use of pharmacoeconomic data in making hospital formulary decisions. Am J Health Syst Pharm. 2002;59(15):1441–4.Google Scholar
  41. 41.
    Späth HM, Charavel M, Morelle M, Carrere MO. A qualitative approach to the use of economic data in the selection of medicines for hospital formularies: a French survey. Pharm World Sci. 2003;25(6):269–75. Scholar
  42. 42.
    Wu O, Knill-Jones R, Wilson P, Craig N. The impact of economic information on medical decision making in primary care. J Eval Clin Pract. 2004;10(3):407–11. Scholar
  43. 43.
    Hailey D. Perceptions of Australian health technology assessments: report of a survey. Int J Technol Assess Health Care. 1993;9(4):588–9.Google Scholar
  44. 44.
    Thurston SJ, Craig D, Wilson P, Drummond MF. Increasing decision-makers’ access to economic evaluations: alternative methods of communicating the information. Int J Technol Assess Health Care. 2008;24(2):151–7.Google Scholar
  45. 45.
    Davies L, Coyle D, Drummond M. Current status of economic appraisal of health technology in the European community: report of the network. Soc Sci Med. 1994;38(12):1601–7. Scholar
  46. 46.
    Alsultan MS. The role of pharmacoeconomics in formulary decision making in different hospitals in Riyadh,Saudi Arabia. Saudi Pharm J. 2011;19(1):51–6.Google Scholar
  47. 47.
    Holtorf A-P, Brixner D, Bellows B, Keskinaslan A, Dye J, Oderda G. Current and future use of HEOR data in healthcare decision-making in the United States and in emerging markets. Am Health Drug Benefits. 2012;5(7):428.Google Scholar
  48. 48.
    Bae EY, Hong JM, Kwon HY, Jang S, Lee HJ, Bae S, et al. Eight-year experience of using HTA in drug reimbursement: South Korea. Health Policy. 2016;120(6):612–20. Scholar
  49. 49.
    Gallego G, Van Gool K, Kelleher D. Resource allocation and health technology assessment in Australia: views from the local level. Int J Technol Assess Health Care. 2009;25(2):134–40. Scholar
  50. 50.
    Chen LC, Ashcroft DM, Elliott RA. Do economic evaluations have a role in decision-making in Medicine Management Committees? A qualitative study. Pharm World Sci. 2007;29(6):661–70. Scholar
  51. 51.
    Martin DK, Hollenberg D, MacRae S, Madden S, Singer P. Priority setting in a hospital drug formulary: a qualitative case study and evaluation. Health Policy. 2003;66(3):295–303.Google Scholar
  52. 52.
    Ross J. The use of economic evaluation in health care: Australian decision makers’ perceptions. Health Policy. 1995;31(2):103–10. Scholar
  53. 53.
    Singer PA, Martin DK, Giacomini M, Purdy L. Priority setting for new technologies in medicine: qualitative case study. BMJ. 2000;321(7272):1316–9.Google Scholar
  54. 54.
    Eddama O, Coast J. Use of economic evaluation in local health care decision-making in England: A qualitative investigation. Health Policy. 2009;89(3):261–70. Scholar
  55. 55.
    Dempsey AF, Cowan AE, Stokley S, Messonnier M, Clark SJ, Davis MM. The role of economic information in decision-making by the Advisory Committee on Immunization Practices. Vaccine. 2008;26(42):5389–92. Scholar
  56. 56.
    Al MJ, Feenstra T, Brouwer WBF. Decision makers’ views on health care objectives and budget constraints: results from a pilot study. Health Policy. 2004;70(1):33–48. Scholar
  57. 57.
    Duthie T, Trueman P, Chancellor J, Diez L. Research into the use of health economics in decision making in the United Kingdom. Phase II: is health economics ‘for good or evil’? Health Policy. 1999;46(2):143–57. Scholar
  58. 58.
    Mitton C, Donaldson C. Setting priorities in Canadian regional health authorities: a survey of key decision makers. Health Policy. 2002;60(1):39–58. Scholar
  59. 59.
    Ijzerman MJ, Reuzel RPB, Severens HL. Pre-assessment to assess the match between cost-effectiveness results and decision makers’ information needs: an illustration using two cases in rehabilitation medicine in the Netherlands. Int J Technol Assess Health Care. 2003;19(1):17–27.Google Scholar
  60. 60.
    Williams IP, Bryan S. Cost-effectiveness analysis and formulary decision making in England: findings from research. Soc Sci Med. 2007;65(10):2116–29. Scholar
  61. 61.
    Berry SR, Hubay S, Soibelman H, Martin DK. The effect of priority setting decisions for new cancer drugs on medical oncologists’ practice in Ontario: a qualitative study. BMC Health Serv Res. 2007;7(1):1.Google Scholar
  62. 62.
    Cheung KL, Evers SM, Hiligsmann M, Vokó Z, Pokhrel S, Jones T, et al. Understanding the stakeholders’ intention to use economic decision-support tools: a cross-sectional study with the Tobacco Return on Investment tool. Health Policy. 2016;120(1):46–54.Google Scholar
  63. 63.
    van Dongen JM, Tompa E, Clune L, Sarnocinska-Hart A, Bongers PM, van Tulder MW, et al. Bridging the gap between the economic evaluation literature and daily practice in occupational health: a qualitative study among decision-makers in the healthcare sector. Implement Sci. 2013;8:57.Google Scholar
  64. 64.
    Martin DK, Pater JL, Singer PA. Priority-setting decisions for new cancer drugs: a qualitative case study. Lancet. 2001;358(9294):1676–81.Google Scholar
  65. 65.
    PausJenssen AM, Singer PA, Detsky AS. Ontario’s formulary committee: how recommendations are made. Pharmacoeconomics. 2003;21(4):285–94. Scholar
  66. 66.
    Teerawattananon Y, Russell S. A difficult balancing act: plicy actors’ perspectives on using economic evaluation to inform health-care coverage decisions under the Universal Health Insurance Coverage scheme in Thailand. Value Health. 2008;11(Suppl. 1):S52–60. Scholar
  67. 67.
    Omar F, Tinghög G, Tinghög P, Carlsson P. Attitudes towards priority-setting and rationing in healthcare: an exploratory survey of Swedish medical students. Scand J Publ Health. 2009;37(2):122–30. Scholar
  68. 68.
    Asim OB, J, Coast J. Towards a greater understanding of priority setting: role of an economic approach? Findings from a workshop to local decision makers. London: Health Economics Study Group; 2006.Google Scholar
  69. 69.
    Gallego G, Casey R, Norman R, Goodall S. Introduction and uptake of new medical technologies in the Australian health care system: a qualitative study. Health Policy. 2011;102(2–3):152–8. Scholar
  70. 70.
    Gallego G, Fowler S, van Gool K. Decision makers’ perceptions of health technology decision making and priority setting at the institutional level. Aust Health Rev. 2008;32(3):520–7.Google Scholar
  71. 71.
    Gallego G, Van Gool K, Casey R, Maddern G. Surgeon’s veiws of health technology assessment in Australia: online pilot survey. Int J Technol Assess Health Care. 2013;29(3):309–14. Scholar
  72. 72.
    Hoffmann C, Stoykova BA, Nixon J, Glanville JM, Misso K, Drummond MF. Do health-care decision makers find economic evaluations useful? The findings of focus group research in UK health authorities. Value Health. 2002;5(2):71–8. Scholar
  73. 73.
    Claxton K, Ginnelly L, Sculpher M, Philips Z, Palmer S. A pilot study on the use of decision theory and value of information analysis as part of the NHS Health Technology Assessment programme. Health Technol Assess. 2004;8(31):1–103.Google Scholar
  74. 74.
    Van Harten WH, Retèl VP. Close cooperation with health technology assessment expertise is crucial for implementation and ultimately reimbursement of innovations in oncology. Ecancermedicalscience. 2016;28(10):686.Google Scholar
  75. 75.
    Anell A. Priority setting for pharmaceuticals: the use of health economic evidence by reimbursement and clinical guidance committees. Eur J Health Econ. 2004;5(1):28–35. Scholar
  76. 76.
    Jenkings KN, Barber N. What constitutes evidence in hospital new drug decision making? Soc Sci Med. 2004;58(9):1757–66. Scholar
  77. 77.
    McDonald R. Street-level bureaucrats? Heart disease, health economics and policy in a primary care group. Health Social Care Commun. 2002;10(3):129–35.Google Scholar
  78. 78.
    Kolasa K, Dziomdziora M, Fajutrao L. What aspects of the health technology assessment process recommended by international health technology assessment agencies received the most attention in Poland in 2008? Int J Technol Assess Health Care. 2011;27(1):84–94. Scholar
  79. 79.
    Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4(1):50.Google Scholar
  80. 80.
    Rich RF. Measuring knowledge utilization: processes and outcomes. Knowledge Policy. 1997;10(3):11–24.Google Scholar
  81. 81.
    Lavis JN, Ross SE, Hurley JE. Examining the role of health services research in public policymaking. Milbank Q. 2002;80(1):125–54.Google Scholar
  82. 82.
    Chubin D, Maienschein J. Staffing science policy-making. Science. 2000;290(5496):1501.Google Scholar
  83. 83.
    Coast J. Is economic evaluation in touch with society’s health values? BMJ. 2004;329(7476):1233–6.Google Scholar
  84. 84.
    Pettigrew AM. Context and action in the transformation of the firm. J Manage Stud. 1987;24(6):649–70.Google Scholar
  85. 85.
    Banta HD. Dissemination of health technology assessment. In: del Llano-Senaris J, Campillo-Artero CC, editors. Health technology assessment and health policy today: a multifaceted view of their unstable crossroads. Heidelberg: Springer; 2015. p. 147–56.Google Scholar
  86. 86.
    Elshaug AG, Hiller JE, Tunis SR, Moss JR. Challenges in Australian policy processes for disinvestment from existing, ineffective health care practices. Aust N Z Health Policy. 2007;4(1):23.Google Scholar
  87. 87.
    Pearson S, Littlejohns P. Reallocating resources: how should the National Institute for Health and Clinical Excellence guide disinvestment efforts in the National Health Service? J Health Serv Res Policy. 2007;12(3):160–5.Google Scholar
  88. 88.
    Hoffmann C. The influence of economic evaluation studies on decision making: a European survey. Health Policy. 2000;52(3):179–92. Scholar
  89. 89.
    Robinson LA, Hammitt JK. Behavioural economics and the conduct of benefit-cost analysis: towards principles and standards. J Benefit Cost Anal. 2011;2(2):1–51.Google Scholar
  90. 90.
    Harris TI. QALYfying the value of life. J Med Ethics. 1987;13(3):117–23.Google Scholar
  91. 91.
    Kirk MA, Kelley C, Yankey N, Birken SA, Abadie B, Damschroder L. A systematic review of the use of the consolidated framework for implementation research. Implement Sci. 2016;11(1):72.Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Queensland University of Technology, Institute of Health and Biomedical InnovationBrisbaneAustralia
  2. 2.Faculty of LawQueensland University of TechnologyBrisbaneAustralia

Personalised recommendations