Abstract
Background and Objective
There is a need for the application of theory in understanding the use of evidence from economic evaluations in healthcare decision making. The purpose of this study is to review the published literature on the use of evidence from economic evaluations for healthcare decision making and to map the findings to the Consolidated Framework for Implementation Research (CFIR).
Methods
A systematic search strategy was used to identify studies investigating the factors that determine the use of evidence from economic evaluation in healthcare decision making. Barriers and facilitators identified in the included studies were mapped across the five CFIR domains, with the “intervention” referring to the use of economic evaluations in decision making. Gaps, inconsistencies and emergent relations were identified through the mapping process.
Results
Fifty-three studies met eligibility criteria and were included in the review. The CFIR constructs associated with the Intervention Characteristics and those associated with the knowledge and beliefs of users of economic evaluations were widely cited in the identified barriers and facilitators. Other constructs from the CFIR had not been reported in the literature, such as ‘organisational networks’ and ‘individual stage of change’. Most of the stages in the implementation process as described by the CFIR were reflected in the identified barriers and facilitators.
Discussion
By categorising barriers and facilitators into domains, the CFIR provides a systematic approach to assess how these factors interact. Literature gaps in the literature regarding the use of economic evaluation in healthcare decision making were identified, specifically issues regarding organisational networks and the role of feedback.
Conclusions
Through mapping findings from studies of the use of evidence from economic evaluations in healthcare decision making, we present an implementation framework based on the CFIR for understanding the use of economic evaluations into practice.
Similar content being viewed by others
References
Elshaug AG, McWilliams JM, Landon BE. The value of low-value lists. JAMA. 2013;309(8):775–6.
Melfi CA, Drake BG, Tierney WM. The role of public opinion in drug resource allocation decisions. Pharmacoeconomics. 1996;9(2):106–12.
Traulsen JM, Almarsdóttir AB. Pharmaceutical policy and the lay public. Pharm World Sci. 2005;27(4):273–7.
Banks G. Evidence-based policy making: what is it? How do we get it?. Canberra (ACT): Productivity Commission; 2009.
Haines IE. Over 150 potentially low-value health care practices: an Australian study. Med J Aust. 2013;198(2):84–5.
Rosenberg W, Donald A. Evidence based medicine: an approach to clinical problem-solving. BMJ. 1995;310(6987):1122.
Merlo G, Page K, Ratcliffe J, Halton K, Graves N. Bridging the gap: exploring the barriers to using economic evidence in healthcare decision making and strategies for improving uptake. Appl Health Econ Health Policy. 2015;13(3):303–9.
Kislov R. Engaging with theory: from theoretically informed to theoretically informative improvement research. BMJ Qual Saf. 2019;28(3):177–9.
Davidoff F, Dixon-Woods M, Leviton L, Michie S. Demystifying theory and its use in improvement. BMJ Qual Saf. 2015;24(3):228–38.
Foy R, Ovretveit J, Shekelle PG, Pronovost PJ, Taylor SL, Dy S, et al. The role of theory in research to develop and evaluate the implementation of patient safety practices. BMJ Qual Saf. 2011;20(5):453–9.
Davies P, Walker AE, Grimshaw JM. A systematic review of the use of theory in the design of guideline dissemination and implementation strategies and interpretation of the results of rigorous evaluations. Implement Sci. 2010;5(1):14.
Colquhoun HL, Brehaut JC, Sales A, Ivers N, Grimshaw J, Michie S, et al. A systematic review of the use of theory in randomized controlled trials of audit and feedback. Implement Sci. 2013;8(1):66.
Williams I, Bryan S. Understanding the limited impact of economic evaluation in health care resource allocation: a conceptual framework. Health Policy. 2007;80(1):135–43. https://doi.org/10.1016/j.healthpol.2006.03.006.
Bryan S, Williams I, McIver S. Seeing the nice side of cost-effectiveness analysis: a qualitative investigation of the use of CEA in nice technology appraisals. Health Econ. 2007;16(2):179–93. https://doi.org/10.1002/hec.1133.
Williams I, Bryan S, McIver S. How should cost-effectiveness analysis be used in health technology coverage decisions? Evidence from the National Institute for Health and Clinical Excellence approach. J Health Serv Res Policy. 2007;12(2):73–9. https://doi.org/10.1258/135581907780279521.
Williams I, McIver S, Moore D, Bryan S. The use of economic evaluations in NHS decision-making: a review and empirical investigation. Health Technol Assess. 2008;12(7):iii–63.
Dixon-Woods M. Using framework-based synthesis for conducting reviews of qualitative studies. BMC Med. 2011;9(1):39.
Kirk MA, Kelley C, Yankey N, Birken SA, Abadie B, Damschroder L. A systematic review of the use of the consolidated framework for implementation research. Implement Sci. 2016;11(1):1.
Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q. 2004;82(4):581–629.
Kitson A. From research to practice: one organisational model for promoting research based practice. EDTNA ERCA J. 1996;23(4):39–45.
Graham ID, Logan J. Innovations in knowledge transfer and continuity of care. Can J Nurs Res. 2004;36(2):89–103.
Damschroder LJ, Lowery JC. Evaluation of a large-scale weight management program using the consolidated framework for implementation research (CFIR). Implementat Sci. 2013;8(1):51.
English M, Nzinga J, Mbindyo P, Ayieko P, Irimu G, Mbaabu L. Explaining the effects of a multifaceted intervention to improve inpatient care in rural Kenyan hospitals-interpretation based on retrospective examination of data from participant observation, quantitative and qualitative studies. Implement Sci. 2011;6(1):124.
Connell LA, McMahon NE, Watkins CL, Eng JJ. Therapists’ use of the graded repetitive arm supplementary program (GRASP) intervention: a practice implementation survey study. Phys Ther. 2014;94(5):632.
Hoffmann C, Graf von der Schulenburg JM. The influence of economic evaluation studies on decision making: a European survey. Health Policy. 2000;52(3):179–92.
Anell A, Svarvar P. Pharmacoeconomics and clinical practice guidelines: a survey of attitudes in Swedish formulary committees. Pharmacoeconomics. 2000;17(2):175–85.
Chaikledkaew U, Lertpitakpong C, Teerawattananon Y, Thavorncharoensap M, Tangcharoensathien V. The current capacity and future development of economic evaluation for policy decision-making: a survey among researchers and decision-makers in Thailand. Value Health. 2009;12(Suppl. 3):S31–5. https://doi.org/10.1111/j.1524-4733.2009.00624.x.
Sloan FA, Whetten-Goldstein K, Wilson A. Hospital pharmacy decisions, cost containment, and the use of cost-effectiveness analysis. Soc Sci Med. 1997;45(4):523–33. https://doi.org/10.1016/S0277-9536(96)00393-0.
West R, Borden EK, Coller JP, Rawson NSB, Tonks RS. “Cost-effectiveness” estimates result in flawed decision-making in listing drugs for reimbursement. Can J Public Health. 2002;93(6):421–5.
Zwart-Van Rijkom JEF, Leufkens HGM, Busschbach JJV, Broekmans AW, Rutten FFH. Differences in attitudes, knowledge and use of economic evaluations in decision-making in the Netherlands: the Dutch results from the EUROMET project. Pharmacoeconomics. 2000;18(2):149–60.
Tarride JE, McCarron CE, Lim M, Bowen JM, Blackhouse G, Hopkins R, et al. Economic evaluations conducted by Canadian health technology assessment agencies: where do we stand? Int J Technol Assess Health Care. 2008;24(4):437–44. https://doi.org/10.1017/S0266462308080574.
Baghbanian A, Hughes I, Khavarpour FA. Resource allocation and economic evaluation in Australia’s healthcare system. Aust Health Rev. 2011;35(3):278–83. https://doi.org/10.1071/AH10890.
Haslé-Pham E, Arnould B, Späth HM, Follet A, Duru G, Marquis P. Role of clinical, patient-reported outcome and medico-economic studies in the public hospital drug formulary decision-making process: results of a European survey. Health Policy. 2005;71(2):205–12. https://doi.org/10.1016/j.healthpol.2004.08.007.
Jansson S, Anell A. The impact of decentralised drug-budgets in Sweden: a survey of physicians’ attitudes towards costs and cost-effectiveness. Health Policy. 2006;76(3):299–311. https://doi.org/10.1016/j.healthpol.2005.06.002.
Fattore G, Torbica A. Economic evaluation in health care: the point of view of informed physicians. Value Health. 2006;9(3):157–67. https://doi.org/10.1111/j.1524-4733.2006.00096.x.
Walley T, Baron S, Cooke J, Drummond M. Economic evaluations of drug therapy: attitudes of primary care prescribing advisors in Great Brittain. Health Policy. 1997;41(1):61–72.
Bloom BS. Use of formal benefit/cost evaluations in health system decision making. Am J Manag Care. 2004;10(5):329–35.
Drummond M, Cooke J, Walley T. Economic evaluation under managed competition: evidence from the U.K. Soc Sci Med. 1997;45(4):583–95. https://doi.org/10.1016/s0277-9536(96)00398-x.
Kulsomboon V, Palumbo FB, Mullins CD. Criteria to request pharmacoeconomic data and data sources for hospital formulary decisions. Drug Inf J. 2001;35(1):231–40.
Odedina FT, Sullivan J, Nash R, Clemmons CD. Use of pharmacoeconomic data in making hospital formulary decisions. Am J Health Syst Pharm. 2002;59(15):1441–4.
Späth HM, Charavel M, Morelle M, Carrere MO. A qualitative approach to the use of economic data in the selection of medicines for hospital formularies: a French survey. Pharm World Sci. 2003;25(6):269–75. https://doi.org/10.1023/B:PHAR.0000006523.22131.69.
Wu O, Knill-Jones R, Wilson P, Craig N. The impact of economic information on medical decision making in primary care. J Eval Clin Pract. 2004;10(3):407–11. https://doi.org/10.1111/j.1365-2753.2004.00490.x.
Hailey D. Perceptions of Australian health technology assessments: report of a survey. Int J Technol Assess Health Care. 1993;9(4):588–9.
Thurston SJ, Craig D, Wilson P, Drummond MF. Increasing decision-makers’ access to economic evaluations: alternative methods of communicating the information. Int J Technol Assess Health Care. 2008;24(2):151–7.
Davies L, Coyle D, Drummond M. Current status of economic appraisal of health technology in the European community: report of the network. Soc Sci Med. 1994;38(12):1601–7. https://doi.org/10.1016/0277-9536(94)90060-4.
Alsultan MS. The role of pharmacoeconomics in formulary decision making in different hospitals in Riyadh,Saudi Arabia. Saudi Pharm J. 2011;19(1):51–6.
Holtorf A-P, Brixner D, Bellows B, Keskinaslan A, Dye J, Oderda G. Current and future use of HEOR data in healthcare decision-making in the United States and in emerging markets. Am Health Drug Benefits. 2012;5(7):428.
Bae EY, Hong JM, Kwon HY, Jang S, Lee HJ, Bae S, et al. Eight-year experience of using HTA in drug reimbursement: South Korea. Health Policy. 2016;120(6):612–20. https://doi.org/10.1016/j.healthpol.2016.03.013.
Gallego G, Van Gool K, Kelleher D. Resource allocation and health technology assessment in Australia: views from the local level. Int J Technol Assess Health Care. 2009;25(2):134–40. https://doi.org/10.1017/S0266462309090187.
Chen LC, Ashcroft DM, Elliott RA. Do economic evaluations have a role in decision-making in Medicine Management Committees? A qualitative study. Pharm World Sci. 2007;29(6):661–70. https://doi.org/10.1007/s11096-007-9125-z.
Martin DK, Hollenberg D, MacRae S, Madden S, Singer P. Priority setting in a hospital drug formulary: a qualitative case study and evaluation. Health Policy. 2003;66(3):295–303.
Ross J. The use of economic evaluation in health care: Australian decision makers’ perceptions. Health Policy. 1995;31(2):103–10. https://doi.org/10.1016/0168-8510(94)00671-7.
Singer PA, Martin DK, Giacomini M, Purdy L. Priority setting for new technologies in medicine: qualitative case study. BMJ. 2000;321(7272):1316–9.
Eddama O, Coast J. Use of economic evaluation in local health care decision-making in England: A qualitative investigation. Health Policy. 2009;89(3):261–70. https://doi.org/10.1016/j.healthpol.2008.06.004.
Dempsey AF, Cowan AE, Stokley S, Messonnier M, Clark SJ, Davis MM. The role of economic information in decision-making by the Advisory Committee on Immunization Practices. Vaccine. 2008;26(42):5389–92. https://doi.org/10.1016/j.vaccine.2008.07.085.
Al MJ, Feenstra T, Brouwer WBF. Decision makers’ views on health care objectives and budget constraints: results from a pilot study. Health Policy. 2004;70(1):33–48. https://doi.org/10.1016/j.healthpol.2004.01.009.
Duthie T, Trueman P, Chancellor J, Diez L. Research into the use of health economics in decision making in the United Kingdom. Phase II: is health economics ‘for good or evil’? Health Policy. 1999;46(2):143–57. https://doi.org/10.1016/s0168-8510(98)00057-8.
Mitton C, Donaldson C. Setting priorities in Canadian regional health authorities: a survey of key decision makers. Health Policy. 2002;60(1):39–58. https://doi.org/10.1016/S0168-8510(01)00190-7.
Ijzerman MJ, Reuzel RPB, Severens HL. Pre-assessment to assess the match between cost-effectiveness results and decision makers’ information needs: an illustration using two cases in rehabilitation medicine in the Netherlands. Int J Technol Assess Health Care. 2003;19(1):17–27.
Williams IP, Bryan S. Cost-effectiveness analysis and formulary decision making in England: findings from research. Soc Sci Med. 2007;65(10):2116–29. https://doi.org/10.1016/j.socscimed.2007.06.009.
Berry SR, Hubay S, Soibelman H, Martin DK. The effect of priority setting decisions for new cancer drugs on medical oncologists’ practice in Ontario: a qualitative study. BMC Health Serv Res. 2007;7(1):1.
Cheung KL, Evers SM, Hiligsmann M, Vokó Z, Pokhrel S, Jones T, et al. Understanding the stakeholders’ intention to use economic decision-support tools: a cross-sectional study with the Tobacco Return on Investment tool. Health Policy. 2016;120(1):46–54.
van Dongen JM, Tompa E, Clune L, Sarnocinska-Hart A, Bongers PM, van Tulder MW, et al. Bridging the gap between the economic evaluation literature and daily practice in occupational health: a qualitative study among decision-makers in the healthcare sector. Implement Sci. 2013;8:57.
Martin DK, Pater JL, Singer PA. Priority-setting decisions for new cancer drugs: a qualitative case study. Lancet. 2001;358(9294):1676–81.
PausJenssen AM, Singer PA, Detsky AS. Ontario’s formulary committee: how recommendations are made. Pharmacoeconomics. 2003;21(4):285–94. https://doi.org/10.2165/00019053-200321040-00006.
Teerawattananon Y, Russell S. A difficult balancing act: plicy actors’ perspectives on using economic evaluation to inform health-care coverage decisions under the Universal Health Insurance Coverage scheme in Thailand. Value Health. 2008;11(Suppl. 1):S52–60. https://doi.org/10.1111/j.1524-4733.2008.00367.x.
Omar F, Tinghög G, Tinghög P, Carlsson P. Attitudes towards priority-setting and rationing in healthcare: an exploratory survey of Swedish medical students. Scand J Publ Health. 2009;37(2):122–30. https://doi.org/10.1177/1403494808100276.
Asim OB, J, Coast J. Towards a greater understanding of priority setting: role of an economic approach? Findings from a workshop to local decision makers. London: Health Economics Study Group; 2006.
Gallego G, Casey R, Norman R, Goodall S. Introduction and uptake of new medical technologies in the Australian health care system: a qualitative study. Health Policy. 2011;102(2–3):152–8. https://doi.org/10.1016/j.healthpol.2011.04.003.
Gallego G, Fowler S, van Gool K. Decision makers’ perceptions of health technology decision making and priority setting at the institutional level. Aust Health Rev. 2008;32(3):520–7.
Gallego G, Van Gool K, Casey R, Maddern G. Surgeon’s veiws of health technology assessment in Australia: online pilot survey. Int J Technol Assess Health Care. 2013;29(3):309–14. https://doi.org/10.1017/S026646231300024X.
Hoffmann C, Stoykova BA, Nixon J, Glanville JM, Misso K, Drummond MF. Do health-care decision makers find economic evaluations useful? The findings of focus group research in UK health authorities. Value Health. 2002;5(2):71–8. https://doi.org/10.1046/j.1524-4733.2002.52109.x.
Claxton K, Ginnelly L, Sculpher M, Philips Z, Palmer S. A pilot study on the use of decision theory and value of information analysis as part of the NHS Health Technology Assessment programme. Health Technol Assess. 2004;8(31):1–103.
Van Harten WH, Retèl VP. Close cooperation with health technology assessment expertise is crucial for implementation and ultimately reimbursement of innovations in oncology. Ecancermedicalscience. 2016;28(10):686.
Anell A. Priority setting for pharmaceuticals: the use of health economic evidence by reimbursement and clinical guidance committees. Eur J Health Econ. 2004;5(1):28–35. https://doi.org/10.1007/s10198-003-0195-0.
Jenkings KN, Barber N. What constitutes evidence in hospital new drug decision making? Soc Sci Med. 2004;58(9):1757–66. https://doi.org/10.1016/S0277-9536(03)00373-3.
McDonald R. Street-level bureaucrats? Heart disease, health economics and policy in a primary care group. Health Social Care Commun. 2002;10(3):129–35.
Kolasa K, Dziomdziora M, Fajutrao L. What aspects of the health technology assessment process recommended by international health technology assessment agencies received the most attention in Poland in 2008? Int J Technol Assess Health Care. 2011;27(1):84–94. https://doi.org/10.1017/S0266462310001236.
Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4(1):50.
Rich RF. Measuring knowledge utilization: processes and outcomes. Knowledge Policy. 1997;10(3):11–24.
Lavis JN, Ross SE, Hurley JE. Examining the role of health services research in public policymaking. Milbank Q. 2002;80(1):125–54.
Chubin D, Maienschein J. Staffing science policy-making. Science. 2000;290(5496):1501.
Coast J. Is economic evaluation in touch with society’s health values? BMJ. 2004;329(7476):1233–6.
Pettigrew AM. Context and action in the transformation of the firm. J Manage Stud. 1987;24(6):649–70.
Banta HD. Dissemination of health technology assessment. In: del Llano-Senaris J, Campillo-Artero CC, editors. Health technology assessment and health policy today: a multifaceted view of their unstable crossroads. Heidelberg: Springer; 2015. p. 147–56.
Elshaug AG, Hiller JE, Tunis SR, Moss JR. Challenges in Australian policy processes for disinvestment from existing, ineffective health care practices. Aust N Z Health Policy. 2007;4(1):23.
Pearson S, Littlejohns P. Reallocating resources: how should the National Institute for Health and Clinical Excellence guide disinvestment efforts in the National Health Service? J Health Serv Res Policy. 2007;12(3):160–5.
Hoffmann C. The influence of economic evaluation studies on decision making: a European survey. Health Policy. 2000;52(3):179–92. https://doi.org/10.1016/S0168-8510(00)00076-2.
Robinson LA, Hammitt JK. Behavioural economics and the conduct of benefit-cost analysis: towards principles and standards. J Benefit Cost Anal. 2011;2(2):1–51.
Harris TI. QALYfying the value of life. J Med Ethics. 1987;13(3):117–23.
Kirk MA, Kelley C, Yankey N, Birken SA, Abadie B, Damschroder L. A systematic review of the use of the consolidated framework for implementation research. Implement Sci. 2016;11(1):72.
Author information
Authors and Affiliations
Contributions
All authors developed the concept and research design. GM conducted the literature review and data extraction and prepared the manuscript. KP and NG reviewed the manuscript and added content.
Corresponding author
Ethics declarations
Funding
This research was funded through a scholarship grant from the Centre of Research Excellence for Reducing Hospital Acquired Infection. The funding agreement ensured the authors’ independence in designing the study, interpreting the data, and writing and publishing the report.
Conflict of Interest
Gregory Merlo, Katie Page, Pauline Zardo and Nicholas Graves have no conflicts of interest that are directly relevant to the content of this article.
Ethics Approval
Not applicable.
Consent for Publication
Not applicable.
Data Availability
The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Merlo, G., Page, K., Zardo, P. et al. Applying an Implementation Framework to the Use of Evidence from Economic Evaluations in Making Healthcare Decisions. Appl Health Econ Health Policy 17, 533–543 (2019). https://doi.org/10.1007/s40258-019-00477-4
Published:
Issue Date:
DOI: https://doi.org/10.1007/s40258-019-00477-4