The Funding, Administrative, and Policy Influences on the Evaluation of Primary Prevention Programs in Australia
- 137 Downloads
Evaluation of primary prevention and health promotion programs contributes necessary information to the evidence base for prevention programs. There is increasing demand for high-quality evaluation of program impact and effectiveness for use in public health decision making. Despite the demand for evidence and known benefits, evaluation of prevention programs can be challenging and organizations face barriers to conducting rigorous evaluation. Evaluation capacity building efforts are gaining attention in the prevention field; however, there is limited knowledge about how components of the health promotion and primary prevention system (e.g., funding, administrative arrangements, and the policy environment) may facilitate or hinder this work. We sought to identify the important influences on evaluation practice within the Australian primary prevention and health promotion system. We conducted in-depth semi-structured interviews with experienced practitioners and managers (n = 40) from government and non-government organizations, and used thematic analysis to identify the main factors that impact on prevention program evaluation. Firstly, accountability and reporting requirements impacted on evaluation, especially if expectations were poorly aligned between the funding body and prevention organization. Secondly, the funding and political context was found to directly and indirectly affect the resources available and evaluation approach. Finally, it was found that participants made use of various strategies to modify the prevention system for more favorable conditions for evaluation. We highlight the opportunities to address barriers to evaluation in the prevention system, and argue that through targeted investment, there is potential for widespread gain through improved evaluation capacity.
KeywordsEvaluation capacity Primary prevention Health promotion Evidence-based public health
We would like to thank Emily McCluskey for her valuable contribution to participant recruitment and data analysis in this study.
This work was supported by the Australian Research Council (ARC) [DP150103575]. Prof. Gabbe was supported by ARC Future Fellowship [FT170100048] during the preparation of this manuscript.
Compliance with Ethical Standards
Conflict of Interest
The authors declare that they have no conflict of interest.
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.
Informed consent was obtained from all individual participants included in the study.
- Baugh Littlejohns, L., Baum, F., Lawless, A., & Freeman, T. (2018). Disappearing health system building blocks in the health promotion policy context in South Australia (2003–2013). Critical Public Health, 1–13. https://doi.org/10.1080/09581596.2017.1418501.
- Dahler-Larsen. (2009). Evaluation and public management. In E. Ferlie, L. E. Lynn, & C. Pollitt (Eds.), The Oxford handbook of public management. Oxford: Oxford University Press.Google Scholar
- Huckel Schneider, C., Milat, A. J., & Moore, G. (2016). Barriers and facilitators to evaluation of health policies and programs: Policymaker and researcher perspectives. Evaluation and Program Planning, 58, 208–215. https://doi.org/10.1016/j.evalprogplan.2016.06.011.CrossRefPubMedGoogle Scholar
- Jackson, H., & Shiell, A. (2017). Preventive health: How much does Australia spend and is it enough? Canberra: Foundation for Alcohol Research and Education.Google Scholar
- Lobo, R., Petrich, M., & Burns, S. K. (2014). Supporting health promotion practitioners to undertake evaluation for program development. BMC Public Health, 14. https://doi.org/10.1186/1471-2458-14-1315.
- Lobo, R., Crawford, G., Hallett, J., Laing, S., Mak, D. B., Jancey, J., …, Doherty, M. (2016). A research and evaluation capacity building model in Western Australia. Health Promotion International https://doi.org/10.1093/heapro/daw088.
- Norton, S., Milat, A., Edwards, B., & Giffin, M. (2016). Narrative review of strategies by organizations for building evaluation capacity. Evaluation and Program Planning, 58, 1–19. https://doi.org/10.1016/j.evalprogplan.2016.04.004.CrossRefPubMedGoogle Scholar
- Oxman, A. D., Bjørndal, A., Becerra-Posada, F., Gibson, M., Block, M. A. G., Haines, A., … Wibulpolprasert, S. (2010). A framework for mandatory impact evaluation to ensure well informed public policy decisions. The Lancet, 375, 427–431. https://doi.org/10.1016/S0140-6736(09)61251-4.
- Pettman, T. L., Armstrong, R., Doyle, J., Burford, B., Anderson, L. M., Hillgrove, T., … Waters, E. (2012). Strengthening evaluation to capture the breadth of public health practice: Ideal vs. real. Journal of Public Health (Oxford, England), 34, 151–155. https://doi.org/10.1093/pubmed/fds014.
- Rychetnik, L., Bauman, A., Laws, R., King, L., Rissel, C., Nutbeam, D., … Caterson, I. (2012). Translating research for evidence-based public health: Key concepts and future directions. Journal of Epidemiology and Community Health, 66, 1187–1192. https://doi.org/10.1136/jech-2011-200038.
- Schwarzman, J., Bauman, A., Gabbe, B., Rissel, C., Shilton, T., & Smith, B. J. (2018). Organizational determinants of evaluation practice in Australian prevention agencies. Health Education Research, Published online ahead of print, 1–13. https://doi.org/10.1093/her/cyy015.
- Spoth, R., Rohrbach, L., Greenberg, M., Leaf, P., Brown, C., Fagan, A., … Hawkins, J. (2013). Addressing Core challenges for the next generation of type 2 translation research and systems: the translation science to population impact (TSci impact) framework. Prevention Science, 14, 319–351. https://doi.org/10.1007/s11121-012-0362-6.
- van Koperen, T. M., Renders, C. M., Spierings, E. J. M., Hendriks, A.-M., Westerman, M. J., Seidell, J. C., & Schuit, A. J. (2016). Recommendations and improvements for the evaluation of integrated community-wide interventions approaches. Journal of Obesity, 2016, 2385698–2385698. https://doi.org/10.1155/2016/2385698.CrossRefPubMedPubMedCentralGoogle Scholar
- Wutzke, S., Morrice, E., Benton, M., Milat, A., Russell, L., & Wilson, A. (2017). Australia's National Partnership Agreement on preventive health: Critical reflections from states and territories. Health Promotion Journal of Australia. https://doi.org/10.1002/hpja.9.