Advertisement

Developing Evaluation Questions: Beyond the Technical Issues

  • Ligia de Salazar
  • Mary Hall

This chapter is a contribution to the evaluation debate in health promotion. It explores the strategies for, and main concerns in, identifying and formulating relevant and answerable evaluation questions. Unlike many previous texts discussing what is involved in developing evaluation questions, in this chapter we argue that a wide range of contextual and highly political factors contribute to the framing of evaluation questions. Furthermore, we strongly believe that the same intervention can lend itself to a variety of different evaluation questions depending upon the stage of intervention development at which the evaluation is conducted. Finally, we contend that the evaluation question or set of questions should reflect the ever-changing context of the intervention, as well as the stage if the intervention, if evaluation is to be truly useful for health promotion.

The Cochrane Collaboration defined the relationship of health promotion and public health in this way: “health promotion...

Keywords

Health Promotion Effectiveness Evaluation Health Promotion Intervention Evaluation Question Holistic Thinking 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References

  1. Abma, T. A. (2005). Responsive evaluation: Its meaning and special contribution to health promotion. Evaluation and Program Planning, 28, 279–289.CrossRefGoogle Scholar
  2. Battista, R. N., Lance, J. M., Lehoux, P., Régnier, G. (1999), “Health Technology Assessment and the Regulation of Medical Devices and Procedures in Quebec”, in International Journal of Technology Assessment in Health Care, vol. 15, pp: 593–601.Google Scholar
  3. Benjamin, K., Perfetto, E., &; Greene, R. (1995). Public policy and the application of outcomes assessments: Paradigms vs politics [Suppl.]. Medical Care, 33(4), AS299–AS306.PubMedGoogle Scholar
  4. Brewer, G., & de Leon, P. (1983). Foundations of policy analysis. Homewood, IL: Dorsey.Google Scholar
  5. Brint, S. (1990). Rethinking the policy influence of experts: From general characterizations to analysis of variation. Sociological Forum, 5(3), 361–385.CrossRefGoogle Scholar
  6. Carvalho, A., Bodstein, R. C., Hartz, Z. & Matida, A. (2004). Concepts and approaches in the evaluation of health promotion. en Ciência e Saúde Coletiva, 9(3), 521–529.Google Scholar
  7. de Salazar, L. (2002). Municipios y Comunidades Saludables. El reto de la Evaluación. Centro para el Desarrollo y Evaluación de Políticas y Tecnología en Salud Pública, Colombia. Cedetes. Universidad del Valle.Google Scholar
  8. de Salazar, L. (2004). Efectividad en Promoción de la Salud. Guía de Evaluación Rápida. Capítulo 8: La Sistematización de Experiencias en Promoción de la Salud. CEDETES – Universidad del Valle. Cali, Colombia.Google Scholar
  9. de Salazar, L., Díaz, C., & Magaña, A. (2002). Municipios y Comunidades Saludables. El reto de la Evaluación. Cali, colombia, Cedetes, Universidad del Valle.Google Scholar
  10. Diez-Roux, A. V. (1998). Bringing context back into epidemiology: variables and fallacies in multilevel analysis. American Journal of Public Health, 88(2), 216–222.CrossRefPubMedGoogle Scholar
  11. Earl, S., Carden, F., & Smutylo, T. (2001). Outcome mapping: building learning and reflection into development programs, International Development Research Centre.Google Scholar
  12. EPPI – Centre (2006). “Cochrane Health Promotion & Public Health Field” [en línea], disponible en: http://eppi.ioe.ac.uk/cms/Default.aspx?tabid=269, recuperado: 4 de febrero de 2008.
  13. Feldstein, P. (1996). The Politics of Health Legislation. An Economic Perspective. Chic: Health Administration Press.Google Scholar
  14. Francke, M., & Morgan, M. (1995). La sistematización: Apuesta por la generación de conocimientos a partir de las experiencias de promoción. Lima, Materiales Didácticos No 1. Escuela para el desarrollo.Google Scholar
  15. Chilean Government, Programa Orígenes (2004). Estudio sistematización participativa de experiencias de salud intercultural en las comunidades Mapuche y establecimientos de salud existentes en las comunas focalizadas por el programa desarrollo integral de comunidades indígenas.Google Scholar
  16. Guba, E. G., & Lincoln, Y. S. (1981). Effective evaluation. Beverly Hills:Sage.Google Scholar
  17. Guba, E. G., & Lincoln, Y. S. (1989). Fourth generation evaluation. Beverly Hills: Sage.Google Scholar
  18. Hawe, P. Degeling, D., Hall, J. & Brierley, A. (2003). Evaluating health promotion: A health worker‘s guide. Sydney, Australia: Maclennan and Petty Ltd.Google Scholar
  19. Jara, O. (2000). Tres posibilidades de sistematización: comprensión, aprendizaje y teorización, Sistematización de experiencias, Aportes. Bogotá, Colombia Dimensión Educativa.Google Scholar
  20. Jasanoff, S. (1993). The fifth branch: Science advisors as policymakers. Cambridge: Harvard University Press.Google Scholar
  21. Kemm, J. (2006, June). The limitations of ‘evidence-based’ public health. Journal of Evaluation in Clinical Practice, 12, 319.CrossRefPubMedGoogle Scholar
  22. Lincoln, Y. S., & Guba, E. G. (1985). Naturalistic inquiry. Beverly Hills: Sage.Google Scholar
  23. Loomis, D., & Wing S. (1990). Is molecular epidemiology a germ theory for the end of the twentieth century? International Journal of Epidemiology, 19, l–3.CrossRefGoogle Scholar
  24. MacDonald, G., Veen, C., & Tones, K. (1996). Evidence for success in health Promotion: Suggestions for improvements. Health Education Research. Theory and Practice, 11(3), 367–376.Google Scholar
  25. McQueen, D. V. & Anderson, L. (2000). What counts as evidence? Issues and debates on evidence, relevance to the evaluation of community health promotion programs.Google Scholar
  26. McQueen, D. V. & Jones, C. (2007). Global Perspective on Health Promotion Effectiveness. An introduction. In D. V. McQueen & C. Jones (Eds.), Global Perspective on Health Promotion Effectiveness (pp. 201–224). New York: Springer.Google Scholar
  27. Madjar, I. & Walton, J. A. (2001), What is problematic about evidence? In J. M. Morse, J. M. Swanson, and A. J. Kuzel (Eds.), The nature of evidence in qualitative research. Thousand Oaks, CA: Sage.Google Scholar
  28. Marmot, M. (2004). Evidence-based policy or policy-based evidence? British Medical Journal, 328,906–907CrossRefPubMedGoogle Scholar
  29. Milio, N. (1990), Nutrition policy for food-rich countries: A strategic analysis. Baltimore: Johns Hopkins University Press.Google Scholar
  30. Organisation for Economic Co-operation and Development (OECD) (1998). Review of the DAC Principles for Evaluation Development Assistance, [en línea], disponible en: http://www.oecd.org/dac/evaluation, recuperado: 5 de enero de 2004.
  31. Pan American Health Organization (PAHO). (2007). Guide to Economic Evaluation in Health Promotion, Washington, PAHO.Google Scholar
  32. Potvin, L., Haddad, S., & Frohlich, K. L. (2001). Beyond process and outcome evaluation: A comprehensive approach for evaluating health promotion programmes, in In I. Rootman, et al. (Eds.), Evaluation in health promotion. Principles and perspectives (pp. 45–62). Copenhague: WHO regional publications. European series; No 92.Google Scholar
  33. Ray, L. D. & Mayan, M. (2001). Who decides what counts as evidence? In J. M. Morse, J. M. Swanson, & A. J. Kuzel (Eds.), The nature of evidence in qualitative research (pp. 50–73). Thousand Oaks: Sage.Google Scholar
  34. Shadish, W. R., Cook, T. D. & Leviton, L. C. (1995). Foundations of Program Evaluation. Theories of Practice, newbury Park. London & New Delhi: Sage.Google Scholar
  35. Smutylo, T. (2001). Impacto latente, atribución oculta: Cómo superar las amenazas al aprendizaje en los programas de desarrollo, Unidad de Evaluación, Centro Internacional de Investigaciones para el Desarrollo, IDRC, Ottawa, Canadá.Google Scholar
  36. Stake, R. E. (1975). To evaluate an arts program. In R. E. Stake (Ed.), Evaluating the arts in education: A responsive approach, Colombus Ohio, Merrill, 13–31.Google Scholar
  37. Stake, R. E., & Abma, T. A. (2005). Responsive evaluation. In S. Mathison (Ed.), Encyclopaedia of evaluation (pp. 376–379). Thousand Oaks: Sage.Google Scholar
  38. Stivers, C. (1991) The politics of public health: The dilemma of a public profession. In T. Litman & S Robins, (Eds.), Health politics and policy (pp. 356–369). Albany, NY: Delmar Pub.Google Scholar
  39. Waters, W., Doyle, J., Jackson, N., Howes, F., Brunton, G. & Oakle, A. (2006, April). Evaluating the effectiveness of public health interventions: the role and activities of the Cochrane collaboration. Journal of Epidemiology and Community Health, 60, 285–289.CrossRefPubMedGoogle Scholar
  40. Weiss, C. H. (1998). Have we learned anything new about the use of evaluation? American Journal of Evaluation, 19(1), 21–33.Google Scholar
  41. World Health Organization (WHO). (2005, October 10–12). Bridging the “Know–Do” Gap Meeting on Knowledge Translation in Global Health. Geneva, Switzerland: WHO.Google Scholar
  42. World Health Organization (WHO). (2001). Evaluation in health promotion. Principles and perspectives. I. Rootman et al. (Ed.), WHO Regional Publications, European Series, No 92.Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2008

Authors and Affiliations

  • Ligia de Salazar
    • 1
  • Mary Hall
    • 1
  1. 1.Universidad del ValleCaliColombia

Personalised recommendations