Advertisement

Evaluating Participatory Health Research

  • John G. Oetzel
  • Jane Springett
  • Nina Wallerstein
  • Laura Parajon
  • Irene Sia
  • Mark Wieland
  • Abigail Reese
  • Rangimahora Reddy
Chapter

Abstract

This chapter explores some of the challenges in evaluating PHR. Firstly, it examines the reasons for evaluating participatory health research (PHR), placing that need in the context of different paradigms of public health research and the ongoing debates about methodology and different models of evaluation. Four key issues/challenges are identified. Secondly, this chapter introduces the community-based participatory research (CBPR) conceptual model as a framework that helps to address these key issues. Thirdly, this chapter includes several cases to illustrate ways of PHR evaluation: from New Zealand, the USA (including communities of immigrants/refugees), and Nicaragua. Finally, this chapter ends with a discussion on ways in which PHR researchers can proactively evaluate their research, so lessons can be learnt and case studies can be developed to showcase alternative ways of documenting process, impact, and outcomes. This discussion highlights how the conceptual model addresses some of the key issues surrounding the evaluation of PHR.

Keywords

Process evaluation Outcome evaluation Community-based participatory research conceptual model 

Notes

Acknowledgements

The following authors are associated with the cases: (a) New Zealand, John Oetzel and Rangimahora Reddy; (b) Rochester, Mark Wieland, Irene Sia, and Abigail Reese; and (c) Nicaragua, Laura Parajon. We also thank our many community partners and colleagues with whom we have collaborated and learnt from over the years. The CBPR model was originally developed and tested with US National Institutes of Health NARCH funding (U261HS300293; U261IHS0036-04-00) and revised under “Engage for Equity” National Institute of Nursing Research funding (1R01NR015241-01A1).

References

  1. Bell, S., Morse, S., & Shah, R. A. (2012). Understanding stakeholder participation in research as part of sustainable development. Journal of Environmental Management, 101, 13–22.CrossRefPubMedCentralGoogle Scholar
  2. Bowen, S. J., & Graham, I. D. (2013). From knowledge translation to engaged scholarship: Promoting research relevance and utilization. Archives of Physical Medicine and Rehabilitation, 94, S3–S8.CrossRefPubMedCentralGoogle Scholar
  3. Bradbury-Huang, H. (2014). Quality in action research. In M. Brydon-Miller, D. Coghlan, & P. Gaya (Eds.), Encyclopedia of action research. London: Sage.Google Scholar
  4. Cahill, C. (2007). The personal is political: Developing new subjectivities through participatory action research. Gend Place Culture, 14, 267–292.CrossRefGoogle Scholar
  5. Campilan, D., & Prain, G. (2000). Self-assessment as an approach to evaluating participatory research: An Asian experience. In N. Lilja & L. Sperling (Eds.), Assessing the impact of participatory research and gender analysis (pp. p172–p182). Cali: CGIAR SWP-PRGA.Google Scholar
  6. Cargo, M., & Mercer, S. L. (2008). The value and challenges of participatory research: Strengthening its practice. Annual Review of Public Health, 29, 325–350.CrossRefPubMedCentralGoogle Scholar
  7. Center for Participatory Research A (n.d.). CBPR project: Research for improvement health 2009–2013. Available via https://cpr.unm.edu/research-projects/cbpr-project/research-for-improved-health.html. Accessed: 2 Nov 2017.
  8. Center for Participatory Research B (n.d.). CBPR model visioning guide for planning and evaluation. Available via https://cpr.unm.edu/research-projects/cbpr-project/index.html. Accessed: 2 Nov 2017.
  9. Center for Participatory Research C (n.d.). Empowerment curriculum. Available via http://cpr.unm.edu/curricula--classes/empowerment-curriculum.html. Accessed: 2 Nov 2017.
  10. Coimbra, C. E. A., Jr. (1999). Scientific production in public health and international bibliographic bases. Caderno Saúde Pública, 5, 883–888.CrossRefGoogle Scholar
  11. Chambers, R. (1998). Foreword. In J. Holland and J. Blackburn (eds.). Whose voice? Participatory research and policy change. London. Immediate Technology Publications.Google Scholar
  12. de Sousa Santos, B. (2014). Epistemologies of the south: Justice against epistemicide. Boulder: Paradigm Publishers.Google Scholar
  13. Daigneault, P.-M., & Jacob, S. (2009). Toward accurate measurement of participation rethinking the conceptualization and operationalization of participatory evaluation. American Journal of Evaluation, 30, 330–348.CrossRefGoogle Scholar
  14. Devia, C., Baker, E., Sanchez-Youngman, S., et al. (2017). CBPR to advance social and racial equity: Urban and rural partnerships in black and Latino communities. International Journal for Equity in Health, 16, 17.CrossRefPubMedCentralGoogle Scholar
  15. Duran, B., Oetzel, J.G., Pearson, C., et al. (under review). Promising practices in community-based participatory research: Evidence from a national study.Google Scholar
  16. Green, L. W., George, M. A., Daniel, M., et al. (1995). Study of participatory research in health promotion: Review and recommendations for development of participatory research in health promotion in Canada. Ottawa: The Royal Society of Canada.Google Scholar
  17. Greene, J. C. (1994). Qualitative program evaluation practices and promise. In N. K. Denzin & Y. S. Lincoln (Eds.), Handbook of qualitative research (pp. 530–544). London: Sage Publications., London.Google Scholar
  18. Guthrie, S., Wamae, W., Diepeveen, S., et al. (2013). Measuring research: A guide to research evaluation frameworks and tools. Europe: RAND.Google Scholar
  19. Hall, B., Tandon, R., & Tremblay, C. (2015). Strengthening community university research partnerships: Global perspectives. Victoria: University of Victoria.Google Scholar
  20. Hicks, S., Duran, B., Wallerstein, N., et al. (2012). Evaluating community-based participatory research (CBPR) to improve community-partnered science and community health. Progress in Community Health Partnerships, 6, 289–299.CrossRefPubMedCentralGoogle Scholar
  21. Home, R., & Rump, N. (2015). Evaluation of a multi-case participatory action research project: The case of SOLINSA. The Journal of Agricultural Education and Extension, 21, 73–89.CrossRefGoogle Scholar
  22. House, E. R. (2015). Evaluating: Values, biases, and practical wisdom. Charlotte: IAP.Google Scholar
  23. Hummelbrunner, R. (2010). Beyond logframe: Critique, variations and alternatives. In N. Fujita (Ed.), Beyond logframe: Using systems concepts in evaluation. Tokyo: Foundation for Advanced Studies on International Development.Google Scholar
  24. International Collaboration for Participatory Health Research (ICPHR). (2013). Position paper 1: What is participatory Health Research? Berlin: International Collaboration for Participatory Health Research.Google Scholar
  25. Israel, B. A., Schulz, A. J., Parker, E. A., et al. (2018). Critical issues in developing and following CBPR principles. In N. Wallerstein, B. Duran, J. G. Oetzel, & M. Minkler (Eds.), Community-based participatory research for health (3rd ed., pp. 31–44). San Francisco: Jossey-Bass.Google Scholar
  26. Johansson, A. W., & Lindhult, E. (2008). Emancipation or workability?: Critical versus pragmatic scientific orientation in action research. Action Research, 6, 95–115.CrossRefGoogle Scholar
  27. Kastelic, S., Wallerstein, N., Duran, B., et al. (2018). Socio-ecologic framework for CBPR: Development and testing of a model. In N. Wallerstein, B. Duran, J. G. Oetzel, & M. Minkler (Eds.), Community-based participatory research for health (3rd ed., pp. 77–94). San Francisco: Jossey-Bass.Google Scholar
  28. Kennedy, V., & Cram, F. (2010). Ethics of researching with Whānau collectives. MAI Review, 2010(3), 1–8.Google Scholar
  29. Khodyakov, D., Stockdale, S., Jones, F., et al. (2011). An exploration of the effect of community engagement in research on perceived outcomes of partnered mental health services projects. Social Mental Health, 1, 185–199.CrossRefGoogle Scholar
  30. Lucero, J., Wallerstein, N., Duran, B., Alegria, M., Greene-Moton, E., Israel, B., Kastelic, S., Magarati, M., Oetzel, J., Pearson, C., Schulz, A., Villegas, M., & White Hat, E. (2018). Development of a mixed methods investigation of process and outcomes of community-based participatory research. Journal of Mixed Methods Research, 12, 55–74.CrossRefPubMedCentralGoogle Scholar
  31. Mantoura, P., & Potvin, L. (2013). A realist–constructionist perspective on participatory research in health promotion. Health Promotion International, 28, 61–72.CrossRefPubMedCentralGoogle Scholar
  32. Oetzel, J. G., Villegas, M., Zenone, H., et al. (2015a). Enhancing stewardship of community-engaged research through governance. American Journal of Public Health, 105, 1161–1167.CrossRefPubMedCentralGoogle Scholar
  33. Oetzel, J. G., Zhou, C., Duran, B., et al. (2015b). Establishing the psychometric properties of constructs in a community-based participatory research conceptual model. American Journal of Health Promotion, 29, e188–e202.CrossRefPubMedCentralGoogle Scholar
  34. Oetzel, J. G., Duran, B., Sussman, A., et al. (2018). Evaluation of CBPR partnerships and outcomes: Lessons and tools from the Research for Improved Health study. In N. Wallerstein, B. Duran, J. G. Oetzel, & M. Minkler (Eds.), Community-based participatory research for health (3rd ed., pp. 237–250). San Francisco: Jossey-Bass.Google Scholar
  35. Oetzel, J.G., Wallerstein, N., Duran, B., et al. (in press). Community-engaged research for health: A test of the CBPR conceptual model. BioMed Research International.Google Scholar
  36. Orange, C. (1987). The story of a treaty. Wellington: Allen & Unwin.Google Scholar
  37. Oetzel, J. G., Wallerstein, N., Duran, B., Villegas, M., Sanchez-Youngman, S., Nguyen, T., Woo, K., Wang, J., Schulz, A., Kaholokula, Israel, B., & Alegria, M. (2018). Community-engaged research for health: A test of the CBPR conceptual model. BioMed Research International. 2018, Article ID: 7281405Google Scholar
  38. Pearson, C. R., Duran, B., Oetzel, J., et al. (2015). Research for improved health: Variability and impact of structural characteristics in federally- funded community engaged research. Progress in Community Health Partnerships, 9, 17–29.CrossRefPubMedCentralGoogle Scholar
  39. Pinto, A. C., & de Andrade, J. B. (1999). Factor of impact in scientific journals: What is the meaning of this parameter? Química Nova, 22, 448–453.CrossRefGoogle Scholar
  40. Renger, R., Wood, S., Williamson, S., et al. (2011). Systemic evaluation, impact, evaluation and logic models. Evaluation Journal of Australasia, 11, 24–30.CrossRefGoogle Scholar
  41. Rochester Healthy Community Partnership (n.d.). Our research model. Available via https://rochesterhealthy.org/our-research-model; Accessed on 2 Nov 2017.
  42. Sanchez-Youngman, S., & Wallerstein, N. (2018). Appendix 7: Partnership river of life: Creating an historical timeline. In N. Wallerstein, B. Duran, J. G. Oetzel, & M. Minkler (Eds.), Community-based participatory research for health (3rd ed., pp. 375–378). San Francisco: Jossey-Bass.Google Scholar
  43. Schulz, A., Israel, B. A., & Lantz, P. (2003). Instrument for evaluating dimensions of group dynamic within community-based participatory research partnerships. Evaluation Program Planning, 26, 249–262.CrossRefGoogle Scholar
  44. Shoveller, J., Viehbeck, S., Di Ruggiero, E., et al. (2016). A critical examination of representations of context within research on population health interventions. Critical Public Health, 26, 487–500.CrossRefGoogle Scholar
  45. Simpson, M., & Ake, T. (2010). Whitiwhiti korero: Exploring the researchers’ relationship in cross-cultural research. Journal of Intercultural Communication Research, 39, 185–205.CrossRefGoogle Scholar
  46. Springett, J., Wright, M. T., & Roche, B. (2011). Developing quality criteria for participatory health research. An agenda for action, WZB Discussion Paper. Berlin: Wissenschaftszentrum Berlin für Sozialforschung.Google Scholar
  47. Staley, K. (2009). Exploring impact: Public involvement in NHS, public health and social care research. Eastleigh: INVOLVE.Google Scholar
  48. Taylor, C. (2010). What would Jim Grant say now? Lancet, 375, 1236–1237.CrossRefPubMedCentralGoogle Scholar
  49. Trickett, E. J. (2011). Community-based participatory research as worldview or instrumental strategy: Is it lost in translation(al) research? American Journal of Public Health, 101, 1353–1355.CrossRefPubMedCentralGoogle Scholar
  50. Turato, E. R. (2005). Qualitative and quantitative methods in the health care field: Definitions, differences and the objects of research. Revista Saúde Pública, 39, 507–514.CrossRefGoogle Scholar
  51. Wallerstein, N., & Duran, B. (2006). Using community based participatory research to address health disparities. Health Promotion Practice, 7, 312–323.CrossRefPubMedCentralGoogle Scholar
  52. Wallerstein, N., & Duran, B. (2010). Community-based participatory research contributions to intervention research: The intersection of science and practice to improve health equity. American Journal of Public Health, 100, S40–S46.CrossRefPubMedCentralGoogle Scholar
  53. Wallerstein, N., Oetzel, J., Duran, B., et al. (2008). CBPR: What predicts outcomes? In M. Minkler & N. Wallerstein (Eds.), Community-based participatory research for health: From process to outcomes (2nd ed., pp. 371–392). San Francisco: Jossey-Bass.Google Scholar
  54. Wallerstein, N., Oetzel, J. G., Duran, B., et al. (under review). Cultural-centeredness in community-based participatory research: Culture as power and agency.Google Scholar
  55. Whitelaw, S. (2012). The emergence of a ‘dose–response’ analogy in the health improvement domain of public health: A critical review. Crit Public Health, 22, 427–440.CrossRefGoogle Scholar
  56. Wiek, A., Talwar, S., O’Shea, M., & Robinson, J. (2014). Toward a methodological scheme for capturing societal effects of participatory sustainability research. Research Evaluation, 23, 117–132.CrossRefGoogle Scholar
  57. Wieland, M. L., Weis, J. A., Yawn, B. P., et al. (2012). Perceptions of tuberculosis among immigrants and refugees at an adult education center: A community-based participatory research approach. Journal of Immigrant and Minority Health/Center for Minority Public Health, 14, 14–22.CrossRefGoogle Scholar
  58. Zarcadoolas, C., Pleasant, A., & Greer, D. S. (2005). Understanding health literacy: An expanded model. Health Promotion International, 20, 195–203.CrossRefPubMedCentralGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • John G. Oetzel
    • 1
  • Jane Springett
    • 2
  • Nina Wallerstein
    • 3
  • Laura Parajon
    • 4
  • Irene Sia
    • 5
  • Mark Wieland
    • 5
  • Abigail Reese
    • 3
  • Rangimahora Reddy
    • 6
  1. 1.University of WaikatoHamiltonNew Zealand
  2. 2.University of AlbertaEdmontonCanada
  3. 3.University of New MexicoAlbuquerqueUSA
  4. 4.AMOS Health and Hope, Managua, Nicaragua and PotomacPotomacUSA
  5. 5.Mayo Clinic College of MedicineRochesterUSA
  6. 6.Rauawaawa Kaumātua Charitable TrustHamiltonNew Zealand

Personalised recommendations