Program Evaluation: Why Process Matters

Abstract

In peace programming in humanitarian contexts, evaluations are essential for learning what worked or did not work and for strengthening the practice of peacebuilding. Despite their importance, evaluations are seldom conducted. Worse yet, even when evaluations have been conducted, they may not be peaceful and may actively undermine or limit the impact of peacebuilding programs. The objectives of this chapter are to raise awareness of how evaluations are frequently not peaceful, increase understanding of why non-peaceful evaluations are prevalent, and provide principles and exemplars that support the use of peaceful approaches to evaluation. In examining how evaluations are frequently not peaceful, the first part of the chapter shows how evaluations are typically designed and implemented with technical considerations such as validity, reliability, and robustness in mind. Although these considerations are important, an exclusive focus on them often leads to evaluations that are driven by outsiders, take an extractive approach, discriminate against people who had not participated in the program, and marginalize the categories that guide the thinking of local people about peace and peacebuilding. When this happens, the evaluation process itself can cause unintended harm and actively set back the cause of peace in the local area. The evaluation process evokes fear, assumes colonial dimensions, and becomes an imposition that leaves local people feeling objectified, marginalized, and exploited. In this manner, the evaluation process becomes part of a system of social injustice that is antithetical to peace. The second part of the chapter will explore why non-peaceful evaluations are so widespread. An understanding of the “why” is critical to the efforts to prevent unintended harm and change the institutions, policies, and practices that enable non-peaceful evaluations. Emphasizing the structure of the humanitarian system and the power asymmetries it embodies, this section will examine the power gap that exists between communities and implementing agencies and also between donors and implementing agencies, the donor driven nature of many interventions, the pressures for quick results and for the implementing agency demonstrating positive results, the culture of technical experts, and the use of prespecified questions, surveys, and scales that have been selected primarily on the basis of scientific merit or the desire to impact policy leaders. The third part of the chapter will examine how to make evaluations more peaceful through processes of relationship building, power sharing, deep community engagement and ownership, giving voice to the voiceless , giving constructive feedback, and inclusivity at stages of evaluation design, implementation, analysis, and use of the data and findings. It shows how the use of qualitative data in a mixed methods approach can simultaneously strengthen the technical quality of the evaluation and the process aspects of the evaluation. Two practical exemplars of peaceful approaches to program evaluation will be used to illustrate key points, provide practical tips on how to make evaluations more peaceful, and encourage readers to use peaceful evaluation processes.