Evaluating Fidelity and Effectiveness of Interventions

The implementation and dissemination of injury control interventions in community settings often follows the publication of findings from formal research trials. In the absence of strong evidence for an intervention, public health administrators must decide to solve problems by creating new solutions or implementing interventions that appear to be promising and model current public health best practice. In either scenario, program officers need to understand if the decision to implement the intervention led to an important public health effect in their community setting. The evaluation of the effectiveness of an intervention or program is particularly important if there are few previous studies of the program’s effectiveness in real-world settings because the future diffusion of the program to other sites depends on its demonstrated effectiveness. The evaluation of the fidelity of the program implementation process is also an important task for program officials. Fidelity is defined as the accuracy or exactness of the replication of an intervention, based on the prototype model. Program administrators must be careful to monitor the fidelity of the program components as they have been developed in previous research models. Failure to monitor the quality of implementation of the intervention procedures could lead to reduced effectiveness of the program.

The purpose of this chapter is to review approaches toward the evaluation of program effectiveness and fi delity. The intended audience for this chapter is primarily injury control and public health program administrators and their staff. In this chapter, we describe two specifi c scenarios frequently faced by program administrators. The fi rst is the development of an evaluation section of a funding proposal from a state government department of health to disseminate an injury prevention intervention. The second involves planning for a site visit to evaluate an injury prevention program in a state or local health department. Both require extensive preparation and coordination of people and activities. Both require careful attention to program implementation and evaluation strategies. We believe that the execution of these scenarios illustrates some of the practical principles regarding the evaluation of program fi delity and effectiveness.