From Field Experiments to Program Implementation: Assessing the Potential Outcomes of an Experimental Intervention Program for Unemployed Persons

Abstract

In field experiments evaluating preventive interventions, nonparticipation of persons who decline to participate at the outset, or later, by not showing up to the experimental treatment, constitutes a serious threat to the validity of the findings. However, in social intervention programs nonparticipation is a commonplace reality. Since nonparticipation is a pervasive reality in virtually every type of social intervention program, field experiments that are used to test the intervention for effectiveness should be designed and evaluated with this reality in mind.

Demonstrated a procedure suggested by Bloom (1984) to provide estimates for the effects of an intervention on its actual participants compared to global effects on study participants in the intervention group, whether or not they showed up. Analyses were based on data collected in a field experiment that tested a preventive intervention for unemployed persons (Caplan, Vinokur, Price, & van Ryn, 1989). Effect size estimates were two to three times larger for the actual participant group than for the entire experimental group on employment outcomes (e.g., earnings) and mental health (anxiety and depression). Further analyses produced results showing that compared to participants, the nonparticipants achieved significantly higher levels of reemployment at posttests and did not differ significantly from participants on all other outcomes. The results suggest that persons who most needed the intervention and benefited from it were drawn into it through self-selection processes.