From Causes to Outcomes: Determining Prevention Can Work

Abstract

To be successful in practice with families and to gain support for further dissemination, maltreatment prevention interventions must accomplish two goals of program design and evaluation: (1) develop and implement a logic model informed by theory that targets known causes of child maltreatment, and (2) demonstrate evidence of effectiveness with rigorous methodological designs that isolate program effects. The current chapter focuses on the role of two types of causal inquiry for research in child maltreatment prevention. The chapter begins with a discussion of how theories on maltreatment etiology inform logic models for existing prevention programs (“causes of known effects”). We then move to a summary of exciting statistical methods that bring us closer to inferring causality in observational studies (“effects of known causes”). The chapter ends with reflections on methodological advances for the future of maltreatment prevention, with a discussion on how to continue to move the field forward.

Reflection: Nature of Evidence and How We Use It

• Lisbeth Schorr 

1. Center for the Study of Social Policy, Washington, DC,, USA

   Lisbeth Schorr

Enormous changes have occurred in the last 50 years in how we define evidence and apply it to public policy. My own direct experience with how these changes have played out began when I joined the War on Poverty at its inception. Dozens of initiatives were inaugurated based on high hopes, innovative ideas, and evidence of varying strengths. Efforts to be more systematic focused on recruiting several of Robert McNamara’s “whiz kids” from the Pentagon to calculate cost-effectiveness ratios. (Their only unambiguous contribution in the early stages was finding that the cost-effectiveness of a proposed family planning program came in at 17 to 1, allowing the Office of Economic Opportunity to establish the first federal line item funding for contraceptive services. Reflecting the climate of the time, these would be provided only to married women over 21!)

The importance of establishing clear, quantifiable evidence of what works has grown throughout my career. But the current narrow view that randomized control trials and similar experiments are the only credible information to inform decisions about what is worth funding has overshot the mark and undermined many efforts to improve outcomes. You cannot unravel the “why” and “how” if all your knowledge comes from studies that hold an intervention constant and isolated, as though it were operating in a laboratory setting. This emphasis on proving that a defined model “works” has obscured how different contexts—the institutional system, the funding streams, and the participants and their community—influence and shape an intervention’s potential to be effective. Insisting that “proven” models be implemented with fidelity discourages local agencies and funders from reaching new populations, addressing emerging issues, and acting on lessons learned.

To make significant progress, we must determine the essential elements of our efforts and then carefully document, on a day to day basis, how our actions impact our ability to achieve outcomes. We need to “steer as we go” and accept the fact that there is no straight line between cause and effect—we need to tolerate some messiness. Failure to be comfortable with “messy” results pushes us away from the type of complex interventions that are needed to successfully reduce child maltreatment and confront other complex problems, like race-, class- and income-based gaps in well-being and achievement.

The key to improvement is not solely a function of creating better program models. Learning about how and why and how well an intervention works, and explicitly taking account of the importance of context, is a different type of learning. This approach takes us beyond yes/no judgments of a defined model and allows us to rank programs not in terms of the elegance with which they have been evaluated, but rather by our understanding of the strategy’s potential to improve defined outcomes. It would allow us to make judgments beyond individual programs to identify strategies, including the interactions among multiple programs and reforms of systems and policies, that could achieve transformative outcomes.

We have come a long way since the War on Poverty. We no longer rely on justifications or anecdotes that the program worked for someone, somewhere as a basis for allocating resources. Gathering, analyzing, and applying a broader range of evidence is particularly important when it comes to initiatives aimed at prevention, where the need to understand context is critical. Building a deeper and wider knowledge base will require the ability to understand the critical elements of diverse interventions that focus on similar outcomes. At present, there are not many vehicles that allow program developers or funders to do that type of cross-strategy learning. In order to develop a broader, deeper understanding of how we might improve outcomes, we need learning communities and learning networks that can build on one another’s experience and learn together from research to build a knowledge base and evidence pool sufficiently sturdy to lead to meaningful improvements at scale in the outcomes we most care about.