Abstract
Evidence-based prevention interventions hold great promise for enhancing the well-being of individuals, families, and society. As these interventions are implemented in new contexts and at wider scales, policymakers and private sector organizations are increasingly interested in understanding the economic returns that programs produce through reductions of burden on public service systems, such as criminal justice and human services. Thus, it is important to ensure that economic models account for factors, such as retention, which are important when interventions are implemented in real-world contexts with selective populations and voluntary participation. Yet the field of prevention has provided little guidance to help researchers and policymakers analyze the economics of interventions so that estimates reflect the impact of implementation factors on intervention cost-effectiveness. This paper discusses the role retention plays in the economic efficiency of interventions when the prevention of child maltreatment is the primary motivation for funding these programs. We present a conceptual model to serve as a guide for explicit inclusion of retention rates when calculating cost estimates to be used in cost-effectiveness analysis. A case study is presented, demonstrating the variability in estimates dependent on the definition of retention and the estimated retention rate. The results underscore the importance of improving our understanding of factors underlying and related to retention, such as engagement, which may improve the precision of cost and cost-effectiveness analysis in applied settings.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Barnett, S. W. (2011). Effectiveness of early educational intervention. Science, 333, 975–978. https://doi.org/10.1126/science.1204534.
Boller, K., Daro, D., Grosso, P. D., Colel, R., Paulsell, D., Hart, B., et al. (2014). Making replication work: Building infrastructure to implement, scale-up, and sustain evidence-based early childhood home visiting programs with fidelity. Washington, D.C.: Children’s Bureau, Administration for Children, Families, U.S. Department of Health, and Human Services.
Burwick, A., Zaveri, H., Shang, L., Boller, K., Daro, D., & Strong, D. A. (2014). Costs of early childhood home visiting: An analysis of programs implemented in the supporting evidence-based home visiting to prevent child maltreatment initiative. Princeton: Mathematica Policy Research. Retrieved August 1, 2017, from http://www.chapinhall.org/sites/default/files/documents/Costs%20of%20EC%20-Home%20Visiting.Final%20Report.January%2030%202014.2.pdf.
Cicchetti, D., & Hinshaw, S. P. (2002). Prevention and intervention science: Contributions to developmental theory. Development and Psychopathology, 14, 667–671. https://doi.org/10.1017/S0954579402004017.
Corbacho, B., Bell, K., Stamuli, E., Richardson, G., Ronaldson, S., Hood, K., ... Torgerson, D. (2017). Cost-effectiveness of the family nurse partnership (FNP) programme in England: Evidence from the building blocks trial. Journal of Evaluation in Clinical Practice. https://doi.org/10.1111/jep.12799
Costa, K. (2012). Washington state shows what works: Data-driven analysis of public programs. Washington State Institute for Public Policy. Retrieved July 10, 2017, from https://www.americanprogress.org/issues/general/news/2012/02/10/11147/washington-state-shows-what-works/.
Crowley, D. M., & Jones, D. (2015). Financing prevention: Opportunities for economic analysis across the translational research cycle. Translational Behavioral Medicine, 6, 1–8. https://doi.org/10.1007/s13142-015-0354-8.
Crowley, D. M., Hill, L. G., Kuklinski, M. R., & Jones, D. E. (2014). Research priorities for economic analyses of prevention: Current issues and future directions. Prevention Science, 15, 789–798. https://doi.org/10.1007/s11121-013-0429-z.
Crowley, D. M., Dodge, K. A., Barnett, W. S., Corso, P., Duffy, S., Graham, P., et al. (2018). Standards of evidence for conducting and reporting economic evaluations in prevention science. Prevention Science, 19, 366–390. https://doi.org/10.1007/s11121-017-0858-1.
Drake, E. K., Aos, S., & Miller, M. G. (2009). Evidence-based public policy options to reduce crime and criminal justice costs: Implications in Washington state. Victims and Offenders, 4, 170–196. https://doi.org/10.1080/15564880802612615.
Dube, S., & Pendergrass, K. (2017). In Colorado, the use of data is changing the way government operates. Retrieved May 20, 2017, from http://www.pewtrusts.org/en/research-and-analysis/analysis/2017/05/09/in-colorado-the-use-of-data-is-changing-the-way-government-operates.
Duggan, A., Fuddy, L., Burrell, L., Higman, S. M., McFarlane, E., Windham, A., & Sia, C. (2004). Randomized trial of a statewide home visiting program to prevent child abuse: Impact in reducing parental risk factors. Child Abuse & Neglect, 28, 623–643. https://doi.org/10.1016/j.chiabu.2003.08.008.
Foster, E. M., & Jones, D. (2006). Can a costly intervention be cost-effective? An analysis of violence prevention. Archives of General Psychiatry, 63, 1284–1291. https://doi.org/10.1001/archpsyc.63.11.1284.
Gomby, D. S. (2007). The promise and limitations of home visiting: Implementing effective programs. Child Abuse & Neglect, 31, 793–799. https://doi.org/10.1016/j.chiabu.2007.07.001.
Gorman, D. M. (2017). The decline effect in evaluations of the impact of the strengthening families program for youth 10-14 (SFP 10-14) on adolescent substance use. Children and Youth Services Review, 81, 29–39. https://doi.org/10.1016/j.childyouth.2017.07.009.
Greenblatt, J., & Donovan, A. (2013). The promise of pay for success. Community Development Investment Review, 9, 19–22.
Haddix, A. C., Teutsch, S. M., & Corso, P. S. (2003). Prevention effectiveness: A guide to decision analysis and economic evaluation (2nd ed.). New York: Oxford University Press.
Heckman, J. J., Holland, M. L., Makino, K. K., Pinto, R., & Rosales-Rueda, M. (2017). An analysis of the Memphis nurse-family partnership program. National Bureau of Economic Research.
Heinrichs, N., Bertram, H., Kuschel, A., & Hahlweg, K. (2005). Parent recruitment and retention in a universal prevention program for child behavior and emotional problems: Barriers to research and program participation. Prevention Science, 6, 275–286. https://doi.org/10.1007/s11121-005-0006-1.
Henggeler, S. W., Schoenwald, S. K., Borduin, C. M., & Swenson, C. C. (2006). Methodological critique and meta-analysis as Trojan horse. Children and Youth Services Review, 28, 447–457. https://doi.org/10.1016/j.childyouth.2005.07.001.
Hoffmann, C., Stoykova, B. A., Nixon, J., Glanville, J. M., Misso, K., & Drummond, M. F. (2002). Do health-care decision makers find economic evaluations useful? The findings of focus group research in UK health authorities. Value in Health, 5, 71–78. https://doi.org/10.1046/j.1524-4733.2002.52109.x.
Holland, M. L., Christensen, J. J., Shone, L. P., Kearney, M. H., & Kitzman, H. J. (2014). Women’s reasons for attrition from a nurse home visiting program. Journal of Obstetric, Gynecologic, & Neonatal Nursing, 43, 61–70. https://doi.org/10.1111/1552-6909.12263.
Holland, M. L., Olds, D. L., Dozier, A. M., & Kitzman, H. J. (2018). Visit attendance patterns in nurse-family partnership community sites. Prevention Science, 1–12. https://doi.org/10.1007/s11121-017-0829-6.
Ingoldsby, E. M. (2010). Review of interventions to improve family engagement and retention in parent and child mental health programs. Journal of Child and Family Studies, 19, 629–645. https://doi.org/10.1007/s10826-009-9350-2.
Ingoldsby, E. M., Baca, P., McClatchey, M. W., Luckey, D. W., Ramsey, M. O., Loch, J. M., . . . Smith, B. J. (2013). Quasi-experimental pilot study of intervention to increase participant retention and completed home visits in the nurse–family partnership. Prevention Science, 14, 525–534. https://doi.org/10.1007/s11121-013-0410-x
Kessler, R., & Glasgow, R. E. (2011). A proposal to speed translation of healthcare research into practice: Dramatic change is needed. American Journal of Preventive Medicine, 40, 637–644. https://doi.org/10.1016/j.amepre.2011.02.023.
Lee, S., Aos, S., Drake, E., Pennucci, A., Miller, M., & Anderson, L. (2012). Return on investment: Evidence-based options to improve statewide outcomes. Olympia: Washington State Institute for Public Policy.
Littell, J. H. (2006). The case for multisystemic therapy: Evidence or orthodoxy? Children and Youth Services Review, 28, 458–472. https://doi.org/10.1016/j.childyouth.2005.07.002.
Lyons-Ruth, K., & Melnick, S. (2004). Dose–response effect of mother–infant clinical home visiting on aggressive behavior problems in kindergarten. Journal of the American Academy of Child & Adolescent Psychiatry, 43, 699–707. https://doi.org/10.1097/01.chi.0000122730.72597.07.
McKay, K. (2013). Evaluating social impact bonds as a new reentry financing mechanism: A case study on reentry programming in Maryland. Annapolis: Department of Legislative Services.
Michalopoulos, C., Lee, H., Duggan, A., Lundquist, E., Tso, A., Crowne, S. S., . . . Knox, V. (2015). The mother and infant home visiting program evaluation: Early findings on the maternal, infant, and early childhood home visiting program. A Report to Congress. (OPRE Report No. 2015–11.). Administration for Children & Families.
Miller, T. R., & Hendrie, D. (2015). Nurse family partnership: Comparing costs per family in randomized trials versus scale-up. The Journal of Primary Prevention, 36, 419–425. https://doi.org/10.1007/s10935-015-0406-3.
O’Brien, R. A., Moritz, P., Luckey, D. W., McClatchey, M. W., Ingoldsby, E. M., & Olds, D. L. (2012). Mixed methods analysis of participant attrition in the nurse-family partnership. Prevention Science, 13, 219–228. https://doi.org/10.1007/s11121-012-0287-0.
Olds, D., Donelan-McCall, N., O’Brien, R., MacMillan, H., Jack, S., Jenkins, T., . . . Thorland, B. (2013). Improving the nurse–family partnership in community practice. Pediatrics, 132, S110–S117. https://doi.org/10.1542/peds.2013-1021l
Peterson, C., Florence, C., Thomas, R., & Klevens, J. (2017). Cost-benefit analysis of two child abuse and neglect primary prevention programs for US states. Prevention Science, 1–11. https://doi.org/10.1007/s11121-017-0819-8.
Results First (2014). Achieving success with the Pew-MacArthur Results First initiative. Retrieved August 1, 2017, from: http://www.pewtrusts.org/en/research-and-analysis/issue-briefs/2014/02/05/achieving-success-with-the-pewmacarthur-results-first-initiative.
Spoth, R. L., & Greenberg, M. T. (2011). Impact challenges in community science-with-practice: Lessons from prosper on transformative practitioner-scientist partnerships and prevention infrastructure development. American Journal of Community Psychology, 48, 106–119. https://doi.org/10.1007/s10464-010-9417-7.
Spoth, R. L., Rohrbach, L. A., Greenberg, M. T., Leaf, P., Brown, C., Fagan, A., . . . Hawkins, J. (2013). Addressing core challenges for the next generation of type 2 translation research and systems: The translation science to population impact (TSci impact) framework. Prevention Science, 14, 319–351. https://doi.org/10.1007/s11121-012-0362-6.
Supplee, L. H., Parekh, J., & Johnson, M. (2018). Principles of precision prevention science for improving recruitment and retention of participants. Prevention Science, 19, 689–694. https://doi.org/10.1007/s11121-018-0884-7.
Washington State Institute for Public Policy. (2017). Benefit-cost technical documentation. Olympia, WA.
Welsh, B. C., Sullivan, C. J., & Olds, D. L. (2010). When early crime prevention goes to scale: A new look at the evidence. Prevention Science, 11, 115–125. https://doi.org/10.1007/s11121-009-0159-4.
Wilson, P., Rush, R., Hussey, S., Puckering, C., Sim, F., Allely, C. S., . . . Gillberg, C. (2012). How evidence-based is an ‘evidence-based parenting program’? A PRISMA systematic review and meta-analysis of Triple P. BMC Medicine, 10, 130. https://doi.org/10.1186/1741-7015-10-130.
Woolf, S. H. (2008). The meaning of translational research and why it matters. JAMA, 299, 211–213. https://doi.org/10.1001/jama.2007.26.
Yates, B. T. (2018). Commentary on “standards of evidence for conducting and reporting economic evaluations in prevention science”. Prevention Science, 19, 396–401. https://doi.org/10.1007/s11121-018-0885-6.
Zhou, F., Santoli, J., Messonnier, M. L., Yusuf, H. R., Shefer, A., Chu, S. Y., . . . Harpaz, R. (2005). Economic evaluation of the 7-vaccine routine childhood immunization schedule in the United States, 2001. Archives of Pediatrics & Adolescent Medicine, 159, 1136–1144. https://doi.org/10.1001/archpedi.159.12.1136
Funding
Funding for this research was provided by the Office of Early Childhood, Colorado Department of Human Services.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of Interest
The authors declare that they have no conflict of interest.
Research Involving Human Participants and/or Animals
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.
Informed Consent
The Colorado State University IRB ruled informed consent was not required given the present study’s exclusive use of secondary de-identified administrative data.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Timpe, Z., Winokur, M. Integrating Retention Rates into Economic Analyses of Prevention Interventions. Prev Sci 20, 566–574 (2019). https://doi.org/10.1007/s11121-018-0958-6
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11121-018-0958-6