Implementation Quality and Positive Experiences in After-School Programs

  • Amanda Brown Cross
  • Denise C. Gottfredson
  • Denise M. Wilson
  • Melissa Rorie
  • Nadine Connell
Original Paper

Abstract

Data collected during an evaluation of a multi-site trial of an enhanced after-school program were used to relate quality of program implementation to student experiences after school. The enhanced after-school program incorporated a drug use and violence prevention component that was shown to be effective in previous research. Building on Durlak and Dupre’s (Am J Community Psychol 41:327–350, 2008) dimensions of implementation, we assessed the level of dosage, quality of management and climate, participant responsiveness, and staffing quality achieved at the five program sites. We evaluated how these characteristics co-varied with self-reported positive experiences after-school. The study illustrates how multiple dimensions of program implementation can be measured, and shows that some but not all dimensions of implementation are related to the quality of student after-school experiences. Measures of quality of management and climate, participant responsiveness, and staffing stability were most clearly associated with youth experiences. The importance of measuring multiple dimensions of program implementation in intervention research is discussed.

Keywords

After-school programs Implementation Evidence based practice 

References

  1. Armstrong, T., & Armstrong, G. (2004). The organizational, community and programmatic characteristics that predict the effective implementation of after-school programs. Journal of School Violence, 3, 93–109.CrossRefGoogle Scholar
  2. Beckett, M., Hawken, A., & Jacknowitz, A. (2001). Accountability for after-school care: Devising standards and measuring adherence to them. Santa Monica, CA: Rand Corporation.Google Scholar
  3. Dane, A. V., & Schneider, B. H. (1998). Program integrity in primary and early secondary prevention: Are implementation effects out of control? Clinical Psychology Review, 18, 23–45.CrossRefPubMedGoogle Scholar
  4. Durlak, J. A., & DuPre, E. P. (2008). Implementation matters: A review of research on the influence of implementation on program outcomes and the factors affecting implementation. American Journal of Community Psychology, 41, 327–350.CrossRefPubMedGoogle Scholar
  5. Durlak, J. A., & Weissberg, R. P. (2007). The impact of after-school programs that promote personal and social skills. Chicago, IL: Collaborative for Academic, Social, and Emotional Learning. Retrieved February 12, 2007, from www.casel.org.
  6. Dusenbury, L., Brannigan, R., Falco, M., & Hansen, W. B. (2003). A review of research on fidelity of implementation: Implications for drug abuse prevention in school settings. Health Education Research, 18, 237–256.CrossRefPubMedGoogle Scholar
  7. Dynarski, M., James-Burdumy, S., Moore, M., Rosenberg, L., Deke, J., & Mansfield, W. (2003). When schools stay open late: The national evaluation of the 21st century community learning centers program: New findings: US Department of Education, National Center for Education Evaluation and Regional Assistance. Washington, DC: US Government Printing Office.Google Scholar
  8. Fashola, O. (1998). Review of extended day and after school programs and their effectiveness. Retrieved January, 2008, from www.csos.jhu.edu/crespar/techReports/Report24.pdf.
  9. Finance Project. (2007). Estimated federal investment in out-of-school time. Retrieved April, 2008, from www.financeproject.org/publications/estimatedfederalOST_TFPflyer.pdf.
  10. Gerstenblith, S. A., Soulé, D. A., Gottfredson, D. C., Lu, S., Kellstrom, M. A., Womer, S. C., et al. (2005). ASPs, antisocial behavior, and positive youth development: An exploration of the relationship between program implementation and changes in youth development. In J. L. Mahoney, R. W. Larson, & J. S. Eccles (Eds.), Organized activities as contexts of development: Extracurricular activities, after-school and community programs (pp. 457–478). Mahwah, NH: Lawrence Erlbaum & Associates.Google Scholar
  11. Gottfredson, D. C., Cross, A. B., & Soulé, D. A. (2007). Distinguishing characteristics of effective and ineffective after-school programs to prevention delinquency and victimization. Criminology and Public Policy, 6, 601–631.Google Scholar
  12. Gottfredson, D. C., Cross, A. B., Wilson, D. M., Rorie, M., & Connell, N. (in press). Effects of participation in after-school programs for middle school students: A randomized trial. Journal of Research on Educational Effectiveness.Google Scholar
  13. Gottfredson, D. C., Gerstenblith, S. A., Soulé, D. A., Womer, S. C., & Lu, S. (2004). Do ASPs reduce delinquency? Prevention Science, 5, 253–266.CrossRefPubMedGoogle Scholar
  14. Gottfredson, D. C., & Gottfredson, G. D. (2002). Quality of school-based prevention programs: Results from a national survey. Journal of Research in Crime and Delinquency, 39, 3–35.CrossRefGoogle Scholar
  15. Gottfredson, D. C., Gottfredson, G. D., & Weisman, S. A. (2001). The timing of delinquent behavior and its implications for ASPs. Criminology & Public Policy, 1(1), 61–80.CrossRefGoogle Scholar
  16. Granger, R. C. (2008). After-school programs and academics: Implications for policy, practice, and research. Social Policy Report, 22, 3–19.Google Scholar
  17. Granger, R. C., Durlak, J. A., Yohalem, N., & Reisner, E. (2007). Improving after-school program quality. The William T. Grant foundation working paper. Retrieved May, 2008, from www.wtgrantfoundation.org/usr_doc/Improving_After-School_Program_Quality.pdf.
  18. Grossman, J., Campbell, M., & Raley, B. (2007). Quality time after School: What instructors can do to enhance learning. Philadelphia: Public/Private Ventures.Google Scholar
  19. Hansen, W. B., & Dusenbury, L. (2004). All Stars Plus: A competence and motivation enhancement approach to prevention. Health Education, 104, 371–381.CrossRefGoogle Scholar
  20. Hansen, W. B., & Larson, R. (2005). The youth experiences survey 2.0: Instrument revisions and validity testing. Urbana-Champaign, IL: University of Illinois.Google Scholar
  21. Harrington, N. G., Giles, S. M., Hoyle, R. H., Feeney, G. J., & Youngbluth, S. C. (2001). Evaluation of the All Stars™ character education and problem behavior prevention program: Effects on mediator and outcome variables for middle school students. Health Education Research, 28, 533–546.Google Scholar
  22. Kane, T. J. (2004). The impact of after school programs: Interpreting the results of four recent evaluations. The William T. Grant foundation working paper. Retrieved May, 2008, from http://www.wtgrantfoundation.org/usr_doc/After-school_paper.pdf.
  23. Lauer, P. A., Akiba, M., Wilkerson, S. B., Apthorp, H. S., Snow, D., & Martin-Glenn, M. L. (2006). Out-of-school time programs: A meta-analysis of effects for at-risk students. Review of Educational Research, 76, 275–313.CrossRefGoogle Scholar
  24. Mahoney, J. L. (2000). School extracurricular activity participation as a moderator in the development of antisocial patterns. Child Development, 71, 502–516.CrossRefPubMedGoogle Scholar
  25. McNeal, R. B., Hansen, W. B., Harrington, N. G., & Giles, S. M. (2004). How All Stars™ works: An examination of program effects on mediating variables. Health Education and Behavior, 31, 165–178.CrossRefPubMedGoogle Scholar
  26. Pierce, K. M., Hamm, J. V., & Vandell, D. L. (1999). Experiences in after-school programs and children’s adjustment in first grade classrooms. Child Development, 70, 756–767.CrossRefPubMedGoogle Scholar
  27. Rosenthal, R., & Vandell, D. L. (1996). Quality of care at school-aged child-care programs: Regulatable features, observed experiences, child perspectives, and parent perspectives. Child Development, 67, 2434–2445.CrossRefPubMedGoogle Scholar
  28. Sickmund, M., Snyder, H. N., & Poe-Yamagata, E. (1997). Juvenile offenders and victims: 1997 update on violence. Washington, DC: Office of Juvenile Justice and Delinquency Prevention.Google Scholar
  29. Simpkins-Chaput, S., Little, P. M. D., & Weiss, H. B. (2004). Understanding and measuring attendance in out-of-school time programs. Issues and opportunities in out-of-school time evaluation brief no. 7. Cambridge, MA: Harvard Family Research Project.Google Scholar
  30. Weisman, S. A., Womer, S. C., Kellstrom, M. A., Bryner, S. L., Kahler, A., Slocum, L., et al. (2002). Maryland after school community grant program, report on the 20012002 school year evaluation of the phase 3 after school programs. Technical report.Google Scholar
  31. Weiss, H. B., Little, P. M., & Bouffard, S. M. (2005). More than just being there: Balancing the participation equation. New Directions for Youth Development, 2005, 15–31.CrossRefGoogle Scholar
  32. Whitebook, M., Howes, C., & Phillips, D. (1998). Worthy work, unlivable wages: The national child care staffing study, 1988–1997. Washington, DC: National Center for the Early Childhood Work Force.Google Scholar

Copyright information

© Society for Community Research and Action 2010

Authors and Affiliations

  • Amanda Brown Cross
    • 1
  • Denise C. Gottfredson
    • 1
  • Denise M. Wilson
    • 1
  • Melissa Rorie
    • 1
  • Nadine Connell
    • 2
  1. 1.Department of Criminology and Criminal JusticeUniversity of MarylandCollege ParkUSA
  2. 2.Rowan UniversityGlassboroUSA

Personalised recommendations