Abstract
This paper examines the relation between the implementation quality of after-school literacy activities and student reading gains. The data are from an evaluation of a multi-site after-school program in California in which continuous program quality improvement strategies were implemented to improve the delivery of a new balanced literacy program. Strategies included: (1) targeted staff training throughout the year, (2) regular observations and coaching of staff, and (3) the use of data to measure progress. Programs struggled to successfully implement these strategies early in the initiative, but gradually improved the quality and consistency of their use. Program quality, as measured through observations, also increased. Results suggested that the size of student reading gains were positively correlated with the quality of literacy programming provided by each instructor.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
After School Program Excellence & Accountability Bill. California Senate Bill 638, 2006.
Arbreton, A. J. A., Goldsmith, J., & Sheldon, J. (2005). Launching literacy in after-school programs: Early lessons from the CORAL initiative. Philadelphia, PA: Public/Private Ventures.
Arbreton, A. J. A., Sheldon, J., Bradshaw, M., Goldsmith, J., Jucovy, L., & Pepper, S. (2008). Advancing achievement: Findings from an independent evaluation of a major after-school initiative. Philadelphia, PA: Public/Private Ventures.
Blau, D. M. (1997). The production of quality in child care centers. Journal of Human Resources, 32, 354–387.
Britsch, B., Martin, N., Stuczynski, A., Tomala, B., & Tucci, P. (2005). Literacy in afterschool programs: Literature review. Portland, OR: Northwest Regional Educational Laboratory.
Buher-Kane, J., Peter, N., Olitsky, S., & Kinnevy, S. (2006). Findings from five out-of-school time focus groups: Professional development preferences, experiences and recommendations for future planning. Journal of Youth Development, 1(2), Article 0602RS003. Retrieved November 14, 2007, from http://www.nae4ha.org/directory/jyd/jyd_article.aspx?id=856c67b2-3a76-428c-95c6-390edae5d1c2.
Clarke-Stewart, K. A., Vandell, D. L., Burchinal, M., O’Brien, M., & McCartney, K. (2002). Do regulable features of child-care homes affect children’s development? Early Childhood Research Quarterly, 17, 52–86.
Cummins, J. (2003). Reading and the bilingual student: Fact and friction. In Gt. Garcia (Ed.), English learners: Reaching the highest level of English literacy. Newark, DE: International Reading Association.
Dennehy, J., & Noam, G. G. (2005). Evidence for action: Strengthening after-school programs for all children and youth: The Massachusetts out-of-school time workforce. A research report of Achieve Boston, an initiative of Boston After School & Beyond. Retrieved November 11, 2007, from http://www.niost.org/Evidence%20for%20%20Action.pdf
DuBois, D. L., Neville, H. A., Parra, G. R., & Pugh-Lilly, A. O. (2002). Testing a new model of mentoring. New Directions for Youth Development, 93, 21–57.
Dynarski, M., Moore, M., Mullens, J., Gleason, P., James-Burdumy, S., Rosenberg, L., et al. (2003). When schools stay open late: The national evaluation of the 21st century learning centers program, first year findings. Jessup, MD: Education Publications Center, US Department of Education.
Eccles, J., & Gootman, J. A. (Eds.). (2002). Community programs to promote youth development. Board on Children, Youth, and Families, Division of Behavioral and Social Sciences and Education. Washington, DC: The National Academic Press.
Gee, J. P. (1999). Critical issues: Reading and the new literacy studies: Reframing the National Academy of Sciences report on reading. Journal of Literacy Research, 31, 355–374.
Grossman, J., Campbell, M., & Raley, B. (2007). Quality time after school: What instructors can do to enhance learning. Philadelphia: Public/Private Ventures.
Halpern, R. (2005). Confronting the big lie: The need to reframe expectations of afterschool programs. New York: Partnership for After School Education.
Hebbeler, K., Briggs, D., Contreras, G., Garza, N., Montgomery, J., Neufeld, S., et al. (2003). Implementation of communities organizing resources to advance learning (CORAL) in 2002–2003. Palo Alto, CA: SRI International.
James-Burdumy, S., Dynarski, M., Moore, M., Dekke, J., Mansfield, W., Pistorino, C., et al. (2005). When schools stay open late: The national evaluation of the 21st century community learning centers program final report. Washington, DC: US Department of Education/Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance (NCEE).
Mahoney, J. L., Lord, H., & Carryl, E. (2005). An ecological analysis of after-school program participation and the development of academic performance and motivational attributes for disadvantaged children. Child Development, 76, 811–825.
Mahoney, J. L., Parente, M. E., & Lord, H. (2007). Program-level differences in afterschool program engagement: Links to child competence, program quality and content. The Elementary School Journal, 107, 385–404.
Miller, B. M. (2005). Pathways to success for youth: What counts in after-school. Massachusetts after-school research study (MARS) report. Wellesley, MA: National Institute for Out-of-School Time.
Miller, B. M., Brigham, R., & Perea, F. (2006). Afterschool literacy coaching initiative of Boston: Final evaluation report. Boston: Massachusetts 2020.
National After School Association. (2006). Understanding the afterschool workforce: Opportunities and challenges for an emerging profession. Houston, TX: Cornerstones for Kids.
Neufeld, B., & Roper, D. (2003). Coaching: A strategy for developing instructional capacity. Providence, RI: Annenberg Institute for School Reform.
No Child Left Behind Act (NCLB). (2007). http://www.whitehouse.gov/news/reports/no-child-letf-behind.html#1. Retrieved August 26, 2007.
Poglinco, S. M., Bach, A., Hovde, K., Rosenblum, S., Saunders, M., & Supovitz, J. (2003). The heart of the matter: The coaching model in America’s choice schools. Philadelphia, PA: Consortium for Policy Research in Education, University of Pennsylvania, Graduate School of Education.
Reisner, E. R., White, R. N., Russell, C. A., & Birmingham, J. (2004). Building quality, scale and effectiveness in after-school programs. Washington, DC: Policy Studies Associates, Inc.
Russo, A. (2004). School-based coaching: A revolution in professional development—Or just the latest fad? Harvard Education Letter, 20, 1–4.
Ryan, L. M., Foster, M. L., & Cohen, J. (2002). Enhancing literacy support in after-school programs. Boston: Boston Plan for Excellence. Retrieved December 7, 2007, from http://www.pearweb.org/research/pdfs/5%20-%20enhancing.pdf.
Spielberger, J., & Halpern, R. (2002). The role of after-school programs in children’s literacy development. Chicago: University of Chicago, Chapin Hall Center for Children.
Vandell, D., Reisner, E., & Pierce, K. (2007). Outcomes linked to high-quality afterschool programs: Longitudinal findings from the study of promising afterschool programs. Washington, DC: Policy Studies Associates.
Acknowledgments
The authors would like to thank their colleagues who collaborated on the evaluation: Molly Bradshaw, Julie Goldsmith, Nora Gutierrez, and Gary Walker. The study also would not have been possible without the contributions of CORAL staff members, students, and parents. A grant from the James Irvine Foundation supported the writing of this piece as well as the longitudinal evaluation on which it is based. That evaluation led to a series of publications that address a variety of topics including the process of adding academics to an after-school program, academic outcomes for study participants, the needs of English language learners in after-school programs, and tools for after-school practitioners. These reports can be downloaded at the Public/Private Ventures’ Web site, http://www.ppv.org.
Author information
Authors and Affiliations
Corresponding author
Appendix
Appendix
Activity Observation Quality Indicators
The observation tool used to measure activity quality involved recording a running narrative of the instruction provided to participants. Researchers then indicated which of the six balanced literacy strategies occurred during the lesson and assigned scores of 1 through 5 for each of the strategies observed, based on the quantity and quality of implementation of key elements in each area. Listed below are examples of the quality indicators that researchers looked for staff to exhibit during each observation. Arbreton et al. (2005) includes a more extensive review of observation methodology.
-
Read aloud indicators. Use an introductory activity to get youth excited about the text. Define important words while reading. Occasionally ask open-ended questions.
-
Book discussion indicators. Select a discussion topic related to the read aloud text. Encourage youth to listen to and respond to peers’ comments. Provide opportunities for youth to make personal connections to the text.
-
Writing indicators. Model the writing activity for youth. Provide adequate writing supplies. Allow at least a few students share their work.
-
Independent reading indicators. Ensure materials are at appropriate (independent reading) levels. Move around the room to assist all youth. Provide 15 min or more for uninterrupted independent reading.
-
Skill development activity indicators. Introduce new activities or games by clearly explaining instructions. If applicable, place students in groups of manageable sizes. Make activities colorful and fun.
-
Vocabulary indicators. Instruct youth to listen for new words during read aloud or other activities. Record new words in journals or a prominent space in room. Use new vocabulary in games, writing, or other parts of lesson.
Composite Literacy Quality Scores
Ratings in each of these six categories were aggregated from observations of each class across the year in order to categorize the overall quality of the literacy instruction experienced in each class. Five levels of composite literacy quality were specified, with the score “1” representing poor implementation of the model and the score “5” representing strong implementation. Research suggests that read alouds and independent reading are the foundation of a successful program (Ryan et al. 2002), thus, their presence was weighted more heavily than the other strategies. Both of these strategies had to be rated as consistently present in order for that observed group to receive a composite score higher than 1. The composite literacy quality score for that class also took account of the frequency of a strategy’s use by counting it as present only if it was observed during at least half of the observations of a given group over the course of the year. The composite literacy quality scores were defined as follows:
-
Composite Score 1. Read alouds and independent reading were implemented at low quality or during fewer than half of the observations of a group.
-
Composite Score 2. Read alouds and independent reading were implemented at a quality rating of at least three during at least half of the observations of a group.
-
Composite Score 3. Read alouds, independent reading, and one other literacy strategy (writing, book discussions, vocabulary, or skill development) were implemented at a quality rating of at least three during at least half of the observations of a group.
-
Composite Score 4. Read alouds and independent reading were implemented at a rating of at least four, and all of the remaining strategies at a rating of at least three, during at least half of the observations of a group.
-
Composite Score 5. All six strategies were implemented at a rating of four or five during at least half of the observations of a group.
Rights and permissions
About this article
Cite this article
Sheldon, J., Arbreton, A., Hopkins, L. et al. Investing in Success: Key Strategies for Building Quality in After-School Programs. Am J Community Psychol 45, 394–404 (2010). https://doi.org/10.1007/s10464-010-9296-y
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10464-010-9296-y