Similar to other organized activities such as school-based extracurricular activities or community-based youth organizations, after-school programs (ASPs) operate outside of the school day and are characterized by structure and opportunities to build competencies (Mahoney et al. 2005a). Different from some other organized activities, ASPs tend to serve school-age children between the ages of 5 and 18, provide service on most afternoons during the school week, and offer a curriculum that often includes nutrition, some form of academic assistance (e.g., help with homework), and/or various enrichment activities such as physical recreation, arts, music, and opportunities to develop leadership and other types of personal and social skills.

Why a special issue devoted to after ASPs? We feel there are several reasons to devote fresh attention to ASPs. First, ASPs have become a part of community life for many children and families. Although exact figures are not available, depending on what type of programs and activities are counted, surveys suggest that as many as 8.4 million school-age youth in the United States participate in ASPs to some degree (http://www.afterschoolalliance.org).

Second, with many parents working well into the after-school hours, a significant function of ASPs is the provision of a safe and adult-supervised environment for children whose parents are working during the afternoon. This supervision allows parents to work without worrying about their children’s well-being during the hours following school dismissal. Third, the federal government offers nearly $1 billion in annual funding to ASPs through the US Department of Education (2009) and ASPs are also financially supported by local and state governments and by private foundations (Mahoney et al. 2009a). Fourth, and most important from a community perspective, is the realization that ASPs can be an excellent opportunity to foster the growth and development of young people. For example, engaging youth in positive activities with their peers while providing them adult support and guidance can be a vehicle for developing or enhancing skills and resources in various personal, social, cultural, artistic, civic, or athletic arenas (e.g., Mahoney et al. 2010). In other words, ASPs are part of the ecological set of influences that can promote young people’s development and well-being. Despite the importance and emphasis on after-school programming, basic empirical questions remain unanswered. Thus, the fifth and final reason for this special issue is to provide a perspective on the current status of research on ASPs and to offer a research agenda for the future.

This special issue is divided into three main sections. In this first section, our introductory piece presents a historical overview of ASP programming, followed by a synopsis of ASP outcome research, and then briefly describes a developmental ecological model illustrating the potentially important factors affecting program outcomes. We also indicate how the various articles in this special issue relate to different aspects of the model in an attempt to understand ASP findings. Then we offer a few recommendations designed to improve future program evaluations in general.

The second and major section of this special issue contains twelve invited articles that reflect many different themes and topics. For instance, several papers focus on evaluating ASP outcomes related to academic as well as social well-being. Others are devoted to systems-level issues including capacity building at the community level, and analysis of organizational and systemic factors that affect ASP outcomes such as staff-youth interactions, program structure, and youth engagement in activities. There is also considerable breadth in terms of methodology and analytic strategies. For example, there is a meta-analysis, two narrative reviews, a qualitative study, a mixed method design employing both quantitative and qualitative procedures, a case study, and a project emphasizing participatory research strategies. The invited papers also attend to diversity issues: there is one paper describing two separate projects involving Latino youth, a historically underserved group by ASPs, and another paper focuses on African-American youth and their parents. ASPs in rural, urban and suburban settings are included as are youth from different SES levels and school-aged populations (elementary, junior high and high school).

The third and final section of this special issue contains two commentaries, one prepared by Granger (2010) and the other by Hirsch et al. (2010) who present their reactions to the included papers plus their insights regarding additional research, practice and policy issues in the field. This special issue is not meant to be a comprehensive summary of all research to date. In fact, there has been an exponential increase in after-school research in the past 10 years, and we were only able to accommodate a small number of those who initially sent in proposals for possible articles. Rather, our hope is that readers will find that the contents of this issue provide them with a useful introduction to several of the most prominent and timely research, practice, and policy issues related to after-school programming.

Historical and Current Context

Before discussing ASP research findings it is helpful to sketch the history of ASPs in the United States and discuss the current political and social climate affecting their operation. ASPs primarily arose from historical changes in the labor force and formal schooling beginning in the late nineteenth century (Halpern 2002). Compulsory education laws and the declining need for American children to participate in the industrialized labor force expanded the amount of leisure time children had outside of school (Kleiber and Powell 2005). “Boy’s clubs,” essentially drop-in after-school centers, arose in response to this increase in discretionary time. The turn of the century, however, brought with it the idea that more structured play activities would be beneficial for children’s development (i.e., Lee 1915). ASPs were subsequently created with mission statements and purposes beyond those of basic child care that included providing support to working families and enhancing children’s social and academic adjustment (Halpern 2002).

The need and desire for ASPs continued to grow in the twentieth century fueled by the rise in maternal employment. By 1955, 38% of mothers with children ages 6–17 were employed, with current estimates reaching 78% (US Department of Labor 2005). This increase in dual-earner families, and later in single-parent families, led to a child supervision gap between the end of the typical workday and the traditional school day. Simultaneously, the child development study movement was growing (White 2000), and concerns over the safety and development of unsupervised children (i.e., “latchkey children”) increased (Halpern 2002; Kleiber and Powell 2005; Mahoney and Parente 2009).

These concerns were augmented by changes in neighborhood contexts in the early twentieth century, including the growth in urban areas and tenement housing (Halpern 2002). Neighborhoods were beginning to be seen as unsafe areas for children and the desire for safe after-school contexts increased. Changes in neighborhood organization and increases in crime, again coupled with the expansion of the child study movement, also increased attention to children who were unsupervised in the after-school hours. Research linking unsupervised care to poor developmental outcomes (e.g., Long and Long 1983; Pettit et al. 1997; Mahoney and Parente 2009) augmented the desire for safe after-school care arrangements.

After-school program participation has seen an enormous growth in the last quarter century and one impetus for the rise in participation is increased political support (Mahoney et al. 2009a). In 1990, the Child Care Development and Block Grant, now called the Child Care Development Fund, offered federal dollars to low-income households to subsidize child care expenses, including after-school care. In 1994, the 21st Century Community Learning Centers (21CCLCs) funding became available to out-of-school programs, including ASPs. The 21CCLC budget has remained around $1 billion since 2002.

What Are Current Programs Like?

There is no standard format or operating procedure for an ASP. Current programs vary considerably in location, size, staffing, funding, hours of operation, activities and structure, and, most important, in their general mission and specific goals. For example, programs may be based on school grounds or various community agencies, serve less than ten or several hundred youth, may focus on youth of all ages or those in elementary, middle or high school, and receive a combination of funding from the federal government and local resources. A typical program is open several weekdays during the normal school year for several hours after school ends. A few also have weekend or summertime hours. Most programs offer several activities that may begin with some type of academic assistance (see below), coupled with different types of personal, social or cultural activities consistent with each program’s specific goals. Many ASPs have multiple goals. For example, some programs focus on academic/cognitive development and personal and social development, or the latter coupled with broadening young people’s cultural or artistic pursuits. Finally, some programs offer services for parents that might include English-language or parenting classes.

Synopsis of Outcome Research

Several promising findings have appeared with respect to the positive impact of ASPs. Compared to those who do not attend ASPs, program participants in several studies have shown significant changes in their levels of academic achievement, attachment to school, problem behaviors, physical health, and various aspects of their social and emotional development. Among the many positive outcomes that have been reported are improved reading and math achievement scores, better grades, greater liking for school, higher rates of homework completion, lower levels of behavioral problems and drug use, decreased levels of body mass index and obesity, increased self-confidence, and gains in social competence, leadership skills, and civic engagement (e.g., Durlak and Weissberg 2007; Lauer et al. 2006; Harvard Family Research Project 2008; Mahoney et al. 2009b, 2010; National Research Council and Institute of Medicine 2002; Riggs and Greenberg 2004; Vandell et al. 2005).

These results are best viewed as only promising because three primary factors complicate the interpretation of current outcome data. First, research findings have been inconsistent. Whereas some programs have yielded positive outcomes, others have not, and the reasons for this inconsistency are not always clear. It is difficult to reconcile findings because of the variability that exists among programs, their participants, and the experimental rigor of outcome research. Second, youth not only spend their time in ASPs but also in other out of school time activities, and may change their level of participation in each of these activities over time. Thus, it is difficult to isolate the specific impact of ASPs. Third, researchers are still in the process of identifying the multiple factors that can affect outcomes. In several cases, a few suggestive studies have appeared indicating which variables might be important, but replications are needed in order to reach more definitive conclusions.

A Model for Understanding the Effects of ASPs

Figure 1 depicts a developmental ecological model that offers a useful perspective for understanding the potential impact of ASPs and that is consistent with other ecological viewpoints (e.g., Bronfenbrenner and Morris 2006; Mahoney et al. 2009b, 2010; Riggs and Greenberg 2004; Vandell and Posner 1999). This model provides a framework for further highlighting areas of needed research. The model contains five main sections that include individual characteristics, social contexts and their ecologies, specific features and processes related to different ASPs, the levels and types of youth participation in programs, and, finally, the types of short and long-term outcomes that may emanate from program participation.

Fig. 1
figure 1

A developmental ecological model illustrating some potentially important factors affecting how after-school programs influence youth outcomes

This model can be helpful in conceptualizing developmental impacts of ASPs by illustrating the dynamic interplay that may occur among persons, program features, and other contexts over time. Therefore, it is important to keep in mind that although there can be a straightforward direct connection between ASPs and youth development, indirect links are also possible and perhaps quite likely. For example, families and peer groups may not only influence youth development directly, but also affect attendance patterns and participation rates that, in turn, will influence program outcomes. In addition, it should be noted that the model allows for recursive cycles of influence such that, over time, the components of the model can feedback on one other. For instance, ASPs impacts on youth development have implications for a young person’s subsequent participation in such programs and can also affect the program context itself. Finally, this more general model can help to guide the development of detailed logic models needed to answer process-oriented questions such as why academic or social development outcomes of ASPs might depend on participants’ entering characteristics or their school and family context, or how staff behaviors influence a youths’ continued participation.

In addition to the four main sections, this model emphasizes three main principles that are important for advancing ASP research and that are addressed by the articles in this special issue. The first principle is that it is essential to apply a holistic approach to developmental outcomes that considers multiple domains of adjustment including cognitive, biological/physical, social, and personal domains. It is not always possible to study all four domains simultaneously, but many studies in this special issue consider multiple outcomes that can help us understand the specificity or breadth of program effects. For example, using diverse methodologies, both the studies by Pierce et al. (2010) and Shernoff (2010) found that program quality features were associated with both improved social and academic outcomes. Similarly, Riggs et al. (2010) report that after-school programming may positively affect the academic and personal development of Latino youth. Durlak et al. (2010) meta-analysis suggests that ASPs can achieve positive outcomes related to personal, social and academic development.

A second principle illustrated by the model in Fig. 1 is that development is multiply determined by factors within and outside individuals that include the interactions that occur between the individual and multiple environmental/social contexts (e.g., family, school, peer group, and community factors). For example, the reports by Sheldon et al. (2010) and Holleman et al. (2010) strongly suggest that the capacity to form collaborative relationships with other community systems and supports that exist between the ASP and community residents and institutions (e.g., families, schools, and other agencies) are critical to program development and success. Likewise, Cornelli Sanderson and Richards (2010) show that developing effective after-school programming requires that efforts be tailored to the needs of multiple stakeholders (e.g., community based organizations, schools, youth, parents) in the particular communities of interest.

A third important principle of the model is that, over time, youth and environmental influences are bidirectional: youth exert an influence on the environments in which they find themselves as well as being affected by each environmental setting. For example, Larson and Walker (2010) found that several of the dilemmas or challenges faced by ASP staff were generated by youth behaviors. In other words youth behavior can affect the ASP setting by making it easier or harder for staff to do their job. In addition, Smith et al. (2010) report how the pedagogical approaches used by staff in ASPs are affected by characteristics of the youth participants. For example, staff were more likely to adopt a youth-centered pedagogical style featuring a supportive environment and active learning opportunities when serving high school students. In contrast, staff-centered approaches that tended to provide fewer opportunities for choice, planning, and reflection was more common during program offerings with elementary school students.

In addition to addressing the three key principles, the articles in this special issue also speak to the five main sections of the model shown in Fig. 1. This is detailed below.

Youth Characteristics

Little information exists regarding how youth characteristics such as gender, age, and prior cognitive or psychological development relates to program outcomes (Riggs and Greenberg 2004; Mahoney et al. 2010). Several articles in this issue address this area. For example, Riggs et al. (2010) found that youth who had greater concentration and emotion regulation skills at the outset of programming benefited most from consistent ASP participation. In addition, results from the Safe Kids After-school Survey (Cornelli Sanderson and Richards 2010) highlight the importance of multiple informants in after-school research. Specifically, what is desired in ASPs (e.g., program goals, attendance, and offerings) varies significantly across different informants (e.g., parents and youth) and according to informant characteristics such as age and gender.

Social Ecologies

The second section of Fig. 1, social ecologies, suggests that consideration should be given to several major groups and social settings that may influence ASP participation and outcomes. (e.g., family, school, peers, and the neighborhood or local community). Although space does not allow us to discuss each of these settings in depth, several articles in this issue indicate the importance of considering their potential impact.

For example, Cross et al. (2010) reported that peer interactions varied across the programs they studied and seemed to be an important influence on program climate and operation. In some programs, youth were disengaged and difficult to manage, while at other sites youth bonded well together and easily followed program rules. This is another example of the potential effect that youth can have on their environment.

Holleman et al. (2010) description of the Safe and Sound initiative in Baltimore makes it clear that linkages between multiple contexts at different levels of organization—city government, public agencies and schools, non-profit organizations, local foundations, community leaders, parents, and youth themselves—have important roles to play in developing large-scale sustainable programming. In addition, Holleman et al. (2010) stress the importance of citywide system planning that needs to be done so that ASPs occupy their most useful niche; that is, that ASPs are developed and spread across a community in order to systematically address issues related to equity, availability and accessibility, and localized needs and preferences.

As noted earlier, youth may spend their time out of school in many different ways and it should come as no surprise that communities differ in the resources and opportunities available to local youth. The article by Shernoff (2010) stresses the importance of examining the other ways that youth spend their after-school time apart from ASPs by examining program participants’ experiences when in program-related activities as compared with other settings.

After-School Program Features

Structure and Process

There are several ways to examine how any organization or social setting influences its members. Within the after-school field, considerable recent attention has been devoted to identifying features that can help us identify programs of high quality. In general, these efforts focus on two broad dimensions: structural features (e. g., physical facilities, staff characteristics, and financial and material resources) and process features (e.g., the day-to-day interactions occurring between staff and participants, the setting’s psychosocial climate including behavioral norms and participants’ sense of belonging and contribution, and the types of activities available to youth).

In this special issue, Yohalem and Wilson-Ahlstrom (2010) detail the relevance of various ASP structure and process features for a variety of youth outcomes. In addition, they provide an overview of youth program quality instruments and specify the program features that are assessed with each of these instruments. Pierce et al. (2010) address several process features in their study including staff-child relations, available activities, and programming flexibility. Only positive staff-child relations were associated with numerous positive academic outcomes. The other process features were related to academic outcomes in two instances only. Shernoff (2010) used experience sampling techniques to capture individual’s perceptions of ASP process features (i.e., engagement, challenge, and importance), and found these to be important factors in social and academic outcomes.

Smith et al. (2010) examined several aspects of process quality used by staff during program offerings (e.g., welcoming atmosphere, group participation, opportunities for active learning and reflection). They demonstrate that these quality aspects tend to cluster together into identifiable pedagogical styles. The pedagogies vary significantly in their emphasis on positive youth development. They also relate to the types of offerings provided (e.g., homework, art, sports, life skills) and, as noted earlier, vary somewhat according to the characteristics of the youth served such as age.

The findings reported by Cornelli Sanderson and Richards (2010) suggest that programs should be developed in response to diverse stakeholder needs. For instance, results from a carefully designed community survey reveal that although there is some agreement between parents and youth about what types of after-school content is most desirable, important differences are also apparent and merit attention so that programs that are offered match the needs of their key stakeholders.

Implementation

No matter how well a program is conceived or designed, it must be conducted well to achieve its intended effects (Durlak and Dupre 2008). Implementation refers to how well an intended program is actually put into practice. Assessing program implementation is now viewed as an essential feature of a program evaluation and should become part and parcel of all ASP outcome research.

Several articles in this special issue focus on implementation. For example, Sheldon et al. (2010) describes efforts to use quality improvement strategies to enhance program implementation in multi-site after-school evaluation in California. As compared with a single point in time initiative, results suggested that program implementation enhancements were associated with ongoing efforts at quality improvement. Similarly, they suggest that having a full-time director who could coordinate training resulted in the greatest implementation improvements and subsequent reading gains. Cornelli Sanderson and Richards’ (2010) results suggest that programs should be implemented with the needs of the community in mind and that key stakeholders should be invited to give input on program design and development. Moreover, Cross et al. (2010) reported that levels of implementation achieved across five different program sites was related to the levels of positive experiences reported by youth. These authors found that some aspects of implementation were more clearly associated with youths’ reported experiences (i.e., quality of management, psychosocial climate, participant responsiveness, and staff stability) whereas others were not. These findings also stress the importance of considering the multi-dimensional aspects of program implementation.

Participation

Whether it is medicine, clinical treatment, or involvement in an ASP, a sufficient dosage of active ingredients is necessary to produce the intended results. Unfortunately, low and sporadic attendance is the norm in many ASPs, and it is unclear how different levels of participation relate to different outcomes. Therefore, we list program participation as a separate section in Fig. 1 to emphasize its critical role in ASP outcomes. Weiss et al. (2005) have discussed the importance of considering the full “participation equation” which consists of enrollment plus attendance, plus engagement. In other words, if ASPs are to be of benefit, youth must first walk through the door, then attend regularly, and finally, become actively engaged when they are present.

Roth et al. (2010) narrative review of outcome research failed to find much support for the notion that participation was related to different developmental outcomes. However, these authors admitted their conclusions were limited by the inconsistent definitions and metrics that evaluators used to assess participation. They also observed that few studies focused on either the breadth of activities in which youth were involved or youths’ level of engagement in different program opportunities.

Short- and Long-Term Outcomes

We have already discussed the different types of short-term outcomes that may be associated with ASPs, and long-term benefits also need examination. Unfortunately, there appears to be a lack of information on the durability of ASP program gains (Mahoney et al. 2010). In this issue, although many studies utilized longitudinal data, not one study addresses whether there are sustained positive effects of programming. In fact, reviews by Durlak et al. (2010), and Roth et al. (2010) found few after-school studies that contained follow-up data, suggesting this is an important priority for future research.

General Recommendations for Future Program Evaluations

This final section offers some general recommendations to improve future program evaluations that can applied to fit the particular aims of different research efforts. For example, the basic experimental rigor of after-school program evaluations can be improved in several ways. More randomized experiments are needed to reach casual conclusions and in particular to rule out selection bias. Youth elect to join ASPs and comparing those who volunteer with controls who have not fails to address the variety of known individual and contextual factors that are linked to why some youth are interested in ASPs while others are not (e.g., Bouffard et al. 2006; Mahoney et al. 2009b; Wimer et al. 2008). It is also important to not only collect baseline data on participants including information on their prior school performance and behavioral adjustment, but also, in control group designs, to test for pre-treatment equivalence and control for any initial differences that are present in subsequent statistical analyses. Because many current programs contain multiple activities or components, comparisons of programs with different components would be helpful (e.g., Mahoney et al. 2007; Smith et al. 2010). Studies that systematically vary what is provided during different parts of ASPs would help us understand the relative effectiveness of program components. For example, some findings have suggested that it is not necessary to include a major academic component in an ASP to boost participants’ academic performance (Rosewater 2009). Therefore, it is also important to examine both the specificity and possible breadth of the impact of different programs.

Furthermore, because it is not always clear how many youth drop out of programs, attrition should be closely monitored. An intent-to-treat analysis in which all those who begin an intervention regardless of their ultimate degree of participation are included in the experimental sample can provide a rigorous test of program impact. However, ASP attendance can be dynamic over time and participation itself is a multifaceted construct (Weiss et al. 2005). Accordingly, to best understand the role of ASPs in the developmental process, multiple approaches to analysis and a variety of data collection methods that involve different informants and use psychometrically sound measures should be employed. Except for inspection of school records, many current studies have restricted their data collection to self-report instruments completed by youth and some of these measures have been of unknown or dubious reliability and validity. As previously indicated, more follow-up studies are needed because few studies have determined the durability of program impact over time. There is also a role for qualitative studies that can increase our understanding of issues that are more difficult to capture through typical quantitative procedures (e.g., Larson et al. 2005). Regardless of the specific research question, theory-driven research is preferred because theories provide a logical framework for what to expect, how and when to measure important processes and variables, and how to interpret the findings and consider alternative explanations. The use of participatory research methods can increase the likelihood that ASPs that are developed and offered in different communities address the needs and wishes of parents and youth.

Finally, interpreting the consequences of any out-of-school activity or arrangement can depend on what it is being compared against. With some notable exceptions (e.g., Mahoney et al. 2005b; National Institute of Child Health and Human Development Early Child Care Research Network 2004; Pedersen 2005) studies of ASPs have been vague about the types of alternative arrangements that serve as the reference comparison. Moreover, those studies that have compared ASP participation to specific alternative arrangements have tended to conduct analyses as if youth experience only one out-of-school context. In fact, youth are often involved in several different activities and arrangements in a given week (Capizzano et al. 2000; Hofferth et al. 1991; Kleiner et al. 2004). Accordingly, ASPs can be part of a patchwork of out-of-school experiences (e.g., Lord and Mahoney 2007) and the specific combination of arrangements experienced can be expected to change across development.

The above methodological issue are not exhaustive but more attention devoted to these matters will strengthen our confidence in ASP process and outcome research and increase our understanding of who benefits and how from program participation. It is important to note that every one of the 12 invited articles in this issue uses one or more of the above mentioned strategies, either in their individual projects or, in the case of review articles, in the variables that were examined when evaluating after-school research.

Concluding Comments

To summarize, a single investigation cannot address all of these potentially important issues, but as the contributions to this special issue illustrate, it is important to begin to study ASPs with specific, ecologically-oriented research questions. Psychotherapy outcome research has profitably shifted from asking broad questions about outcomes toward more nuanced inquiry regarding the benefits of treatment (i.e., which specific techniques administered by whom helps which clients with what kinds of problems?). We feel the time has quickly arrived for research on ASPs to assume the same discerning perspective. In terms of a future research agenda, after-school research can advance by assessing outcomes in consideration of the many factors that might affect program impact. We need more data on all of the following issues: Who does what to whom, in what ways, in what types of settings, within what broader context; what level of participation or engagement is needed by which populations to achieve what types of outcomes, and what are the most effective ways to improve current programs?