Skip to main content
Log in

Reassessing findings from the Fast Track study: problems of method and analysis

  • Published:
Journal of Experimental Criminology Aims and scope Submit manuscript



A 2010 article by the Conduct Problems Prevention Research Group considers an important social and public policy problem: can early psychosocial intervention prevent delinquency? The article examines the effects of the Fast Track preventive intervention on youth arrests and self-reported delinquent behavior through age 19. The article reports that the intervention reduced court-recorded juvenile arrest activity as well as a range of other disparate effects.


This comment assesses the methodology employed in the article.


The original article suffers from a range of methodological problems. First and foremost, the article includes a large number of statistical tests and highlights only a subset that are statistically significant. As this article demonstrates, these findings likely involve “false discoveries” or chance findings. Uncertainty about the study’s findings is increased still further by problems of randomization, the treatment of site, the handling of missing data, and the inclusion of a collider as a covariate in key analyses. A proper assessment of the study’s meaning and implications has been impeded by inaccuracies in how the study’s methodology has been described over time.


The original article offers chance findings, suffers from methodological errors and builds on a flawed study design. As a result, it is impossible to conclude that “that a comprehensive preventive intervention can prevent juvenile arrest rates”. What the intervention would accomplish were it implemented in a new community is unknown.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1


  1. Only the values of observations close to this percentile contribute to its estimation.

  2. In fairness to the authors, their description of randomization has become more accurate over time. Original study reports indicated that the study randomly assigned schools: “Because part of the intervention (described below) involved a school-based intervention, we assigned entire schools (n = 54) to either the intervention condition or the control condition” (Conduct Problems Prevention Research Group,1999, p. 634).

  3. Indeed, observational methods such as propensity scores may balance observed factors more effectively than randomization (Freedman et al. 2009).

  4. That study indicates that “Children's sum scores on the two screening measures (teacher and parent ratings of behavior problems) were averaged. Children who fell in the top 10 % at each site on the combined screen ("high risk") were invited to participate in the longitudinal study”. (Conduct Problems Prevention Research Group 1999, p. 634)

  5. One irony of the study is that the ability to examine the high-risk interaction highlighted in the paper is that it involves comparing children originally targeted with those children added to the study to reach recruitment goals or for some other purpose. Note that Foster (2010) finds that the interaction between initial risk status does not generalize across outcomes.

  6. The authors seem to reverse themselves in an erratum subsequently published in the journal.

  7. This problem has been known for some time; the clearest recent statement can be found in Sobel (2008). In other words, the standard treatment of mediation assumes that—conditional on the covariates—detention is determined as if it is randomly assigned.

  8. There are several archetypical examples used to illustrate this point. In one, the outcome of interest is the event that the golf course is wet. There are two causes—rain and a sprinkler system. The sprinkler system is on a timer and runs every other day, whether it rains or not. It runs even on rainy days. However, if one limits one’s analyses to when the golf course is wet, there is a spurious relationship between rain and the sprinkler running. In particular, on those days, the chance that the sprinkler ran on days when it did not rain is 100 %. If it rained, then the chance that the sprinkler ran is 50 %.


  • Baron, R. M., & Kenny, D. A. (1986). The moderator-mediator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations. Journal of Personality and Social Psychology, 51(6), 1173–1182.

    Article  Google Scholar 

  • Benjamini, Y., & Hochberg, Y. (1995). Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the Royal Statistical Society. Series B (Methodological), 289–300.

  • Benjamini, Y., & Yekutieli, D. (2001). The control of the false discovery rate in multiple testing under dependency. The Annals of Statistics, 29(4), 1165–1188.

    Article  Google Scholar 

  • Cohen, M. A. (2005). The costs of crime and justice. New York: London.

    Book  Google Scholar 

  • Cole, S. R., & Hernan, M. A. (2008). Constructing inverse probability weights for marginal structural models. American Journal of Epidemiology, 168(6), 656–664.

    Article  Google Scholar 

  • Conduct Problems Prevention Research Group. (1999). Initial impact of the Fast Track prevention trial for conduct problems: I The High-risk sample. Journal of Consulting and Clinical Psychology, 67(5), 631–647.

    Article  Google Scholar 

  • Conduct Problems Prevention Research Group. (2010). Fast Track intervention effects on youth arrests and delinquency. Journal of Experimental Criminology, 6(2), 131–157.

    Article  Google Scholar 

  • Dixon, D. O., & Simon, R. (1991). Bayesian subset analysis. Biometrics, 47(3), 871–881.

    Article  Google Scholar 

  • Dixon, D. O., & Simon, R. (1992). Bayesian subset analysis in a colorectal cancer clinical trial. Statistics in Medicine, 11(1), 13–22.

    Google Scholar 

  • Efron, B., & Gong, G. (1983). A leisurely look at the bootstrap, the jackknife, and cross-validation. The American Statistician, 37(1), 36–48.

    Google Scholar 

  • Foster, E. M. (2010). Costs and effectiveness of the Fast Track intervention for antisocial behavior. The Journal of Mental Health Policy and Economics, 13(3), 101–119.

    Google Scholar 

  • Foster, E. M., Jones, D., & Conduct Problems Prevention Research Group. (2006). Can a costly intervention be cost-effective?: An analysis of violence prevention. Archives of General Psychiatry, 63, 1284–1291.

    Article  Google Scholar 

  • Freedman, D., Collier, D., Sekhon, J. S., & Stark, P. B. (2009). Statistical models and causal inference: a dialogue with the social sciences. New York: Cambridge University Press.

    Book  Google Scholar 

  • Robins, J. M. (1997). Marginal structural models. Paper presented at the 1997 American Statistical Association, Section on Bayesian Statistical Science, Washington, DC.

  • Robins, J. M. (1999). Marginal structural models versus structural nested models as tools for causal inference. Statistical Models in Epidemiology: The Environment and Clinical Trials, 16, 95–134.

    Google Scholar 

  • Robins, J. M., Blevins, D., Ritter, G., & Wulfsohn, M. (1992). G-estimation of the effect of prophylaxis therapy for Pneumocystis carinii pneumonia on the survival of AIDS patients. Epidemiology, 3(4), 319–336.

    Article  Google Scholar 

  • Sato, T., & Matsuyama, Y. (2003). Marginal structural models as a tool for standardization. Epidemiology, 14(6), 680–686.

    Article  Google Scholar 

  • Simon, R. (2002). Bayesian subset analysis: application to studying treatment-by-gender interactions. Statistics in Medicine, 21(19), 2909–2916.

    Article  Google Scholar 

  • Sobel, M. E. (2008). Identification of causal parameters in randomized studies with mediating variables. Journal of Educational and Behavioral Statistics, 33(2), 230–251.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations


Corresponding author

Correspondence to E. Michael Foster.

Additional information

This article is commentary for article “Fast track intervention effects on youth arrests and delinquency”, published in Volume 6, Issue 2, under doi:10.1007/s11292-010-9091-7.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Foster, E.M. Reassessing findings from the Fast Track study: problems of method and analysis. J Exp Criminol 9, 109–117 (2013).

Download citation

  • Published:

  • Issue Date:

  • DOI: