Journal of Experimental Criminology

, Volume 15, Issue 3, pp 397–419 | Cite as

A conceptual replication of the Strategic Training Initiative in Community Supervision (STICS)

  • James Bonta
  • Tanya Rugge
  • Guy Bourgon
  • Kayla A. Wanamaker



This study was an attempt to replicate the findings from an earlier experimental evaluation of a probation officer training program by Bonta et al. (Criminal Justice and Behavior, 38: 1127–1148, 2011). An experimental design was used with an improvement in the random assignment of clients and was tested with a sample of probation officers from a new jurisdiction.


Probation officers from the Canadian province of Alberta were randomly assigned to training or probation-as-usual. Officer behavior was measured by audio recordings of supervision sessions and recidivism was defined as a new conviction within 2 years of the initial recording. Attrition resulted in 27 probation officers submitting audio recordings of supervision sessions over a 6-month period (15 in the experimental group and 12 in the control). There were 160 recordings of 81 probationers submitted.


The audio recordings showed inconsistent changes in officer behavior and no differences in recidivism between the clients of the experimental and control probation officers. However, the use of cognitive techniques by the probation officers was associated with a longer time to recidivism. In addition, by 10 months, more than half of the trained officers stopped their involvement in ongoing professional development activities.


Although the study failed to replicate the major findings reported by Bonta et al., it did highlight the importance of cognitive techniques in officer training. The results are interpreted with respect to the replication literature and the difficulties inherent in direct and conceptual replications especially in real-world settings.


Cognitive-behavioral Community supervision Offender rehabilitation Replication STICS 



We would like to thank Leticia Gutierrez for training and overseeing the team that coded the audio recordings (Stephanie Desson, Greg Moreau, Nick Chadwick, Arshina Kassam, Jordan Monnink, and Benjamin Gallant). Thanks to Richelle Budd, Kelsey Burrow, Rebecca Grace, and Julia Voju for their work in various aspects of data collection and organization, and to Liz Bourguignon and Colleen Hamilton who assisted with the probation officers’ ongoing clinical support activities. Our gratitude is extended to senior managers James Donaghue and Kim Sanderson, Jim Cook, and the probation officers along with their managers who made this study possible.

Compliance with ethical standards

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. The study was conducted under the auspices of the Government of Canada and the Department of Public Safety (Canada) does not have a formal Research Ethics Review Board. However, researchers are obligated to follow the ethical guide for research with human subjects as outlined in the Tri-Council Policy Statement and the Values and Ethics Code for the Public Service (Government of Canada).

Informed consent

Informed written consent was obtained from all individual participants included in the study.

Conflict of interest

James Bonta receives royalties on sales of the Level of Service Inventory-Revised cited in this article.


The opinions expressed are those of the authors and do not necessarily represent the views of Public Safety Canada. The datasets generated during and/or analyzed during the current study are available on reasonable request from

Supplementary material

11292_2019_9371_MOESM1_ESM.docx (67 kb)
ESM 1 (DOCX 66 kb)


  1. Aebi, M. F., Akdeniz, G., Barclay, G., Campistol, C., Caneppele, S., Gruszczyńska, B., Harrendorf, S., Heiskanen, M., Hysi, V., Jehle, J.-M., Jokinen, A., Kensey, A., Killas, M., Lewis, C. G., Savano, E., Smit, P., & Ϸórisdóttir, R. (2014). European sourcebook of crime and criminal justice statistics 2014. Helsinki: European Institute for Crime Prevention and Control affiliated with the United Nations (HEUNI).Google Scholar
  2. Andrews, D. A. (2011). The impact of nonprogrammatic factors on criminal justice interventions. Legal and Criminological Psychology, 16, 1–23. Scholar
  3. Andrews, D. A., & Bonta, J. (1995). The level of service inventory – revised. Toronto: Multi-Health Systems.Google Scholar
  4. Andrews, D. A., & Dowden, C. (2005). Managing correctional treatment for reduced recidivism: a meta-analytic review of program integrity. Legal and Criminological Psychology, 10, 173–187. Scholar
  5. Andrews, D. A., Bonta, J., & Hoge, R. D. (1990a). Classification for effective rehabilitation: rediscovering psychology. Criminal Justice and Behavior, 17, 19–52.CrossRefGoogle Scholar
  6. Andrews, D. A., Zinger, I., Hoge, R. D., Bonta, J., Gendreau, P., & Cullen, F. T. (1990b). Does correctional treatment work?: a clinically relevant and psychologically informed meta-analysis. Criminology, 28, 369–404.CrossRefGoogle Scholar
  7. Appelbaum, M., Cooper, H., Kline, R. B., Mayo-Wilson, E., Nezu, A. M., & Rao, S. M. (2018). Journal article reporting standards for quantitative research in psychology: the APA publications and communications board task force report. American Psychologist, 73, 3–25. Scholar
  8. Australian Bureau of Statistics (2016). Persons in corrective services. Corrective Services, Australia, June Quarter.Google Scholar
  9. Barnes, J. C., TenEyck, M. F., Pratt, T. C., & Cullen, F. T. (2019). How powerful is the evidence in criminology? On whether we should fear a coming crisis of confidence. Justice Quarterly.
  10. Beck, A. T. (2016). Cognitive therapy: nature and relation to behavior therapy. Behavior Therapy, 47, 776–784. Scholar
  11. Beidas, R. S., & Kendall, P. C. (2010). Training therapists in evidence-based practice: a critical review of studies from a systems-contextual perspective. Clinical Psychology: Science and Practice, 17, 1–30.Google Scholar
  12. Bogue, B. M., Nandi, A., & Jongma, A. E., Jr. (2003). The probation and parole treatment planner. Hoboken: John Wiley & Sons.Google Scholar
  13. Bonta, J., & Andrews, D. A. (2017). The psychology of criminal conduct (6th ed.). New York: Routledge.Google Scholar
  14. Bonta, J., Rugge, T., Scott, T., Bourgon, G., & Yessine, A. (2008). Exploring the black box of community supervision. Journal of Offender Rehabilitation, 47, 248–270. Scholar
  15. Bonta, J., Bourgon, G., Rugge, T., Scott, T.-L., Yessine, A., Gutierrez, L., & Li, J. (2011). An experimental demonstration of training probation officers in evidence-based community supervision. Criminal Justice and Behavior, 38, 1127–1148. Scholar
  16. Bonta, J., Bourgon, G., Rugge, T., Gress, C., & Gutierrez, L. (2013). Taking the leap: from pilot project to wide-scale implementation of the Strategic Training Initiative in Community Supervision (STICS). Justice Research and Policy, 15, 17–35. Scholar
  17. Bourgon, G., & Gutierrez, L. (2012). The general responsivity principle in community supervision: The importance of probation officers using cognitive intervention techniques and its influence on recidivism. Journal of Crime and Justice, 35, 149–166. Scholar
  18. Bourgon, G., & Gutierrez, L. (2013). The importance of building good relationships in community corrections: Evidence, theory, and practice of the therapeutic alliance. In P. Ugwudike & P. Raynor (Eds.), What works in offender compliance: International perspectives and evidence-based practice (pp. 256–278). Basingstoke: Palgrave Macmillan.CrossRefGoogle Scholar
  19. Bourgon, G., Bonta, J., Rugge, T., & Gutierrez, L. (2010). Technology transfer: The importance of on-going clinical supervision in translating ‘what works’ to everyday community supervision. In F. McNeil, P. Raynor, & C. Trotter (Eds.), Offender supervision. New directions in theory, research and practice (pp. 88–106). New York: Willan Publishing.Google Scholar
  20. Bourgon, G., Rugge, T., Chadwick, N., & Bonta, J. (2018). The living laboratory studies: providing insights into community supervision practices. Federal Probation, 82, 3–12.Google Scholar
  21. Butler, A. C., Chapman, J. E., Forman, E. M., & Beck, A. T. (2006). The empirical status of cognitive-behavioral therapy: a review of meta-analysis. Clinical Psychology Review, 26, 17–31. Scholar
  22. Cesario, J. (2014). Priming, replication, and the hardest science. Perspectives on Psychological Science, 9, 40–48. Scholar
  23. Chadwick, N., Dewolf, A., & Serin, R. (2015). Effectively training community supervision officers: a meta-analytic review of the impact on offender outcome. Criminal Justice and Behavior, 42, 977–989. Scholar
  24. Cicchetti, D. V. (1994). Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological Assessment, 6, 284–290.CrossRefGoogle Scholar
  25. Council of State Governments Justice Centre. (2013). Lessons from the states: reducing recidivism and curbing corrections costs through justice reinvestment. New York: Council of State Governments Justice Centre.Google Scholar
  26. Cullen, F. T. (2013). Rehabilitation: beyond nothing works. In M. Tonry (Ed.), Crime and justice, Vol. 42, Crime and justice in America 1975-2025 (pp. 299–376). Chicago: University of Chicago Press.Google Scholar
  27. Dowdon, C., & Andrews, D. A. (2004). The importance of staff practices in delivering effective correctional treatment: a meta-analysis of core correctional practices. International Journal of Offender Therapy and Comparative Criminology, 48, 203–214. Scholar
  28. Drake, E. (2013). Inventory of evidence-based and research-based programs for adult corrections. Olympia: Washington State Institute for Public Policy.Google Scholar
  29. Edens, D. (2002). From the editors: Replication, meta-analysis, scientific progress, and AMJ’s publication policy. The Academy of Management Journal, 45, 841–846.CrossRefGoogle Scholar
  30. Ellis, A. (1962). Reason and emotion in psychotherapy. Oxford: Lyle Stuart.Google Scholar
  31. Farrington, D. P., Lösel, F., Boruch, R. F., Gottfredson, D. C., Mazerolle, L., Sherman, L. W., & Weisburd. (2018). Advancing knowledge about replication in criminology. Journal of Experimental Criminology.
  32. Gendreau, P., & Ross, R. R. (1979). Effective correctional treatment: bibliotherapy for cynics. Crime & Delinquency, 25, 463–489.CrossRefGoogle Scholar
  33. Gendreau, P., Smith, P., & Thériault, Y. L. (2009). Chaos theory and correctional treatment: Common sense, correctional quackery, and the law of fartcatchers. Journal of Contemporary Criminal Justice, 25, 384–393. Scholar
  34. Gleicher, L., Manchak, S. M., & Cullen, F. T. (2013). Creating a supervision kit: How to improve probation and parole. Federal Probation, 77, 22–27.Google Scholar
  35. Jones, N. J., Brown, S. L., Robinson, D., & Frey, D. (2015). Incorporating strengths into quantitative assessments of criminal risk for adult offenders: the service planning instrument. Criminal Justice and Behavior, 42, 321–338. Scholar
  36. Kaeble, D. (2018). Correctional populations in the United States, 2016. Bureau of Justice Statistics Bulletin (April, 2018). Washington, DC: Office of Justice Programs, U. S. Department of Justice .Google Scholar
  37. Kennealy, P. J., Skeem, J. L., Manchak, S. M., & Louden, J. E. (2012). Firm, fair, and caring officer-offender relationships protect against supervision failure. Law and Human Behavior, 36, 496–505. Scholar
  38. Kleiman, M. A. R. (2011). Justice reinvestment in community supervision. Criminology & Public Policy, 10, 651–659. Scholar
  39. Kroner, D. G., & Morgan, R. D. (2014). An overview of strategies for the assessment and treatment of criminal thinking. In R. C. Tafrate & D. Mitchell (Eds.), Forensic CBT: A handbook for clinical practice (pp. 87–103). Chichester: John Wiley & Sons.Google Scholar
  40. Labrecque, R. M., & Smith, P. (2017). Does training and coaching matter? An 18-month evaluation of a community supervision model. Victims & Offenders: An International Journal of Evidence-based Research, Policy, and Practice, 12, 233–252. Scholar
  41. Labrecque, R. M., Smith, P., Schweitzer, M., & Thompson, C. (2013). Targeting antisocial attitudes in community supervision using the EPICS model: an examination of change scores on the Criminal Sentiment Scale. Federal Probation, 77, 15–20.Google Scholar
  42. Latessa, E. J., Smith, P., Schweitzer, M., & Labrecque, R. M. (2013). Evaluation of the effective practices in community supervision model (EPICS) in Ohio. Available from the author, School of Criminal Justice, Center for Criminal Justice Research, University of Cincinnati, PO Box 210389, Cincinnati, Ohio.Google Scholar
  43. Lipsey, M. W., & Cullen, F. T. (2007). The effectiveness of correctional rehabilitation: a review of systematic reviews. Annual Review of Law and Social Science, 3, 297–320. Scholar
  44. Lösel, F. (2018). Evidence comes by replication, but needs differentiation: the reproducibility issue in science and its relevance to criminology. Journal of Experimental Criminology, 14, 257–278. Scholar
  45. Lowenkamp, C. T., Latessa, E. J., & Smith, P. (2006). Does correctional program quality really matter? The impact of adhering to the principles of effective interventions. Criminology & Public Policy, 5, 575–594. Scholar
  46. Maxwell, S. E., Lau, M. Y., & Howard, G. S. (2015). Is psychology suffering from a replication crisis? What does “failure to replicate” really mean? American Psychologist, 70, 487–498. Scholar
  47. McGuire, J., Bilby, C. A. L., Hatcher, R. M., Hollin, C. R., Hounsome, J., & Palmer, E. J. (2008). Evaluation of structured cognitive-behavioral treatment programmes in reducing criminal recidivism. Journal of Experimental Criminology, 4, 21–40. Scholar
  48. Meichenbaum, D. (1977). Cognitive-behavior modification: An integrative approach. New York: Plenum Press.CrossRefGoogle Scholar
  49. Miller, W. R., Yahne, C. E., Moyers, T. B., Martinez, J., & Pirritano, M. (2004). A randomized trial of methods to help clinicians learn motivational interviewing. Journal of Consulting and Clinical Psychology, 72, 1050–1062. Scholar
  50. New Zealand Department of Corrections. (2016). Community sentences and orders statistics – September 2016. Auckland: New Zealand Department of Corrections.Google Scholar
  51. Ollendick, T. H. (2016). A commentary on cognitive behavior therapy: where we have been, where we are, and where we need to go from here. Cognitive and Behavioral Practice, 23, 436–440. Scholar
  52. Olver, M. E., Stockdale, K. C., & Wormith, J. S. (2014). Thirty years of research on the level of service scales: a meta-analytic examination of predictive accuracy and sources of variability. Psychological Assessment, 26, 156–176. Scholar
  53. Orbis Partners. (2003). Service Planning Instrument (SPIn). Ottawa: Author.Google Scholar
  54. Pashler, H., & Wagenmaker, E.-J. (2012). Editors’ introduction to the special section on replicability in psychological science: a crisis of confidence? Perspectives on Psychological Science, 7, 528–530. Scholar
  55. Polaschek, D. L. L. (2012). An appraisal of the risk-need-responsivity (RNR) model of offender rehabilitation and its application in correctional treatment. Legal and Criminological Psychology, 17, 1–17. Scholar
  56. Public Safety Canada. (2016). The corrections and conditional release statistical overview 2016. Ottawa: Public Safety Canada.Google Scholar
  57. Richards, B. C. (2016). Cognitive Therapy Special Interest Group: 50 years of cognitive therapy: A brief history. The Behavior Therapist, 39, 283.Google Scholar
  58. Robinson, C. J., VanBenschoten, S., Alexander, M., & Lowenkamp, C. T. (2011). A random (almost) study of Staff Training Aimed at Reducing Re-arrest (STARR): reducing recidivism through intentional design. Federal Probation, 2, 57–63.Google Scholar
  59. Robinson, C. J., Lowenkamp, C. T., Holsinger, A. M., VanBenschoten, S., Alexander, M., & Oleson, J. C. (2012). A random study of staff training aimed at reducing re-arrest (STARR): using core correctional practice in probation interactions. Journal of Crime and Justice, 35, 167–188. Scholar
  60. Rugge, T., & Bonta, J. (2014). Training community corrections officers in cognitive-behavioral intervention strategies. In R. C. Tafrate & D. Mitchell (Eds.), Forensic CBT: A handbook for clinical practice (pp. 122–136). Chichester: John Wiley & Sons.Google Scholar
  61. Schmidt, F. L. (1996). Statistical significance testing and cumulative knowledge in psychology: implications for training researchers. Psychological Methods, 1, 115–129. Scholar
  62. Schmidt, F. L., & Hunter, J. E. (2015). Methods of meta-analysis: Correcting error and bias in research findings. Thousand Oaks: Sage.CrossRefGoogle Scholar
  63. Schmidt, D. J., & Oh, I.-S. (2016). The crisis of confidence in research findings in psychology: is lack of replication the real problem? Or is it something else? Archives of Scientific Psychology, 4, 32–37. Scholar
  64. Simons, D. J. (2014). The value of direct replication. Perspectives on Psychological Science, 9, 76–80. Scholar
  65. Strobe, W., & Strack, F. (2014). The alleged crisis and the illusion of exact replication. Perspectives on Psychological Science, 9, 59–71. Scholar
  66. Tafrate, R. C., & Mitchell, D. (Eds.). (2014). Forensic CBT: A handbook for clinical practice. Chichester. UK: John Wiley & Sons.Google Scholar
  67. Tafrate, R. C., Mitchell, D., & Simourd, D. J. (2018). CBT with justice involved clients. New York: Guilford Publications.Google Scholar
  68. Taxman, F. S. (2008). No illusions: offender and organizational change in Maryland’s Proactive Community Supervision efforts. Criminology & Public Policy, 7, 275–302. Scholar
  69. Taxman, F. S., Shepardson, E. S., & Byrne, J. M. (2004). Tools of the trade: a guide to incorporating science into practice. Washington: National Institute of Corrections, Washington, DC and Maryland Department of Public Safety and Correctional Services, Baltimore.Google Scholar
  70. Trafimow, D., & Marks, M. (2015). Editorial. Basic and Applied Social Psychology, 37, 1–2. Scholar
  71. Trotter, C. (1996). The impact of different supervision practices in community corrections. Australian and New Zealand Journal of Criminology, 29, 1–18.CrossRefGoogle Scholar
  72. Vazire, S. (2018). Implications of the credibility revolution for productivity, creativity, and progress. Perspectives on Psychological Science, 13, 411–417. Scholar
  73. Walters, G. D. (2014). Applying CBT to the criminal thought process. In R. C. Tafrate & D. Mitchell (Eds.), Forensic CBT: A handbook for clinical practice (pp. 104–121). Chichester: John Wiley & Sons.Google Scholar
  74. Walters, S. T., Matson, S. A., Baer, J. S., & Ziedonis, D. M. (2005). Effectiveness of workshop training for psychosocial addiction treatment: a systematic review. Journal of Substance Abuse Treatment, 29, 283–293. Scholar
  75. Wampold, B. E. (2007). Psychotherapy: the humanistic (and effective) treatment. American Psychologist, 62, 857–873. Scholar
  76. Wormith, S. J., & Bonta, J. (2018). The Level of Service (LS) instruments. In J. P. Singh, D. G. Kroner, J. S. Wormith, S. L. Desmarais, & Z. Hamilton (Eds.), Recidivism risk assessment: a handbook for practitioners (pp. 117–145). New York: Wiley.Google Scholar
  77. Wormith, S. J., Ferguson, M., & Bonta, J. (2013). Offender classification and case management and their application in Canadian Corrections. In J. Winterdyck & M. Z. Weinrath (Eds.), Adult corrections in Canada (pp. 171–198). Whitby: de Sitter Publications.Google Scholar

Copyright information

© Crown 2019

Authors and Affiliations

  • James Bonta
    • 1
  • Tanya Rugge
    • 1
  • Guy Bourgon
    • 1
  • Kayla A. Wanamaker
    • 1
  1. 1.OttawaCanada

Personalised recommendations