Advertisement

Prevention Science

, Volume 20, Issue 6, pp 914–935 | Cite as

Adapting a Compilation of Implementation Strategies to Advance School-Based Implementation Research and Practice

  • Clayton R. CookEmail author
  • Aaron R. Lyon
  • Jill Locke
  • Thomas Waltz
  • Byron J. Powell
Article

Abstract

Schools, like other service sectors, are confronted with an implementation gap, with the slow adoption and uneven implementation of evidence-based practices (EBP) as part of routine service delivery, undermining efforts to promote better youth behavioral health outcomes. Implementation researchers have undertaken systematic efforts to publish taxonomies of implementation strategies (i.e., methods or techniques that are used to facilitate the uptake, use, and sustainment of EBP), such as the Expert Recommendations for Implementing Change (ERIC) Project. The 73-strategy ERIC compilation was developed in the context of healthcare and largely informed by research and practice experts who operate in that service sector. Thus, the comprehensibility, contextual appropriateness, and utility of the existing compilation to other service sectors, such as the educational setting, remain unknown. The purpose of this study was to initiate the School Implementation Strategies, Translating ERIC Resources (SISTER) Project to iteratively adapt the ERIC compilation to the educational sector. The results of a seven-step adaptation process resulted in 75 school-adapted strategies. Surface-level changes were made to the majority of the original ERIC strategies (52 out of 73), while five of the strategies required deeper modifications for adaptation to the school context. Six strategies were deleted and seven new strategies were added based on existing school-based research. The implications of this study’s findings for prevention scientists engaged in implementation research (e.g., creating a common nomenclature for implementation strategies) and limitations are discussed.

Keywords

Implementation science Implementation strategies School-based mental and behavioral health Evidence-based practices 

Notes

Funding

This publication was supported in part by funding from the Institute of Education Sciences (R305A160114 PIs - Lyon and Cook; R305A170292 PIs – Cook and Lyon). BJP was supported by K01MH113806, R25MH080916, and UL1TR001111. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health or Institute of Education Sciences.

Compliance with Ethical Standards

Conflict of Interest

The authors declare that they have no conflict of interest.

Ethical Approval

Exempt status was obtained from the university IRB prior to conducting the study.

Informed Consent

No informed consent was necessary as part of this study.

References

  1. Aarons, G. A., Hurlburt, M., & Horwitz, S. M. (2011). Advancing a conceptual model of evidence-based practice implementation in child welfare. Administration and Policy in Mental Health and Mental Health Services Research, 38, 4–23.CrossRefGoogle Scholar
  2. Aarons, G. A., Ehrhart, M. G., & Farahnak, L. R. (2014). The implementation leadership scale (ILS): Development of a brief measure of unit level implementation leadership. Implementation Science, 9, 45.CrossRefGoogle Scholar
  3. Abrahamson, E. (2004). Change without pain: How managers can overcome initiative overload, organizational chaos, and employee burnout. Boston: Harvard Business School Press.Google Scholar
  4. Barrera, M., & González-Castro, F. (2006). A heuristic framework for the cultural adaptation of interventions. Clinical Psychology: Science and Practice, 13, 311–316.Google Scholar
  5. Barrish, H. H., Saunders, M., & Wolfe, M. D. (1969). Good behavior game. Effects of individual contingencies for group consequences and disruptive behavior in a classroom. Journal of Applied Behavioral Analysis, 2, 119–124.CrossRefGoogle Scholar
  6. Beidas, R. S., & Kendall, P. C. (2010). Training therapists in evidence-based practice: A critical review of studies from a systems-contextual perspective. Clinical Psychology: Science and Practice, 17, 1–30.Google Scholar
  7. Bernal, G., Bonilla, J., & Bellido, C. (1995). Ecological validity and cultural sensitivity for outcome research: Issues for the cultural adaptation and development of psychosocial treatments with Hispanics. Journal of Abnormal Child Psychology, 23, 67–82.CrossRefGoogle Scholar
  8. Bernal, G., Jimenez-Chafey, M. I., & Domenech-Rodríguez, M. M. (2009). Cultural adaptation of treatments: A resource for considering culture in evidence-based practice. Professional Psychology: Research and Practice, 40, 361–368.CrossRefGoogle Scholar
  9. Bookstein, A. (1990). Informetric distributions, part I: Unified overview. Journal of the American Society for Information Science, 41, 368–375.CrossRefGoogle Scholar
  10. Boyd, M. R., Powell, B. J., Endicott, D., & Lewis, C. C. (2017). A method for tracking implementation strategies: An exemplar implementing measurement-based care in community behavioral health clinics. Behavior Therapy.  https://doi.org/10.1016/j.beth.2017.11.012.
  11. Brackett, M. A., & Rivers, S. E. (2013). Transforming students’ lives with social and emotional learning. In R. Pekrun & L. Linnenbrink-Garcia (Eds.), International handbook of emotions in education (pp. 368–388). New York: Taylor & Francis.Google Scholar
  12. Bradshaw, C. P., Mitchell, M. M., & Leaf, P. J. (2010). Examining the effects of schoolwide positive behavioral interventions and supports on student outcomes: Results from a randomized controlled effectiveness trial in elementary schools. Journal of Positive Behavior Interventions, 12, 133–148.CrossRefGoogle Scholar
  13. Bruns, E. J., Duong, M. T., Lyon, A. R., Pullmann, M. D., Cook, C. R., Cheney, D., & McCauley, E. (2016). Fostering SMART partnerships to develop an effective continuum of behavioral health services and supports in schools. American Journal of Orthopsychiatry, 86, 156–170.CrossRefGoogle Scholar
  14. Bunger, A. C., Powell, B. J., Robertson, H. A., MacDowell, H., Birken, S. A., & Shea, C. (2017). Tracking implementation strategies: A description of a practical approach and early findings. Health Research Policy and Systems, 15, 1–12.CrossRefGoogle Scholar
  15. Burnes, B. (2004). Emergent change and planned change: Competitors or allies? The case of XYZ construction. International Journal of Operations & Production Management, 24, 886–902.CrossRefGoogle Scholar
  16. Cook, B. G., & Odom, S. L. (2013). Evidence-based practices and implementation science in special education. Exceptional Children, 79, 135–144.Google Scholar
  17. Cook, C. R., Lyon, A. R., Kubergovic, D., Wright, D. B., & Zhang, Y. (2015). A supportive beliefs intervention to facilitate the implementation of evidence-based practices within a multi-tiered system of supports. School Mental Health, 7, 49–60.CrossRefGoogle Scholar
  18. Cook, C. R., Miller, F., Fiat, A., Renshaw, T. L., Frye, M., & Joseph, G. (2017a). Promoting secondary teachers’ well-being and intentions to implement evidence-based practices: Randomized evaluation of the ACHIEVER Resilience Curriculum. Psychology in the Schools, 54, 13–28.CrossRefGoogle Scholar
  19. Cook, C. R., Pauling, S., McCaslin, S., Larson, M., Thayer, A. J., & Fiat, A. (2017b). Brief reminders as a low threshold strategy to facilitate teacher delivery of evidence-based classroom practices. Manuscript in preparation.Google Scholar
  20. Dalkey, N. C., & Helmer, O. (1963). An experimental application of the Delphi method to the use of experts. Management Science, 9, 458–467.CrossRefGoogle Scholar
  21. Damian, J., Gallo, J., Leaf, P., & Mendelson, T. (2017). Organizational and provider level factors in implementation of trauma-informed care after a city-wide training: An explanatory mixed methods assessment. BMC Health Services Research, 17, 750.CrossRefGoogle Scholar
  22. Damschroder, L. J., Aron, D. C., Keith, R. E., Kirsh, S. R., Alexander, J. A., & Lowery, J. C. (2009). Fostering implementation of health services research findings into practice: A consolidated framework for advancing implementation science. Implement Science, 4, 50.CrossRefGoogle Scholar
  23. Dart, E., Cook, C. R., Gresham, F. M., & Cheneir, J. (2012). Test driving to increase treatment integrity and student outcomes. School Psychology Review, 41, 467–481.Google Scholar
  24. DiGennaro, F. D., Martens, B. K., & McIntyre, L. L. (2005). Increasing treatment integrity through negative reinforcement: Effects on teacher and student behavior. School Psychology Review, 34, 220–231.Google Scholar
  25. Dingfelder, H. E., & Mandell, D. S. (2011). Bridging the research-to-practice gap in autism intervention: An application of diffusion of innovation theory. Journal of Autism and Developmental Disorders, 41, 597–609.CrossRefGoogle Scholar
  26. Durlak, J. A., & DuPre, E. P. (2008). Implementation matters: A review of research on the influence of implementation on program outcomes and the factors affecting implementation. American Journal of Community Psychology, 41, 327–350.CrossRefGoogle Scholar
  27. Ehrhart, M. G., Aarons, G. A., & Farahnak, L. R. (2014). Assessing the organizational context for EBP implementation: The development and validity testing of the Implementation Climate Scale (ICS). Implementation Science, 9, 157.CrossRefGoogle Scholar
  28. Ehrhart, M. G., Aarons, G. A., & Farahnak, L. R. (2015). Going above and beyond for implementation: The development and validity testing of the Implementation Citizenship Behavior Scale (ICBS). Implementation Science, 10, 65.CrossRefGoogle Scholar
  29. Farmer, E. M., Burns, B. J., Phillips, S. D., Angold, A., & Costello, E. J. (2003). Pathways into and through mental health services for children and adolescents. Psychiatric Services, 54, 60–66.CrossRefGoogle Scholar
  30. Fishbein, D. H., Ridenour, T. A., Stahl, M., & Sussman, S. (2016). The full translational spectrum of prevention science: Facilitating the transfer of knowledge to practices and policies that prevent behavioral health problems. Translational Behavioral Medicine, 6, 5–16.CrossRefGoogle Scholar
  31. Fixsen, D., Blase, K., Metz, A., & Van Dyke, M. (2013). Statewide implementation of evidence-based programs. Exceptional Children, 79, 213–232.Google Scholar
  32. Flottorp, S. A., Oxman, A. D., Krause, J., et al. (2013). A checklist for identifying determinants of practice: A systematic review and synthesis of frameworks and taxonomies of factors that prevent or enable improvements in healthcare professional practice. Implementation Science, 8, 1–11.  https://doi.org/10.1186/1748-5908-8-35 [PMC free article] [PubMed] [CrossRef].CrossRefGoogle Scholar
  33. Forman, S. G., Shapiro, E. S., Codding, R. S., Gonzales, J. E., Reddy, L. A., Rosenfield, S. A., et al. (2013). Implementation science and school psychology. School Psychology Quarterly, 28, 77–100.CrossRefGoogle Scholar
  34. Franks, R., & Bory, T. (2015). Who supports the successful implementation and sustainability of evidence-based practices? Defining and understanding the roles of intermediary and purveyor organisations. New Directions for Child and Adolescent Development, 149, 41–56.CrossRefGoogle Scholar
  35. Gottfredson, D., & Gottfredson, G. (2002). Quality of school-based prevention programs: Results from a national survey. Journal of Research in Crime & Delinquency, 39, 3–35.CrossRefGoogle Scholar
  36. Grol, R. (2001). Successes and failures in the implementation of evidence-based guidelines for clinical practice. Medical Care, 39, 46–54.CrossRefGoogle Scholar
  37. Herlihy, B., & Corey, G. (2006). Boundary issues in counseling: Multiple roles and responsibilities. Alexandria: American Counseling Association.Google Scholar
  38. Herschell, A. D., Kolko, D. J., Baumann, B. L., & Davis, A. C. (2010). The role of therapist training in the implementation of psychosocial treatments: A review and critique with recommendations. Clinical Psychology Review, 30, 448–466.CrossRefGoogle Scholar
  39. Horner, R. H., Sugai, G., & Fixsen, D. L. (2017). Implementing effective educational practices at scales of social importance. Clinical Child & Family Psychology Review, 20, 25–35.CrossRefGoogle Scholar
  40. Johnson, L. D., Wehby, J. H., Symons, F. J., Moore, T. C., Maggin, D. M., & Sutherland, K. S. (2014). An analysis of preference relative to teacher implementation of intervention. The Journal of Special Education, 48, 214–224.CrossRefGoogle Scholar
  41. Kane, M., & Trochim, W. M. K. (2007). Concept mapping for planning and evaluation. Thousand Oaks: Sage Publications.CrossRefGoogle Scholar
  42. Kazdin, A. E. (2007). Mediators and mechanisms of change in psychotherapy research. Annual Review of Clinical Psychology, 3, 1–27.CrossRefGoogle Scholar
  43. Khamisa, N., Peltzer, K., & Oldenburg, B. (2013). Burnout in relation to specific contributing factors and health outcomes among nurses: A systematic review. International Journal of Environmental Research and Public Health, 10, 2214–2240.CrossRefGoogle Scholar
  44. Kohler, F., Crillery, K., Shearer, D., & Good, G. (1997). Effects of peer coaching on teacher and student outcomes. Journal of Educational Research, 90, 240–250.CrossRefGoogle Scholar
  45. Krause, J., Van Lieshout, J., Klomp, R., Huntink, E., Aakhus, E., Flottorp, S., et al. (2014). Identifying determinants of care for tailoring implementation in chronic diseases: An evaluation of different methods. Implementation Science, 9, 102.CrossRefGoogle Scholar
  46. Leeman, J., Birken, S. A., Powell, B. J., Rohweder, C., & Shea, C. M. (2017). Beyond “implementation strategies”: Classifying the full range of strategies used in implementation science and practice. Implementation Science, 12, 125–134.CrossRefGoogle Scholar
  47. Leong, F. T. (1996). Toward an integrative model of cross-cultural counseling and psychotherapy. Applied and Preventive Psychology, 5, 189–209.CrossRefGoogle Scholar
  48. Lewis, C. C. (2017). Systematic review of D & I Mechanisms in health. Keynote address at 4th biennial Society for Implementation Research Collaboration Conference.Google Scholar
  49. Lewis, C. C., Klasnja, P., Powell, B. J., Lyon, A. R., Tuzzio, L., Jones, S., et al. (2018). From classification to causality: advancing understanding of mechanisms of change in implementation science. Frontiers of Public Health, 6, 1–6.CrossRefGoogle Scholar
  50. Lochman, J. E., & Wells, K. C. (2002). Contextual social–cognitive mediators and child outcome: A test of the theoretical model in the Coping Power program. Development and Psychopathology, 14, 945–967.CrossRefGoogle Scholar
  51. Locke, J., Beidas, R. S., Marcus, S., Stahmer, A., Aarons, G. A., Lyon, A. R., et al. (2016). A mixed methods study of individual and organizational factors that affect implementation of interventions for children with autism in public schools. Implementation Science, 11, 135.CrossRefGoogle Scholar
  52. Lyon, A. R., & Bruns, E. J. (2019). From evidence to impact: Joining our best school mental health practices with our best implementation strategies. School Mental Health, 11, 106–114.CrossRefGoogle Scholar
  53. Lyon, A. R., Lewis, C. C., Melvin, A., Boyd, M., Nicodimos, S., Liu, F. F., & Jungbluth, N. (2016). Health Information Technologies—Academic and Commercial Evaluation (HIT-ACE) methodology: Description and application to clinical feedback systems. Implementation Science, 11, 128.CrossRefGoogle Scholar
  54. Lyon, A. R., Pullmann, M. D., Walker, S. C., & D’Angelo, G. (2017). Community-sourced intervention programs: Review of submissions in response to a statewide call for “promising practices.”. Administration and Policy in Mental Health and Mental Health Services Research, 44, 16–28.CrossRefGoogle Scholar
  55. Lyon, A., Cook, C. R., Brown, E., Locke, J., Ehrhart, M., Davis, C., & Aarons, G. (2018). Assessing the organizational implementation context in the education sector: Confirmatory factor analysis of measures of implementation leadership, climate, and citizenship. Implementation Science, 13, 5.CrossRefGoogle Scholar
  56. Lyon, A. R., Comtois, K. A., Kerns, S. E. U., Landes, S. J., & Lewis, C. C. (2019). Closing the science-practice gap in implementation before it widens. In A. Shlonsky, R. Mildon, & B. Albers (Eds.), The science of implementation. Springer (in press).Google Scholar
  57. McHugh, R. K., & Barlow, D. H. (2010). The dissemination and implementation of evidence-based psychological treatments. A review of current efforts. American Psychologist, 65, 73–84.CrossRefGoogle Scholar
  58. McKleroy, V. S., Galbraith, J. S., Cummings, B., Jones, P., Harshbarger, C., et al. (2006). Adapting evidence-based behavioral interventions for new settings and target populations. AIDS Education & Prevention, 18, 59–73.CrossRefGoogle Scholar
  59. Morina, N., Koerssen, R., & Pollet, T. V. (2016). Interventions for children and adolescents with posttraumatic stress disorder: A meta-analysis of comparative outcome studies. Clinical Psychology Review, 47, 41–54.CrossRefGoogle Scholar
  60. Nadeem, E., & Ringle, V. (2016). De-adoption of an evidence-based trauma intervention in schools: A retrospective report from an urban school district. School Mental Health, 8, 132–143.CrossRefGoogle Scholar
  61. Novins, D. K., Green, A. E., Legha, R. K., et al. (2013). Dissemination and implementation of evidence-based practices for child and adolescent mental health: A systematic review. Journal of the American Academy of Child & Adolescent Psychiatry, 52, 1009–1025.CrossRefGoogle Scholar
  62. Owens, J., Lyon, A. R., Brandt, N. E., Maisa Warner, M., Nadeem, E., Spiel, C., & Wagner, M. (2014). Implementation science in school mental health: Key constructs and a proposed research agenda. School Mental Health, 6, 99–111.CrossRefGoogle Scholar
  63. Powell, B. J., McMillen, J. C., Proctor, E. K., Carpenter, C. R., Griffey, R. T., Bunger, A. C., et al. (2012). A compilation of strategies for implementing clinical innovations in health and mental health. Medical Care Research and Review, 69, 123–157.CrossRefGoogle Scholar
  64. Powell, B. J., Proctor, E. K., & Glass, J. E. (2014). A systematic review of strategies for implementing empirically supported mental health interventions. Research Social Work & Practice., 24, 192–212.CrossRefGoogle Scholar
  65. Powell, B. J., Waltz, T. J., Chinman, M. J., Damschroder, L. J., Smith, J. L., et al. (2015). A refined compilation of implementation strategies: Results from the Expert Recommendations for Implementing Change (ERIC) project. Implementation Science, 10, 1–14.CrossRefGoogle Scholar
  66. Powell, B. J., Beidas, R., Lewis, C. C., Aarons, G. A., McMillen, J. C., Proctor, E. K., & Mandell, D. S. (2017). Methods to improve the selection and tailoring of implementation strategies. The Journal of Behavioral Health Services & Research, 44, 177–194.CrossRefGoogle Scholar
  67. Proctor, E. K., Landsverk, J., Aarons, G., Chambers, D., Glisson, C., & Mittman, C. (2009). Implementation research in mental health services: An emerging science with conceptual, methodological, and training challenges. Administration and Policy in Mental Health and Mental Health Services Research, 36, 24–34.CrossRefGoogle Scholar
  68. Proctor, E. K., Powell, B. J., & McMillen, J. C. (2013). Implementation strategies: Recommendations for specifying and reporting. Implementation Science, 8, 1–11.CrossRefGoogle Scholar
  69. Resnicow, K., Baranowski, T., Ahluwalia, J., & Braithwaite, R. (1990). Cultural sensitivity in public health: Defined and demystified. Ethnicity and Disease, 9, 10–21.Google Scholar
  70. Ringwalt, C. L., Vincus, A., Ennett, S., Johnson, R., & Rohrbach, L. A. (2004). Reasons for teachers’ adaptation of substance use prevention curricula in schools with non-white student populations. Prevention Science, 5, 61–67.CrossRefGoogle Scholar
  71. Ringwalt, C. R., Vincus, A., Ennett, S. T., Hanley, S., Bowling, J. M., & Rohrbach, L. A. (2009). The prevalence of evidence-based substance use prevention curricula in U.S. middle schools in 2005. Prevention Science, 10, 33–40.CrossRefGoogle Scholar
  72. Russ, S. J., Sevdalis, N., Moorthy, K., et al. (2014). A qualitative evaluation of the barriers and facilitators toward implementation of the WHO surgical safety checklist across hospitals in England: Lessons from the “surgical checklist implementation project”. Annual Surgery, 261, 81–91.CrossRefGoogle Scholar
  73. Sholomskas, D. E., Syracuse-Siewert, G., Rounsaville, B. J., Ball, S. A., Nuro, K. F., & Carroll, K. M. (2005). We don’t train in vain: A dissemination trial of three strategies of training clinicians in cognitive-behavioral therapy. Journal of Consulting and Clinical Psychology, 73, 106–115.CrossRefGoogle Scholar
  74. St. Peter Pipkin, C., Vollmer, T. R., & Sloman, K. N. (2010). Effects of treatment integrity failures during differential reinforcement of alternative behavior: A translational model. Journal of Applied Behavior Analysis, 43, 47–70.CrossRefGoogle Scholar
  75. Stahmer, A. C., Reed, S., & Lee, E. (2015). Training teachers to use evidence-based practices for autism: Examining procedural implementation fidelity. Psychology in the School, 52, 181–195.CrossRefGoogle Scholar
  76. Stetler, C. B., Legro, M. W., Rycroft-Malone, J., Bowman, C., Curran, G., & Guihan, M. (2006). Role of “external facilitation” in implementation of research findings: A qualitative evaluation of facilitation experiences in the Veterans Health Administration. Implementation Science, 1, 23.CrossRefGoogle Scholar
  77. Urbach, J., Moore, B. A., Klinger, J. K., Galman, S., Haager, D., Brownell, M. T., & Dingle, M. (2015). “That’s my job”: Comparing the beliefs of more and less accomplished special educators related to their roles and responsibilities. Teacher Education and Special Education, 38, 323–336.CrossRefGoogle Scholar
  78. Vescio, V., Ross, D., & Adams, A. (2008). A review of research on the impact of professional learning communities on teaching practice and student learning. Teaching and Teacher Education, 24, 80e91.CrossRefGoogle Scholar
  79. Walker, V. L., Chung, Y.C., & Bonnet, L. K. (2017). Function-based intervention in inclusive school settings: A meta-analysis. Journal of Positive Behavior Interventions.Google Scholar
  80. Waltz, T. J., Powell, B. J., Matthieu, M. M., Damschroder, L. J., Chinman, M. J., Smith, J. L., et al. (2015). Use of concept mapping to characterize relationships among implementation strategies and assess their feasibility and importance: Results from the Expert Recommendations for Implementing Change (ERIC) study. Implementation Science, 10, 1–8.CrossRefGoogle Scholar
  81. Weiner, B. J., Lewis, C. C., Stanick, C. F., Powell, B. J., Dorsey, C. N., Clary, A. S., Boynton, M. H., & Halko, H. (2017). Psychometric assessment of three newly developed implementation outcome measures. Implementation Science, 12, 1–12.CrossRefGoogle Scholar
  82. Weisz, J., & Kazdin, A. (2017). Evidence-based psychotherapies for children and adolescents. Guilford.Google Scholar
  83. Wensing, M., & Grol, R. (2005). Methods to identify implementation problems. In R. Grol, M. Wensing, & M. Eccles (Eds.), Improving patient care: The implementation of change in clinical practice (pp. 109–120). Edinburgh: Elsevier.Google Scholar
  84. Wensing, M., Oxman, A., Baker, R., et al. (2011). Tailored implementation for chronic diseases (TICD): A project protocol. Implementation Science, 6, 1–8.CrossRefGoogle Scholar
  85. Williams, N. J. (2016). Multilevel mechanisms of implementation strategies in mental health: Integrating theory, research, and practice. Administration and Policy in Mental Health and Mental Health Services Research, 43, 783–798.CrossRefGoogle Scholar

Copyright information

© Society for Prevention Research 2019

Authors and Affiliations

  1. 1.University of Minnesota, Twin CitiesMinneapolisUSA
  2. 2.University of WashingtonSeattleUSA
  3. 3.Eastern Michigan UniversityYpsilantiUSA
  4. 4.Brown SchoolWashington University in St. LouisSt. LouisUSA

Personalised recommendations