Abstract
The purpose of this study was to examine common and unique factors influencing implementation process for two evidence-based interventions for children with autism spectrum disorder (ASD) in mental health and education service contexts. This study prospectively collected qualitative data from intervention developers and research staff on the implementation process within the context of two separate ASD intervention effectiveness trials. Results reveal common and unique factors influencing implementation in both study contexts. Implementation leadership and provider attitudes and motivation emerge as key influences on implementation across systems. These findings provide promising targets for modular implementation interventions that can be leveraged within growing, large-scale translation efforts in usual care.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Aarons, G. A., Ehrhart, M. G., Farahnak, L. R., & Hurlburt, M. S. (2015). Leadership and Organizational Change for Implementation (LOCI): A randomized mixed method pilot study of a leadership and organization development intervention for evidence-based practice implementation. Implementation Science, 10(1), 1–11. https://doi.org/10.1186/s13012-014-0192-y.
Aarons, G. A., Ehrhart, M. G., Farahnak, L. R., & Sklar, M. (2014). Aligning leadership across systems and organizations to develop a strategic climate for evidence-based practice implementation. Annual Review of Public Health, 35, 255–274.
Aarons, G. A., Green, A. E., Trott, E., Willging, C. E., Torres, E. M., Ehrhart, M. G., & Roesch, S. C. (2016). The roles of system and organizational leadership in system-wide evidence-based intervention sustainment: A mixed-method study. Administration and Policy in Mental Health and Mental Health Services Research, 43, 991–1008. https://doi.org/10.1007/s10488-016-0751-4.
Aarons, G. A., Hurlburt, M., & Horwitz, S. M. (2011). Advancing a conceptual model of evidence-based practice implementation in public service sectors. Administration and Policy in Mental Health and Mental Health Services Research, 38, 4–23. https://doi.org/10.1007/s10488-010-0327-7.
Baio, J., Wiggins, L., Christensen, D. L., Maenner, M. J., Daniels, J., Warren, Z., et al. (2018). Prevalence of autism spectrum disorder among children aged 8 years - autism and developmental disabilities monitoring network, 11 Sites, United States, 2014. MMWR Surveillance Summaries, 67(6), 1–23. https://doi.org/10.15585/mmwr.ss6706a1.
Barnett, M., Stadnick, N., Saifan, D., Brookman-Frazee, L., Lau, A., & Regan, J. (2017). How intervention and implementation characteristics relate to community therapists’ attitudes toward evidence-based practices: A mixed methods study. Administration and Policy in Mental Health and Mental Health Services Research, 44(6), 824–837. https://doi.org/10.1007/s10488-017-0795-0.
Bass, B. M., & Avolio, B. J. (1990). The implications of transformational and transactional leadership for individual, team, and organizational development. In W. Pasmore & R. W. Woodman (Eds.), Research in Organizational Change and Development (pp. 231–272). Greenwich: JAI Press.
Berkowitz, S. (1997). Analyzing qualitative data. In User-friendly handbook for mixed method evaluations. http://www.ehr.nsf.gov/EHR/REC/pubs/NSF97-153/CHAP_3.HTM.
Brookman-Frazee, L., Baker-Ericzén, M., Stadnick, N., & Taylor, R. (2012). Parent perspectives on community mental health services for children with autism spectrum disorders. Journal of Child and Family Studies, 21, 533–544. https://doi.org/10.1007/s10826-011-9506-8.
Brookman-Frazee, L., Baker-Ericzén, M., Stahmer, A., Mandell, D., Haine, R. A., & Hough, R. L. (2009). Involvement of youths with autism spectrum disorders or intellectual disabilities in multiple public service systems. Journal of Mental Health Research in Intellectual Disabilities, 2, 201–219. https://doi.org/10.1080/19315860902741542.
Brookman-Frazee, L., & Drahota, A. (2010). An individualized mental health intervention for children with autism spectrum disorders (AIM HI): A model to address challenging behaviors in children with ASD—A therapist manual. San Diego: University of California.
Brookman-Frazee, L., Drahota, A., Stadnick, N., & Palinkas, L. A. (2012). Therapist perspectives on community mental health services for children with autism spectrum disorders. Administration and Policy in Mental Health and Mental Health Services Research, 39, 365–373. https://doi.org/10.1007/s10488-011-0355-y.
Brookman-Frazee., L. & Stahmer, A. C. (2018). Effectiveness of a multi-level implementation strategy for ASD interventions: Study protocol for two linked cluster randomized trials. Implementation Science, 13(1), 66.
Brookman-Frazee, L., Stahmer, A., Stadnick, N., Chlebowski, C., Herschell, A., & Garland, A. F. (2016). Characterizing the use of research-community partnerships in studies of evidence-based interventions in children’s community services. Administration and Policy in Mental Health and Mental Health Services Research, 43, 93–104. https://doi.org/10.1007/s10488-014-0622-9.
Brookman-Frazee, L., Taylor, R., & Garland, A. F. (2010). Characterizing community-based mental health services for children with autism spectrum disorders and disruptive behavior problems. Journal of Autism and Developmental Disorders, 40, 1188–1201. https://doi.org/10.1007/s10803-010-0976-0.
Chlebowski, C., Magana, S., Wright, B., & Brookman-Frazee, L. (2018). Implementing an intervention to address challenging behaviors for Autism Spectrum Disorder in publicly- funded mental health services: Therapist and parent perceptions of delivery with Latinx families. Cultural Diversity and Ethnic Minority Psychology, 24(4), 552–563.
Curran, G. M., Bauer, M., Mittman, B., Pyne, J. M., & Stetler, C. (2012). Effectiveness-implementation hybrid designs: Combining elements of clinical effectiveness and implementation research to enhance public health impact. Medical Care, 50, 217–226. https://doi.org/10.1097/MLR.0b013e3182408812.
Dingfelder, H. E., & Mandell, D. S. (2011). Bridging the research-to-practice gap in autism intervention: An application of diffusion of innovation theory. Journal of Autism and Developmental Disorders, 41, 597–609. https://doi.org/10.1007/s10803-010-1081-0.
Drahota, A., Meza, R. D., Brikho, B., Naaf, M., Estabillo, J. A., Gomez, E. D.,.. . Aarons, G. A. (2016). Community-Academic partnerships: A systematic review of the state of the literature and recommendations for future research. The Milbank Quarterly, 94, 163–214. https://doi.org/10.1111/1468-0009.12184.
Drahota, A., Stadnick, N., & Brookman-Frazee, L. (2014). Therapist perspectives on training in a package of evidence-based practice strategies for children with autism spectrum disorders served in community mental health clinics. Administration and Policy in Mental Health and Mental Health Services Research, 41(1), 114–125. https://doi.org/10.1007/s10488-012-0441-9.
Dyson, M., Chlebowski, C., Wright, B., & Brookman-Frazee, L. (2017). How do certified and uncertified therapists differ in their perceptions of a mental health intervention for ASD? Evidence-Based Practice in Child & Adolescent Mental Health, 2, 179–194.
Dyson, M. W., Chlebowski, C., & Brookman-Frazee, L. (2019). Therapists’ adaptations to an intervention to reduce challenging behaviors in children with autism spectrum disorder in publicly funded mental health services. Journal of Autism and Developmental Disorders, 49(3), 924–934. https://doi.org/10.1007/s10803-018-3795-3.
Eaves, L. C., & Ho, H. H. (2008). Young adult outcome of autism spectrum disorders. Journal of Autism and Developmental Disorders, 38, 739–747. https://doi.org/10.1007/s10803-007-0441-x.
Edmondson, A. C. (2003). Speaking up in the operating room: How team leaders promote learning in interdisciplinary action teams. Journal of Management Studies, 40, 1419–1452. https://doi.org/10.1111/1467-6486.00386.
Finley, E. P., Huynh, A. K., Farmer, M. M., Bean-Mayberry, B., Moin, T., Oishi, S. M., et al. (2018). Periodic reflections: A method of guided discussions for documenting implementation phenomena. BMC Medical Research Methodology, 18(1), 153. https://doi.org/10.1186/s12874-018-0610-y.
Forman, S. G., Shapiro, E. S., Codding, R. S., Gonzales, J. E., Reddy, L. A., Rosenfield, S. A.,.. . Stoiber, K. C. (2013). Implementation science and school psychology. School Psychology Quarterly, 28, 77–100. https://doi.org/10.1037/spq0000019.
Garland, A. F., Hurlburt, M. S., Brookman-Frazee, L., Taylor, R. M., & Accurso, E. C. (2010). Methodological challenges of characterizing usual care psychotherapeutic practice. Administration and Policy in Mental Health and Mental Health Services Research, 37, 208–220. https://doi.org/10.1007/s10488-009-0237-8.
Green, A. E., Albanese, B. J., Shapiro, N. M., & Aarons, G. A. (2014). The roles of individual and organizational factors in burnout among community-based mental health service providers. Psychological Services, 11, 41–49. https://doi.org/10.1037/a0035299.
Klein, K. J., Conn, A. B., & Sorra, J. S. (2001). Implementing computerized technology: An organizational analysis. Journal of Applied Psychology, 86(5), 811.
Leigh, J. P., & Du, J. (2015). Brief report: Forecasting the economic burden of autism in 2015 and 2025 in the United States. Journal of Autism and Developmental Disorders, 45, 4135–4139. https://doi.org/10.1007/s10803-015-2521-7.
Mancini, J. A., & Marek, L. I. (2004). Sustaining community-based programs for families: Conceptualization and measurement. Family Relations, 53, 339–347. https://doi.org/10.1111/j.0197-6664.2004.00040.x.
Mandell, D. S., Walrath, C. M., Manteuffel, B., Sgro, G., & Pinto-Martin, J. (2005). Characteristics of children with autistic spectrum disorders served in comprehensive community-based mental health settings. Journal of Autism and Developmental Disorders, 35, 313–321. https://doi.org/10.1007/s10803-005-3296-z.
Moullin, J. C., Dickson, K. S., Stadnick, N. A., Rabin, B., & Aarons, G. A. (2019). Systematic review of the Exploration, Preparation, Implementation, Sustainment, (EPIS) framework. Implementation Science, 14(1), 1. https://doi.org/10.1186/s13012-018-0842-6.
National Autism Center. (2015). National Standards Project, Phase 2, from http://www.nationalautismcenter.org/national-standards-project/phase-2/.
NVivo qualitative data analysis Software (Version 10) [Computer software]. (2012). Retrieved from http://www.qsrinternational.com/.
Orsmond, G. I., Shattuck, P. T., Cooper, B. P., Sterzing, P. R., & Anderson, K. A. (2013). Social participation among young adults with an autism spectrum disorder. Journal of Autism and Developmental Disorders, 43, 2710–2719. https://doi.org/10.1007/s10803-013-1833-8.
Palinkas, L. A., Schoenwald, S. K., Hoagwood, K., Landsverk, J., Chorpita, B. F., & Weisz, J. R. (2008). An ethnographic study of implementation of evidence-based treatments in child mental health: First steps. Psychiatric Services, 59, 738–746. https://doi.org/10.1176/appi.ps.59.7.738.
Powell, B. J., McMillen, J. C., Proctor, E. K., Carpenter, C. R., Griffey, R. T., Bunger, A. C.,.. . York, J. L. (2012). A compilation of strategies for implementing clinical innovations in health and mental health. Medical Care Research and Review, 69, 123–157. https://doi.org/10.1177/1077558711430690.
Proctor, E. K., Landsverk, J., Aarons, G., Chambers, D., Glisson, C., & Mittman, B. (2009). Implementation research in mental health services: An emerging science with conceptual, methodological, and training challenges. Administration and Policy in Mental Health and Mental Health Services Research, 36, 24–34. https://doi.org/10.1007/s10488-008-0197-4.
Proctor, E. K., Silmere, H., Raghavan, R., Hovmand, P., Aarons, G., Bunger, A.,.. . Hensley, M. (2011). Outcomes for implementation research: Conceptual distinctions, measurement challenges, and research agenda. Administration and Policy in Mental Health and Mental Health Services Research, 38, 65–76. https://doi.org/10.1007/s10488-010-0319-7.
Roux, A. M., Shattuck, P. T., Cooper, B. P., Anderson, K. A., Wagner, M., & Narendorf, S. C. (2013). Postsecondary employment experiences among young adults with an autism spectrum disorder. Journal of the American Academy of Child and Adolescent Psychiatry, 52, 931–939. https://doi.org/10.1016/j.jaac.2013.05.019.
Seale, C., & Silverman, D. (1997). Ensuring rigour in qualitative research. The European Journal of Public Health, 7(4), 379–384.
Shattuck, P. T., Narendorf, S. C., Cooper, B., Sterzing, P. R., Wagner, M., & Taylor, J. L. (2012). Postsecondary education and employment among youth with an autism spectrum disorder. Pediatrics, 129, 1042–1049. https://doi.org/10.1542/peds.2011-2864.
Simonoff, E., Pickles, A., Charman, T., Chandler, S., Loucas, T., & Baird, G. (2008). Psychiatric disorders in children with autism spectrum disorders: Prevalence, comorbidity, and associated factors in a population-derived sample. Journal of the American Academy of Child and Adolescent Psychiatry, 47, 921–929. https://doi.org/10.1097/CHI.0b013e318179964f.
Stahmer, A. C., & Aarons, G. A. (2009). Attitudes toward adoption of evidence-based practices: A comparison of autism early intervention providers and children’s mental health providers. Psychological Services, 6(3), 223–234. https://doi.org/10.1037/a0010738.
Stahmer, A. C., Collings, N. M., & Palinkas, L. A. (2005). Early intervention practices for children with autism: Descriptions from community providers. Focus on Autism and Other Developmental Disabilities, 20, 66–79. https://doi.org/10.1177/10883576050200020301.
Stahmer, A. C., Suhrheinrich, J., Reed, S., & Schreibman, L. (2012). What works for you? Using teacher feedback to inform adaptations of pivotal response training for classroom use. Autism Research and Treatment, 2012, 709861. https://doi.org/10.1155/2012/709861.
Stahmer, A. C., Suhrheinrich, J., & Reith, S. (2016). A pilot examination of the adapted protocol for classroom pivotal response teaching. Journal of the American Academy of Special Education Professionals, 119, 139.
Stogdill, R. M. (1974). Handbook of leadership: A survey of theory and research. New York: The Free Press.
Suhrheinrich, J., Stahmer, A. C., Reed, S., Schreibman, L., Reisinger, E., & Mandell, D. S. (2013). Implementation challenges in translating pivotal response training into community settings. Journal of Autism and Developmental Disorders, 43, 2970–2976. https://doi.org/10.1007/s10803-013-1826-7.
Wong, C., Odom, S. L., Hume, K. A., Cox, A. W., Fettig, A., Kucharczyk, S.,.. . Schultz, T. R. (2015). Evidence-based practices for children, youth, and young adults with autism spectrum disorder: A comprehensive review. Journal of Autism and Developmental Disorders, 45, 1951–1966. https://doi.org/10.1007/s10803-014-2351-z.
Wood, J. J., McLeod, B. D., Klebanoff, S., & Brookman-Frazee, L. (2015). Toward the implementation of evidence-based interventions for youth with autism spectrum disorders in schools and community agencies. Behavior Therapy, 46, 83–95. https://doi.org/10.1016/j.beth.2014.07.003.
Funding
This study was funded by NIMH Grants R01MH111950, R01MH111981, R01MH094317, and IES Grant R324B070027.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
All authors declare that they have no conflict of interest.
Ethical Approval
This article does not contain any studies with human participants performed by any of the authors.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Brookman-Frazee, L., Chlebowski, C., Suhrheinrich, J. et al. Characterizing Shared and Unique Implementation Influences in Two Community Services Systems for Autism: Applying the EPIS Framework to Two Large-Scale Autism Intervention Community Effectiveness Trials. Adm Policy Ment Health 47, 176–187 (2020). https://doi.org/10.1007/s10488-019-00931-4
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10488-019-00931-4