Abstract
This study examined patterns of evidence-based treatment (EBT) delivery following a county-wide EBT reform initiative. Data were gathered from 60 youth and their 21 providers, who were instructed to deliver therapy as they normally would under the EBT initiative. Results showed limited applicability of county-supported EBTs to this service sample, and that most youth did not receive traditional delivery of EBTs. Findings suggest that it may be unrealistic to expect providers to deliver EBTs with fidelity with all clients, and that EBT implementation may be best thought of as a strategy for improving mental health services rather than a goal.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
The LACDMH PEI Plan is intended to promote effective mental health services for underserved cultural populations, individuals experiencing onset of serious psychiatric illness, children and youth in stressed families, trauma-exposed individuals, children and youth at risk of school failure, and children and youth at risk of or experiencing juvenile justice involvement. Given the aims of this study, we have focused on aspects of the PEI Plan related to early intervention for children and youth.
The EBTs supported by the LACDMH PEI Plan were selected through a feedback loop between community stakeholders (e.g., caregivers, community forum participants) and decision-making bodies (e.g., LACDMH staff, advisory groups). Although MAP—an evidence-informed services support system that offers a collection of “how to” guides for delivering common EBT practices to address a variety of mental health concerns and a searchable database that can identify the EBT practices that may be appropriate for specific clients—has been supported by the LACDMH PEI Plan since July 2010, MAP was not included in analyses because: (a) the randomized effectiveness trial from which data were gathered was designed to exclude providers who were trained in MAP; and (b) this decision enhances the generalizability of our findings to mental health service systems promoting the use of EBTs—as MAP is not an EBT per se.
References
Alexander, J. F., & Robbins, M. S. (2011). Functional family therapy. In A. D. Kidman, T. H. Ollendick & R. C. Murrihy (Eds.), Clinical handbook of assessing and treating conduct problems in youth (pp. 245–271). New York: Springer.
Asarnow, J. R., Jaycox, L. H., Duan, N., LaBorde, A. P., Rea, M. M., Murray, P., … Wells, K. B. (2005). Effectiveness of a quality improvement intervention for adolescent depression in primary care clinics: A randomized controlled trial. JAMA, 293(3), 311–319. https://doi.org/10.1001/jama.293.3.311.
Beidas, R. S., Koerner, K., Weingardt, K. R., & Kendall, P. C. (2011). Training research: Practical recommendations for maximum impact. Administration and Policy in Mental Health and Mental Health Services Research, 38(4), 223–237. https://doi.org/10.1007/s10488-011-0338-z.
Brookman-Frazee, L., Stadnick, N., Roesch, S., Regan, J., Barnett, M., Bando, L., … Lau, A. (2016). Measuring sustainment of multiple practices fiscally mandated in children’s mental health services. Administration and Policy in Mental Health and Mental Health Services Research, 43(6), 1009–1022. https://doi.org/10.1007/s10488-016-0731-8.
California Department of Mental Health (2016). Mental Health Services Act expenditure report: Fiscal year 2016–2017. Retrieved from http://www.dhcs.ca.gov/formsandpubs/Documents/Legislative%20Reports/Mental%20Health/MHSAExpend_Report_Jan2016.pdf.
Chen, H., Reeves, J. H., Fincham, J. E., Kennedy, W. K., & Martin, B. C. (2006). Off-label use of antidepressant, anticonvulsant, and antipsychotic medications among Georgia Medicaid enrollees in 2001. The Journal of Clinical Psychiatry, 67(6), 972–982. Retrieved from http://www.psychiatrist.com/JCP/article/Pages/2006/v67n06/v67n0615.aspx.
Chorpita, B. F., Bernstein, A., & Daleiden, E. L. (2008). Driving with roadmaps and dashboards: Using information resources to structure the decision models in service organizations. Administration and Policy in Mental Health and Mental Health Services Research, 35(1–2), 114–123. https://doi.org/10.1007/s10488-007-0151-x.
Chorpita, B. F., Bernstein, A., & Daleiden, E. L. (2011). Empirically guided coordination of multiple evidence-based treatments: An illustration of relevance mapping in children’s mental health services. Journal of Consulting and Clinical Psychology, 79(4), 470–480. https://doi.org/10.1037/a0023982.
Chorpita, B. F., Bernstein, A. D., Becker, K. D., Ebesutani, C., Park, A. L., Brown, T., Tsai, K., Moskowitz, A. L., & Daleiden, E. L. (2016). Measurement feedback and beyond: The place for evidence in clinical decision support. In J. Persons (Chair), Monitoring progress in psychotherapy: Why and how. Symposium presented at the 50th Annual Convention of the Association for Behavioral and Cognitive Therapies. New York, NY.
Chorpita, B. F., & Daleiden, E. L. (2014). Structuring the collaboration of science and service in pursuit of a shared vision. Journal of Clinical Child and Adolescent Psychology, 43(2), 323–338. https://doi.org/10.1080/15374416.2013.828297.
Chorpita, B. F., Daleiden, E. L., Park, A. L., Ward, A. M., Levy, M. C., Cromley, T., … Krull, J. L. (2017). Child STEPs in California: A cluster randomized effectiveness trial comparing modular treatment with community implemented treatment for youth with anxiety, depression, conduct problems, or traumatic stress. Journal of Consulting and Clinical Psychology, 85(1), 13–25. https://doi.org/10.1037/cco0000133.
Chorpita, B. F., Moffitt, C. E., & Gray, J. (2005). Psychometric properties of the revised child anxiety and depression scale in a clinical sample. Behaviour Research and Therapy, 43(3), 309–322. https://doi.org/10.1016/j.brat.2004.02.004.
Cohen, J. A., & Mannarino, A. P. (1996). A treatment outcome study for sexually abused preschool children: Initial findings. Journal of the American Academy of Child and Adolescent Psychiatry, 35(1), 42–50. https://doi.org/10.1097/00004583-199601000-00011.
Cooper, J. L., & Aratani, Y. (2009). The status of states’ policies to support evidence-based practices in children’s mental health. Psychiatric Services, 60(12), 1672–1675. https://doi.org/10.1176/ps.2009.60.12.1672.
Daleiden, E. L., Chorpita, B. F., Donkervoet, C., Arensdorf, A. M., Brogan, M., & Hamilton, J. D. (2006). Getting better at getting them better: Health outcomes and evidence-based practice within a system of care. Journal of the American Academy of Child and Adolescent Psychiatry, 45(6), 749–756. https://doi.org/10.1097/01.chi.0000215154.07142.63.
Damschroder, L. J., Aron, D. C., Keith, R. E., Kirsh, S. R., Alexander, J. A., & Lowery, J. C. (2009). Fostering implementation of health services research findings into practice: A consolidated framework for advancing implementation science. Implementation Science, 4, 50–65. https://doi.org/10.1186/1748-5908-4-50.
Ebesutani, C., Bernstein, A., Nakamura, B. J., Chorpita, B. F., & Weisz, J. R. (2010). A psychometric analysis of the revised child anxiety and depression scale—parent version in a clinical sample. Journal of Abnormal Child Psychology, 38(2), 249–260. https://doi.org/10.1007/s10802-009-9363-8.
Ebesutani, C., Chorpita, B. F., Higa-McMillan, C. K., Nakamura, B. J., Regan, J., & Lynch, R. E. (2011). A psychometric analysis of the revised child anxiety and depression scales—parent version in a school sample. Journal of Abnormal Child Psychology, 39(2), 173–185. https://doi.org/10.1007/s10802-010-9460-8.
Ebesutani, C., Tottenham, N., & Chorpita, B. (2015). The revised child anxiety and depression scale-parent version: Extended applicability and validity for use with younger youth and children with histories of early-life caregiver neglect. Journal of Psychopathology and Behavioral Assessment, 37(4), 705–718. https://doi.org/10.1007/s10862-015-9494-x.
Ehrenreich-May, J., Southam-Gerow, M. A., Hourigan, S. E., Wright, L. R., Pincus, D. B., & Weisz, J. R. (2011). Characteristics of anxious and depressed youth seen in two different clinical contexts. Administration and Policy in Mental Health and Mental Health Services Research, 38(5), 398–411. https://doi.org/10.1007/s10488-010-0328-6.
Eyberg, S. M., Boggs, S. R., & Algina, J. (1995). Parent-child interaction therapy: A psychosocial model for the treatment of young children with conduct problem behavior and their families. Psychopharmacology Bulletin, 31(1), 83–91.
Garland, A. F., Brookman-Frazee, L., Hurlburt, M. S., Accurso, E. C., Zoffness, R. J., Haine-Schlagel, R., & Ganger, W. (2010). Mental health care for children with disruptive behavior problems: A view inside therapists’ offices. Psychiatric Services, 61(8), 788–795. https://doi.org/10.1176/ps.2010.61.8.788.
Glisson, C., Schoenwald, S. K., Kelleher, K., Landsverk, J., Hoagwood, K. E., Mayberg, S., & Green, P. (2008). Therapist turnover and new program sustainability in mental health clinics as a function of organizational culture, climate, and service structure. Administration and Policy in Mental Health and Mental Health Services Research, 35(1–2), 124–133. https://doi.org/10.1007/s10488-007-0152-9.
Goodman, R. (2001). Psychometric properties of the strengths and difficulties questionnaire. Journal of the American Academy of Child and Adolescent Psychiatry, 40(11), 1337–1345. https://doi.org/10.1097/00004583-200111000-00015.
Greenberg, M. T., Mihalic, S. F., & Kusché, C. A. (1998). Promoting alternative thinking strategies (PATHS). Center for the Study and Prevention of Violence, Institute of Behavioral Science, University of Colorado at Boulder.
Guan, K., Levy, M. C., Kim, R. E., Brown, T. E., Reding, M. E., Rith-Najarian, L., … Chorpita, B. F. (2017). Managing in-session “surprises:” Provider responses to emergent life events during evidence-based treatment implementation. Administration and Policy in Mental Health and Mental Health Services Research, 44(2), 164–176. https://doi.org/10.1007/s10488-015-0692-3.
Gyani, A., Shafran, R., Myles, P., & Rose, S. (2014). The gap between science and practice: How therapists make their clinical decisions. Behavior Therapy, 45(2), 199–211. https://doi.org/10.1016/j.beth.2013.10.004.
Hawley, K. M., & Weisz, J. R. (2002). Increasing the relevance of evidence-based treatment review to practitioners and consumers. Clinical Psychology: Science and Practice, 9(2), 225–230. https://doi.org/10.1093/clipsy.9.2.225.
Huey, S. J. Jr., & Polo, A. J. (2008). Evidence-based psychosocial treatments for ethnic minority youth. Journal of Clinical Child and Adolescent Psychology, 37(1), 262–301. https://doi.org/10.1080/15374410701820174.
Lewinsohn, P. M., Clarke, G. N., Hops, H., & Andrews, J. (1990). Cognitive-behavioral treatment for depressed adolescents. Behavior Therapy, 21(4), 385–401. https://doi.org/10.1016/S0005-7894(05)80353-3.
Lieberman, A. F., Van Horn, P., & Ippen, C. G. (2005). Toward evidence-based treatment: Child-parent psychotherapy with preschoolers exposed to marital violence. Journal of the American Academy of Child and Adolescent Psychiatry, 44(12), 1241–1248. https://doi.org/10.1097/01.chi.0000181047.59702.58.
Los Angeles County Department of Mental Health (2011). Prevention and Early Intervention (PEI) evidence-based practices, promising practices, and community-defined evidence practices: Resource Guide 2.0. Retrieved from http://file.lacounty.gov/SDSInter/dmh/159575_EBPPPCDEResourceGuide2_0.pdf.
Los Angeles County Department of Mental Health (2013). Prevention and Early Intervention (PEI) evidence-based practices (EBP) outcome measures. Retrieved from http://file.lacounty.gov/SDSInter/dmh/201946_PEI_Outcome_Measures_REV._07-22-13.pdf.
Los Angeles County Department of Mental Health (2014). Mental Health Services Act: Three year program and expenditure plan: Fiscal years 2014–2015 through 2016–2017. Retrieved from http://file.lacounty.gov/SDSInter/dmh/211892_ThreeYearPlan_072414.pdf.
Los Angeles County Department of Mental Health (2016a). Countywide data report for PEI programs. Retrieved from http://file.lacounty.gov/SDSInter/dmh/1003274_CW-Report-March-31-2016.pdf.
Los Angeles County Department of Mental Health (2016b). Training protocols for Prevention and Early Intervention practices. Retrieved from http://file.lacounty.gov/SDSInter/dmh/201947_PEITrainingProtocolsrevised4-1-16.pdf.
McHugh, R. K., & Barlow, D. H. (2010). The dissemination and implementation of evidence-based psychological treatments: A review of current efforts. American Psychologist, 65(2), 73–84. https://doi.org/10.1037/a0018121.
Najavits, L. M., Gallop, R. J., & Weiss, R. D. (2006). Seeking Safety therapy for adolescent girls with PTSD and substance use disorder: A randomized controlled trial. The Journal of Behavioral Health Services and Research, 33(4), 453–463. https://doi.org/10.1007/s11414-006-9034-2.
National Advisory Mental Health Council Workgroup on Child and Adolescent Mental Health Intervention and Deployment. (2001). Blueprint for change: Research on child and adolescent mental health. Rockville, MD: U.S. Department of Health and Human Services, Public Health Service, National Institutes of Health.
Palinkas, L. A., Weisz, J. R., Chorpita, B. F., Levine, B., Garland, A. F., Hoagwood, K. E., & Landsverk, J. (2013). Continued use of evidence-based treatments after a randomized controlled effectiveness trial: A qualitative study. Psychiatric Services, 64(11), 1110–1118. https://doi.org/10.1176/appi.ps.004682012.
Park, A. L., Moskowitz, A. L., & Chorpita, B. F. (2016). Community-based providers’ selection of practices for children and adolescents with comorbid mental health problems. Journal of Clinical Child and Adolescent Psychology. https://doi.org/10.1080/15374416.2016.1188706.
Powell, B. J., Beidas, R. S., Rubin, R. M., Stewart, R. E., Wolk, C. B., Matlin, S. L., … Mandell, D. S. (2016). Applying the policy ecology framework to Philadelphia’s behavioral health transformation efforts. Administration and Policy in Mental Health and Mental Health Services Research, 43(6), 909–926. https://doi.org/10.1007/s10488-016-0733-6.
Radley, D. C., Finkelstein, S. N., & Stafford, R. S. (2006). Off-label prescribing among office-based physicians. Archives of Internal Medicine, 166(9), 1021–1026. https://doi.org/10.1001/archinte.166.9.1021.
Raghavan, R., Bright, C. L., & Shadoin, A. L. (2008). Toward a policy ecology of implementation of evidence-based practices in public mental health settings. Implementation Science, 3(1), 26–35. https://doi.org/10.1186/1748-5908-3-26.
Reding, M. E., Chorpita, B. F., Lau, A. S., & Innes-Gomberg, D. (2014). Providers’ attitudes toward evidence-based practices: Is it just about providers, or do practices matter, too? Administration and Policy in Mental Health and Mental Health Services Research, 41(6), 767–776. https://doi.org/10.1007/s10488-013-0525-1.
Reding, M. E. J., Guan, K., Regan, J., Palinkas, L. A., Lau, A. S., & Chorpita, B. F. (2017). Implementation in a changing landscape: Provider experiences during rapid scaling of use of evidence-based treatments. Cognitive and Behavioral Practice. https://doi.org/10.1016/j.cbpra.2017.05.005.
Rotheram-Borus, M. J., Swendeman, D., & Chorpita, B. F. (2012). Disruptive innovations for designing and diffusing evidence-based interventions. American Psychologist, 67(6), 463–476. https://doi.org/10.1037/a0028180.
Sanders, M. R., Markie-Dadds, C., Turner, K. M., & Ralph, A. (2004). Using the triple P system of intervention to prevent behavioural problems in children and adolescents. In P. Barrett & T. H. Ollendick (Eds.), Handbook of interventions that work with children and adolescents: Prevention and treatment (pp. 489–516). Malden: Wiley Interscience.
Shah, S. S., Hall, M., Goodman, D. M., Feuer, P., Sharma, V., Fargason, C., … Levin, J. E. (2007). Off-label drug use in hospitalized children. Archives of Pediatrics and Adolescent Medicine, 161(3), 282–290. https://doi.org/10.1001/archpedi.161.3.282.
Southam-Gerow, M. A., Chorpita, B. F., Miller, L. M., & Gleacher, A. A. (2008). Are children with anxiety disorders privately referred to a university clinic like those referred from the public mental health system? Administration and Policy in Mental Health and Mental Health Services Research, 35(3), 168–180. https://doi.org/10.1007/s10488-007-0154-7.
Southam-Gerow, M. A., Daleiden, E. L., Chorpita, B. F., Bae, C., Mitchell, C., Faye, M., & Alba, M. (2014). MAPping Los Angeles County: Taking an evidence-informed model of mental health care to scale. Journal of Clinical Child & Adolescent Psychology, 43(2), 190–200. https://doi.org/10.1080/15374416.2013.833098.
Stein, B. D., Jaycox, L. H., Kataoka, S. H., Wong, M., Tu, W., Elliott, M. N., & Fink, A. (2003). A mental health intervention for schoolchildren exposed to violence: A randomized controlled trial. JAMA, 290(5), 603–611. https://doi.org/10.1001/jama.290.5.603.
Steinberg, A. M., Brymer, M. J., Decker, K. B., & Pynoos, R. S. (2004). The University of California at Los Angeles post-traumatic stress disorder reaction index. Current Psychiatry Reports, 6(2), 96–100. https://doi.org/10.1007/s11920-004-0048-2.
Ward, A. M., Regan, J., Chorpita, B. F., Starace, N., Rodriguez, A., & Okamura, K. … Research Network on Youth Mental Health. (2013). Tracking evidence based practice with youth: Validity of the MATCH and standard manual consultation records. Journal of Clinical Child and Adolescent Psychology, 42(1), 44–55. https://doi.org/10.1080/15374416.2012.700505.
Webster-Stratton, C. (2006). The incredible years: A trouble-shooting guide for parents of children aged 2–8 years. Seattle, WA: Incredible Years.
Weisz, J. R., & Chorpita, B. F. (2012). “Mod squad” for youth psychotherapy: Restructuring evidence-based treatment for clinical practice. In P. C. Kendall (Ed.), Child and adolescent therapy: Cognitive-behavioral procedures (4th edn., pp. 379–397, Chapter xiv, 538 Pages). New York, NY: Guilford Press.
Weisz, J. R., Chorpita, B. F., Frye, A., Ng, M. Y., Lau, N., Bearman, S. K., … Hoagwood, K. E. (2011). Youth Top Problems: Using idiographic, consumer-guided assessment to identify treatment needs and to track change during psychotherapy. Journal of Consulting and Clinical Psychology, 79(3), 369. https://doi.org/10.1037/a0023307.
Weisz, J. R., Chorpita, B. F., Palinkas, L. A., Schoenwald, S. K., Miranda, J., Bearman, S. K., … Gray, J. (2012). Testing standard and modular designs for psychotherapy treating depression, anxiety, and conduct problems in youth: A randomized effectiveness trial. Archives of General Psychiatry, 69(3), 274–282. https://doi.org/10.1001/archgenpsychiatry.2011.147.
Wittich, C. M., Burkle, C. M., & Lanier, W. L. (2012). Ten common questions (and their answers) about off-label drug use. Mayo Clinic Proceedings, 87(10), 982–990. https://doi.org/10.1016/j.mayocp.2012.04.017.
Zima, B. T., Hurlburt, M. S., Knapp, P., Ladd, H., Tang, L., Duan, N., … Wells, K. B. (2005). Quality of publicly-funded outpatient specialty mental health care for common childhood psychiatric disorders in California. Journal of the American Academy of Child and Adolescent Psychiatry, 44(2), 130–144. https://doi.org/10.1097/00004583-200502000-00005.
Funding
This work was supported by the John D. and Catherine T. MacArthur Foundation, who did not shape the design or conduct of the study; collection, management, analysis, or interpretation of the data, or preparation, review, or approval of the manuscript.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Ethical Approval
Approval by the institutional review board of the University of California, Los Angeles, as well as those institutional review boards of participating service agencies that requested independent reviews, was obtained for this research. All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. This article does not contain any studies with animals performed by any of the authors.
Informed Consent
Informed consent was obtained from all individual participants included in the study.
Rights and permissions
About this article
Cite this article
Park, A.L., Tsai, K.H., Guan, K. et al. Unintended Consequences of Evidence-Based Treatment Policy Reform: Is Implementation the Goal or the Strategy for Higher Quality Care?. Adm Policy Ment Health 45, 649–660 (2018). https://doi.org/10.1007/s10488-018-0853-2
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10488-018-0853-2