Interactions Between Clinician and Organizational Characteristics to Predict Cognitive-Behavioral and Psychodynamic Therapy Use

  • Emily M. Becker-HaimesEmail author
  • Nathaniel J. Williams
  • Kelsie H. Okamura
  • Rinad S. Beidas
Original Article


Conceptual models of implementation posit contextual factors and their associations with evidence-based practice (EBP) use at multiple levels and suggest these factors exhibit complex cross-level interactions. Little empirical work has examined these interactions, which is critical to advancing causal implementation theory and optimizing implementation strategy design. Mixed effects regression examined cross-level interactions between clinician (knowledge, attitudes) and organizational characteristics (culture, climate) to predict cognitive-behavioral and psychodynamic therapy use with youth (N = 247 clinicians across 28 agencies). Results indicated several interactions, highlighting the importance of attending to interactions between variables at multiple levels to advance multilevel implementation theory and strategies.


Implementation science Youth Cognitive-behavioral therapy Psychodynamic therapy 



Funding for this research project was supported by the National Institute of Mental Health (NIMH) Grant K23MH099179 to Dr. Beidas.

Compliance with Ethical Standards

Conflict of interest

Dr. Beidas receives royalties from Oxford University Press and has served as a consultant for Merck, Dohme, & Sharp. All other authors have nothing to disclose.


  1. Aarons, G. A. (2004). Mental health provider attitudes toward adoption of evidence-based practice: The Evidence-Based Practice Attitude Scale (EBPAS). Mental Health Services Research, 6(2), 61–74. Scholar
  2. Aarons, G. A. (2005). Measuring provider attitudes toward evidence-based practice: Consideration of organizational context and individual differences. Child and Adolescent Psychiatric Clinics of North America, 14(2), 255–271. Scholar
  3. Aarons, G. A., Ehrhart, M. G., Farahnak, L. R., & Hurlburt, M. S. (2015). Leadership and organizational change for implementation (LOCI): A randomized mixed method pilot study of a leadership and organization development intervention for evidence-based practice implementation. Implementation Science, 10(1), 11. Scholar
  4. Aarons, G. A., Glisson, C., Green, P. D., Hoagwood, K., Kelleher, K. J., & Landsverk, J. A. (2012). The organizational social context of mental health services and clinician attitudes toward evidence-based practice: A United States national study. Implementation Science, 7(1), 56. Scholar
  5. Aarons, G. A., Glisson, C., Hoagwood, K., Kelleher, K., Landsverk, J., & Cafri, G. (2010). Psychometric properties and US National norms of the Evidence-Based Practice Attitude Scale (EBPAS). Psychological Assessment, 22(2), 356. Scholar
  6. Abbass, A. A., Rabung, S., Leichsenring, F., Refseth, J. S., & Midgley, N. (2013). Psychodynamic psychotherapy for children and adolescents: A meta-analysis of short-term psychodynamic models. Journal of the American Academy of Child and Adolescent Psychiatry, 52(8), 863–875. Scholar
  7. Becker, E. M., Smith, A. M., & Jensen-Doss, A. (2013). Who’s using treatment manuals? A national survey of practicing therapists. Behaviour Research and Therapy, 51(10), 706–710. Scholar
  8. Beidas, R. S., Aarons, G., Barg, F., Evans, A., Hadley, T., Hoagwood, K., et al. (2013). Policy to implementation: Evidence-based practice in community mental health - study protocol. Implementation Science. Scholar
  9. Beidas, R. S., Marcus, S., Aarons, G. A., Hoagwood, K. E., Schoenwald, S., Evans, A. C., … & Adams, D. R. (2015). Predictors of community therapists’ use of therapy techniques in a large public mental health system. JAMA Pediatrics169(4), 374-382. CrossRefGoogle Scholar
  10. Beidas, R., Skriner, L., Adams, D., Wolk, C. B., Stewart, R. E., Becker-Haimes, E., et al. (2017). The relationship between consumer, clinician, and organizational characteristics and use of evidence-based and non-evidence-based therapy strategies in a public mental health system. Behaviour Research and Therapy, 99, 1–10. Scholar
  11. Beidas, R. S., Williams, N. J., Green, P. D., Aarons, G. A., Becker-Haimes, E. M., Evans, A. C., … & Marcus, S. C. (2018). Concordance between administrator and clinician ratings of organizational culture and climate. Administration and Policy in Mental Health and Mental Health Services Research45(1), 142-151. CrossRefGoogle Scholar
  12. Benjamin Wolk, C., Marcus, S. C., Weersing, V. R., Hawley, K. M., Evans, A. C., Hurford, M. O., et al. (2016). Therapist-and client-level predictors of use of therapy techniques during implementation in a large public mental health system. Psychiatric Services, 67(5), 551–557.CrossRefGoogle Scholar
  13. Brookman-Frazee, L., Haine, R. A., Baker-Ericzén, M., Zoffness, R., & Garland, A. F. (2010). Factors associated with use of evidence-based practice strategies in usual care youth psychotherapy. Administration and Policy in Mental Health and Mental Health Services Research, 37(3), 254–269. Scholar
  14. Brookman-Frazee, L., & Stahmer, A. C. (2018). Effectiveness of a multi-level implementation strategy for ASD interventions: Study protocol for two linked cluster randomized trials. Implementation Science, 13(1), 66. Scholar
  15. Busse, C., Kach, A. P., & Wagner, S. M. (2016). Boundary conditions: What they are, how to explore them, why we need them, and when to consider them. Organizational Research Methods, 20(4), 574–609. Scholar
  16. Creed, T. A., Wolk, C. B., Feinberg, B., Evans, A. C., & Beck, A. T. (2016). Beyond the label: Relationship between community therapists’ self-report of a cognitive behavioral therapy orientation and observed skills. Administration and Policy in Mental Health and Mental Health Services Research, 43(1), 36–43. Scholar
  17. Czincz, J., & Romano, E. (2013). Childhood sexual abuse: Community-based treatment practices and predictors of use of evidence-based practices. Child and Adolescent Mental Health, 18(4), 240–246. Scholar
  18. Damschroder, L. J., Aron, D. C., Keith, R. E., Kirsh, S. R., Alexander, J. A., & Lowery, J. C. (2009). Fostering implementation of health services research findings into practice: A consolidated framework for advancing implementation science. Implementation Science, 4(1), 50. Scholar
  19. De Nadai, A. S., & Storch, E. A. (2013). Design considerations related to short-term psychodynamic psychotherapy. Journal of the American Academy of Child and Adolescent Psychiatry, 52(11), 1213–1214. Scholar
  20. Dorsey, S., McLaughlin, K. A., Kerns, S. E., Harrison, J. P., Lambert, H. K., Briggs, E. C., et al. (2017). Evidence base update for psychosocial treatments for children and adolescents exposed to traumatic events. Journal of Clinical Child & Adolescent Psychology, 46(3), 303–330. Scholar
  21. Durlak, J. A., & DuPre, E. P. (2008). Implementation matters: A review of research on the influence of implementation on program outcomes and the factors affecting implementation. American Journal of Community Psychology, 41(3–4), 327–350. Scholar
  22. Egger, N., Konnopka, A., Beutel, M. E., Herpertz, S., Hiller, W., Hoyer, J., … & Wiltink, J. (2016). Long‐term cost‐effectiveness of cognitive behavioral therapy versus psychodynamic therapy in social anxiety disorder. Depression and Anxiety33(12), 1114-1122. CrossRefGoogle Scholar
  23. Ehrhart, M. G., Aarons, G. A., & Farahnak, L. R. (2014). Assessing the organizational context for EBP implementation: The development and validity testing of the Implementation Climate Scale (ICS). Implementation Science, 9(1), 157. Scholar
  24. Ehrhart, M. G., Torres, E. M., Wright, L. A., Martinez, S. Y., & Aarons, G. A. (2016). Validating the implementation climate scale (ICS) in child welfare organizations. Child Abuse and Neglect, 53, 17–26. Scholar
  25. Garland, A. F., Brookman-Frazee, L., Hurlburt, M. S., Accurso, E. C., Zoffness, R. J., Haine-Schlagel, R., et al. (2010). Mental health care for children with disruptive behavior problems: A view inside therapists’ offices. Psychiatric Services, 61(8), 788–795. Scholar
  26. Glisson, C. (2002). The organizational context of children’s mental health services. Clinical Child and Family Psychology Review, 5(4), 233–253. Scholar
  27. Glisson, C., Green, P., & Williams, N. J. (2012). Assessing the organizational social context (OSC) of child welfare systems: Implications for research and practice. Child Abuse and Neglect, 36(9), 621–632. Scholar
  28. Glisson, C., Landsverk, J., Schoenwald, S., Kelleher, K., Hoagwood, K. E., Mayberg, S., et al. (2008). Assessing the organizational social context (OSC) of mental health services: Implications for research and practice. Administration and Policy in Mental Health and Mental Health Services Research, 35(1–2), 98. Scholar
  29. Glisson, C., & Williams, N. J. (2015). Assessing and changing organizational social contexts for effective mental health services. Annual Review of Public Health, 36(1), 507–523. Scholar
  30. Glisson, C., Williams, N. J., Hemmelgarn, A., Proctor, E., & Green, P. (2016). Increasing clinicians’ EBT exploration and preparation behavior in youth mental health services by changing organizational culture with ARC. Behaviour Research and Therapy, 76, 40–46. Scholar
  31. Henggeler, S. W., Chapman, J. E., Rowland, M. D., Halliday-Boykins, C. A., Randall, J., Shackelford, J., et al. (2008). Statewide adoption and initial implementation of contingency management for substance-abusing adolescents. Journal of Consulting and Clinical Psychology, 76(4), 556. Scholar
  32. Higa-McMillan, C. K., Francis, S. E., Rith-Najarian, L., & Chorpita, B. F. (2016). Evidence base update: 50 years of research on treatment for child and adolescent anxiety. Journal of Clinical Child & Adolescent Psychology, 45(2), 91–113. Scholar
  33. Higa-McMillan, C. K., Nakamura, B. J., Morris, A., Jackson, D. S., & Slavin, L. (2015). Predictors of use of evidence-based practices for children and adolescents in usual care. Administration and Policy in Mental Health and Mental Health Services Research, 42(4), 373–383. Scholar
  34. Hofmann, S. G., Asnaani, A., Vonk, I. J., Sawyer, A. T., & Fang, A. (2012). The efficacy of cognitive behavioral therapy: A review of meta-analyses. Cognitive Therapy and Research, 36(5), 427–440. Scholar
  35. Hurlburt, M. S., Garland, A. F., Nguyen, K., & Brookman-Frazee, L. (2010). Child and family therapy process: Concordance of therapist and observational perspectives. Administration and Policy in Mental Health and Mental Health Services Research, 37(3), 230–244. Scholar
  36. Jacobs, S. R., Weiner, B. J., & Bunger, A. C. (2014). Context matters: Measuring implementation climate among individuals and groups. Implementation Science, 9(1), 46. Scholar
  37. Jensen-Doss, A., Hawley, K. M., Lopez, M., & Osterberg, L. D. (2009). Using evidence-based treatments: The experiences of youth providers working under a mandate. Professional Psychology: Research and Practice, 40(4), 417. Scholar
  38. Kam, C.-M., Greenberg, M. T., & Walls, C. T. (2003). Examining the role of implementation quality in school-based prevention using the PATHS curriculum. Prevention Science, 4(1), 55–63. Scholar
  39. Kolko, D. J., Cohen, J. A., Mannarino, A. P., Baumann, B. L., & Knudsen, K. (2009). Community treatment of child sexual abuse: A survey of practitioners in the National Child Traumatic Stress Network. Administration and Policy in Mental Health and Mental Health Services Research, 36(1), 37–49. Scholar
  40. LeBreton, J. M., & Senter, J. L. (2008). Answers to 20 questions about interrater reliability and interrater agreement. Organizational Research Methods, 11(4), 815–852. Scholar
  41. Lewis, C. C., Klasnja, P., Powell, B., Tuzzio, L., Jones, S., Walsh-Bailey, C., et al. (2018). From classification to causality: Advancing understanding of mechanisms of change in implementation science. Frontiers in Public Health, 6, 136. Scholar
  42. Lim, A., Nakamura, B. J., Higa-McMillan, C. K., Shimabukuro, S., & Slavin, L. (2012). Effects of workshop trainings on evidence-based practice knowledge and attitudes among youth community mental health providers. Behaviour Research and Therapy, 50(6), 397–406. Scholar
  43. Lyon, A. R., Ludwig, K., Romano, E., Leonard, S., Vander Stoep, A., & McCauley, E. (2013). “If it’s worth my time, I will make the time”: School-based providers’ decision-making about participating in an evidence-based psychotherapy consultation program. Administration and Policy in Mental Health and Mental Health Services Research, 40(6), 467–481. Scholar
  44. McCart, M. R., & Sheidow, A. J. (2016). Evidence-based psychosocial treatments for adolescents with disruptive behavior. Journal of Clinical Child & Adolescent Psychology, 45(5), 529–563. Scholar
  45. McCrae, R. R., & Costa, P. T. (1987). Validation of the five-factor model of personality across instruments and observers. Journal of Personality and Social Psychology, 52(1), 81–90. Scholar
  46. Midgley, N., O’Keeffe, S., French, L., & Kennedy, E. (2017). Psychodynamic psychotherapy for children and adolescents: an updated narrative review of the evidence base. Journal of Child Psychotherapy, 43(3), 307–329. Scholar
  47. Nelson, T. D., & Steele, R. G. (2007). Predictors of practitioner self-reported use of evidence-based practices: Practitioner training, clinical setting, and attitudes toward research. Administration and Policy in Mental Health and Mental Health Services Research, 34(4), 319–330. Scholar
  48. Niven, D. J., Mrklas, K. J., Holodinsky, J. K., Straus, S. E., Hemmelgarn, B. R., Jeffs, L. P., et al. (2015). Towards understanding the de-adoption of low-value clinical practices: A scoping review. BMC Medicine, 13(1), 255. Scholar
  49. Novins, D. K., Green, A. E., Legha, R. K., & Aarons, G. A. (2013). Dissemination and implementation of evidence-based practices for child and adolescent mental health: A systematic review. Journal of the American Academy of Child and Adolescent Psychiatry, 52(10), 1009–1025. Scholar
  50. Olin, S. S., Williams, N., Pollock, M., Armusewicz, K., Kutash, K., Glisson, C., et al. (2014). Quality indicators for family support services and their relationship to organizational social context. Administration and Policy in Mental Health and Mental Health Services Research, 41(1), 43–54. Scholar
  51. Powell, B. J., Beidas, R. S., Rubin, R. M., Stewart, R. E., Wolk, C. B., Matlin, S. L., et al. (2016). Applying the policy ecology framework to Philadelphia’s behavioral health transformation efforts. Administration and Policy in Mental Health and Mental Health Services Research, 43(6), 909–926. Scholar
  52. Powell, B. J., Mandell, D. S., Hadley, T. R., Rubin, R. M., Evans, A. C., Hurford, M. O., et al. (2017). Are general and strategic measures of organizational context and leadership associated with knowledge and attitudes toward evidence-based practices in public behavioral health settings? A cross-sectional observational study. Implementation Science, 12(1), 64. Scholar
  53. Prasad, V., & Ioannidis, J. P. (2014). Evidence-based de-implementation for contradicted, unproven, and aspiring healthcare practices. Implementation Science, 9(1), 1. Scholar
  54. Proctor, E., Silmere, H., Raghavan, R., Hovmand, P., Aarons, G., Bunger, A., et al. (2011). Outcomes for implementation research: Conceptual distinctions, measurement challenges, and research agenda. Administration and Policy in Mental Health and Mental Health Services Research, 38(2), 65–76. Scholar
  55. Schoenwald, S. K., & Garland, A. F. (2013). A review of treatment adherence measurement methods. Psychological Assessment, 25(1), 146. Scholar
  56. Stumpf, R. E., Higa-McMillan, C. K., & Chorpita, B. F. (2009). Implementation of evidence-based services for youth: Assessing provider knowledge. Behavior Modification, 33(1), 48–65. Scholar
  57. Tabak, R. G., Khoong, E. C., Chambers, D. A., & Brownson, R. C. (2012). Bridging research and practice: Models for dissemination and implementation research. American Journal of Preventive Medicine, 43(3), 337–350. Scholar
  58. Wandersman, A., Duffy, J., Flaspohler, P., Noonan, R., Lubell, K., Stillman, L., … Saul, J. (2008). Bridging the gap between prevention research and practice: The interactive systems framework for dissemination and implementation. American Journal of Community Psychology, 41(3-4), 171-181. CrossRefGoogle Scholar
  59. Weersing, V. R., Weisz, J. R., & Donenberg, G. R. (2002). Development of the therapy procedures checklist: A therapist-report measure of technique use in child and adolescent treatment. Journal of Clinical Child and Adolescent Psychology, 31(2), 168–180. Scholar
  60. Weisz, J. R., Kuppens, S., Ng, M. Y., Eckshtain, D., Ugueto, A. M., Vaughn-Coaxum, R., … & Weersing, V. R. (2017). What five decades of research tells us about the effects of youth psychological therapy: A multilevel meta-analysis and implications for science and practice. American Psychologist, 72(2), 79. CrossRefGoogle Scholar
  61. Whetten, D. A. (1989). What constitutes a theoretical contribution? Academy of Management Review, 14(4), 490–495.CrossRefGoogle Scholar
  62. Williams, N. J., Ehrhart, M. G., Aarons, G. A., Marcus, S. C., & Beidas, R. S. (2018). Linking molar organizational climate and strategic implementation climate to clinicians’ use of evidence-based psychotherapy techniques: Cross-sectional and lagged analyses from a 2-year observational study. Implementation Science, 13(1), 85.CrossRefGoogle Scholar
  63. Williams, N. J., & Glisson, C. (2013). Reducing turnover is not enough: The need for proficient organizational cultures to support positive youth outcomes in child welfare. Children and Youth Services Review, 35(11), 1871–1877. Scholar
  64. Williams, N. J., & Glisson, C. (2014a). The role of organizational culture and climate in the dissemination and implementation of empirically supported treatments for youth. In R. Beidas & P. Kendall (Eds.), Dissemination and implementation of evidence based practices in child and adolescent mental health (pp. 61–81). New York: Oxford University Press.Google Scholar
  65. Williams, N. J., & Glisson, C. (2014b). Testing a theory of organizational culture, climate and youth outcomes in child welfare systems: A United States national study. Child Abuse and Neglect, 38(4), 757–767. Scholar
  66. Williams, N. J., Glisson, C., Hemmelgarn, A., & Green, P. (2017). Mechanisms of change in the ARC organizational strategy: Increasing mental health clinicians’ EBP adoption through improved organizational culture and capacity. Administration and Policy in Mental Health and Mental Health Services Research, 44(2), 269–283. Scholar
  67. Zhou, X., Hetrick, S. E., Cuijpers, P., Qin, B., Barth, J., Whittington, C. J., et al. (2015). Comparative efficacy and acceptability of psychotherapies for depression in children and adolescents: A systematic review and network meta-analysis. World Psychiatry, 14(2), 207–222. Scholar
  68. Zyzanski, S. J., Flocke, S. A., & Dickinson, L. M. (2004). On the nature and analysis of clustered data. The Annals of Family Medicine, 2(3), 199–200.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  • Emily M. Becker-Haimes
    • 1
    Email author
  • Nathaniel J. Williams
    • 2
  • Kelsie H. Okamura
    • 1
  • Rinad S. Beidas
    • 1
    • 3
    • 4
  1. 1.Department of PsychiatryUniversity of Pennsylvania Perelman School of MedicinePhiladelphiaUSA
  2. 2.School of Social WorkBoise State UniversityBoiseUSA
  3. 3.Department of Medical Ethics and Health PolicyPerelman School of Medicine, University of PennsylvaniaPhiladelphiaUSA
  4. 4.Penn Implementation Science Center at the Leonard Davis Institute of Health Economics (PISCE@LDI)University of PennsylvaniaPhiladelphiaUSA

Personalised recommendations