Revisiting How We Assess Therapist Competence in Cognitive Therapy
We report on two approaches to addressing the problem of low reliability in the assessment of therapist competence in cognitive therapy (CT). In Study 1, we utilized ratings of a session from each of six therapists and investigated how CT experts evaluate specific therapist behaviors in making ratings on the Cognitive Therapy Scale (CTS). Departures from the consensus on judgments of these therapist behaviors were associated with more discrepant competence ratings. We take these results to suggest that, at least among experts, providing greater clarity about which therapist behaviors are relevant to each CTS item may enhance reliability. In Study 2, we utilize a sample of 14 therapists and pilot test a standardized assessment of therapist competence. The standardized assessment exhibited an intra-class correlation coefficient of 0.89 for a single rater. Standardized assessments of competence were related to four measures of therapist experience and qualifications, whereas the CTS was only related to two of these measures. With these promising initial results, we encourage future research to examine the predictive validity of standardized competence assessments in CT.
KeywordsTherapist competence Assessment Cognitive therapy Depression
Compliance with Ethical Standards
Conflict of Interest
Iony D. Schmidt, Daniel R. Strunk, Robert J. DeRubeis, Laren R. Conklin, and Justin D. Braun declare that they have no conflict of interest.
All procedures performed in studies involving human participants were in accordance with the ethical standards of the Institutional and/or National Research Committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.
Informed consent was obtained from all individual participants included in the study.
This article does not contain any studies with animals performed by any of the authors.
- Academy of Cognitive Therapy. (2018). Certification: Certification process. Retrieved from http://www.academyofct.org/.
- Baldwin, S. A., & Imel, Z. E. (2013). Therapist effects: Findings and methods. In M. E. Lambert (Ed.), Bergin and Garfield’s handbook of psychotherapy and behavior change (6th ed., pp. 258–297). New York: Wiley.Google Scholar
- Beck, A. T., Rush, A. J., Shaw, B. F., & Emery, G. (1979). Cognitive therapy of depression. New York: Guilford.Google Scholar
- Beck Institute for Cognitive Behavior Therapy. (2018). Cognitive Therapy Scale rating manual. Retrieved from https://www.beckinstitute.org/wp-content/uploads/2015/10/CTRS-current-10-2011-Cognitive-Therapy-Rating-Scale-2.pdf.
- Beck Institute for Cognitive Behavior Therapy. (2018). Cognitive Therapy Rating Scale. Retrieved from https://www.beckinstitute.org/wp-content/uploads/2015/10/CTRS-12-2011_portrait-Cognitive-Therapy-Rating-Scale-1.pdf.
- Cohen, J., Cohen, P., West, S. G., & Aiken, L. S. (2003). Applied multiple correlation/regression analysis for the behavioral sciences. Mahwah, NJ: Lawrence Erlbaum.Google Scholar
- Dennhag, I., Gibbons, M. B. C., Barber, J. P., Gallop, R., & Crits-Christoph, P. (2012). How many treatment sessions and patients are needed to create a stable score of adherence and competence in the treatment of cocaine dependence? Psychotherapy Research, 22, 475–488.CrossRefPubMedPubMedCentralGoogle Scholar
- DeRubeis, R. J., Gelfand, L. A., German, R. E., Fournier, J. C., & Forand, N. R. (2013). Understanding processes of change: How some patients reveal more than others—And some groups of therapists less—About what matters in psychotherapy. Psychotherapy Research, 24, 419–428. https://doi.org/10.1080/10503307.2013.838654.CrossRefPubMedPubMedCentralGoogle Scholar
- DeRubeis, R. J., Hollon, S. D., Amsterdam, J. D., Shelton, R. C., Young, P. R., Salomon, R. M., … Gallop, R. (2005). Cognitive therapy vs medications in the treatment of moderate to severe depression. Archives of General Psychiatry, 62, 409–416. https://doi.org/10.1001/archpsyc.62.4.409.CrossRefPubMedGoogle Scholar
- Dobson, K. S., Shaw, B. F., & Vallis, T. M. (1985). Reliability of a measure of quality of cognitive therapy. British Journal of Clinical Psychology, 24, 295–300. https://doi.org/10.1111/j.2044-8260.1985.tb00662.x.CrossRefPubMedGoogle Scholar
- Hollon, S. D., Evans, M. D., Auerbach, A., DeRubeis, R. J., Elkin, I., Lowery, A., … Piasecki, J. (1988). Development of a system for rating therapies for depression: Differentiating cognitive therapy, interpersonal psychotherapy and clinical management pharmacotherapy. Unpublished manuscript. Nashville: Vanderbilt University.Google Scholar
- Imel, Z. E., Baldwin, S. A., Baer, J. S., Hartzler, B., Dunn, C., Rosengren, D. B., & Atkins, D. C. (2014). Evaluating therapist adherence in motivational interviewing by comparing performance with standardized and real patients. Journal of Consulting and Clinical Psychology, 82, 472–481. https://doi.org/10.1037/a0036158.CrossRefPubMedPubMedCentralGoogle Scholar
- Jacobson, N. S., & Gortner, E. T. (2000). Can depression be de-medicalized in the 21st century: Scientific revolutions, counter-revolutions and the magnetic field of normal science. Behaviour Research and Therapy, 38, 103–117. https://doi.org/10.1016/S0005-7967(99)00029-7.CrossRefPubMedGoogle Scholar
- Lee, I. A., & Preacher, K. J. (2013). Calculation for the test of the difference between two dependent correlations with one variable in common [Computer software]. Available from http://quantpsy.org.
- Littell, R. C., Milliken, G. A., Stroup, W. W., Wolfinger, R. D., & Schabenberger, O. (2006). SAS system for mixed models (2nd ed.). Cary, NC: SAS Institute.Google Scholar
- McManus, F., Rakovshik, S., Kennerley, H., Fennell, M., & Westbrook, D. (2012). An investigation of the accuracy of therapists’ self-assessment of cognitive-behaviour therapy skills. British Journal of Clinical Psychology, 51, 292–306. https://doi.org/10.1111/j.2044-8260.2011.02028.x.CrossRefPubMedGoogle Scholar
- Shaw, B. F., Elkin, I., Yamaguchi, J., Olmstead, M., Vallis, T. M., Dobson, K. S., Lowery, A., … Imber, S. D. (1999). Therapist competence ratings in relation to clinical outcome in cognitive therapy of depression. Journal of Consulting and Clinical Psychology, 67, 837–846. https://doi.org/10.1037/0022-006X.67.6.837.CrossRefPubMedGoogle Scholar
- Sholomskas, D. E., Syracuse-Siewert, G., Rounsaville, B. J., Ball, S. A., Nuro, K. F., & Carroll, K. M. (2005). We don’t train in vain: A dissemination trial of three strategies of training clinicians in cognitive-behavioral therapy. Journal of Consulting and Clinical Psychology, 73, 106–115. https://doi.org/10.1037/0022-006X.73.1.106.CrossRefPubMedPubMedCentralGoogle Scholar
- Strunk, D. R., Brotman, M. A., DeRubeis, R. J., & Hollon, S. D. (2010). Therapist competence in cognitive therapy for depression: predicting subsequent symptom change. Journal of Consulting and Clinical Psychology, 78, 429–437. https://doi.org/10.1037/a0019631.CrossRefPubMedPubMedCentralGoogle Scholar
- Trepka, C., Rees, A., Shapiro, D. A., Hardy, G. E., & Barkham, M. (2004). Therapist competence and outcome of cognitive therapy for depression. Cognitive Therapy & Research, 28, 143–157. https://doi.org/10.1023/B:COTR.0000021536.39173.66.CrossRefGoogle Scholar