Cognitive Therapy and Research

, Volume 42, Issue 4, pp 385–397 | Cite as

The Cognitive Therapy Scale and Cognitive Therapy Scale-Revised as Measures of Therapist Competence in Cognitive Behavior Therapy for Depression: Relations with Short and Long Term Outcome

  • Nikolaos Kazantzis
  • Xavier Clayton
  • Timothy. J. Cronin
  • Davide Farchione
  • Karina Limburg
  • Keith. S. Dobson
Original Article


Therapist competence is an important factor in treatment integrity. This study reports on a direct comparison of the original Cognitive Therapy Scale (CTS) with the revised version, the Cognitive Therapy Scale-Revised (CTS-R), as observational instruments designed to evaluate therapist competence in a completed trial of Cognitive Behavior Therapy (CBT) for depression. Treatment sessions (N = 94) from 50 depressed participants (M age = 39.2 years, 76% female, with an average of 5.9 depressive episodes) were evaluated on the CTS and CTS-R by trained independent observers who were blind to treatment outcome, as well as to subject, and session numbers. A comprehensive training program and inter-rater reliability monitoring protocol were employed. Two models were used to compare CTS and CTS-R in relation to change in depression symptoms at termination, 12 and 24 month follow-up while controlling for pre-treatment depression levels and working alliance. Both the CTS and CTS-R demonstrated comparable internal reliability, interrater reliability, and when assessed in early treatment phase, both predicted a statistically significant reduction in depressive symptomatology at termination. No significant competence-outcome relations were detected with late CTS and CTS-R ratings, and the significant positive interaction terms indicated relations with depressive symptomatology were not maintained at follow-up. Given these findings, we encourage future research to examine specific competence domains and “therapist drift” with an increased number of session assessments.


Cognitive Behavior Therapy Therapist competence Major depressive disorder Cognitive Therapy Scale Cognitive Therapy Scale-Revised 



Funding was provided by Helen Macpherson Smith Trust and National Institute of Mental Health.

Compliance with Ethical Standards

Conflict of interest

The authors state that there are no conflicts of interests to disclose in this submitted work to Cognitive Therapy & Research.

Ethical Approval

This research was reviewed and approved by the Monash University Human Ethics Committee (Project ID #2589). The parent study was conducted at the University of Washington (Jacobson et al., 1996) and had received prior ethical approval.

Informed Consent

All research participants provided informed consent prior to participation. All procedures performed were in accordance with the ethical standards of the institutional research committees and with the 1964 Helsinki declaration and its later amendments. In addition, the attribution of authorship on this manuscript has also been carefully considered and is being made in a manner that is consistent with ethical research practice.

Animal Rights

No animal studies were carried out by the authors for this article.


  1. Andony, L. J., Tay, E., Allen, K. L., Wade, T. D., Hay, P., Touyz, S., McIntosh, V. V. W., Treasure, J., Schmidt, U. H., Fairburn, C. G., Erceg-Hurn, D. M., Fursland, A., Crosby, R. D., & Byrne, S. M. (2015). Therapist adherence in the strong without anorexia nervosa (SWAN) study: A randomized controlled trial of three treatments for adults with anorexia nervosa. International Journal of Eating Disorders, 48(8), 1170–1175. Scholar
  2. Aviram, A., & Westra, H. A. (2011). The impact of motivational interviewing on resistance in cognitive behavioural therapy for generalized anxiety disorder. Psychotherapy Research, 21, 698–708. Scholar
  3. Barber, J. P., Liese, B. S., & Abrams, M. J. (2003). Development of the cognitive therapy adherence and competence scale. Psychotherapy Research, 13, 205–221. Scholar
  4. Barlow, D. H. (1996). Health care policy, psychotherapy research, and the future of psychotherapy. American Psychologist, 51, 1050–1058. Scholar
  5. Beck, A. T., & Dozois, D. J. (2011). Cognitive therapy: Current status and future directions. Annual Review of Medicine, 62, 397–409. doi.CrossRefPubMedGoogle Scholar
  6. Beck, A. T., Rush, A. J., Shaw, B. F., & Emery, G. (1979). Cognitive therapy of depression. New York: Guilford Press.Google Scholar
  7. Beck, A. T., Steer, R. A., & Garbin, M. G. (1988). Psychometric properties of the Beck depression inventory: Twenty-five years of evaluation. Clinical Psychology Review, 8, 77–100. Scholar
  8. Blackburn, I. M., James, I. A., Milne, D. L., Baker, C., Standart, S., Garland, A., & Reichelt, F. K. (2001). The revised cognitive therapy scale (CTS-R): Psychometric properties. Behavioural and Cognitive Psychotherapy, 29, 431–446. Scholar
  9. Bordin, E. S. (1980). Of human bonds that bind or free. Presidential address delivered at the meeting of the Society for Psychotherapy Research, Pacific Grove, CA.Google Scholar
  10. Boswell, J. F., Gallagher, M. W., Sauer-Zavala, S. E., Bullis, J., Gorman, J. M., Shear, M. K., Woods, S., & Barlow, D. H. (2013). Patient characteristics and variability in adherence and competence in cognitive-behavioral therapy for panic disorder. Journal of Consulting and Clinical Psychology, 81(3), 443. Scholar
  11. Branson, A., Shafran, R., & Myles, P. (2015). Investigating the relationship between competence and patient outcome with CBT. Behaviour Research and Therapy, 68, 19–26. Scholar
  12. Carter, J. D., McIntosh, V. V., Jordan, J., Porter, R. J., Frampton, C. M., & Joyce, P. R. (2013). Psychotherapy for depression: A randomized clinical trial comparing schema therapy and cognitive behavior therapy. Journal of Affective Disorders, 151, 500–505. Scholar
  13. Chambless, D. L., & Hollon, S. D. (1998). Defining empirically supported therapies. Journal of Consulting and Clinical Psychology, 66(1), 7. Scholar
  14. Clark, D. M. (2012). The English improving access to psychological therapies (IAPT) program: History and progress. In R. K. McHugh & D. H. Barlow (Eds.), Dissemination and implementation of evidence-based psychological interventions (pp. 61–77). Oxford: Oxford University Press, Inc.Google Scholar
  15. Clark, D. M., Layard, R., Smithies, R., Richards, D. A., Suckling, R., & Wright, B. (2009). Improving access to psychological therapy: Initial evaluation of two UK demonstration sites. Behaviour Research and Therapy, 47, 910–910. Scholar
  16. Cohen, J., Cohen, P., West, S. G., & Aiken, L. S. (2003). Applied multiple regression/correlation analysis for the behavioral sciences (3rd ed.). Mahwah, NJ: Lawrence Erlbaum Associates.Google Scholar
  17. Crits-Christoph, P., Baranackie, K., Kurcias, J., Beck, A., Carroll, K., Perry, K., … Gallagher, D. (1991). Meta-analysis of therapist effects in psychotherapy outcome studies. Psychotherapy Research, 1(2), 81–91. Scholar
  18. Crits-Christoph, P., Cooper, A., & Luborsky, L. (1988). The accuracy of therapists interpretations and the outcome of dynamic therapy. Journal of Consulting and Clinical Psychology, 56, 490–495. Scholar
  19. Cronbach, L. J., & Meehl, P. E. (1955). Construct validity in psychological tests. Psychological Bulletin, 52, 281–302. Scholar
  20. Dennhag, I., Gibbons, M. B. C., Barber, J. P., Gallop, R., & Crits-Christoph, P. (2012). How many treatment sessions and patients are needed to create a stable score of adherence and competence in the treatment of cocaine dependence? Psychotherapy Research, 22, 475–488. Scholar
  21. Diggle, P. J., Heagerty, P., Liang, K., & Zeger, S. L. (2002). Analysis of longitudinal data (2nd ed.). Oxford: Oxford University Press.Google Scholar
  22. Dobson, K. S. (2000). Chronic processes in depression: Differentiating self and other influences in onset, maintenance, and relapse/recurrence. Clinical Psychology: Science and Practice, 7, 236–239. Scholar
  23. Dobson, K. S., & Kazantzis, N. (2003). The therapist in cognitive-behavioral therapy: Introduction to a special section. Psychotherapy Research, 13, 131–134.CrossRefGoogle Scholar
  24. Dobson, K. S., & Shaw, B. F. (1988). The use of treatment manuals in cognitive therapy: Expertise and issues. Journal of Consulting and Clinical Psychology, 56, 673–680.CrossRefPubMedGoogle Scholar
  25. Dobson, K. S., Shaw, B. F., & Vallis, T. M. (1985). The reliability of competency ratings on cognitive-behaviour therapists. British Journal of Clinical Psychology, 24, 295–300. Scholar
  26. Dobson, K. S., & Singer, A. R. (2005). Definitional and practical issues in the assessment of treatment integrity. Clinical Psychology: Science and Practice, 12, 384–387.Google Scholar
  27. Dozois, D. J. A., Dobson, K. S., & Ahnberg, J. L. (1998). A psychometric evaluation of the Beck depression inventory-II. Psychological Assessment, 10, 83–89. Scholar
  28. Fairburn, C. G., & Cooper, Z. (2011). Therapist competence, therapy quality, and therapist training. Behaviour Research and Therapy, 49, 373–378. Scholar
  29. Finn, R. H. (1970). A note on estimating the reliability of categorical data. Educational and Psychological Measurement, 30, 71–76. Scholar
  30. Flückiger, C., Del Re, A. C., Wampold, B. E., Symonds, D., & Horvath, A. O. (2012). How central is the alliance in psychotherapy? A multilevel longitudinal meta-analysis. Journal of Counseling Psychology, 59(1), 10. Scholar
  31. Fowler, D., Rollinson, R., & Fremch, P. (2011). Adherence and competence assessment in studies of CBT for psychosis: Current status and future directions. Epidemiology and Psychiatric Sciences, 20, 121–126. Scholar
  32. Gordon, P. K. (2006). A comparison of two versions of the cognitive therapy scale. Behavioural and Cognitive Psychotherapy, 35, 343–353. Scholar
  33. Gortner, E. T., Gollan, J. K., Dobson, K. S., & Jacobson, N. S. (1998). Cognitive-behavioral treatment for depression: Relapse prevention. Journal of Consulting and Clinical Psychology, 66, 377–384.CrossRefPubMedGoogle Scholar
  34. Hamilton, M. (1960). A rating scale for depression. Journal of Neurology, Neurosurgery and Psychiatry, 12, 56–62. Scholar
  35. Hamilton, M. (1967). Development of a rating scale for primary depressive illness. British Journal of Social and Clinical Psychology, 6, 278–296. Scholar
  36. Haubert, L. C., & Dobson, K. S. (2007). Treatment of depression and mechanisms of change: Strengthening the links among theory, research, and practice. Clinical Psychology: Science and Practice, 14, 247–251.Google Scholar
  37. Hofmann, S. G. (2004). An introduction to modern CBT: Psychological solutions to mental health problems. Oxford: Wiley.Google Scholar
  38. Hofmann, S. G., Asnaani, A., Vonk, I. J. J., Sawyer, A. T., & Fang, A. (2012). The efficacy of cognitive behavioral therapy: A review of meta-analyses. Cognitive Therapy & Research, 36(5), 427–440. Scholar
  39. Horvath, A. O., Re., D., Fluckiger, A. C., C., & Symonds, D. (2011). Alliance in individual psychotherapy. Psychotherapy, 48, 9–16. Scholar
  40. Jacobson, N. S. (1998). How important are cognitive interventions in cognitive therapy? Presented at the World Congress of Behavioral and Cognitive Therapies, July 21–26. Acapulco, Mexico.Google Scholar
  41. Jacobson, N. S. (1999). The role of the allegiance effect in psychotherapy research: Controlling and accounting for it. Clinical Psychology: Science and Practice, 6, 116–119. Scholar
  42. Jacobson, N. S., Dobson, K. S., Fruzzetti, A. E., Schmaling, K. B., & Salusky, S. (1991). Marital therapy as a treatment for depression. Journal of Consulting and Clinical Psychology, 59, 547–557.CrossRefPubMedGoogle Scholar
  43. Jacobson, N. S., Dobson, K. S., Truax, P. A., Addis, M. E., Koerner, K., Gollan, J. K., … Prince, S. E. (1996). A component analysis of cognitive-behavioural treatment for depression. Journal of Consulting and Clinical Psychology, 64, 295–304. Scholar
  44. Jacobson, N. S., & Gortner, E. T. (2000). Can depression be de-medicalized in the 21st century: Scientific revolutions, counter-revolutions and the magnetic field of normal science. Behaviour Research and Therapy, 38, 103–117. Scholar
  45. Jacobson, N. S., & Hollon, S. D. (1996). Prospects for future comparisons between drugs and psychotherapy: Lessons from the CBT-versus-pharmacotherapy exchange. Journal of Consulting and Clinical Psychology, 64, 104–108. Scholar
  46. Kazantzis, N. (2003). Therapist competence in cognitive-behavioural therapies: Review of the contemporary empirical evidence. Behaviour Change, 20, 1–12. Scholar
  47. Kazantzis, N. (2018). Introduction to the special issue on processes of cognitive behavioral therapy: Does “necessary, but not sufficient” still capture it? Cognitive Therapy & Research, 42(2), 115–120. Scholar
  48. Kazantzis, N., Cronin, T. J., Farchione, D., & Dobson, K. S. (2018). Working alliance in cognitive behavior therapy for depression: Can brief observer alliance assessments yield reliable and valid data? Manuscript submitted for publication.Google Scholar
  49. Kazantzis, N., Dattilio, F. M., & Dobson, K. S. (2017). The therapeutic relationship in cognitive behavior therapy: A clinician’s guide. New York: Guilford.Google Scholar
  50. Kazantzis, N., Whittington, C. J., Zelencich, L., Norton, P. J., Kyrios, M., & Hofmann, S. G. (2016). Quantity and quality of homework compliance: A meta-analysis of relations with outcome in cognitive behavior therapy. Behavior Therapy, 47, 755–772. Scholar
  51. Kendall, P. C. (1998). Empirically supported psychological therapies. Journal of Consulting and Clinical Psychology, 66(1), 3. Scholar
  52. Kline, P. (1999). The handbook of psychological testing (2nd ed.). London: Routledge.Google Scholar
  53. Kuyken, W., & Tsivrikos, D. (2009). Therapist competence, comorbidity and cognitive-behavioural therapy for depression. Psychotherapy and Psychosomatics, 78, 42–48. Scholar
  54. Lombard, M., Snyder-Duch, J., & Bracken, C. C. (2002). Content analysis in mass communication: Assessment and reporting of intercoder reliability. Human Communication Research, 28, 587–604. Scholar
  55. Lorenzo-Luaces, L., & DeRubeis, R. J. (2018). Miles to go before we sleep: Advancing the understanding of psychotherapy by modeling complex processes. Cognitive Therapy & Research, 42(2), 212–217.CrossRefGoogle Scholar
  56. McGlinchey, J. B., & Dobson, K. S. (2003). Treatment integrity concerns in cognitive therapy for depression. International Journal of Cognitive Psychotherapy, 17(4), 299–318.CrossRefGoogle Scholar
  57. McHugh, R. K., & Barlow, D. H. (2010). The dissemination and implementation of evidence-based psychological treatments: A review of current efforts. American Psychologist, 65, 73–84. Scholar
  58. Moncher, F. J., & Prinz, R. J. (1991). Treatment fidelity in outcome studies. Clinical Psychology Review, 11, 247–266.CrossRefGoogle Scholar
  59. Muse, K., & McManus, F. (2013). A systematic review of methods for assessing competence in cognitive-behavioural therapy. Clinical Psychology Review, 33, 484–499. Scholar
  60. Muse, K., McManus, F., Rakovshik, S., & Thwaites, R. (2017). Development and psychometric evaluation of the assessment of core CBT skills (ACCS): An observation-based tool for assessing cognitive behavioral therapy competence. Psychological Assessment, 29(5), 542–555. Scholar
  61. Neuendorf, K. A. (2002). The content analysis guidebook. Thousand Oaks, CA: Sage.Google Scholar
  62. Nezu, A. M., & Nezu, C. M. (2008). Treatment integrity. Handbook of research methods in abnormal and clinical psychology, pp. 351–363. New York: Sage.Google Scholar
  63. Nezu, C. M., Martell, C. R., & Nezu, A. M. (2013). Specialty competencies in cognitive and behavioral psychology. Oxford: Oxford University Press.Google Scholar
  64. Padesky, C. A. (1996). Developing cognitive therapist competency: Teaching and supervision models. In P. M. Salkovskis (Ed.), Frontiers of cognitive therapy (pp. 266–292). New York: The Guilford Press.Google Scholar
  65. Papageorgiou, C., & Wells, A. (2015). Group metacognitive therapy for sever antidepressant and CBT resistant depression: A baseline-controlled trial. Cognitive Therapy & Research, 39, 14–22. Scholar
  66. Perepletchikova, F., & Kazdin, A. E. (2005). Treatment integrity of therapeutic change: Issues and research recommendations. Clinical Psychology: Science and Practice, 12, 365–383. Scholar
  67. Rabe-Hesketh, S., & Skrondal, A. (2012). Multilevel and longitudinal modeling using stata, Volume I: Continuous responses (3rd ed.). Texas: Stata Press Publications.Google Scholar
  68. Reichelt, F., James, I. A., & Blackburn, I.-M. (2003). Impact of training on rating competence in cognitive therapy. Journal of Behavior Therapy and Experimental Psychiatry, 34, 87–99. Scholar
  69. Schmidt, N. B., & Woolaway-Bickel, K. (2000). The effect of treatment compliance on outcome in cognitive-behavioral therapy for panic disorder: Quantity Versus Quality. Journal of Consulting and Clinical Psychology, 68(1), 13–18.CrossRefPubMedGoogle Scholar
  70. Schoenwald, S. K., & Garland, A. F. (2013). A review of treatment adherence measurement methods. Psychological Assessment, 25, 146–156. Scholar
  71. Shaw, B. F. (1984). Specification of the training and evaluation of cognitive therapists for outcome studies. In J. Williams & R. Spitzer (Eds.), Psychotherapy research (pp. 173–188). New York: Guilford.Google Scholar
  72. Shaw, B. F., Elkin, I., Yamaguchi, J., Olmsted, M., Vallis, T. M., Dobson, K. S., … Imber, S. D. (1999). Therapist competence ratings in relation to clinical outcome in cognitive therapy of depression. Journal of Consulting and Clinical Psychology, 67, 837–846. Scholar
  73. Simons, A. D., Padesky, C. A., Montemarano, J., Lewis, C. C., Murakami, J., Lamb, K., … Beck, A. T. (2010). Training and dissemination of cognitive behavior therapy for depression in adults: A preliminary examination of therapist competence and client outcomes. Journal of Consulting and Clinical Psychology, 78, 751–756. Scholar
  74. Simons, A. D., Rozek, D. C., & Serrano, J. L. (2013). Wanted: Reliable and valid measures for the science of cognitive behavioral therapy dissemination and implementation. Clinical Psychology: Science and Practice, 20(2), 181–194. Scholar
  75. Spitzer, R. L., Williams, J. B. W., & Gibbon, M. (1987). Instruction manual for the structured clinical interview for DSM-III-R. New York: New York State Psychiatric Institute.Google Scholar
  76. Stiles, W. B. (2009). Responsiveness as an obstacle for psychotherapy outcome research: It’s worse than you think. Clinical Psychology: Science and Practice, 16(1), 86–91. Scholar
  77. Stirman, S. W., Buchhofer, R., McLaulin, M. S. J. B., Evans, A. C., & Beck, A. T. (2009). Public-academic partnerships: The Beck initiative: A partnership to implement cognitive therapy in a community behavioral health system. Psychiatric Services, 60(10), 1302–1304. Scholar
  78. Strunk, D. R., Brotman, M. A., DeRubeis, R. J., & Hollon, S. D. (2010). Therapist competence in cognitive therapy for depression: Predicting subsequent symptom change. Journal of Consulting and Clinical Psychology, 78, 429–437. Scholar
  79. Tichenor, V., & Hill, C. E. (1989). A comparison of six measures of working alliance. Psychotherapy, 26, 195–199. Scholar
  80. Tracey, T. J., & Kokotovic, A. M. (1989). Factor structure of the working alliance inventory. Psychological Assessment: A Journal of Consulting and Clinical Psychology, 1, 207–210. Scholar
  81. Tracey, T. J., Wampold, B. E., Lichtenberg, J. W., & Goodyear, R. K. (2014). Expertise in psychotherapy: An elusive goal? American Psychologist, 69(3), 218. Scholar
  82. Trepka, C., Rees, A., Shapiro, D. A., Hardy, G. E., & Barkham, M. (2004). Therapist competence and outcome of cognitive therapy for depression. Cognitive Therapy & Research, 28, 143–157. Scholar
  83. Vallis, T. M., Shaw, B. F., & Dobson, K. S. (1986). The Cognitive Therapy Scale: Psychometric properties. Journal of Consulting and Clinical Psychology, 54, 381–385. Scholar
  84. Waller, G. (2009). Evidence-based treatment and therapist drift. Behaviour Research and Therapy, 47, 119–127. Scholar
  85. Waller, G., Stringer, H., & Meyer, C. (2012). What cognitive behavioral techniques do therapists report using when delivering cognitive behavioral therapy for the eating disorders? Journal of Consulting and Clinical Psychology, 80, 171–175. Scholar
  86. Waltz, J., Addis, M. E., Koerner, K., & Jacobson, N. S. (1993). Testing the integrity of a psychotherapy protocol: Assessment of adherence and competence. Journal of Consulting and Clinical Psychology, 61, 620–630. Scholar
  87. Webb, C. A., DeRubeis, R. J., & Barber, J. P. (2010). Therapist adherence/competence and treatment outcome: A meta-analytic review. Journal of Consulting and Clinical Psychology, 78, 200–211. Scholar
  88. Weck, F., Grikscheit, F., Höfling, V., Kordt, A., Hamm, A. O., Gerlach, A. L., Alpers, G. W., Arolt, V., Kircher, T., Pauli, P., Rief, W., & Lang, T. (2016). The role of treatment delivery factors in exposure-based cognitive behavioral therapy for panic disorder with agoraphobia. Journal of Anxiety Disorders, 42, 10–18. Scholar
  89. Weck, F., Richtberg, S., Esch, S., Hofling, V., & Stangier, U. (2013). The relationship between therapist competence and homework compliance in maintenance cognitive therapy for recurrent depression: Secondary analysis of a randomized trial. Behavior Therapy, 44, 162–172. Scholar
  90. Weck, F., Richtberg, S., Jakob, M., Neng, J. M., & Höfling, V. (2015). Therapist competence and therapeutic alliance are important in the treatment of health anxiety (hypochondriasis). Psychiatry Research, 228(1), 53–58. Scholar
  91. Whisman, M. A. (1993). Mediators and moderators of change in cognitive therapy of depression. Psychological Bulletin, 114, 248–265. Scholar
  92. Whitehurst, G. J. (1984). Interrater agreement for journal manuscript reviews. American Psychologist, 39, 22–28. Scholar
  93. Wiles, N., Thomas, L., Ridgway, N., Turner, N., Campbell, J., Garland, J., … Lewis, G. (2013). Cognitive behavioural therapy as an adjunct to pharmacotherapy for primary care based patients with treatment resistant depression: Results of the CoBalT randomised controlled trial. The Lancet, 381, 375–384. Scholar
  94. Williams, R. M., Moorey, S., & Cobb, J. (1991). Training in cognitive-behavior therapy: Pilot evaluation of a training course using the Cognitive Therapy Scale. Behavior Psychotherapy, 19, 373–376.CrossRefGoogle Scholar
  95. Young, J. E., & Beck, A. T. (1980). Cognitive Therapy Scale: Rating manual. Philadelphia, PA: University of Pennsylvania.Google Scholar
  96. Young, R. K., Shammaa, K. E., & Beck, A. T. (1979). The competency checklist for cognitive therapists. Philadelphia, PA: Center for Cognitive Therapy, University of Pennsylvania.Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Cognitive Behavior Therapy Research Unit, School of Psychological SciencesMonash UniversityMelbourneAustralia
  2. 2.Department of Mathematics and StatisticsLa Trobe UniversityMelbourneAustralia
  3. 3.Department of Psychosomatic Medicine and PsychotherapyTechnische UniversitätMunichGermany
  4. 4.Department of PsychologyUniversity of CalgaryCalagryCanada

Personalised recommendations