Advertisement

Methodological Issues in the Evaluation of Therapies

  • Ric Ric G. Steele
  • Ric Jennifer A. Mize Nelson
  • Ric Timothy D. Nelson
Part of the Issues in Clinical Child Psychology book series (ICCP)

Keywords

Mental Health Service Consumer Satisfaction Applied Setting Boot Camp Parent Management Training 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Aarons, G. A. (2005). Measuring provider attitudes toward evidence-based practice: Considerations of organizational context and individual differences. Child and Adolescent Psychiatric Clinics of North America, 14, 255–271.PubMedCrossRefGoogle Scholar
  2. Achenbach, T. M. (2001). What are norms and why do we need valid ones? Clinical Psychology: Science and Practice, 8, 446–450.CrossRefGoogle Scholar
  3. Addis, M. E., & Krasnow, A. D. (2000). A national survey of practicing psychologists’ attitudes toward psychotherapy treatment manuals. Journal of Consulting and Clinical Psychology, 68, 331–339.PubMedCrossRefGoogle Scholar
  4. American Psychological Association Presidential Task Force on Evidence-Based Practice (2006). Evidence-based practice in psychology. American Psychologist, 61, 271–285.Google Scholar
  5. Anderson, C. M., & Kim, C. (2003). Evaluating treatment efficacy with single-case designs. In M. C. Roberts & S. S. Ilardi (Eds.), Handbook of Research Methods in Clinical Psychology (pp. 73–91). Malden, MA: Blackwell Publishing.Google Scholar
  6. Attkisson, C. C., & Greenfield, T. K. (1999). The UCSF Client Satisfaction Scales: I. The Client Satisfaction Questionnaire-8. In M. E. Maruish (Ed.), The Use of Psychological Testing for Treatment Planning and Outcomes Assessment, 2nd ed. (pp. 1333–1346). Mahwah, NJ: Lawrence Erlbaum Associates.Google Scholar
  7. Attkisson, C. C., & Greenfield, T. K. (2004). The UCSF Client Satisfaction Scales: I. The Client Satisfaction Questionnaire-8. In M. E. Maruish (Ed.), The Use of Psychological Testing for Treatment Planning and Outcomes Assessment: Volume 3: Instruments for Adults, 3rd ed. (pp. 799–811). Mahwah, NJ: Lawrence Erlbaum Associates.Google Scholar
  8. Beidel, D. C., Turner, S. M., & Morris, T. L. (2000). Behavioral treatment of childhood social phobia. Journal of Consulting and Clinical Psychology, 68, 1072–1080.PubMedCrossRefGoogle Scholar
  9. Bickman, L., Guthrie, P. R., Foster, E. M., Lambert, E. W., Summerfelt, W. T., Breda, C. S., et al. (1995). Evaluating Managed Mental Health Services: The Fort Bragg Experiment. New York: Plenum.Google Scholar
  10. Bickman, L., & Mulvaney, S. (2005). Large-scale evaluations of children’s mental health services: The Ft. Bragg and Stark County studies. In R. G. Steele & M. C. Roberts (Eds.), Handbook of Mental Health Services for Children, Adolescents, and Families (pp. 371–386). New York: Kluwer.CrossRefGoogle Scholar
  11. Bickman, L., Noser, K., & Summerfelt, W. T. (1999). Long-term effects of a system of care on children and adolescents. The Journal of Behavioral Health Services and Research, 26, 185–202.CrossRefGoogle Scholar
  12. Birmaher, B., Brent, D. A., Kolko, D., Baugher, M., Bridge, J., Holder, D., et al. (2000). Clinical outcome after short-term psychotherapy for adolescents with major depressive disorder. Archives of General Psychiatry, 57, 29–35.PubMedCrossRefGoogle Scholar
  13. Borrelli, B., Sepinwall, D., Ernst, D., Bellg, A. J., Czajkowski, S., Breger, R., et al. (2005). A new tool to assess treatment fidelity and evaluation of treatment fidelity across 10 years of health behavior research. Journal of Consulting and Clinical Psychology, 73, 852–860.PubMedCrossRefGoogle Scholar
  14. Bottcher, J., & Ezell, M. E. (2005). Examining the effectiveness of boot camps: A randomized experiment with a long-term follow-up. Journal of Research in Crime and Delinquency, 42, 309–332.CrossRefGoogle Scholar
  15. Center for Mental Health Services (1997). The Family Satisfaction Questionnaire—Abbreviated Version (FSQ-A). Unpublished manuscript.Google Scholar
  16. Chambless, D. L., & Hollon, S. D. (1998). Defining empirically supported therapies. Journal of Consulting and Clinical Psychology, 66, 7–18.PubMedCrossRefGoogle Scholar
  17. Chorpita, B. F., & Donkervoet, C. (2005). Implementation of the Felix Consent Decree in Hawaii: The impact of policy and practice development efforts on service delivery. In R. G. Steele & M. C. Roberts (Eds.), Handbook of Mental Health Services for Children, Adolescents, and Families} (pp. 317–332). New York: Kluwer.CrossRefGoogle Scholar
  18. Chorpita, B. F., Yim, L. M., Donkervoet, J. C., Arensdorf, A., Amundsen, M. J., McGee, C., et al. (2002). Toward large-scale implementation of empirically supported treatments for children: A review and observations by the Hawaii Empirical Basis to Services Task Force. Clinical Psychology: Science and Practice, 9, 165–190.CrossRefGoogle Scholar
  19. Clinical Treatment and Services Research Workgroup (CTSRW) (1998). Bridging Science and Service: A Report by the National Advisory Mental Health Council’s Clinical Treatment and Services Research Workgroup .Bethesda, MD: National Institutes of Health, National Institute of Mental Health. Retrieved May 19, 2006, from http://www.nimh.nih.gov/publicat/nimhbridge.pdf.Google Scholar
  20. Connor-Smith, J. K., & Weisz, J. R. (2003). Applying treatment outcome research in clinical practice: Techniques for adapting interventions to the real world. Child and Adolescent Mental Health, 8, 3–10.CrossRefGoogle Scholar
  21. Crosbie, J. (1993). Interrupted time-series analyses with brief single-subject data. Journal of Consulting and Clinical Psychology, 61, 966–974.PubMedCrossRefGoogle Scholar
  22. Des Jarlais, D. C., Lyles, C., Crepaz, N., & the TREND Group (2004). Improving the reporting quality of nonrandomized evaluations of behavioral and public health interventions: The TREND statement. American Journal of Public Health, 94, 361–366.PubMedCrossRefGoogle Scholar
  23. Graczyk, P. A., Domitrovich, C. E., & Zins, J. E. (2003). Facilitating the implementation of evidence-based prevention and mental health promotion efforts in schools. In M. D. Weist, S. W. Evans, & N. A. Lever (Eds.), Handbook of School Mental Health: Advancing Practice and Research (pp. 301–318). New York: Kluwer Academic/Plenum Publishers.Google Scholar
  24. Greenfield, T. K., & Attkisson, C. C. (2004). The UCSF Client Satisfaction Scales: II. The Service Satisfaction Scale-30. In M.E. Maruish (Ed.), The Use of Psychological Testing for Treatment Planning and Outcomes Assessment: Volume 3: Instruments for Adults, 3rd ed. (pp. 813–837). Mahwah, NJ: Lawrence Erlbaum Associates.Google Scholar
  25. Hawley, K. M., & Weisz, J. R. (2005). Youth versus parent working alliance in usual clinical care: Distinctive associations with retention, satisfaction, and treatment outcome. Journal of Clinical Child and Adolescent Psychology, 34, 117–128.PubMedCrossRefGoogle Scholar
  26. Henderson, J. L., MacKay, S., & Peterson-Badali, M. (2006). Closing the research-practice gap: Factors affecting adoption and implementation of a children’s mental health program. Journal of Clinical Child and Adolescent Psychology, 35, 2–12.PubMedCrossRefGoogle Scholar
  27. Henggeler, S. W., Melton, G. B., Brondino, M. J., Scherer, D. G., & Hanley, J. H. (1997). Multisystemic therapy with violent and chronic juvenile offenders and their families: The role of treatment fidelity in successful dissemination. Journal of Consulting and Clinical Psychology, 65, 821–833.PubMedCrossRefGoogle Scholar
  28. Hernandez, M., Hodges, S., & Cascardi, M. (1998). The ecology of outcomes: System accountability in children’s mental health. The Journal of Behavioral Health Services & Research, 25, 136–150.CrossRefGoogle Scholar
  29. Hoagwood, K., Jensen, P. S., Petti, T., & Burns, B. J. (1996). Outcomes of mental health care for children and adolescents: I. A comprehensive conceptual model. Journal of the American Academy of Child and Adolescent Psychiatry, 35, 1055–1063.PubMedCrossRefGoogle Scholar
  30. Holden, E. W., & Brennan, A. M. (Eds.) (2002). Special issue: Evaluating systems of care: The National Evaluation of the Comprehensive Community Mental Health Services for Children and Their Families Program. Children’s Services: Social Policy, Research, and Practice, 5, 1–74.Google Scholar
  31. Holden, E. W., De Carolis, G., & Huff, B. (2002). Policy implications of the National Evaluation of the Comprehensive Community Mental Health Services for Children and Their Families Program. Children’s Services: Social Policy, Research, and Practice, 5, 57–66.Google Scholar
  32. Holden, E. W., Stephens, R. L., & Santiago, R. L. (2005). Methodological challenges in the national evaluation of the Comprehensive Community Mental Health Services for Children and Their Families program. In R. G. Steele & M. C. Roberts (Eds.), Handbook of Mental Health Services for Children, Adolescents, and Families (pp. 387–401). New York: Kluwer.CrossRefGoogle Scholar
  33. Huebner, C. E., Barlow, W. E., Tyll, L. T., Johnston, B. D., & Thompson, R. S. (2004). Expanding developmental and behavioral services for newborns in primary care: Program design, delivery, and evaluation framework. American Journal of Preventive Medicine, 26, 344–355.PubMedCrossRefGoogle Scholar
  34. Jacobson, N. S., & Truax, P. (1991). Clinical significance: A statistical approach to defining meaningful change in psychotherapy research. Journal of Consulting and Clinical Psychology, 59, 12–19.PubMedCrossRefGoogle Scholar
  35. Jensen, P. S., Hoagwood, K., & Petti, T. (1996). Outcomes of mental health care for children and adolescents: II. Literature review and application of a comprehensive model. Journal of the American Academy of Child and Adolescent Psychiatry, 35, 1064–1077.PubMedCrossRefGoogle Scholar
  36. Jensen-Doss, A., & Weisz, J. R. (2006). Syndrome co-occurrence and treatment outcomes in youth mental health clinics. Journal of Consulting and Clinical Psychology, 74, 416–425.CrossRefGoogle Scholar
  37. Kallestad, J. H., & Olweus, D. (2003). Predicting teachers’ and schools’ implementation of the Olweus Bullying Prevention Program: A multilevel study. Prevention and Treatment, 6.Google Scholar
  38. Kazdin, A. E. (1997). A model for developing effective treatments: Progression and interplay of theory, research, and practice. Journal of Clinical Child Psychology, 26, 114–129.PubMedCrossRefGoogle Scholar
  39. Kazdin, A. E. (2001). Almost clinically significant (p≪.10): Current measures may only approach clinical significance. Clinical Psychology: Science and Practice, 8, 455–462.CrossRefGoogle Scholar
  40. Kazdin, A. E. (2003a). Problem-solving skills training and parent management training for conduct disorder. In A. E. Kazdin & J. R Weisz (Eds.), Evidence-Based Psychotherapies for Children and Adolescents (pp. 241–262). New York: Guilford.Google Scholar
  41. Kazdin, A. E. (2003b). Psychotherapy for children and adolescents. Annual Review of Psychology, 54, 253–276.CrossRefGoogle Scholar
  42. Kazdin, A. E. (2006). Arbitrary metrics: Implications for identifying evidence-based treatments. American Psychologist, 61, 42–49.PubMedCrossRefGoogle Scholar
  43. Kazdin, A. E., Siegel, T., & Bass, D. (1990). Drawing on clinical practice to inform research on child and adolescent psychotherapy: Survey of practitioners. Professional Psychology: Research and Practice, 21, 189–198.CrossRefGoogle Scholar
  44. Kazdin, A. E., Siegel, T., & Bass, D. (1992). Cognitive problem-solving skills training and parent management training in the treatment of antisocial behavior in children. Journal of Consulting and Clinical Psychology, 60, 733–747.PubMedCrossRefGoogle Scholar
  45. Kazdin, A. E., & Weisz, J. R. (1998). Identifying and developing empirically supported child and adolescent treatments. Journal of Consulting and Clinical Psychology, 66, 19–36.PubMedCrossRefGoogle Scholar
  46. Kendall, P. C. (Ed.) (1999). Special section: Clinical significance. Journal of Consulting and Clinical Psychology, 67, 283–339.PubMedCrossRefGoogle Scholar
  47. Matos, M., Torres, R., Santiago, R., Jurado, M., & Rodriguez, I. (2006). Adaptation of parent-child interaction therapy for Puerto Rican families: A preliminary study. Family Process, 45, 205–222.PubMedCrossRefGoogle Scholar
  48. Miranda, J., Guillermo, B., Lau, A., Kohn, L., Hwang, W., & LaFromboise, T. (2005). State of the science on psychosocial interventions for ethnic minorities. Annual Review of Clinical Psychology, 1, 113–142.PubMedCrossRefGoogle Scholar
  49. Moher, D., Schulz, K. F., & Altman, D. (2001). The CONSORT statement: Revised recommendations for improving the quality of reports of parallel-group randomized trials. Journal of the American Medical Association, 285, 1987–1991.PubMedCrossRefGoogle Scholar
  50. Moore, K. E., Geffken, G. R., & Royal, G. P. (1995). Behavioral intervention to reduce child distress during self-injection. Clinical Pediatrics, 34, 530–534.PubMedCrossRefGoogle Scholar
  51. Nelson, T. D., & Steele, R. G. (2006). Beyond efficacy and effectiveness: A multifaceted approach to treatment evaluation. Professional Psychology: Research and Practice, 37, 389–397.CrossRefGoogle Scholar
  52. Nelson, T. D., Steele, R. G., & Mize, J. A. (2006). Practitioner attitudes toward evidence-based practice. Administrative Policy in Mental Health and Mental Health Services Research, 33, 398–409.CrossRefGoogle Scholar
  53. Ogles, B. M., Lambert, M. J., & Masters, K. S. (1996). Assessing Outcome in Clinical Practice. Boston: Allyn & Bacon.Google Scholar
  54. Olson, R., & Netherton, S. D. (1996). Consultation and liaison in a children’s hospital. In M. C. Roberts (Ed.), Model Programs in Child and Family Mental Health (pp. 249–264). Mahwah: NJ: Lawrence Erlbaum Associates.Google Scholar
  55. Olweus, D., Limber, S., & Mihalic, S. F. (1999). Blueprints for Violence Prevention, Book Nine: Bullying Prevention Program. Boulder, CO: Center for the Study and Prevention of Violence.Google Scholar
  56. Paul, G. L. (1967). Outcome research in psychotherapy. Journal of Consulting Psychology, 31, 109–118.PubMedCrossRefGoogle Scholar
  57. Persons, J. B., & Silberschatz, G. (1998). Are results of randomized controlled trials useful to psychotherapists? Journal of Consulting and Clinical Psychology, 66, 126–135.PubMedCrossRefGoogle Scholar
  58. Plumer, P. J., & Stoner, G. (2005). The relative effects of classwide peer tutoring and peer coaching on the positive social behaviors of children with ADHD. Journal of Attention Disorders, 9, 290–300.PubMedCrossRefGoogle Scholar
  59. Schiffman, J., Becker, K. D., & Daleiden, E. L. (2006). Evidence-based services in a statewide public mental health system: Do the services fit the problems? Journal of Clinical Child and Adolescent Psychology, 35, 13–19.PubMedCrossRefGoogle Scholar
  60. Schoenwald, S. K., Sheidow, A. J., & Letourneau, E. J. (2004). Toward effective quality assurance in evidence-based practice: Links between expert consultation, therapist fidelity, and child outcomes. Journal of Clinical Child and Adolescent Psychology, 33, 94–104.PubMedCrossRefGoogle Scholar
  61. Sheldrick, R. C., Kendall, P. C., & Heimberg, R. G. (2001). The clinical significance of treatments: A comparison of three treatments for conduct disordered children. Clinical Psychology: Science and Practice, 8, 418–430.CrossRefGoogle Scholar
  62. Shirk, S. R. (2005). President’s message: Dialogue and persistent polarities between research and practice. In Balance: Society of Clinical Child and Adolescent Psychology Newsletter, 20(3), 1–2.Google Scholar
  63. Simons, A. D., & Wildes, J. E. (2003). Therapy and interventions research with adults. In M. C. Roberts & S. S. Ilardi (Eds.), Handbook of Research Methods in Clinical Psychology (pp. 329–351). Malden, MA: Blackwell Publishing.Google Scholar
  64. Slifer, K. J., Eischen, S. E., & Busby, S. (2002). Using counterconditioning to treat behavioural distress during subcutaneous injections in a paediatric rehabilitation patient. Brain Injury, 16, 901–916.PubMedCrossRefGoogle Scholar
  65. Southam-Gerow, M. A., Weisz, J. R., & Kendall, P. C. (2003). Youth with anxiety disorders in research and service clinics: Examining client differences and similarities. Journal of Clinical Child and Adolescent Psychology, 32, 375–385.PubMedCrossRefGoogle Scholar
  66. Stirman, S. W., Crits-Christoph, P., & DeRubeis, R. J. (2004). Achieving successful dissemination of empirically supported psychotherapies: A synthesis of dissemination theory. Clinical Psychology: Science and Practice, 11, 343–359.CrossRefGoogle Scholar
  67. Varni, J. W., Burwinkle, T. M., Dickinson, P., Sherman, S. A., Dixon, P., Ervice, J. A., et al. (2004). Evaluation of the built environment at a children’s convalescent hospital: Development of the Pediatric Quality of Life InventoryTM parent and staff satisfaction measures for pediatric health care facilities. Journal of Developmental and Behavioral Pediatrics, 25, 10–20.PubMedCrossRefGoogle Scholar
  68. Walrath, C. M., Ybarra, M. L., & Holden, E. W. (2006). Understanding the pre-referral factors associated with differential 6-month outcomes among children receiving system-of-care services. Psychological Services, 3, 35–50.CrossRefGoogle Scholar
  69. Weisz, J. R., Jensen, A. L., & McLeod, B. D. (2005). Development and dissemination of child and adolescent psychotherapies: Milestones, methods, and a new deployment-focused model. In E. D. Hibbs & P. S. Jensen (Eds.), Psychosocial Treatments for Child and Adolescent Disorders: Empirically Based Strategies for Clinical Practice, 2nd ed. (pp. 9–39). Washington, DC: American Psychological Association.Google Scholar
  70. Wells, K. B. (1999). Treatment research at the crossroads: The scientific interface of clinical trials and effectiveness research. American Journal of Psychiatry, 156, 5–10.PubMedGoogle Scholar
  71. Wierzbicki, M., & Pekarik, G. (1993). A meta-analysis of psychotherapy dropout. Professional Psychology: Research and Practice, 24, 190–195.CrossRefGoogle Scholar
  72. Yates, B. T. (2003). Toward the incorporation of costs, cost-effectiveness analysis, and cost-benefit analysis into clinical research. In A. E. Kazdin (Ed.), Methodological Issues and Strategies in Clinical Research, 3rd ed. (pp. 711–727). Washington, DC: American Psychological Association.Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2008

Authors and Affiliations

  • Ric Ric G. Steele
    • 1
  • Ric Jennifer A. Mize Nelson
    • 1
  • Ric Timothy D. Nelson
    • 1
  1. 1.University of Kansas, Clinical Child Psychology Program

Personalised recommendations