Skip to main content
SpringerLink
Log in
Book cover

Routine Outcome Monitoring in Couple and Family Therapy pp 261–279Cite as

How Can Outcome Data Inform Change? Experiences from the Child Mental Health Context in Great Britain, Including Barriers and Facilitators to the Collection and Use of Data

  • Chapter
  • First Online:

Part of the book series: European Family Therapy Association Series ((EFTAS))

Abstract

Within the context of child mental health across the UK, a practice research network, Child Outcomes Research Consortium (CORC), has been formed and is committed to collating and using outcome data to inform clinical and service decision making . CORC began with 4 subscribing organizations in 2004 and has grown to approximately 65 in 2015. Based on this experience, learning about how such data are to be used safely and effectively to support performance management and service improvement are discussed. One challenge is that any attempt to measure the “impact” of a service using a given “outcome” is complex. Within psychotherapy in general, and in work with children and families where there are multiple perspectives in particular, this question raises a number of dilemmas. For some professionals, the whole notion of quantitatively measuring such human qualities is questioned. Despite these challenges, CORC has been pioneering in supporting its members to collect and cautiously make use of information with a particular emphasis on outcomes from the service user perspective. Such challenges may be recognized in other countries as well, and CORC aims to share learning to help mitigate these barriers.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   54.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   69.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  • Abrines-Jaume, N., Midgley, N., Hopkins, K., Hoffman, J., Martin, K., Law, D., et al. (2016). A qualitative analysis of implementing shared decision making in child and adolescent mental health services in the united kingdom: stages and facilitators. Clinical Child Psychology and Psychiatry, 21(1), 19–31. doi:10.1177/1359104514547596.

    Article  PubMed  Google Scholar 

  • Batty, M. J., Moldavsky, M., Foroushani, P. S., Pass, S., Marriott, M., Sayal, K., et al. (2013). Implementing routine outcome measures in child and adolescent mental health services: From present to future practice. Child and Adolescent Mental Health, 18(2), 82–87. doi:10.1111/j.1475-3588.2012.00658.x.

    Article  Google Scholar 

  • Bevan, G., & Hood, C. (2006). What’s measured is what matters: targets and gaming in the English public health care system. Public Administration, 84(3), 517–538. doi:10.1111/j.1467-9299.2006.00600.x.

    Article  Google Scholar 

  • Bickman, L., Kelley, S. D., Breda, C., de Andrade, A. R., & Riemer, M. (2011). Effects of routine feedback to clinicians on mental health outcomes of youths: Results of a randomized trial. Psychiatric Services, 62(12), 1423–1429. doi:10.1176/appi.ps.002052011.

    Article  PubMed  Google Scholar 

  • Boswell, J. F., Kraus, D. R., Miller, S. D., & Lambert, M. J. (2013). Implementing routine outcome monitoring in clinical practice: Benefits, challenges, and solutions. Psychotherapy Research, 25(1), 6–19. doi:10.1080/10503307.2013.817696.

    Article  PubMed  Google Scholar 

  • Carlier, I. V. E., Meuldijk, D., Van Vliet, I. M., Van Fenema, E., Van der Wee, N. J. A., & Zitman, F. G. (2012). Routine outcome monitoring and feedback on physical or mental health status: Evidence and theory. Journal of Evaluation in Clinical Practice, 18(1), 104–110. doi:10.1111/j.1365-2753.2010.01543.x.

    Article  PubMed  Google Scholar 

  • Children and Young people’s Mental Health and Well-being Taskforce. (2015). Retrieved January 11, 2016, from http://www.gov.uk/government/groups/children-and-young-peoples-mental-health-and-well-being-taskforce.

  • CORC. (2013). CORC’s position on commissioning for quality and innovation (CQUIN) targets concerning outcome measures. Retrieved from http://www.corc.uk.net/wp-content/uploads/2012/03/CORCs-Position-on-CQUIN-targets-03042013.pdf.

  • CORC. (2015). Homepage. Retrieved 16 Oct 2015, from http://www.corc.uk.net/.

  • de Jong, K., van Sluis, P., Nugter, M. A., Heiser, W. J., & Spinhoven, P. (2012). Understanding the differential impact of outcome monitoring: Therapist variables that moderate feedback effects in a randomized clinical trial. Psychotherapy Research, 22(4), 464–474. doi:10.1080/10503307.2012.673023.

    Article  PubMed  Google Scholar 

  • Deighton, J., Croudace, T., Fonagy, P., Brown, J., Patalay, P., & Wolpert, M. (2014). Measuring mental health and wellbeing outcomes for children and adolescents to inform practice and policy: A review of child self-report measures. Child and Adolescent Psychiatry and Mental Health, 8(1), 14. doi:10.1186/1753-2000-8-14.

    Article  PubMed  PubMed Central  Google Scholar 

  • Department of Health, Department for Education, & NHS England. (2015). Future in mind: Promoting, protecting and improving our children and young people’s mental health and wellbeing. London: Department of Health.

    Google Scholar 

  • EBPU. (2015). EBPU training, from http://www.ucl.ac.uk/ebpu/events/training.

  • Edbrooke-Childs, J., Jacob, J., Argent, R., Patalay, P., Deighton, J., & Wolpert, M. (2016a). The relationship between child- and parent-reported shared decision making and child-, parent-, and clinician-reported treatment outcome in routinely collected child mental health services data. Clinical Child Psychology and Psychiatry, 21(2), 324–338. doi:10.1177/1359104515591226.

    Article  PubMed  Google Scholar 

  • Edbrooke-Childs, J., Jacob, J., Law, D., Deighton, J., & Wolpert, M. (2015). Interpreting standardized and idiographic outcome measures in CAMHS: What does change mean and how does it relate to functioning and experience? Child and Adolescent Mental Health, 20(3), 142–148. doi:10.1111/camh.12107.

    Article  Google Scholar 

  • Edbrooke-Childs, J., Wolpert, M., & Deighton, J. (2016b). Using patient reported outcome measures to improve service effectiveness (UPROMISE): Training clinicians to use outcome measures in child mental health. Administration and Policy in Mental Health and Mental Health Services Research, 43(3), 302–308. doi:10.1007/s10488-014-0600-2.

    Article  PubMed  Google Scholar 

  • Edmondson, A. C., Bohmer, R. M., & Pisano, G. P. (2001). Disrupted routines: Team learning and new technology implementation in hospitals. Administrative Science Quarterly, 46(4), 685–716. doi:10.2307/3094828.

    Article  Google Scholar 

  • Fleming, I., Jones, M., Bradley, J., & Wolpert, M. (2016). Learning from a learning collaboration: The CORC approach to combining research, evaluation and practice in child mental health. Administration and Policy in Mental Health and Mental Health Services Research, 43(3), 297–301. doi:10.1007/s10488-014-0592-y.

    Article  PubMed  Google Scholar 

  • Grenville, J., & Lyne, P. (1995). Patient-centred evaluation and rehabilitative care. Journal of Advanced Nursing, 22(5), 965–972. doi:10.1111/j.1365-2648.1995.tb02649.x.

    Article  PubMed  Google Scholar 

  • Hall, C. L., Moldavsky, M., Baldwin, L., Marriott, M., Newell, K., Taylor, J., et al. (2013). The use of routine outcome measures in two child and adolescent mental health services: A completed audit cycle. BMC Psychiatry, 13(1), 270. doi:10.1186/1471-244X-13-270.

    Article  PubMed  PubMed Central  Google Scholar 

  • Handy, C. (1993). Understanding organizations. Harmondsworth, UK: Penguin.

    Google Scholar 

  • Hannan, C., Lambert, M. J., Harmon, C., Nielsen, S. L., Smart, D. W., Shimokawa, K., et al. (2005). A lab test and algorithms for identifying clients at risk for treatment failure. Journal of Clinical Psychology, 61(2), 155–163. doi:10.1002/jclp.20108.

    Article  PubMed  Google Scholar 

  • Hawley, K. M., & Weisz, J. R. (2005). Youth versus parent working alliance in usual clinical care: distinctive associations with retention, satisfaction, and treatment outcome. Journal of Clinical Child & Adolescent Psychology, 34(1), 117–128. doi:10.1207/s15374424jccp3401_11.

    Article  Google Scholar 

  • House of Commons Health Committee. (2014). Children’s and adolescents’ mental health and CAMHS: Third report of session 2014–15. London: The Stationery Office Limited. Retrieved from http://www.publications.parliament.uk/pa/cm201415/cmselect/cmhealth/342/34202.htm.

  • Hubble, M. A., Duncan, B. L., & Miller, S. D. (1999). The heart and soul of change: What works in therapy. Washington DC: American Psychological Association.

    Book  Google Scholar 

  • Jacob, J., Edbrooke-Childs, J., Holley, S., Law, D., & Wolpert, M. (2015a). Horses for courses? A qualitative exploration of goals formulated in mental health settings by young people, parents and clinicians. Clinical Child Psychology and Psychiatry. doi:10.1177/1359104515577487. (Online first).

    Google Scholar 

  • Jacob, J., Edbrooke-Childs, J., Law, D., & Wolpert, M. (2015b). Measuring what matters to patients: using goal content to inform measure choice and development. Clinical Child Psychology and Psychiatry. doi:10.1177/1359104515615642. (Online first).

    Google Scholar 

  • Johnston, C., & Gowers, S. (2005). Routine outcome measurement: A survey of UK child and adolescent mental health services. Child and Adolescent Mental Health, 10(3), 133–139. doi:10.1111/j.1475-3588.2005.00357.x.

    Article  Google Scholar 

  • Lambert, M. J. (2010). Yes, it is time for clinicians to routinely monitor treatment outcome. In B. L. Duncan, S. D. Miller, B. E. Wampold, & M. A. Hubble (Eds.), The heart and soul of change (2nd ed.). Washington, DC: American Psychological Association.

    Google Scholar 

  • Lambert, M. J., Harmon, C., Slade, K., Whipple, J. L., & Hawkins, E. J. (2005). Providing feedback to psychotherapists on their patients’ progress: clinical results and practice suggestions. Journal of Clinical Psychology, 61(2), 165–174. doi:10.1002/jclp.20113.

    Article  PubMed  Google Scholar 

  • Lambert, M. J., & Shimokawa, K. (2011). Collecting client feedback. Psychotherapy, 48(1), 72–79. doi:10.1037/a0022238.

    Article  PubMed  Google Scholar 

  • Law, D., & Jacob, J. (2015). Goals and goal based outcomes (GBOs): some useful information (3rd ed.). London: CAMHS Press.

    Google Scholar 

  • Law, D., & Wolpert, M. (Eds.). (2014). Guide to using outcomes and feedback tools with children, young people and families (2nd ed.). London: CAMHS Press.

    Google Scholar 

  • Lowe, T. (2013). New development: The paradox of outcomes—The more we measure, the less we understand. Public Money & Management, 33(3), 213–216. doi:10.1080/09540962.2013.785707.

    Article  Google Scholar 

  • Margison, F. R., McGrath, G., Barkham, M., Clark, J. M., Audin, K., Connell, J., et al. (2000). Measurement and psychotherapy. Evidence-Based Practice and Practice-Based Evidence, 177(2), 123–130. doi:10.1192/bjp.177.2.123.

    Google Scholar 

  • Mellor-Clark, J., Cross, S., Macdonald, J., & Skjulsvik, T. (2016). Leading horses to water: Lessons from a decade of helping psychological therapy service use routine outcome measurement to improve practice. Administration and Policy in Mental Health and Mental Health Services Research, 43(3), 279–285. doi:10.1007/s10488-014-0587-8.

    Article  PubMed  Google Scholar 

  • Meltzer, H., Gatward, R., Goodman, R., & Ford, T. (2000). Mental health of children and adolescents in Great Britain. London: The Stationery Office.

    Book  Google Scholar 

  • Miller, S. D., Duncan, B. L., Brown, J., Sorrell, R., & Chalk, M. B. (2006). Using formal client feedback to improve retention and outcome: Making ongoing, real-time assessment feasible. Journal of Brief Therapy, 5, 5–22.

    Google Scholar 

  • Moran, P., Kelesidi, K., Guglani, S., Davidson, S., & Ford, T. (2012). What do parents and carers think about routine outcome measures and their use? A focus group study of CAMHS attenders. Clinical Child Psychology and Psychiatry, 17, 65–79. doi:10.1177/1359104510391859.

    Article  PubMed  Google Scholar 

  • Norman, S., Dean, S., Hansford, L., & Ford, T. (2013). Clinical practitioner’s attitudes towards the use of routine outcome monitoring within child and adolescent mental health services: A qualitative study of two Child and adolescent mental health services. Clinical Child Psychology and Psychiatry, 19(4), 576–595. doi:10.1177/1359104513492348.

    Article  PubMed  Google Scholar 

  • CYP IAPT Outcomes and Evaluation Task and Finish Group. (2011). Children & young people’s IAPT routine outcome monitoring briefing note. Retrieved October 16, 2015, from http://www.iapt.nhs.uk/silo/files/rom-dec11-03.pdf.

  • Pinnock, H., Epiphaniou, E., Pearce, G., Parke, H., Greenhalgh, T., Sheikh, A., et al. (2015). Implementing supported self-management for asthma: A systematic review and suggested hierarchy of evidence of implementation studies. BMC Medicine, 13, 1–18. doi:10.1186/s12916-015-0361-0.

    Article  Google Scholar 

  • Powell, A. E., Davies, H. T. O., & Thomson, R. G. (2003). Using routine comparative data to assess the quality of health care: Understanding and avoiding common pitfalls. Quality and Safety in Health Care, 12(2), 122–128. doi:10.1136/qhc.12.2.122.

    Article  PubMed  PubMed Central  Google Scholar 

  • Raby, E., & Sheppard, D. (2015, October). Meaningful presentation and use of data from ROMs (Routine Outcome Measures). Paper presented at the BPS Children, Young People and Families Conference, Crowne Plaza Birmingham NEC UK.

    Google Scholar 

  • Ruble, L., McGrew, J. H., & Toland, M. D. (2012). Goal attainment scaling as an outcome measure in randomized controlled trials of psychosocial interventions in autism. Journal of Autism and Developmental Disorders, 42(9), 1974–1983. doi:10.1007/s10803-012-1446-7.

    Article  PubMed  PubMed Central  Google Scholar 

  • Sapyta, J., Riemer, M., & Bickman, L. (2005). Feedback to clinicians: Theory, research, and practice. Journal of Clinical Psychology, 61(2), 145–153. doi:10.1002/jclp.20107.

    Article  PubMed  Google Scholar 

  • Shimokawa, K., Lambert, M. J., & Smart, D. W. (2010). Enhancing treatment outcome of patients at risk of treatment failure: Meta-analytic and mega-analytic review of a psychotherapy quality assurance system. Journal of Consulting and Clinical Psychology, 78(3), 298–311. doi:10.1037/a0019247.

    Article  PubMed  Google Scholar 

  • Slade, M. (2002). Routine outcome assessment in mental health services. Psychological Medicine, 32(8), 1339–1343.

    Article  PubMed  Google Scholar 

  • Spiegelhalter, D. J. (2005). Funnel plots for comparing institutional performance. Statistics in Medicine, 24(8), 1185–1202. doi:10.1002/sim.1970.

    Article  PubMed  Google Scholar 

  • Stasiak, K., Parkin, A., Seymour, F., Lambie, I., Crengle, S., Pasene-Mizziebo, E., et al. (2013). Measuring outcome in child and adolescent mental health services: Consumers’ views of measures. Clinical Child Psychology and Psychiatry, 18(4), 519–535. doi:10.1177/1359104512460860.

    Article  PubMed  Google Scholar 

  • Stratton, P., Bland, J., Janes, E., & Lask, J. (2010). Developing an indicator of family function and a practicable outcome measure for systemic family and couple therapy: The SCORE. Journal of Family Therapy, 32(3), 232–258. doi:10.1111/j.1467-6427.2010.00507.x.

    Article  Google Scholar 

  • Tong Gong, A. (2015). An evaluation of CORC report: Staff’s attitudes towards its use and its impact on various areas of clinical and service work. MSc dissertation, University College London and Anna Freud Centre.

    Google Scholar 

  • Troupp, C. (2013). Distinguishing patient-generated outcome measures in CAMHS from a bespoke sandwich. Child and Family Clinical Psychology Review, 1, 19–28.

    Google Scholar 

  • Whale, A. (2015). Using funnel plots: Understanding your data in context. Retrieved October 16, 2015, from http://www.corc.uk.net/blog/using-funnel-plots-understanding-your-data-in-context/.

  • Wolpert, M. (2014). Uses and abuses of patient reported outcome measures (PROMs): Potential iatrogenic impact of proms implementation and how it can be mitigated. Administration and Policy in Mental Health and Mental Health Services Research, 41(2), 141–145. doi:10.1007/s10488-013-0509-1.

    Article  PubMed  Google Scholar 

  • Wolpert, M. (2015, November, 12–13 ). How can outcome data inform service-level management? Experiences from the child mental health context in GB that may have relevance in Norway. Paper presented at the first national conference on the use of feedback systems in Norway, Scandic Oslo Airport Hotel, Gardermoen.

    Google Scholar 

  • Wolpert, M., Cheng, H., & Deighton, J. (2015). Measurement issues: Review of four patient reported outcome measures: SDQ, RCADS, C/ORS and GBO—Their strengths and limitations for clinical use and service evaluation. Child and Adolescent Mental Health, 20(1), 63–70. doi:10.1111/camh.12065.

    Article  Google Scholar 

  • Wolpert, M., Curtis-Tyler, K., & Edbrooke-Childs, J. (2016). A qualitative exploration of patient and clinician views on patient reported outcome measures in child mental health and diabetes services. Administration and Policy in Mental Health and Mental Health Services Research, 43(3), 309–315. doi:10.1007/s10488-014-0586-9.

    Article  PubMed  Google Scholar 

  • Wolpert, M., Deighton, J., De Francesco, D., Martin, P., Fonagy, P., & Ford, T. (2014). From ‘reckless’ to ‘mindful’ in the use of outcome data to inform service-level performance management: perspectives from child mental health. BMJ Quality & Safety, 23(4), 272–276. doi:10.1136/bmjqs-2013-002557.

    Article  Google Scholar 

  • Wolpert, M., Fugard, A., & Deighton, J. (2013). Issues in evaluation of psychotherapies. In P. Graham & S. Reynolds (Eds.), Cognitive behaviour therapy for children and families (3rd ed., pp. 34–47). Cambridge: Cambridge University Press.

    Chapter  Google Scholar 

  • Worthen, V. E., & Lambert, M. J. (2007). Outcome oriented supervision: Advantages of adding systematic client tracking to supportive consultations. Counselling and Psychotherapy Research, 7(1), 48–53. doi:10.1080/14733140601140873.

    Article  Google Scholar 

  • Yeh, M., & Weisz, J. R. (2001). Why are we here at the clinic? Parent-child (Dis) agreement on referral problems at out-patient treatment entry. Journal of Consulting and Clinical Psychology, 69, 1018–1025. doi:10.1037/0022-006X.69.6.1018.

    Article  PubMed  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Child Outcomes Research Consortium, University College London and Anna Freud Centre, 4-8 Rodney Street, London, N19JH, UK

    Jenna Jacob, Elisa Napoleone, Victoria Zamperoni, Lily Levy, Matt Barnard & Miranda Wolpert

  2. Evidence Based Practice Unit, University College London and Anna Freud Centre, London, UK

    Miranda Wolpert

Authors
  1. Jenna Jacob
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Elisa Napoleone
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Victoria Zamperoni
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Lily Levy
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Matt Barnard
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Miranda Wolpert
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Miranda Wolpert .

Editor information

Editors and Affiliations

  1. Modum Bad, Vikersund, Norway

    Terje Tilden

  2. University of Wisconsin, Madison, Wisconsin, USA

    Bruce E. Wampold

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this chapter

Cite this chapter

Jacob, J., Napoleone, E., Zamperoni, V., Levy, L., Barnard, M., Wolpert, M. (2017). How Can Outcome Data Inform Change? Experiences from the Child Mental Health Context in Great Britain, Including Barriers and Facilitators to the Collection and Use of Data. In: Tilden, T., Wampold, B. (eds) Routine Outcome Monitoring in Couple and Family Therapy. European Family Therapy Association Series. Springer, Cham. https://doi.org/10.1007/978-3-319-50675-3_15

Download citation

Publish with us

Policies and ethics

Search

Navigation