Abstract
The term “effective,” on its own, is honorific but vague. Interventions against serious mental illness may be “effective” at goals as diverse as reducing “apparent sadness” or providing housing. Underexamined use of “effective” and other success terms often obfuscates differences and incompatibilities in interventions, degrees of effectiveness, key omissions in effectiveness standards, and values involved in determining what counts as “effective.” Yet vague use of such success terms is common in the research, clinical, and policy realms, with consequences that negatively affect the care offered to individuals experiencing serious mental illness. A pragmatist-oriented solution to these problems suggests that when people use success terms, they need to explain and defend the goals and supporting values embedded in the terms, asking and answering the questions, “Effective at what? For whom? How effective? And why that goal?” Practical and epistemic standards for effectiveness will likely remain plural for good reasons, but each standard should be well explained and well justified.
Similar content being viewed by others
Notes
We use the definition of serious mental illness proposed by the National Institute of Mental Health (NIMH): “a mental, behavioral, or emotional disorder resulting in serious functional impairment, which substantially interferes with or limits one or more major life activities” [46].
The document does sketch a range of questions that need addressing in research, such as “the efficacy and effectiveness of psychological treatments with children and youths at different developmental stages,” [2, p. 275]. It also notes the variability of interventions—for example, the report notes that “…psychological services are most likely to be effective when they are responsive to the patient’s specific problems, strengths, personality, sociocultural context, and preferences” (p. 278). Even with such refinements, however, the criteria for effectiveness can be specified in multiple ways.
Perhaps this blandness explains why the use of general success terms like “effectiveness” has not been much examined in the literature we surveyed. Sandra Tanenbaum’s work is a notable exception. Within the context of critique of evidence-based practice, Tanenbaum presents well the basic insight that the definition of “effective” is both unclear and contentious [62]. She points out that “effectiveness” is defined in objective terms in EBP, and that these measures to do not reflect treatment outcomes pursued by many. She also raises the critical issue of who is defining effectiveness, and she points out the potential coerciveness of implementing treatment or policy with “effectiveness” defined in certain ways. Our discussion is broader than Tanenbaum’s, notably in pointing out that the need to define effectiveness is critical for all involved in SMI intervention, not just those doing EBP. In addition, while Tanenbaum points out that “what works” has moral connotations, she doesn’t enlarge on this or explain the moral connotations of other approaches, nor does she demand specification of goals. Cartwright and Stegenga are another exception; they consider the term in discussing criteria and methods for translating scientific efficacy to policy effectiveness [10].
A third conceptual issue concerns the complexities of translating scientifically demonstrated efficacy or clinically demonstrated need to effective policy; on this issue we offer a pragmatist-oriented direction, but details are beyond the scope of this paper.
With respect to psychological research, Paul independently developed a similar statement of the problem in 1967 [51].
Some “non-clients” are people experiencing SMI who cannot access care. Others are non-clients because they prefer to avoid intervention, tend to their own care, or work with peer interventions; their perspectives are represented in this section and elsewhere.
The term “effectiveness” implies an endpoint, while “process” and “relationship” do not necessarily. Nevertheless, processes and relationships can be better or worse (i.e., more effective) at achieving other goals, and more or less satisfying in the present (hence, more effective at an overall goal of a life well lived). Attempts to assess the quality of processes and relationships are therefore common in research and practice.
Additionally, a few bioethicists argue that effective treatment goals are not appropriate for all clients with SMI. For example, Wijsbek argues that euthanasia may be appropriate for intractably depressed patients [65], and Draper suggests that treatment refusal, rather than forced feeding [18], should be allowed for some patients who suffer from severe and prolonged anorexia nervosa. It is not as clear that providers endorse such positions.
Drug companies have in recent years decreased their spending on psychiatric drug development. This is not necessarily a good thing, given that improved medications, with fewer side effects, could be useful.
The knowledge base on interventions for serious mental illness is weak even taken according to commonly accepted “rules”; that is, accepting DSM categories and the standards of evidence-based medicine. For example, the 2005-2015 Cochrane database entries for bipolar disorder and schizophrenia find “strong” evidence that antipsychotic medications relieve positive symptoms of these disorders in the short term. But that is the only strong evidence found among hundreds of articles that review dozens of medication types, psychosocial interventions, and endpoints.[Unpublished analysis by one of the authors, SCH] Morlino et al. [44] undertook a study of systematic reviews of interventions for schizophrenia in more databases, including the Ovid and Pubmed databases as well as Cochrane. They determined that only 25% of the reviews produced definitive conclusions. In addition to these concerns, Leucht et al. also point out that the entire Cochrane database (comprising meta-analyses including 13,542 articles, by their count) concerns only “first pass” efficacy—that is, the articles consider the effectiveness of initial interventions, but don’t provide guidance on the clinically crucial question of what to do if the approach fails [39]. Turning to the literature, the group found only 10 articles that addressed switching antipsychotics, and found that none of these gave conclusive information.
The Montgomery-Asberg Depression Rating Scale (MADRS) is scored on a 60-point scale. The items examined include apparent sadness, reported sadness, inner tension, reduced sleep, reduced appetite, concentration difficulties, lassitude, inability to feel, pessimistic thoughts, and suicidal thoughts. For example, the “apparent sadness” section assesses “despondency, gloom, and despair (more than just ordinary transient low spirits) reflected in speech, posture, facial expression, and posture.” The rater is instructed to “rate by depth and inability to brighten up” with the following scale: 0 points for “No sadness”; 2 points for “Looks dispirited but does brighten up without difficulty”; 4 points for “Appears sad and unhappy most of the time”; and 6 points for “Looks miserable all the time. Extremely despondent.”
Thank you to an anonymous reviewer for making this point.
Relative to high-quality quantitative research, qualitative research has less control and generalizability. Both qualitative and quantitative research embed biases; interpretive methodologies make qualitative research especially prone to observer biases.
Given the likely connection of these “lifestyle” variations to mental illness, drug side effects, or both, the term “lifestyle,” suggesting full agency and responsibility, is likely misleading here. Still, well-targeted interventions might enhance quality or duration of life for people experiencing schizophrenia.
According to a recent meta-analysis, effect sizes are mostly modest and acceptability is lower relative to placebo [12].
Debates over the rationality of goals expressed by individuals with some forms of SMI, or by individuals at some stages of illness, introduce complex issues involving judgments of goal rationality and whether goals need to be rational. We will not debate that issue here, but we point out that it applies to a minority of people with SMI.
We say “at least partially inconsistent” because complications of up-front versus net costs, among other factors, make costs difficult to determine.
Tanenbaum makes a similar point in the context of EBP “…whenever EBP researchers state their outcome measures, [T]hey should say why they have chosen those measures and whose interests they represent (p. 171).” [62] Our point is that it is not just EBP researchers that need to do this, but all researchers, and providers and policy-makers as well.
Other questions might also point to values. For example, asking “Effective at what?” and “For whom?” jointly might point toward effectiveness standards that benefit caregivers or investors more than patients.
See also Footnote 7.
References
Abassi, M., Majdzadeh, R., Zali, A., Karimi, A., & Akrami, F. (2018). The evolution of public health ethics frameworks: Systematic review of moral values and norms in public health policy. Medicine, Health Care and Philosophy, 21(3), 387–402.
American Psychological Association Presidential Task Force on Evidence-Based Practice. (2006). Evidence-based practice in psychology. American Psychologist, 61(4), 271–285. https://doi.org/10.1037/0003-066X.61.4.271.
Andresen, R., Caputi, P., & Oades, L. G. (2010). Do clinical outcome measures assess consumer-defined recovery? Psychiatry Research, 177, 309–317.
Arts, B., Simons, C. J., Drukker, M., & van Os, J. (2013). Antipsychotic medications and cognitive functioning in bipolar disorder: Moderating effects of COMT Val108/158 Met genotype. BMC Psychiatry, 13, 63. https://doi.org/10.1186/1471-244X-13-63.
Bauer, M., El-Khalili, N., Datto, C., Szamosi, J., & Eriksson, H. (2010). A pooled analysis of two randomised, placebo-controlled studies of extended release quetiapine fumarate adjunctive to antidepressant therapy in patients with major depressive disorder. Journal of Affective Disorders, 127(1–3), 19–30. https://doi.org/10.1016/j.jad.2010.08.032.
Berg, H., & Slaattelid, R. (2017). Facts and values in psychotherapy—A critique of the empirical reduction of psychotherapy within evidence-based practice. Journal of Evaluation in Clinical Practice, 23, 1075–1080.
Cabassa, L. J., Ezell, J. M., & Lewis-Fernandez, R. (2010). Lifestyle interventions for adults with serious mental illness: A systematic literature review. Psychiatric Services (Washington, DC), 61(8), 774–782. https://doi.org/10.1176/ps.2010.61.8.774.
Carlier, I. V. E., Meuldijk, D., Van Vliet, I. M., Van Fenema, E., Van der Wee, N. J. A., & Zitman, F. G. (2012). Routine outcome monitoring and feedback on physical or mental health status: Evidence and theory. Journal of Evaluation in Clinical Practice, 18(1), 104–110.
Carpenter, J. (2002). Mental health recovery paradigm: Implications for social work. Health and Social Work, 27(2), 86–94.
Cartwright, N., & Stegenga, J. (2011). A theory of evidence for evidence-based policy. In P. Dawid, T. Vasiliki, & M. Vasiliki (Eds.), Evidence, Inference and Enquiry (pp. 289–319, Proceedings of the British Academy, Vol. 171): Oxford University Press/British Academy.
Chamberlin, J., & Rogers, J. A. (2006). Planning a community-based mental health system, perspective of service recipients. In L. Davidson, C. Harding, & L. Spaniol (Eds.), Recovery from severe mental illnesses: Research evidence and implications for practice (Vol. 2, pp. 405–413). Boston: Center for Psychiatric Research.
Cipriani, A., Furukawa, T. A., Salanti, G., Chaimani, A., Atkinson, L. Z., Ogawa, Y., et al. (2018). Comparative efficacy and acceptability of 21 antidepressant drugs for the acute treatment of adults with major depressive disorder: A systematic review and network meta-analysis. The Lancet, 391(10128), 1357–1366.
Davidson, L., & Roe, D. (2007). Recovery from versus recovery in serious mental illness: One strategy for lessening confusion plaguing recovery. Journal of Mental Health, 16(4), 459–470.
Davidson, L., Harding, C., & Spaniol, L. (2005). Preface. In L. Davidson, C. Harding, & L. Spaniol (Eds.), Recovery from severe mental illnesses: Research evidence and implications for practice (Vol. 1, p. xix). Boston: Center for Psychiatric Rehabilitation.
Davidson, L., O’Connell, M., Tondora, J., Staeheli, M., & Evans, A. C. (2005). Recovery in serious mental illness: Paradigm shift or shibboleth? In L. Davidson, C. Harding, & L. Spaniol (Eds.), Recovery from severe mental illnesses: Research evidence and implications for practice (Vol. 1, pp. 5–26). Boston: Center for Psychiatric Rehabilitation.
Douglas, H. E. (2016). Science, policy, values: Exploring the nexus. Perspectives on Sciene, 24(5), 475–480.
Drake, R. E., Bond, G. R., Goldman, H. H., Hogan, M. F., & Karakus, M. (2016). Individual placement and support services boost employment for people with serious mental illnesses, but funding is lacking. Health Affairs; Chevy Chase, 35(6), 1098–1105.
Draper, H. (2000). Anorexia nervosa and respecting a refusal of life-prolonging therapy: A limited justification. Bioethics, 14(2), 120–133.
El-Khalili, N., Joyce, M., Atkinson, S., Buynak, R. J., Datto, C., Lindgren, P., et al. (2010). Extended-release quetiapine fumarate (quetiapine XR) as adjunctive therapy in major depressive disorder (MDD) in patients with an inadequate response to ongoing antidepressant treatment: A multicentre, randomized, double-blind, placebo-controlled study. International Journal of Neuropsychopharmacology, 13(7), 917–932. https://doi.org/10.1017/S1461145710000015.
Essock, S. M., Goldman, H. H., Van Tosh, L., Anthony, W. A., Appell, C. R., Bond, G. R., et al. (2006). Evidence-based practices: Setting the context and responding to concerns. In L. Davidson, C. Harding, & L. Spaniol (Eds.), Recovery from severe mental illnesses: Research evidence and implications for practice (Vol. 2, pp. 414–434). Boston: Center for Psychiatric Rehabilitation.
Frese, F. J., III, Jonathan, S., Kress, K., & Vogel-Scibilia, S. (2006). Integrating evidence-based practices and the recovery model. In L. Davidson, C. Harding, & L. Spaniol (Eds.), Recovery from severe mental illnesses: Research evidence and implications for practice (Vol. 2, pp. 375–390). Boston: Center for Psychiatric Research.
Garcia, A. M. (2010). What does “work” mean? Reopening the debate about clinical significance. Clinical Psychology: Science and Practice, 17(1), 48–51.
Green, C. A., Estroff, S. E., Yarborough, B. J. H., Spofford, M., Solloway, M. R., Kitson, R. S., et al. (2014). Directions for future patient-centered and comparative effectiveness research for people with serious mental illness in a learning mental health care system. Schizophrenia Bulletin, 40(Suppl 1), S1–S94.
Gupta, M. (2013). Psychiatry and evidence-based psychiatry: A distinction with a difference. Philosophy, Psychiatry, and Psychology, 19(4), 309–312.
Hawthorne, S., & Ihlan, A. (2018). Rethinking civil commitment: The radical resources of the ethics of care. Public Philosophy Journal. https://doi.org/10.25335/m5/ppj.1.1-4.
Insel, T. R. (2016). The anatomy of NIMH funding. https://www.nimh.nih.gov/funding/funding-strategy-for-research-grants/the-anatomy-of-nimh-funding.shtml. Accessed 16 December 2016.
Institute of Medicine: Division of Health Promotion and Disease Prevention. (1992). The second fifty years: Promoting health and preventing disability. In R. L. Berg & J. S. Cassells (Eds.), National Academy Press.
Ioannidis, J. P. A. (2008). Effectiveness of antidepressants: An evidence myth constructed from a thousand randomized trials? Philosophy, Ethics, and Humanities in Medicine, 3(1), 14.
Jacobson, N., & Curtis, L. (2006). Recovery as policy in mental health services: Strategies emerging from the states. In L. Davidson, C. Harding, & L. Spaniol (Eds.), Recovery from severe mental illness: Research evidence and implications for practice (Vol. 2, pp. 358–374). Boston: Center for Psychiatric Rehabilitation.
Jacobson, N. S., Roberts, L. J., Berns, S. B., & McGlinchey, J. B. (1999). Methods for defining and determining the clinical significance of treatment effects: Description, application, and alternatives. Journal of Consulting and Clinical Psychology, 67(3), 300–307.
Joint Commission on Mental Illness and Health. (1961). Action for mental health; final report, 1961. New York: Basic Books.
Kessler, R. C., Chiu, W. T., Demler, O., Merikangas, K. R., & Walters, E. E. (2005). Prevalence, severity, and comorbidity of 12-month DSM-IV disorders in the National Comorbidity Survey Replication. Archives of General Psychiatry, 62(6), 617–627. https://doi.org/10.1001/archpsyc.62.6.617.
Khandaker, G. M., Gandamaneni, P. K., Dibben, C., Cherukuru, S., Cairns, P., & Manaan, K. R. (2013). Evaluating care pathways for community psychiatry in England: A qualitative study. Journal of Evaluation in Clinical Practice, 19(2), 289–303.
Kincaid, H., Dupre, J., & Wylie, A. (Eds.). (2007). Value-free science? Ideals and illusions. Oxford: Oxford University Press.
Kravitz, R. L., Duan, N., & Braslow, J. (2004). Evidence-based medicine, heterogeneity of treatment effects, and the trouble with averages. THe Milbank Quarterly, 82(4), 661–687.
Krimsky, S. (2003). Science in the private interest: Has the lure of corporate profit corrupted biomedical research?. Lanham: Rowman & Littlefield.
Lambert, M. J., & Ogles, B. M. (2009). Using clinical significance in psychotherapy outcome research: The need for a common procedure and validity data. Psychotherapy Research, 19(4–5), 493–501. https://doi.org/10.1080/10503300902849483.
Larsen-Barr, M. (2016). Experiencing antipsychotic medication: From first prescriptions to attempted discontinuation, dissertation. Auckland: University of Auckland.
Leucht, S., Winter-van Rossum, I., Heres, S., Arango, C., & Fleischhacker, W. W. (2015). The optimization of treatment and management of schizophrenia in Europe (OPTiMiSE) trial: Rationale for its methodology and a review of the effectiveness of switching antipsychotics. Schizophrenia Bullettin, 41(3), 549–558.
Little, M., Lipworth, W., Gordon, J., Markham, P., & Kerridge, I. (2012). Values-based medicine and modest foundationalism. Journal of Evaluation in Clinical Practice, 18(5), 1020–1026.
Longino, H. E. (1990). Science as social knowledge: Values and objectivity in scientific inquiry. Princeton, NJ: Princeton University Press.
Longino, H. E. (2002). The fate of knowledge. Princeton, NJ: Princeton University Press.
Mead, S., & Copeland, M. E. (2005). What recovery means to us: Consumers’ perspectives. In L. Davidson, C. Harding, & L. Spaniol (Eds.), Recovery from severe mental illnesses: Research evidence and implications for practice (Vol. 1). Boston: Center for Psychiatric Rehabilitation.
Morlino, M., Calento, A., Pannone, G., Ventrella, G., & Schiavone, V. (2009). Systematic reviews to support evidence-based psychiatry: What about schizophrenia? Journal of Evaluation in Clinical Practice, 15(6), 1025–1028. https://doi.org/10.1111/j.1365-2753.2009.01146.x.
Morrison, L. J. (2005). Talking back to psychiatry: The psychiatric consumer/survivor/ex-patient movement (New Approaches in Sociology: Studies in Social Inequality, Social Change, and Social Justice, N. Naples, Ed.). New York: Routledge.
National Institute of Mental Health. (2017). Mental illness. https://www.nimh.nih.gov/health/statistics/mental-illness.shtml. Accessed 17 September 2018.
Network, H. V. (2015). Hearing voices network. https://www.hearing-voices.org/about-us/real-change-campaign/. Accessed 17 September 2018.
Nys, T. R. V., & Nys, M. G. (2006). Psychiatry under pressure: Reflections on psychiatry’s drift towards a reductionist biomedical conception of mental illness. Medicine, Health Care and Philosophy, 9(1), 107–115.
Oaks, D. W. (2011). The moral imperative for dialogue with organizations of survivors of coerced psychiatric human rights violations. In T. W. Kallert, J. E. Mezzich, & J. Monahan (Eds.), Coercive treatment in psychiatry: Clinical, legal and ethical aspects (pp. 187–211). Chichester/Hoboken, NJ: Wiley/Blackwell.
Padgett, D. K., Stanhope, V., Henwood, B. F., & Stefanic, A. (2011). Substance use outcomes among homeless clients with serious mental illness: Comparing housing first with treatment first programs. Community Mental Health Journal, 47, 227–232.
Paul, G. (1967). Strategy of outcome research in psychotherapy. Journal of Consulting Psychology, 31(2), 109–118.
Payne, M. (2016). Modern social work theory (4th ed.). New York: Oxford University Press.
Pickersgill, M. D. (2014). Debating DSM-5: Diagnosis and the sociology of critique. Journal of Medical Ethics, 40(8), 521–525. https://doi.org/10.1136/medethics-2013-101762.
Reardon, S. (2017). US mental-health agency’s push for basic research has slashed support for clinical trials. Nature, 546(7658), 339.
Reznek, L. (1991). The philosophical defence of psychiatry (Philosophical Issues in Science, ed. W. H. Newton-Smith). London: Routledge.
Robotham, D., Wykes, T., Rose, D., Doughty, L., Strange, S., Neale, J., et al. (2016). Service user and carer priorities in a biomedical research centre for mental health. Journal of Mental Health, 25(3), 185–188.
Rog, D. J. (2005). The evidence on supported housing. In L. Davidson, C. Harding, & L. Spaniol (Eds.), Recovery from severe mental illnesses: Research evidence and implications for practice (Vol. 1, pp. 358–376). Boston: Center for Psychiatric Rehabilitation.
Shearer-Underhill, C., & Marker, C. (2010). The use of the number needed to treat (NNT) in randomized clinical trials in psychological treatment. Clinical Psychology: Science and Practice, 17(1), 41–47.
Sisti, D. A., Segal, A. G., & Emanuel, E. J. (2015). Improving long-term psychiatric care: Bring back the asylum (Historical Article). JAMA, 313(3), 243–244. https://doi.org/10.1001/jama.2014.16088.
Solomon, M. (2015). Making medical knowledge. Oxford: Oxford University Press.
Stettin, B., Geller, J., Ragosta, K., Cohen, K., & Ghowrwal, J. (2014). Mental health commitment laws: A survey of the states. Arlington, VA: Treatment Advocacy Center.
Tanenbaum, S. J. (2005). Evidence-based practice as mental health policy: Three controversies and a caveat. Health Affairs (Millwood), 24(1), 163–173. https://doi.org/10.1377/hlthaff.24.1.163.
Tanenbaum, S. J. (2008). Perspectives on evidence-based practice from consumers in the US public mental health system. Journal of Evaluation in Clinical Practice, 14, 699–706.
Vanheule, S., & Devisch, I. (2014). Mental suffering and the DSM-5: A critical review. Journal of Evaluation in Clinical Practice, 20, 975–980.
Wijsbek, H. (2012). ‘To thine own self be true’: On the loss of integrity as a kind of suffering. Bioethics, 26(1), 1–7.
Yanos, P. T., Barrow, S. M., & Tsemberis, S. (2004). Community integration in the early phase of housing among homeless persons diagnosed with severe mental illness: Successes and challenges. Community Mental Health Journal, 40(2), 133–150.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Hawthorne, S., Williams-Wengerd, A. ‘Effective’ at What? On Effective Intervention in Serious Mental Illness. Health Care Anal 27, 289–308 (2019). https://doi.org/10.1007/s10728-019-00367-9
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10728-019-00367-9