Community Mental Health Journal

, Volume 55, Issue 7, pp 1165–1172 | Cite as

Psychiatric Risk Assessment from the Clinician’s Perspective: Lessons for the Future

  • Alex S. CohenEmail author
  • Taylor Fedechko
  • Elana K. Schwartz
  • Thanh P. Le
  • Peter W. Foltz
  • Jared Bernstein
  • Jian Cheng
  • Elizabeth Rosenfeld
  • Brita Elvevåg
Original Paper


Accurate prediction of risk-states in Serious Mental Illnesses (SMIs) is critical for reducing their massive societal burden. Risk-state assessments are notably inaccurate. Recent innovations, including widely available and inexpensive mobile technologies for ambulatory “biobehavioral” data, can reshape risk assessment. To help understand and accelerate clinician involvement, we surveyed 90 multi-disciplinary clinicians serving SMI populations in various settings to evaluate how risk assessment is conducted and can improve. Clinicians reported considerable variability in conducting risk assessment, and few clinicians explicated their procedures beyond tying it to broader mental status examinations or interviews. Very few clinicians endorsed using currently-available standardized risk measures, and most reported low confidence in their utility. Clinicians also reported spending approximately half the time conducting individual risk assessments than optimally needed. When asked about improvement, virtually no clinicians acknowledged biobehavioral, objective technologies, or ambulatory recording. Overall, clinicians seemed unaware of meaningful ways to improve risk assessment.


Risk assessment Serious mental illness Technology Machine learning Objective 



This project was funded by Grant 231395 from the Research Council of Norway awarded to Brita Elvevåg.


  1. Beal, A. C. (2012). The Patient-Centered Outcomes Research Institute (PCORI) national priorities for research and initial research agenda. JAMA, 307(15), 1583. Scholar
  2. Ben-Zeev, D., Davis, K. E., Kaiser, S., Krzsos, I., & Drake, R. E. (2013). Mobile technologies among people with serious mental illness: Opportunities for future services. Administration and Policy in Mental Health and Mental Health Services Research, 40(4), 340–343. Scholar
  3. Ben-Zeev, D., Drake, R. E., Corrigan, P. W., Rotondi, A. J., Nilsen, W., & Depp, C. (2012). Using contemporary technologies in the assessment and treatment of serious mental illness. American Journal of Psychiatric Rehabilitation, 15(4), 357–376. Scholar
  4. Bernert, R. A., Hom, M. A., & Roberts, L. W. (2014). A review of multidisciplinary clinical practice guidelines in suicide prevention: Toward an emerging standard in suicide risk assessment and management, training and practice. Academic Psychiatry, 38(5), 585–592. Scholar
  5. Boudreaux, E. D., & Horowitz, L. M. (2014). Suicide risk screening and assessment: Designing instruments with dissemination in mind. American Journal of Preventive Medicine, 47(3 SUPPL. 2), S163–S169. Scholar
  6. Bridge, J. A., Horowitz, L. M., & Campo, J. V. (2017). ED-SAFE-can suicide risk screening and brief intervention initiated in the emergency department save lives? JAMA Psychiatry, 74(6), 555–556. Scholar
  7. Brodsky, S. L. (1988). Fear of litigation in mental health professionals. Criminal Justice and Behavior, 15(4), 492–500. Scholar
  8. Campbell, M. A., French, S., & Gendreau, P. (2009). The prediction of violence in adult offenders: A meta-analytic comparison of instruments and methods of assessment. Criminal Justice and Behavior, 36(6), 567–590. Scholar
  9. Cannon, T. D., Cadenhead, K., Cornblatt, B., Woods, S. W., Addington, J., Walker, E., … Heinssen, R. (2008a). Prediction of psychosis in youth at high clinical risk: A multisite longitudinal study in North America. Archives of General Psychiatry, 65(1), 28–37.
  10. Cannon, T. D., Cadenhead, K., Cornblatt, B., Woods, S. W., Addington, J., Walker, E., … Heinssen, R. (2008b). Prediction of psychosis in youth at high clinical risk: A multisite longitudinal study in North America. Archives of General Psychiatry, 65(1), 28–37.
  11. Cannon, T. D., Yu, C., Addington, J., Bearden, C. E., Cadenhead, K. S., Cornblatt, B. A., … Kattan, M. W. (2016). An individualized risk calculator for research in prodromal psychosis. American Journal of Psychiatry, 173(10), 980–988.
  12. Carrión, R. E., Cornblatt, B. A., Burton, C. Z., Tso, I. F., Auther, A. M., Adelsheim, S., … McFarlane, W. R. (2016). Personalized prediction of psychosis: External validation of the NAPLS-2 psychosis risk calculator with the EDIPPP project. American Journal of Psychiatry, 173(10), 989–996.
  13. Chan, M. K. Y., Bhatti, H., Meader, N., Stockton, S., Evans, J., O’Connor, R. C., … Kendall, T. (2016). Predicting suicide following self-harm: Systematic review of risk factors and risk scales. British Journal of Psychiatry, 209(4), 277–283.
  14. Cohen, A. S. (2019). Advancing ambulatory biobehavioral technologies beyond “proof of concept”: Introduction to the special section. Psychological Assessment, 31(3), 277.CrossRefGoogle Scholar
  15. Cohen, A. S., & Elvevåg, B. (2014). Automated computerized analysis of speech in psychiatric disorders. Current Opinion in Psychiatry, 27(3), 203–209.CrossRefGoogle Scholar
  16. Gaynes, B. N., West, S. L., Ford, C. A., Frame, P., Klein, J., & Lohr, K. N. (2004). Screening for suicide risk in adults: A summary of the evidence for the U.S. Preventive Services Task Force. Annals of Internal Medicine, 140(10), 822–835+I. Scholar
  17. Grove, W. M., Zald, D. H., Lebow, B. S., Snitz, B. E., Nelson, C., Snitz, B. E., et al. (2000). Clinical versus mechanical prediction: A meta-analysis. Hedges & Olkin, 12(1), 19–30. Scholar
  18. Hafeman, D. M., Merranko, J., Goldstein, T. R., Axelson, D., Goldstein, B. I., Monk, K., … Birmaher, B. (2017). Assessment of a person-level risk calculator to predict new-onset bipolar spectrum disorder in youth at familial risk. JAMA Psychiatry, 74(8), 841–847.
  19. Harrison, S. L., de Craen, A. J. M., Kerse, N., Teh, R., Granic, A., Davies, K., … Stephan, B. C. M. (2017). Predicting risk of cognitive decline in very old adults using three models: The framingham stroke risk profile; the cardiovascular risk factors, aging, and dementia model; and oxi-inflammatory biomarkers. Journal of the American Geriatrics Society, 65(2), 381–389.
  20. Heeringa, S. G., Gebler, N., Colpe, L. J., Fullerton, C. S., Hwang, I., Kessler, R. C., … Ursano, R. J. (2013). Field procedures in the Army Study to assess risk and resilience in servicemembers (Army STARRS). International Journal of Methods in Psychiatric Research, 22(4), 276–287.
  21. Hoffmire, C., Stephens, B., Morley, S., Thompson, C., Kemp, J., & Bossarte, R. M. (2016). VA suicide prevention applications network: A national health care system-based suicide event tracking system. Public Health Reports, 131(6), 816–821. Scholar
  22. Holmlund, T. B., Foltz, P. W., Cohen, A. S., Johansen, H. D., Sigurdsen, R., Fugelli, P., … & Elvevåg, B. (2019). Moving psychological assessment out of the controlled laboratory setting: Practical challenges. Psychological Assessment, 31(3), 292.Google Scholar
  23. Insel, T. R. (2008). Assessing the economic costs of serious mental illness. American Journal of Psychiatry, 165(6), 663–665. Scholar
  24. Joseph, J., Kremen, W. S., Franz, C. E., Glatt, S. J., van de Leemput, J., Chandler, S. D., … Twamley, E. W. (2017). Predictors of current functioning and functional decline in schizophrenia. Schizophrenia Research, 188, 158–164.
  25. Kennedy, M., & Gill, M. (1997). Patient litigation following a homicide—Implications for the assessment and managment of risk. International Review of Psychiatry, 9(2–3), 179–186.
  26. Kessler, R. C., Aguilar-Gaxiola, S., Alonso, J., Chatterji, S., Lee, S., Ormel, J., … Wang, P. S. (2009). The global burden of mental disorders: An update from the WHO World Mental Health (WMH) surveys. Epidemiologia E Psichiatria Sociale, 18(1), 23–33.
  27. Kessler, R., & Glasgow, R. E. (2011). A proposal to speed translation of healthcare research into practice: Dramatic change is needed. American Journal of Preventive Medicine, 40(6), 637–644. Scholar
  28. Kessler, R. C., Warner, C. H., Ivany, C., Petukhova, M. V., Rose, S., Bromet, E. J., … Ursano, R. J. (2015). Predicting suicides after psychiatric hospitalization in US army soldiers: The Army Study to Assess Risk and Resilience in Servicemembers (Army STARRS). JAMA Psychiatry, 72(1), 49–57.
  29. Large, M., Kaneson, M., Myles, N., Myles, H., Gunaratne, P., & Ryan, C. (2016). Meta-analysis of longitudinal cohort studies of suicide risk assessment among psychiatric patients: Heterogeneity in results and lack of improvement over time. PLoS ONE, 11(6), e0156322. Scholar
  30. Nelson, H. D., Denneson, L. M., Low, A. R., Bauer, B. W., O’Neil, M., Kansagara, D., et al. (2017a). Suicide risk assessment and prevention: A systematic review focusing on veterans. Psychiatric Services, 68(10), 1003–1015. Scholar
  31. Nelson, B., McGorry, P. D., Wichers, M., Wigman, J. T. W., & Hartmann, J. A. (2017b). Moving from static to dynamic models of the onset of mental disorder a review. JAMA Psychiatry, 74(5), 528–534. Scholar
  32. O’Hagan, M., Cornelius, V., Young, A. H., & Taylor, D. (2017). Predictors of rehospitalization in a naturalistic cohort of patients with bipolar affective disorder. International Clinical Psychopharmacology, 32(3), 115–120. Scholar
  33. Passmore, K., & Leung, W. C. (2002). Defensive practice among psychiatrists: A questionnaire survey. Postgraduate Medical Journal, 78(925), 671–673. Scholar
  34. Peek, C. J., Glasgow, R. E., Stange, K. C., Klesges, L. M., Peyton Purcell, E., & Kessler, R. S. (2014). The 5 r’s: An emerging bold standard for conducting relevant research in a changing world. Annals of Family Medicine, 12(5), 447–455. Scholar
  35. Pestian, J. P., Sorter, M., Connolly, B., Bretonnel Cohen, K., McCullumsmith, C., Gee, J. T., … Murphy, C. (2017). A machine learning approach to identifying the thought markers of suicidal subjects: A prospective multicenter trial. Suicide and Life-Threatening Behavior, 47(1), 112–121.
  36. Rothwell, P. M. (2005). External validity of randomised controlled trials: “To whom do the results of this trial apply?”. Lancet, 365(9453), 82–93. Scholar
  37. Rubin, A., Dolev, T., & Zilcha-Mano, S. (2016). Patient demographics and psychological functioning as predictors of unilateral termination of psychodynamic therapy. Psychotherapy Research. Scholar
  38. Schoenbaum, M., Kessler, R. C., Gilman, S. E., Colpe, L. J., Heeringa, S. G., Stein, M. B., … Cox, K. L. (2014a). Predictors of suicide and accident death in the army study to assess risk and resilience in servicemembers (army starrs) results from the army study to assess risk and resilience in servicemembers (Army STARRS). JAMA Psychiatry, 71(5), 493–503.
  39. Schoenbaum, M., Kessler, R. C., Gilman, S. E., Colpe, L. J., Heeringa, S. G., Stein, M. B., … Cox, K. L. (2014b). Predictors of suicide and accident death in the army study to assess risk and resilience in servicemembers (Army STARRS) results from the army study to assess risk and resilience in servicemembers (Army STARRS). JAMA Psychiatry, 71(5), 493–503.
  40. Seidman, L. J., Shapiro, D. I., Stone, W. S., Woodberry, K. A., Ronzio, A., Cornblatt, B. A., … Woods, S. W. (2016). Association of neurocognition with transition to psychosis: Baseline functioning in the second phase of the north American prodrome longitudinal study. JAMA Psychiatry, 73(12), 1239–1248.
  41. Stuart, M. E., & Weinrich, M. (1998). Beyond managing medicaid costs: Restructuring care. The Milbank Quarterly, 76(2), 251–280.CrossRefGoogle Scholar
  42. Torous, J., & Baker, J. T. (2016). Why psychiatry needs data science and data science needs psychiatry connecting with technology. JAMA Psychiatry, 73(1), 3–4. Scholar
  43. Vassos, E., Di Forti, M., Coleman, J., Iyegbe, C., Prata, D., Euesden, J., … Breen, G. (2017). An examination of polygenic score risk prediction in individuals with first-episode psychosis. Biological Psychiatry, 81(6), 470–477.
  44. Velligan, D. I., Weiden, P. J., Sajatovic, M., Scott, J., Carpenter, D., Ross, R., … Expert Consensus Panel on Adherence Problems in Serious and Persistent Mental Illness. (2009). The expert consensus guideline series: Adherence problems in patients with serious and persistent mental illness. The Journal of Clinical Psychiatry, 70(Suppl. 4), 1–46.
  45. Vitacco, M. J., Tabernik, H. E., Zavodny, D., Bailey, K., & Waggoner, C. (2016). Projecting risk: The importance of the HCR-20 risk management scale in predicting outcomes with forensic patients. Behavioral Sciences & The Law, 34(2–3), 308–320. Scholar
  46. Walsh, C. G., Ribeiro, J. D., & Franklin, J. C. (2017). Predicting risk of suicide attempts over time through machine learning. Clinical Psychological Science, 5(3), 457–469. Scholar
  47. Wand, T. (2012). Investigating the evidence for the effectiveness of risk assessment in mental health care. Issues in Mental Health Nursing, 33(1), 2–7. Scholar
  48. Wilbert, J. R., & Fulero, S. M. (1988). Impact of malpractice litigation on professional psychology: Survey of practitioners. Professional Psychology: Research and Practice, 19(4), 379–382. Scholar
  49. Wu, E. Q., Birnbaum, H. G., Shi, L., Ball, D. E., Kessler, R. C., Moulis, M., et al. (2005). The economic burden of schizophrenia in the United States in 2002. Journal of Clinical Psychiatry, 66(9), 1122–1129. Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  • Alex S. Cohen
    • 1
    Email author
  • Taylor Fedechko
    • 1
  • Elana K. Schwartz
    • 1
  • Thanh P. Le
    • 1
  • Peter W. Foltz
    • 2
  • Jared Bernstein
    • 3
  • Jian Cheng
    • 3
  • Elizabeth Rosenfeld
    • 3
  • Brita Elvevåg
    • 4
    • 5
  1. 1.Department of PsychologyLouisiana State UniversityBaton RougeUSA
  2. 2.Institute of Cognitive ScienceUniversity of ColoradoBoulderUSA
  3. 3.Analytic Measures IncPalo AltoUSA
  4. 4.Department of Clinical MedicineUniversity of Tromsø – The Arctic University of NorwayTromsøNorway
  5. 5.The Norwegian Centre for eHealth ResearchUniversity Hospital of North NorwayTromsøNorway

Personalised recommendations