Back to top Open menu arrow
Purchase PDF Reprints

The Journal of
Clinical Psychiatry
Members only

The Journal of Clinical Psychiatry

Original Research Focus on Suicide October 16, 2026

Assessing Suicide Risk Reduction in the Interval Before Death by Suicide Using Natural Language Processing

; ; ; ;

J Clin Psychiatry 2026;87(4):25m16249

Abstract

Objective: High-suicide-risk patients who quickly respond to psychotherapy, such that they cease being classified as high-risk, frequently die by suicide soon after risk declassification. It is accordingly difficult to distinguish symptom improvement and changes that may be fleeting or associated with evading treatment. This exploratory study used natural language processing to examine care patterns preceding suicide among Veterans Affairs (VA) patients whose classified-risk status remained elevated and those whose classified risk diminished in the interval before death by suicide.

Methods: We identified (n=89) patients classified as high-suicide-risk whose high-risk status was later removed and died by suicide within 6 months, between 2015–2018 (Reduced Clinical Risk Classification [RCRC]). We then selected comparison (n=89) patients who retained high-risk status and died within the same interval (Sustained Clinical Risk Classification [SCRC]). We analyzed electronic health records using statistical models and topic modeling, a natural language processing method that gauges lexical patterns, and monitored risk fluctuations.

Results: RCRC patients were younger, with fewer psychiatric diagnoses. Topics included “Psychotherapy,” “Medication,” “Outpatient,” “Follow-up,” and “Suicide-Assessment.” After risk declassification, RCRC patients demonstrated sharp declines across all topics, whereas SCRC patients maintained or increased engagement.

Conclusion: Our preliminary results suggest that RCRC patient records featured more psychotherapy-related terminology, while SCRC patient records featured more structured and medically oriented terminology. This dichotomy may infer distinct care trajectories. Perceived symptom improvement could prompt reduced surveillance, creating unrecognized vulnerability. Our findings are VA specific and should be interpreted cautiously. Future work will compare patients who died and did not die after risk declassification.

J Clin Psychiatry 2026;87(4):25m16249

Author affiliations are listed at the end of this article.

Also from JCP

Life’s Essential 8 Cardiovascular Health and Suicide Mortality Association of Lithium Use With Suicide and Self-Harm Among Chinese Adolescents and Adults With Bipolar Disorder Impact of VNS on Suicidal Ideation in Markedly Treatment-Resistant Major Depression: A RECOVER Study Report

From the Editors

New Series The JCP Podcast Stalling XT at 100 mg may actually raise the risk of GI-related dropout, while moving to 125 mg can improve tolerability and retention. This practical conversation gets into the dosing, ondansetron timing, and side effect clues that can make or break real-world use in schizophrenia.

Members Only Content

This full article is available exclusively to Professional tier members. Subscribe now to unlock the HTML version and gain unlimited access to our entire library plus all PDFs. If you're already a subscriber, please log in below to continue reading.

Subscribe Now

Already a member? Log in

  1. Mittal V, Brown WA, Shorter E. Are patients with depression at heightened risk of suicide as they begin to recover? PS. 2009;60(3): 384–386. PubMed CrossRef
  2. Obegi JH. Differentiating genuine from feigned suicidality in corrections: a necessary but perilous task. Int J Law Psychiatr. 2020;71:101573. PubMed CrossRef
  3. Simon R, Gutheil TG. Sudden improvement among high-risk suicidal patients: should it be trusted? PS. 2009;60(3):387–389. PubMed CrossRef
  4. McCoy TH, Castro VM, Roberson AM, et al. Improving prediction of suicide and accidental death after discharge from general hospitals with natural language processing. JAMA Psychiatry. 2016;73(10):1064–1071. PubMed CrossRef
  5. Riblet NB, Shiner B, Schnurr P, et al. A pilot study of an intervention to prevent suicide after psychiatric hospitalization. J Nerv Ment Dis. 2019;207(12): 1031–1038. PubMed CrossRef
  6. Rufino KA, Beyene H, Poa E, et al. A false sense of security: rapid improvement as a red flag for death by suicide. J Consult Clin Psychol. 2022; 90(5):405–412. CrossRef
  7. Wagner TH, Schmidt A, Belli F, et al. Health insurance enrollment among US veterans, 2010-2021. JAMA Netw Open. 2024;7(8): e2430205. PubMed CrossRef
  8. Smith AR, Forrest LN, Kinkel-Ram SS, et al. A longitudinal network analysis of suicide risk factors among service members and veterans sampled for suicidal ideation or attempt. Psychol Med. 2024;54(10): 2623–2633. CrossRef
  9. VA. 2022 National Veteran Suicide Prevention Annual Report. 2022. Accessed January 6, 2026. https://www.mentalhealth.va.gov/docs/data-sheets/2022/2022-National-Veteran-Suicide-Prevention-Annual-Report-FINAL-508.pdf
  10. Xu B, Basha SAJ, Xie Z, et al. Suicide risk in U.S. adults: alcohol misuse, veteran status, and urbanization comparison. J Rural Ment Health. 2024; 48(3):156–167. PubMed CrossRef
  11. Nock MK, Ramirez F, Rankin O. Advancing our understanding of the who, when, and why of suicide risk. JAMA Psychiatry. 2019;76(1):11–12. PubMed CrossRef
  12. Rocky Mountain MIRECC for Suicide Prevention, U.S. Department of Veterans Affairs, Cannizzaro K. REACH VET and the possible impact on integrated healthcare. 2018. Accessed January 6, 2026. https://avapl.org/conference/pubs/2018%20Conference%20Presentations/Cannizzaro%20-%20slides%20-%20Open%20Plenary.pdf
  13. Kessler RC, Hwang I, Hoffmire CA, et al. Developing a practical suicide risk prediction model for targeting high-risk patients in the Veterans Health Administration. Int J Methods Psychiatr Res. 2017;26(3):e1575. PubMed CrossRef
  14. McCarthy JF, Bossarte RM, Katz IR, et al. Predictive modeling and concentration of the risk of suicide: implications for preventive interventions in the US Department of Veterans Affairs. Am J Public Health. 2015;105(9):1935–1942. PubMed CrossRef
  15. Hein TC, Peltzman T, Hallows J, et al. Suicide mortality among Veterans Health Administration care recipients with suicide risk record flags. PS. 2021: appi.ps.2020007.
  16. Jobes DA, Gregorian MJ, Colborn VA. A stepped care approach to clinical suicide prevention. Psychol Serv. 2018;15(3):243–250. PubMed CrossRef
  17. Chen JA, Devendorf AR, Du Pont A, et al. Provider perspectives on care for veterans with electronic health record flags for high suicide risk: a mixed methods study. Psychol Serv. 2025.
  18. Landes SJ, Matarazzo BB, Pitcock JA, et al. Impact of implementation facilitation on the REACH VET clinical program for veterans at risk for suicide. PS. 2024;75(8): 726–732. PubMed CrossRef
  19. Bentley KH, Madsen EM, Song E, et al. Determining distinct suicide attempts from recurrent electronic health record codes: classification study. JMIR Form Res. 2024;8:e46364. CrossRef
  20. Berman AL, Silverman MM. Suicide risk assessment and risk formulation part II : suicide risk formulation and the determination of levels of risk. Suicide Life Threat Behav. 2014;44(4):432–443. PubMed CrossRef
  21. Fowler JC. Suicide risk assessment in clinical practice: pragmatic guidelines for imperfect assessments. Psychotherapy. 2012;49(1):81–90. PubMed CrossRef
  22. Østlie K, Stänicke E, Haavind H. Assessments in psychotherapy with suicidal patients: the precedence of alliance work. J Contemp Psychother. 2022;52(2): 89–97. CrossRef
  23. VA DoD. Center of Excellence for Suicide Prevention. Joint Department of Veterans Affairs (VA) and Department of Defense (DoD) Mortality Data Repository - National Death Index (NDI). MIRECC; 2020. https://www.mirecc.va.gov/suicideprevention/documents/VA_DoD-MDR_Flyer.pdf. Accessed December 31, 2020
  24. Kessler RC, Bauer MS, Bishop TM, et al. Using administrative data to predict suicide after psychiatric hospitalization in the Veterans Health Administration system. Front Psychiatry. 2020;11:390. PubMed CrossRef
  25. Kessler RC, Bauer MS, Bishop TM, et al. Evaluation of a model to target high-risk psychiatric inpatients for an intensive postdischarge suicide prevention intervention. JAMA Psychiatry. 2023;80(3):230–240. CrossRef
  26. Britton PC, Ilgen MA, Bohnert KM, et al. Shorter-and longer-term risk for suicide among male US military veterans in the year after discharge from psychiatric hospitalization. J Clin Psychiatry. 2021;82(2):19m13228. PubMed CrossRef
  27. Pratt JW. Remarks on zeros and ties in the Wilcoxon signed rank procedures. J Am Stat Assoc. 1959;54(287):655–667. CrossRef
  28. Health Services Research & Development. Corporate Data Warehouse (CDW). 2026. Accessed January 6, 2026. https://www.hsrd.research.va.gov/for_researchers/cdw.cfm
  29. Kessler RC, Bossarte RM, Luedtke A, et al. Suicide prediction models: a critical review of recent research with recommendations for the way forward. Mol Psychiatry. 2020;25(1):168–179. PubMed CrossRef
  30. Andrade C. Mean difference, standardized mean difference (SMD), and their use in meta-analysis: as simple as it gets. J Clin Psychiatry. 2020; 81(5):20f13681. CrossRef
  31. Bhandarkar AR, Arya N, Lin KK, et al. Building a natural language processing artificial intelligence to predict suicide-related events based on patient portal message data. Mayo Clin Proc Digit Health. 2023;1(4):510–518. CrossRef
  32. Li S, Levis M, DiMambro M, et al. Preprocessing of natural language process variables using a data-driven method improves the association with suicide risk in a large Veterans Affairs population. Comput Biol Med. 2025;189:109939. CrossRef
  33. Levy O, Goldberg Y, Dagan I. Improving distributional similarity with lessons learned from word embeddings. TACL. 2015;3:211–225. CrossRef
  34. Baayen RH, Linke M. Generalized additive mixed models. In: Paquot M, Gries ST, eds. A Practical Handbook of Corpus Linguistics. Springer International Publishing; 2020:563–591.
  35. Maestrini L, Hui FKC, Welsh AH. Restricted maximum likelihood estimation in generalized linear mixed models; 2024. Arxiv. Preprint Posted Online
  36. Miller DL. Bayesian views of generalized additive modelling. Methods Ecol Evolu. 2025;16(3):446–455. CrossRef
  37. Sievert C, Shirley KE. LDAvis: a method for visualizing and interpreting topics. In: Proceedings of the Workshop on Interactive Language Learning, Visualization, and Interfaces; 2014:63–70.
  38. Lenth RV. Emmeans: estimated marginal means, aka least-squares means. Published online October 20, 2017:1.10.3.
  39. Afshartous D, Preston RA. Confidence intervals for dependent data: equating non-overlap with statistical significance. Comput Stat Data Anal. 2010;54(10): 2296–2305. CrossRef
  40. Rehurek R, Sojka P. Software framework for topic modelling with large corpora. Proceedings of the LREC 2010 Workshop on New Challenges for NLP Frameworks. Accessed January 6, 2026; 2010:45–50.
  41. scikit-learn: machine learning in Python — scikit-learn 0.22.1 documentation. Accessed February 23, 2020. https://scikit-learn.org/stable/
  42. R Foundation for Statistical Computing. R: a language and environment for statistical. 2021. Accessed January 6, 2026. https://www.R-project.org
  43. R Core Team. R: a language and environment for statistical computing. 2024. Accessed January 6, 2026. https://www.R-project.org
  44. Méndez-Bustos P, Calati R, Rubio-Ramírez F, et al. Effectiveness of psychotherapy on suicidal risk: a systematic review of observational studies. Front Psychol. 2019;10:277. CrossRef
  45. Huggett C, Peters S, Gooding P, et al. A systematic review and meta-ethnography of client and therapist perspectives of the therapeutic alliance in the context of psychotherapy and suicidal experiences. Clin Psychol Rev. 2024;113:102469. CrossRef
  46. Michaud L, Greenway KT, Corbeil S, et al. Countertransference towards suicidal patients: a systematic review. Curr Psychol. 2021.
  47. Soulié T, Bell E, Jenkin G, et al. Systematic exploration of countertransference phenomena in the treatment of patients at risk for suicide. Archives Suicide Res. 2020;24(1):96–118. CrossRef
  48. Tishby O, Wiseman H. Countertransference types and their relation to rupture and repair in the alliance. Psychotherapy Res. 2022;32(1):16–31. CrossRef
  49. Swartz MS, Swanson JW, Wagner HR, et al. Can involuntary outpatient commitment reduce hospital recidivism?: findings from a randomized trial with severely mentally ill individuals. Aust J Pharm. 1999;156(12):1968–1975. PubMed CrossRef
  50. Che SE, Gwon YG, Kim KH. Follow-up timing after discharge and suicide risk among patients hospitalized with psychiatric illness. JAMA Netw Open. 2023; 6(10):e2336767. PubMed CrossRef
  51. Riblet N, Richardson JS, Shiner B, et al. Death by suicide in the first year after irregular discharge from inpatient hospitalization. PS. 2018;69(9):1032–1035. PubMed CrossRef
  52. Van Orden KA, Cukrowicz KC, Witte TK, et al. Thwarted belongingness and perceived burdensomeness: construct validity and psychometric properties of the Interpersonal Needs Questionnaire. Psychol Assess. 2012;24(1):197–215. PubMed CrossRef
  53. Dunster-Page C, Haddock G, Wainwright L, et al. The relationship between therapeutic alliance and patient’s suicidal thoughts, self-harming behaviours and suicide attempts: a systematic review. J Affect Disord. 2017;223:165–174. PubMed CrossRef
  54. McCarthy JF, Cooper SA, Dent KR, et al. Evaluation of the Recovery Engagement and Coordination for Health–Veterans Enhanced Treatment suicide risk modeling clinical program in the Veterans Health Administration. JAMA Netw Open. 2021; 4(10):e2129900. PubMed CrossRef
  55. Ruiz F, Burgo-Black L, Hunt SC, et al. A practical review of suicide among veterans: preventive and proactive measures for health care institutions and providers. Public Health Rep. 2023;138(2):223–231. PubMed CrossRef
  56. Bloch-Elkouby S, Barzilay S. Alliance-focused safety planning and suicide risk management. Psychotherapy. 2022;59(2):157–162. PubMed CrossRef
  57. Levis M, Levy J, Dimambro M, et al. Using natural language processing to evaluate temporal patterns in suicide risk variation among high-risk Veterans. Psychiatry Res. 2024;339:116097. PubMed CrossRef
  58. Levis M, Levy J, Dent KR, et al. Leveraging natural language processing to improve electronic health record suicide risk prediction for Veterans Health Administration users. J Clin Psychiatry. 2023;84(4):22m14568. PubMed CrossRef
  59. Levis M, Dimambro M, Levy J, et al. Using natural language processing to assess MDD psychotherapy patterns among Veterans Affairs patients with suicide risk. Psychiatry Res. 2026;361:117148. CrossRef
  60. Levy J, Dimambro M, Diallo A, et al. Investigating the differential impact of psychosocial factors by patient characteristics and demographics on veteran suicide risk through machine learning extraction of cross-modal interactions. In: Biocomputing 2025. WORLD SCIENTIFIC; 2024:167–184.
  61. Carroll D, Kearney LK, Miller MA. Addressing suicide in the veteran population: engaging a public health approach. Front Psychiatry. 2020;11:569069. CrossRef
  62. Hawton K, Pirkis J. Preventing suicide: a call to action. Lancet Public Health. 2024;9(10):e825–e830. CrossRef
  63. Wasserman D, Iosue M, Petros NG, et al. Incorporating suicide prevention in the WPA Action Plan 2023-2026. World Psychiatry. 2025;24(2):284–285. PubMed CrossRef
  64. AlSumait L, Barbará D, Gentle J, et al. Topic significance ranking of LDA generative models. In: Buntine W, Grobelnik M, Mladenić D, et al., eds. Machine Learning and Knowledge Discovery in Databases Lecture Notes in Computer Science. 5781. Berlin Heidelberg: Springer; 2009:67–82.
  65. Sounchio SS, Geneste L, Foguem BK. A hypotheses-driven framework for human–machine expertise process. Cognitive Syst Res. 2024;87: 101255. CrossRef
  66. Ingram WM, Baker AM, Bauer CR, et al. Defining major depressive disorder cohorts using the EHR: multiple phenotypes based on ICD-9 codes and medication orders. Neurology, Psychiatry Brain Res. 2020; 36:18–26. PubMed CrossRef
  67. Sharifian N, Kolaja CA, LeardMann CA, et al. Racial, ethnic, and sex disparities in mental health among US service members and veterans: findings from the millennium cohort Study. Am J Epidemiol. 2023;15(3):kwad221–kwad515.
  68. National Academies of Sciences, Engineering, and Medicine; Health and Medicine Division; Board on Health Care Services; Committee to Evaluate the Department of Veterans Affairs Mental Health Services. 6, Department of Veterans Affairs Mental Health Services: Need, Usage, and Access and Barriers to Care. Evaluation of the Department of Veterans Affairs Mental Health Services. Washington (DC): National Academies Press (US); 2018. Accessed January 6, 2026 Available from: https://www.ncbi.nlm.nih.gov/books/NBK499497/
  69. Lizardi D, Stanley B. Treatment engagement: a neglected aspect in the psychiatric care of suicidal patients. PS. 2010;61(12):1183–1191. PubMed CrossRef
  70. Barzilay S, Schuck A, Bloch-Elkouby S, et al. Associations between clinicians’ emotional responses, therapeutic alliance, and patient suicidal ideation. Depress Anxiety. 2020;37(3):214–223. PubMed CrossRef

About the Authors

Buy PDF for $40

Please sign in or purchase this PDF for $40.

The Journal of Clinical Psychiatry September 2026 cover featuring key articles on depression therapy

Vol 87 • 2026 • Number 3

Read the Current Issue

Original Research

Inhaled Mebufotenin (GH001) for Adult Patients with Postpartum Depression

Original Research

US Mortality Involving Obesity and Psychiatric Disorders

Academic Highlights

Treatment-Resistant Depression as a Dynamic Continuum: A Consensus Panel Report

Original Research

Effects of Brexpiprazole on Functioning in Patients with Schizophrenia Who Have Hostility and Agitation Symptoms

Original Research

Exposure-Based Video Therapy for OCD and PTSD

Original Research

Lumateperone as Adjunctive Therapy in Patients With Major Depressive Disorder and Anxious Distress
View current Issue

Insightful. Evidence-based. Human-centered.

Listen Now
The Journal of Clinical Psychiatry podcast cover with Benjamin Brody, MD

Subscribe to read
more articles

Subscribe