Back to top Open menu arrow
Download PDF Reprints

The Journal of
Clinical Psychiatry
Members only

Original Research Focus on Suicide April 8, 2026

Using Natural Language Processing to Evaluate Differences in Psychotherapeutic Services for Posttraumatic Stress Disorder in a Suicide-Risk-Stratified Veteran Sample

; ; ; ;

J Clin Psychiatry 2026;87(2):25m15904

Abstract

Objective: Posttraumatic stress disorder (PTSD) is a common psychiatric condition, especially among US Veterans. Individuals diagnosed with PTSD have higher likelihoods of experiencing suicidal thoughts, attempting suicide, and dying by suicide compared to those without PTSD diagnoses. Although the US Department of Veterans Affairs (VA) emphasizes psychotherapy as a leading treatment for PTSD and associated suicide risk, there is limited understanding about how these treatments are used by VA patients who did and did not die by suicide. To better assess these patients’ psychotherapy usage differences, we used Latent Dirichlet Allocation, a natural language processing topic modeling methodology, to study patients’ unstructured electronic health record (EHR) note corpus.

Methods: We evaluated VA suicide-risk-stratified patients (high-, moderate-, and low-suicide-risk) who died by suicide in 2017–2018 (cases) and suicide-risk-matched patients with similar demographics, diagnoses, and care who did not die by suicide (controls). After collecting all psychotherapy EHR notes within 1 year of case death and completing corpus preprocessing, we derived topics and used a binomial logistic regression model to evaluate topic differences, calculate odds ratios and P values, and examine topic clinical relevance.

Results: We identified 5 topics: Risk, Treatment Planning, Evaluation, Psychosocial, and Medication. Cases and controls had several significantly different topic patterns, including Risk differences for moderate-risk patients, Treatment Planning differences for moderate- and high-risk patients, Evaluation differences for high-risk patients, Psychosocial differences for low- and moderate-risk patients, and Medication differences for all patient subgroups.

Conclusion: Topic differences help distinguish closely matched cases and controls, aiding understanding of psychotherapy utilization and risk monitoring. Our findings suggest divergent care priorities, such that evaluation and risk monitoring are more central for high-risk cases while collaborative treatment planning and medication management are more central for high-risk controls.

J Clin Psychiatry 2026;87(2):25m15904

Author affiliations are listed at the end of this article.

Members Only Content

This full article is available exclusively to Professional tier members. Subscribe now to unlock the HTML version and gain unlimited access to our entire library plus all PDFs. If you're already a subscriber, please log in below to continue reading.

Subscribe Now

Already a member? Log in

  1. Herrman H, Patel V, Kieling C, et al. Time for united action on depression: a Lancet–World Psychiatric Association Commission. Lancet. 2022;399(10328):957–1022. PubMed CrossRef
  2. National Institute of Mental Health. Major depression. 2022. https://www.nimh.nih.gov/health/statistics/major-depression
  3. Lehavot K, Katon JG, Chen JA, et al. Post-traumatic stress disorder by gender and veteran status. Am J Prev Med. 2018;54(1):e1–e9. PubMed CrossRef
  4. Harpaz-Rotem I, Hoff R. FY2021 overview of PTSD patient population data sheet. In: Northeast Program Evaluation Center, VA Office of Mental Health and Suicide Prevention (11MHSP); 2022.
  5. Hamblen JL, Norman SB, Sonis JH, et al. A guide to guidelines for the treatment of posttraumatic stress disorder in adults: an update. Psychotherapy. 2019;56(3):359–373. PubMed CrossRef
  6. Holliday R, Holder N, Williams AM, et al. Treating PTSD in the context of concurrent suicide risk: current evidence and navigating complicating factors. Curr Treat Options Psychiatry. 2021;8(4):213–228. CrossRef
  7. McKinney JM, Hirsch JK, Britton PC. PTSD symptoms and suicide risk in veterans: serial indirect effects via depression and anger. J Affect Disord. 2017;214:100–107. PubMed CrossRef
  8. National Center for PTSD. Ptsd. 2025. https://veteranptsdmedical-gov.us/ptsd/about/work/operational.php
  9. American Psychological Association. Evidence-based practice in psychology. Am Psychol. 2006;61(4):271–285. PubMed CrossRef
  10. Resick PA, Monson CA, Chard KM. Cognitive Processing Therapy for PTSD: A Comprehensive Therapist Manual. Guilford Press; 2024.
  11. Foa EB. Prolonged exposure therapy: past, present, and future. Depress Anxiety. 2011;28(12):1043–1047. PubMed CrossRef
  12. Merz J, Schwarzer G, Gerger H. Comparative efficacy and acceptability of pharmacological, psychotherapeutic, and combination treatments in adults with posttraumatic stress disorder: a network meta-analysis. JAMA Psychiatry. 2019;76(9):904–913. PubMed CrossRef
  13. Lee DJ, Schnitzlein CW, Wolf JP, et al. Psychotherapy versus pharmacotherapy for posttraumatic stress disorder: systemic review and meta-analyses to determine first-line treatments: research article: comparison of PTSD guidelines. Depress Anxiety. 2016;33(9):792–806. PubMed CrossRef
  14. Maguen S, Li Y, Madden E, et al. Factors associated with completing evidence-based psychotherapy for PTSD among veterans in a national healthcare system. Psychiatry Res. 2019;274:112–128. PubMed CrossRef
  15. Schnurr PP, Chard KM, Ruzek JI, et al. Comparison of prolonged exposure vs cognitive processing therapy for treatment of posttraumatic stress disorder among US veterans: a randomized clinical trial. JAMA Netw Open. 2022;5(1):e2136921. PubMed CrossRef
  16. Rosen CS, Matthieu MM, Wiltsey SS, et al. A review of studies on the system-wide implementation of evidence-based psychotherapies for posttraumatic stress disorder in the Veterans Health Administration. Adm Pol Ment Health Ment Health Serv Res. 2016;43(6):957–977.
  17. Department of Veterans Affairs, Department of Defense. VA/DOD Clinical Practice Guideline for the Assessment and Management of Patients at Risk for Suicide. The Assessment and Management of Suicide Risk Work Group; 2019. https://www.healthquality.va.gov/guidelines/mh/srb/
  18. Rozek DC, Baker SN, Rugo KF, et al. Addressing co-occurring suicidal thoughts and behaviors and posttraumatic stress disorder in evidence-based psychotherapies for adults: a systematic review. J Trauma Stress. 2022;35(2):729–745. PubMed CrossRef
  19. Levis M, Dimambro M, Levy J, et al. Characterizing Veteran suicide decedents that were not classified as high-suicide-risk. Psychol Med. 2024;54(11):3135–3144. PubMed CrossRef
  20. Levis M, Leonard Westgate C, Gui J, et al. Natural language processing of clinical mental health notes may add predictive value to existing suicide risk models. Psychol Med. 2020;17(8):1–10. PubMed CrossRef
  21. Levis M, Levy J, Dufort V, et al. Dynamic suicide topic modelling: deriving population-specific, psychosocial and time-sensitive suicide risk variables from Electronic Health Record psychotherapy notes. Clin Psychol Psychother. 2023;30(4):795–810. PubMed CrossRef
  22. Glaser BG, Strauss AL. The Discovery of Grounded Theory: Strategies for Qualitative Research. 1st ed. Routledge; 2017. CrossRef
  23. Li Y, Nair P, Lu XH, et al. Inferring multimodal latent topics from electronic health records. Nat Commun. 2020;11(1):2536. PubMed CrossRef
  24. Nock MK, Ramirez F, Rankin O. Advancing our understanding of the who, when, and why of suicide risk. JAMA Psychiatry. 2019;76(1):11–12. doi:10.1001/jamapsychiatry.2018.3164. PubMed CrossRef
  25. Kessler RC, Hwang I, Hoffmire CA, et al. Developing a practical suicide risk prediction model for targeting high-risk patients in the Veterans Health Administration. Int J Methods Psychiatr Res. 2017;26(3):e1575. doi:10.1002/mpr.1575. PubMed CrossRef
  26. Rocky Mountain MIRECC for Suicide Prevention, U.S. Department of Veterans Affairs, Cannizzaro K. REACH VET and the possible impact on integrated healthcare. 2018. https://avapl.org/conference/pubs/2018%20Conference%20Presentations/Cannizzaro%20-%20slides%20-%20Open%20Plenary.pdf
  27. VA DoD. Center of excellence for suicide prevention. In: Joint Department of Veterans Affairs (VA) and Department of Defense (DoD) Mortality Data Repository - National Death Index (NDI). MIRECC; 2020. Accessed December 31, 2020. https://www.mirecc.va.gov/suicideprevention/documents/VA_DoD-MDR_Flyer.pdf
  28. World Health Organization. International Statistical Classification of Diseases and Related Health Problems. 2nd ed.; 2004.
  29. Holowka DW, Marx BP, Gates MA, et al. PTSD diagnostic validity in Veterans Affairs electronic records of Iraq and Afghanistan veterans. J Consult Clin Psychol. 2014;82(4):569–579. PubMed CrossRef
  30. Lavigne JE, Lagerberg T, Ambrosi JW, et al. Study designs and statistical approaches to suicide and prevention research in real-world data. Suicide Life Threat Behav. 2021;51(1):127–136. PubMed CrossRef
  31. Moser BK, Halabi S. Estimation and testing of the relative risk of disease in case-control studies with a set of k matched controls per case with known prevalence of disease. Stat Med. 2012;31(1):29–44. PubMed CrossRef
  32. Andrade C. Mean difference, Standardized mean difference (SMD), and their use in meta-analysis: as simple as it gets. J Clin Psychiatry. 2020;81(5):20f13681. PubMed CrossRef
  33. Levis M, Dimambro M, Levy J, et al. Evaluating evidence-based psychotherapy utilization patterns among suicide-risk-stratified veterans diagnosed with posttraumatic stress disorder. Clin Psychol Psychother. 2025. Published online.
  34. Office of Information and Technology. Text Integration Utilities (TIU) Clinical Coordinator and User Manual; 2024. https://www.va.gov/vdl/documents/Clinical/CPRS-Text_Integration_Utility_(TIU)/tiuum.pdf
  35. Levy O, Goldberg Y, Dagan I. Improving distributional similarity with lessons learned from word embeddings. Trans Assoc Comput Linguist. 2015;3:211–225. CrossRef
  36. Gan J, Qi Y. Selection of the optimal number of topics for LDA topic model—taking patent policy analysis as an example. Entropy. 2021;23(10):1301. PubMed CrossRef
  37. Rehurek R, Sojka P. Software framework for topic modelling with large corpora. In: Proceedings of the LREC 2010 Workshop on New Challenges for NLP Frameworks; 2010:45–50. http://is.muni.cz/publication/884893/en
  38. Scikit-learn: machine learning in Python — Scikit-learn 0.22.1 documentation. Accessed February 23, 2020. https://scikit-learn.org/stable/
  39. Sievert C, Shirley KE. LDAvis: a method for visualizing and interpreting topics. In: Proceedings of the Workshop on Interactive Language Learning, Visualization, and Interfaces; 2014:63–70.
  40. R foundation for statistical computing. R: A Language and Environment for Statistical Computing. 2021. https://www.R-project.org
  41. R Core Team. R: a Language and environment for statistical computing. 2024. https://www.R-project.org
  42. Lenth RV. Emmeans: estimated marginal means, aka least-squares means. 2017.
  43. McCarthy JF, Bossarte RM, Katz IR, et al. Predictive modeling and concentration of the risk of suicide: implications for preventive interventions in the US Department of Veterans Affairs. Am J Public Health. 2015;105(9):1935–1942. PubMed CrossRef
  44. Wyder M, Ray MK, Russell S, et al. Suicide risk assessment in a large public mental health service: do suicide risk classifications identify those at risk?. Australas Psychiatry. 2021;29(3):322–325. PubMed CrossRef
  45. Saulnier KG, Brabbs S, Szymanski BR, et al. Suicide risk among veterans who receive evidence-based therapy for posttraumatic stress disorder. JAMA Netw Open. 2024;7(12):e2452144. PubMed CrossRef
  46. Wampold BE. How important are the common factors in psychotherapy? An update. World Psychiatry. 2015;14(3):270–277. PubMed CrossRef
  47. Altavini CS, Asciutti APR, Solis ACO, et al. Revisiting evidence of primary prevention of suicide among adult populations: a systematic overview. J Affect Disord. 2022;297:641–656. PubMed CrossRef
  48. Bohart AC. The client is the most important common factor: clients’ self-healing capacities and psychotherapy. J Psychother Integr. 2000;10(2):127–149.
  49. Lizardi D, Stanley B. Treatment engagement: a neglected aspect in the psychiatric care of suicidal patients. Psychiatr Serv. 2010;61(12):1183–1191. PubMed CrossRef
  50. Schnyder U, Ehlers A, Elbert T, et al. Psychotherapies for PTSD: what do they have in common?. Eur J Psychotraumatology. 2015;6(1):28186. PubMed CrossRef
  51. Short NA, Allan NP, Ashrafioun L, et al. Beliefs about mental health treatment, treatment initiation, and suicidal behaviors among veterans and service members at-risk for suicide and not in treatment. Suicide Life Threat Behav. 2024;54(6):1083–1091. PubMed CrossRef
  52. Maguen S, Madden E, Holder N, et al. Effectiveness and comparative effectiveness of evidence-based psychotherapies for posttraumatic stress disorder in clinical practice. Psychol Med. 2021;18(2):1–10. Published online May. doi:10.1017/S0033291721001628. PubMed CrossRef
  53. Bryan CJ. Treating PTSD within the context of heightened suicide risk. Curr Psychiatry Rep. 2016;18(8):73. PubMed CrossRef
  54. Michel K, Gysin-Maillart A, Breit S, et al. Psychopharmacological treatment is not associated with reduced suicide ideation and reattempts in an observational follow-up study of suicide attempters. J Psychiatr Res. 2021;140:180–186. PubMed CrossRef
  55. Zisook S, Domingues I, Compton J. Pharmacologic approaches to suicide prevention. Focus. 2023;21(2):137–144. PubMed CrossRef
  56. Leonard Westgate C, Shiner B, Thompson P, et al. Evaluation of veterans’ suicide risk with the use of linguistic detection methods. Psychiatr Serv. 2015;66(10):1051–1056. PubMed CrossRef
  57. Barzilay S, Schuck A, Bloch-Elkouby S, et al. Associations between clinicians’ emotional responses, therapeutic alliance, and patient suicidal ideation. Depress Anxiety. 2020;37(3):214–223. PubMed CrossRef
  58. Bloch-Elkouby S, Barzilay S. Alliance-focused safety planning and suicide risk management. Psychotherapy. 2022;59(2):157–162. PubMed CrossRef
  59. Ingram WM, Baker AM, Bauer CR, et al. Defining major depressive disorder cohorts using the EHR: multiple phenotypes based on ICD-9 codes and medication orders. Neurol Psychiatry Brain Res. 2020;36:18–26. PubMed CrossRef
  60. Weathers FW, Bovin MJ, Lee DJ, et al. The Clinician-Administered PTSD Scale for DSM–5 (CAPS-5): development and initial psychometric evaluation in military veterans. Psychol Assess. 2018;30(3):383–395. PubMed CrossRef
  61. Barnett MJ, Shadi D, Vista K, et al. Multiple comparisons: to compare or not to compare, that is the question. Res Soc Adm Pharm. 2022;18(2):2331–2334.
  62. Rosenthal JA. Qualitative descriptors of strength of association and effect size. J Soc Serv Res. 1996;21(4):37–59. CrossRef
  63. Chen H, Cohen P, Chen S. How big is a big odds ratio? Interpreting the magnitudes of odds ratios in epidemiological studies. Commun Stat - Simul Comput. 2010;39(4):860–864. CrossRef
  64. Sharifian N, Kolaja CA, LeardMann CA, et al. Racial, ethnic, and sex disparities in mental health among US service members and veterans: findings from the Millennium Cohort Study. Am J Epidemiol. 2024;193(3):kwad221–kwad515. PubMed CrossRef
  65. National Academies of Sciences. Department of Veterans Affairs mental health services: need, usage, and access and barriers to care. In. In: National Academies of Sciences, Engineering, and Medicine; Health and Medicine Division; Board on Health Care Services; Committee to Evaluate the Department of Veterans Affairs Mental Health Services. National Academies Press (US); 2018. www.ncbi.nlm.nih.gov/books/NBK499497/

About the Authors

Buy PDF for $40

Please sign in or purchase this PDF for $40.

Cover features March 2026 issue highlighting articles on psychiatric drugs, peer interventions for perinatal depression, and metformin.

Vol 87 • 2026 • Number 1

Read the Current Issue

Original Research

Innovation in Psychiatric Drug Development: A Quantitative Analysis of FDA-Approved Psychiatric Drugs, 2012–2024

Meta-Analysis

Effectiveness of Peer-Administered Interventions for Perinatal Depression or Anxiety

Original Research

Effect of Adjuvant Metformin on Adherence to and Persistence of Treatment With SGAs in Nondiabetic Patients

Editorial

Thank You to the JCP Community

Editorial

Psychotherapy: Integral to Clinical Psychiatry

Original Research

Trends in Benzodiazepine Prescribing to Adults in the United States
View current Issue

Insightful. Evidence-based. Human-centered.

Listen to the JCP Podcast Now
Figure: Digital illustration shows a human head with neural connections and podcast title over a dark blue patterned background.

Subscribe to read
more articles

Subscribe