How to Reliably Predict Relapse After Electroconvulsive Therapy?

J Clin Psychiatry 2022;83(1):21lr14269

To cite: Lambrichts S, Vansteelandt K, Obbels J, et al. How to reliably predict relapse after electroconvulsive therapy? J Clin Psychiatry. 2022;83(1):21lr14269.
To share: https://doi.org/10.4088/JCP.21lr14269

© Copyright 2021 Physicians Postgraduate Press, Inc.

^aKU Leuven, Department of Neurosciences, Research Group Psychiatry, Neuropsychiatry, Academic Center for ECT and Neuromodulation (AcCENT), University Psychiatric Center KU Leuven (UPC KU Leuven), Kortenberg, Belgium
*Corresponding author: Simon Lambrichts, MD, University Psychiatric Center KU Leuven (UPC KU Leuven), Leuvensesteenweg 517, 3070 Kortenberg, Belgium (simon.lambrichts@upckuleuven.be).

See reply by Andrade and article by Andrade

To the Editor: We thank Dr Andrade for his comment¹ on our study of relapse after abrupt discontinuation of maintenance electroconvulsive therapy (M-ECT) during the COVID-19 pandemic.²

Dr Andrade declares his concerns about the protection against a type I error in our analyses examining the association between several clinical and treatment characteristics and relapse. For both models used in our study, the results of the test of the overall statistical significance (testing the null hypothesis that all regression coefficients are zero) were reported. This test protects against a type I error. As we reported two models, one could rightly argue that the Bonferroni procedure should have been used for both overall tests. For both models used in our study, the P value for the overall test (< .0001) was below the Bonferroni-corrected significance threshold (.05/2). Besides, both models should not be seen as “additive” evidence for the same hypothesis. We used the same model twice with “diagnosis” in the first model being substituted by “indication” in the second model because both seemed relevant from a clinical point of view, yet it was not possible to include both predictors in the same model as they were highly correlated.

Contrary to the impression of Dr Andrade, the reference category for categorical predictor variables was clearly stated in both models, as illustrated in Table 2 of our article.² In this table, we reported all possible pairwise comparisons, each time indicating which categories were compared.

Dr Andrade raises two highly relevant issues in data analysis: overfitting and confounding. In our “naturalistic experiment,” we aimed to avoid using more predictor variables in our model than the sample size allowed for (ie, prevention of overfitting). At the same time, we intended to include important predictor variables (ie, prevention of confounding). In our search for an optimal balance, we decided to include all 5 predictor variables (based on their clinical relevance, or evidence base, or both), yielding an events per variable value of 7.2 (36 patients relapsed; 5 predictor variables), which is below the suggested 10–15. Dr Andrade correctly emphasizes that recruiting larger samples contributes to the solution of overfitting and confounding (partially). However, in our study, following the forced discontinuation of M-ECT, the sample size was determined by the exceptional circumstances. Nevertheless, we fully agree with Dr Andrade’s plea for the recruitment of larger samples. Clearly, recruiting large samples is a major challenge in the ECT research field in general, and in post-ECT follow-up studies in particular. This calls for multicenter collaborations for conducting randomized controlled trials and prospective cohort studies.

Altogether, improving long-term outcome following ECT in patients with severe mood and psychotic disorders—which is high priority for the field³—requires methodologically rigorously conducted studies. More specifically, further work is needed to develop prognostic models,⁴ as reliable prediction of individuals’ risk of relapse may enable more efficient allocation of interventions to prevent relapse, improve quality of life for patients, and reduce economic cost on society.

Published online: December 7, 2021.
Potential conflicts of interest: None.
Funding/support: None.