The article you requested is

Easy and Low-Cost Identification of Metabolic Syndrome in Patients Treated With Second-Generation Antipsychotics: Artificial Neural Network and Logistic Regression Models

J Clin Psychiatry 2010;71(3):225-234
10.4088/JCP.08m04628yel

Objective: Metabolic syndrome (MetS) is an important side effect of second-generation antipsychotics (SGAs). However, many SGA-treated patients with MetS remain undetected. In this study, we trained and validated artificial neural network (ANN) and multiple logistic regression models without biochemical parameters to rapidly identify MetS in patients with SGA treatment.

Method: A total of 383 patients with a diagnosis of schizophrenia or schizoaffective disorder (DSM-IV criteria) with SGA treatment for more than 6 months were investigated to determine whether they met the MetS criteria according to the International Diabetes Federation. The data for these patients were collected between March 2005 and September 2005. The input variables of ANN and logistic regression were limited to demographic and anthropometric data only. All models were trained by randomly selecting two-thirds of the patient data and were internally validated with the remaining one-third of the data. The models were then externally validated with data from 69 patients from another hospital, collected between March 2008 and June 2008. The area under the receiver operating characteristic curve (AUC) was used to measure the performance of all models.

Results: Both the final ANN and logistic regression models had high accuracy (88.3% vs 83.6%), sensitivity (93.1% vs 86.2%), and specificity (86.9% vs 83.8%) to identify MetS in the internal validation set. The mean±SD AUC was high for both the ANN and logistic regression models (0.934±0.033 vs 0.922±0.035, P=.63). During external validation, high AUC was still obtained for both models. Waist circumference and diastolic blood pressure were the common variables that were left in the final ANN and logistic regression models.

Conclusion: Our study developed accurate ANN and logistic regression models to detect MetS in patients with SGA treatment. The models are likely to provide a noninvasive tool for large-scale screening of MetS in this group of patients.


Submitted: August 13, 2008; accepted November 26, 2008.

Online ahead of print: October 6, 2009.

Corresponding authors: Hung-Wen Chiu, PhD, 250, Wusing St, Sinyi District, Taipei 110, Taiwan (hwchiu@tmu.edu.tw), and Yu-Chuan Li, MD, PhD, 250, Wunsing St, Sinyi District, Taipei 110, Taiwan (jack@tmu.edu.tw).

Easy and Low-Cost Identification of Metabolic Syndrome in Patients Treated With Second-Generation Antipsychotics: Artificial Neural Network and Logistic Regression Models

Objective: Metabolic syndrome (MetS) is an important side effect of second-generation antipsychotics (SGAs). However, many SGA-treated patients with MetS remain undetected. In this study, we trained and validated artificial neural network (ANN) and multiple logistic regression models without biochemical parameters to rapidly identify MetS in patients with SGA treatment.

Method: A total of 383 patients with a diagnosis of schizophrenia or schizoaffective disorder (DSM-IV criteria) with SGA treatment for more than 6 months were investigated to determine whether they met the MetS criteria according to the International Diabetes Federation. The data for these patients were collected between March 2005 and September 2005. The input variables of ANN and logistic regression were limited to demographic and anthropometric data only. All models were trained by randomly selecting two-thirds of the patient data and were internally validated with the remaining one-third of the data. The models were then externally validated with data from 69 patients from another hospital, collected between March 2008 and June 2008. The area under the receiver operating characteristic curve (AUC) was used to measure the performance of all models.

Results: Both the final ANN and logistic regression models had high accuracy (88.3% vs 83.6%), sensitivity (93.1% vs 86.2%), and specificity (86.9% vs 83.8%) to identify MetS in the internal validation set. The mean±SD AUC was high for both the ANN and logistic regression models (0.934±0.033 vs 0.922±0.035, P=.63). During external validation, high AUC was still obtained for both models. Waist circumference and diastolic blood pressure were the common variables that were left in the final ANN and logistic regression models.

Conclusion: Our study developed accurate ANN and logistic regression models to detect MetS in patients with SGA treatment. The models are likely to provide a noninvasive tool for large-scale screening of MetS in this group of patients.

J Clin Psychiatry 2010;71(3):225-234

Submitted: August 13, 2008; accepted November 26, 2008.

Online ahead of print: October 6, 2009 (doi:10.4088/JCP.08m04628yel).

Corresponding authors: Hung-Wen Chiu, PhD, 250, Wusing St, Sinyi District, Taipei 110, Taiwan (hwchiu@tmu.edu.tw), and Yu-Chuan Li, MD, PhD, 250, Wunsing St, Sinyi District, Taipei 110, Taiwan (jack@tmu.edu.tw).

Metabolic syndrome (MetS) has been found to be prevalent in patients treated with atypical or second-generation antipsychotics (SGAs), but many patients with MetS have not been detected. Increased identification of MetS and providing adequate interventions are important to decrease patients’ risk of mortality and morbidity. However, large-scale screening for MetS on all patients taking SGAs may be expensive and time consuming. Blood drawing for those without MetS may cause inconvenience, especially when their psychotic symptoms are unstable. Therefore, the need arises to find an easy, fast, and low-cost method to screen MetS in patients with SGA treatment.

Second-generation antipsychotics represent an important advancement in the treatment of schizophrenia. These drugs show comparable or greater efficacy for positive symptoms, superior efficacy for negative symptoms, favorable effects on cognitive function, lower risk of extrapyramidal side effects, and a modest but significantly lower rate of relapse or treatment failure.1–5 However, as the use of SGAs has increased, it has been recognized that certain SGAs are associated with a higher risk of MetS in schizophrenia patients: for example, excessive weight gain, glucose dysregulation, disrupted lipid metabolism, and cardiovascular disease.6–8 Previous studies have shown that approximately 19%–60% of patients with schizophrenia-related disorders have MetS.9–11 Most studies have shown that the prevalence of MetS in patients with schizophrenia or schizophrenia-related disorder is at least twice as high as that in the general population.12 Analyses of the US Food and Drug Administration’s MedWatch pharmacovigilance database have revealed cases of emergent diabetes in patients treated with clozapine,13 olanzapine,14 and risperidone15 that appear within 6 months or less of treatment initiation and resolve on discontinuation of the drug, suggesting a causal link for the association.16 The risk of type 2 diabetes mellitus increases by 9% with SGAs compared to first-generation antipsychotics, particularly with clozapine and olanzapine.17 Based on the Clinical Antipsychotic Trials of Intervention Effectiveness (CATIE) study and a growing number of other randomized clinical trials, clozapine and olanzapine treatment were found to be associated with the highest risk of metabolic disturbances, and risperidone treatment produced intermediate changes.8,18

Metabolic syndrome is associated with weight gain. Being overweight usually leads to lower self-image and self-esteem, decreased quality of life, and social disadvantages,19,20 and it is associated with medication noncompliance.21 Patients with schizophrenia have increased somatic morbidity and mortality risks relative to the general population.22,23 Weight gain might contribute to their risk of morbidity and mortality24 by leading to an increase in lipid dysregulation, hypertension, type 2 diabetes mellitus, cardiovascular disease, and other related diseases.25 Since MetS is an important risk factor for cardiovascular disease and diabetes, regular monitoring of the symptoms associated with MetS is recommended.26,27 However, considerable evidence indicates that mentally ill patients often do not receive adequate recognition and monitoring of or care for their medical illnesses.8,27 In the CATIE schizophrenia trial, 88% of patients with hyperlipidemia, 62.4% with hypertension, and 30.2% of schizophrenia patients with diabetes were not receiving treatment.28 Psychiatrists may also consider the problem of reimbursement for routine screening for MetS in all patients with SGA treatment. In addition, the results of laboratory tests may require days or a week before they are known. And more, psychotic patients may often reject or be reluctant to receive blood drawing. Therefore, psychiatric care systems are now challenged to develop methods of surveillance for MetS.27 The question is raised whether MetS in patients with schizophrenia receiving SGA treatment can be screened simply with some clinical parameters that are both easily available and cheap. The second question is equally important: what methods can best predict the result?

In this work, the use of artificial neural network (ANN) was investigated for its ability to identify MetS. Artificial neural network is a form of artificial intelligence that employs nonlinear mathematical models to mimic the human brain’s own problem-solving process. A neural network takes previously solved examples to build a system of “neurons” that makes classifications and forecasts. The classification rules are not written into algorithms but rather are learned by the network from examples. An ANN comprises layers of neurons. The most commonly used ANN, multilayer perceptron, for example, consists of the input layer, the hidden layer, and the output layer. The input layer is formed by neurons that may receive features for a specified problem. The hidden layer of neurons receives the data from the input layer and is connected to the output layer, with multiple connections between neurons among the layers by weights. The inputs of a neuron are first multiplied by a weighting factor that determines the extent to which each input influences the output, and the weighted inputs are summed to be inserted through the transfer function, resulting in the neuronal output. During the supervised training stage, a data set is presented to the ANN with the correct outputs available. The ANN is trained by first randomly initializing the connection weights between the neurons and then running the data through the network and comparing the output with the known responses. The process repeats and the network alters the weights between connections so that the errors in the outputs are reduced to negligible values. The ANN can then be used for prediction. Logistic regression, which fits the data to a descriptive function, is inherently different from ANN, which raises the question of whether one approach has a better predictive performance than the other.

To our knowledge, there are no published articles to date regarding the identification of SGA-associated MetS by means of ANN and logistic regression. To investigate this problem, we applied ANN and logistic regression to the analysis of data from patients with schizophrenia taking SGAs in an attempt to achieve accurate, rapid identification of MetS in unseen individual patients.

METHOD

Participants

We recruited 400 inpatients who had a diagnosis of schizophrenia or schizoaffective disorder based on the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition criteria29 and who had used risperidone, olanzapine, or clozapine for at least 6 months at Yuli Veterans Hospital, Taiwan. Seventeen patients with concurrent hypoglycemic medications were excluded. Therefore, the first part of the study consisted of 383 patients. The data were used for training and internal validation of ANN and logistic regression analyses and were collected between March 2005 and September 2005. Yuli Veterans Hospital is mainly a psychiatric hospital located in a rural area in eastern Taiwan. The second part of the study consisted of 69 patients from psychiatric day care and outpatient clinic at National Taiwan University Hospital. The data were used for external validation of the established predictive models and were subsequently collected between March 2008 and June 2008. National Taiwan University Hospital is a medical center located at the center of urban Taipei. The study was approved by the institutional review boards at both hospitals and was performed in accordance with the Declaration of Helsinki. All participants provided informed consent to participate.

Procedures

Demographic data; concomitant use of mood stabilizers, other antipsychotics, antihypertensive medications, and hypoglycemic medications; and date and body weight at initiation of SGA treatment were obtained by retrospective chart reviews for all included patients at Yuli Veterans Hospital. Over the course of their hospitalization, all patients had their body weight monitored monthly and recorded. Drug adherence could be optimally controlled for the hospitalized patients. Alcohol consumption was prohibited in general, and smoking was only allowed with some limitations of time and place in the male wards. Age was calculated as the difference in years between the date of the assessment and the participant’s date of birth. Duration of SGA treatment was calculated as the difference in months between the date of the assessment and the date of initiation of SGA treatment. Anthropometric and biochemical assessments were performed to determine whether patients fulfilled our “gold standard” of MetS: the 2005 International Diabetes Federation (IDF) criteria.30 The IDF criteria for MetS are central obesity (criteria for Chinese, waist circumference ≥90 cm for men and ≥80 cm for women) plus any 2 of the following 4 factors: (1) raised triglyceride level, ≥150 mg/dL; (2) reduced high-density lipoprotein (HDL) cholesterol, <40 mg/dL in men and <50 mg/dL in women; (3) raised blood pressure, systolic blood pressure (SBP) ≥130 or diastolic blood pressure (DBP) ≥85 mm Hg; or (4) raised fasting plasma glucose, ≥100 mg/dL.

Anthropometry and Biochemical Assessments

Body weight was measured to the nearest 0.1 kg; waist circumference, to the nearest 0.1 cm; and height, to the nearest 0.5 cm. Body mass index was calculated as weight in kilograms divided by the height in meters squared. All patients fasted overnight prior to the taking of their blood sample, which was drawn between 7:00 am and 8:00 am. Serum glucose, triglyceride, and cholesterol levels were measured using a glucose oxidase autoanalyzer, a triglyceride enzyme autoanalyzer, and a cholesterol oxidase autoanalyzer, respectively (Dimension RxL, DADE Behring Company, Inc, Newark, Delaware).

Data Processing

Table 1

Click figure to enlarge

The input variables, ie, independent variables, are shown in Table 1. The dependent variable, ie, output variable, was a dichotomous variable representing MetS (coded with 1) or non-MetS (coded with 0). Second-generation antipsychotic agents were coded into 2 dummy variables with risperidone as reference in logistic regression analysis and were coded into 3 binary variables representing each SGA in ANN analysis. Binary predictors were coded with 0 or 1 as shown in Table 2. Laboratory data were used to define whether a patient fulfilled the IDF criteria of MetS and were not used as predictor variables. The data set at Yuli Veterans Hospital was randomly divided into 2 separate groups for cross- validation: 255 patients (about two-thirds) as the training set and 128 patients (about one-third) as the internal validation set. Eighty-three patients with MetS (21.7%) were randomly distributed to the 2 sets proportionally. The training set was used to build logistic regression and ANN models. The internal validation set was set aside for later evaluation as a blind data set. Compared to logistic regression, ANN models are more flexible and thus more susceptible to overfitting. To avoid overfitting, we adopted an early stopping method that requires a subset of the training data to be used as a holdout set or selection set.31 Therefore, 65 patients from the training set (about one quarter) were set aside as a selection set for ANN training. Patients with MetS in the selection set remained the same proportion and were selected randomly.

Table 2

Click figure to enlarge

Multiple Logistic Regression Analysis

Logistic regression analysis was first performed using the same training data set of 255 patients as the ANN analysis with maximum likelihood estimation. Although logistic regression does not involve training, we used a “training set” to refer to that portion of the data set used to derive the regression equations.32 The backward stepwise method was used for the selection of variables. The model was then applied, using the statistically significant variables obtained, to predict the occurrence of MetS in the internal validation set of 128 patients. Categorical covariates were contrasted with reference to the first category. The likelihood ratio test was used to assess the covariate-adjusted P value.

Artificial Neural Network Analysis

We constructed several architectures of feed-forward networks, including linear, multilayer perceptrons and radial basis function networks. The networks consisted of 3 layers—an input layer, a hidden layer, and an output layer. The back-propagation algorithm was used as a supervised learning algorithm to train the network, which adjusted the internal parameters of the network over repeated training cycles to reduce the overall error. One iteration consists of a single presentation of each set of inputs for all cases followed by automatic adjustments of the weight connections to minimize the total error for all patients whose data were used in the training. The estimation of error was based on the sum-squared error.33

Artificial neural network models were first trained using the training data set of 190 patients and selection data set of 65 patients. The selection set was used to terminate training if the selection error stopped dropping or, indeed, started to rise. This indicated that the network was starting to overfit the training data. To find an optimal network, different ANN architectures, with 5–25 hidden neurons, were constructed and trained. To identify the input variables that contributed most in the prediction, feature selection was performed with the backward elimination method, which starts training with all input variables and sequentially deletes the next variable that most increases or least decreases the area under the receiver operating characteristic curve (AUC). The network with the highest AUC on the selection set was kept (best network, architecture, and the optimum set of input variables were retained). Then, the optimal model was tested with the internal validation set to determine their predictive accuracy of MetS. Clinical factors were ranked according to their importance.

External Validation

Data from 69 patients at National Taiwan University hospital were used for external validation of our final ANN and logistic regression models. Demographic information and data required for our models and the diagnostic criteria for MetS were collected. The predictive performance of our models with the new data was examined.

Performance of Models

Although there are several ways of evaluating the performance of a predictive model, the AUC provides the best measure of the global accuracy of the model. The performance of logistic regression and ANN on a per patient basis was plotted as receiver operating characteristic (ROC) curves. The area under the curve34 was used as a quantitative measure of the ability of the predictor models to distinguish between MetS and non-MetS. The performance of the final ANN model was compared with the logistic regression model on the internal validation set and external validation set.35 Other measures of performance (accuracy, sensitivity, specificity, positive predictive value [PPV], and negative predictive value [NPV]) were also computed for the ANN and logistic regression analyses. The optimal cut point on the ROC curves was determined by the following rule: sensitivity greater than 85%, specificity not less than 80%, and accuracy as high as possible.

Statistical Analysis

Univariate analysis was performed to compare the differences of demographic, clinical, and metabolic characteristics between patients with MetS and those without MetS. All statistical tests performed were 2-tailed, and the final significance level was set at .05. A software program (SPSS for Windows, version 15.0; SPSS Inc, Chicago, Illinois) was used for statistical analyses. The ANNs were run by STATISTICA Neural Networks (Statistica 7.0, StatSoft Inc., Hamburg, Germany). The AUCs were estimated and compared with MedCalc for Windows, version 9.3.9.0 (MedCalc Software, Mariakerke, Belgium).

RESULTS

Of the 383 patients at Yuli Veterans Hospital, 254 (66.3%) were male. Mean±SD age was 47.5±13.5 years (range, 23–82); mean±SD baseline body weight, 63.1±13.1 kg (range, 37–117); mean±SD baseline BMI, 23.8±4.4 (range, 14.5–43.4); and mean±SD duration of SGAs use, 47.4±27.6 months (range, 6–96). The distribution of SGAs was 38.4% risperidone (n=147), 20.9% olanzapine (n=80), and 40.7% clozapine (n=156). Seventeen patients (4.4%) took antihypertensive medications, 99 (25.8%) took a mood stabilizer, and 38 (9.9%) had concomitant use of other antipsychotics.

Table 2 summarizes the demographic and clinical characteristics according to whether MetS was present. There were no statistically significant between-group differences with respect to sex, age, duration of SGA use, and concomitant use of a mood stabilizer or other antipsychotics. However, patients with MetS were more likely than those without MetS to have significantly higher waist circumference, DBP, SBP, triglyceride level, fasting glucose level, baseline BMI, cross-sectional BMI, baseline body weight, and weight gain; lower HDL level; and a greater concomitant use of mood stabilizers and antihypertensive medications. These predictors, not including laboratory data, were used to establish logistic regression and ANN models.

The data in Table 2 were further stratified by sex. For patients with MetS, male patients were more likely to have a higher waist circumference and lower HDL level, but there was no statistically significant difference for the proportion of central obesity and reduced HDL between female and male patients. There was no statistically significant difference for the occurrence of MetS between female and male patients (20.2% vs 22.4%, c2=0.263, P=.608). For patients without MetS, female patients had significantly higher HDL and BMI than male patients, and male patients had significantly higher waist circumference, SBP, and baseline body weight than female patients.

Prediction of MetS by Logistic Regression

Table 3

Click figure to enlarge

Table 4

Click figure to enlarge

We next sought to determine whether data without laboratory variables would be useful to predict MetS. The final logistic regression model for training data set is shown in Table 3. The optimal cut point for predicted values was 0.21. The logistic regression model had high accuracy (87.5%), sensitivity (87.0%), and specificity (87.6%) in the training data set and high accuracy (83.6%), sensitivity (86.2%), and specificity (83.8%) in the internal validation set (Table 4). The remaining statistically significant covariates included waist circumference, DBP, and female gender (overall model, χ2=129.2, P<.0005). All remaining variables, except sex, were identified in the univariate analysis. One standard deviation increase in waist circumference resulted in about a 10-fold increase in the likelihood of MetS. Similar results, generally showing more modest effects, were observed for female gender and DBP. The following is the derived equation for the prediction of MetS in patients treated with SGAs:

Logit (odds of MetS)=−27.07+0.193×
waist circumference (cm)+0.109×
DBP (mm Hg)+1.47×female.

We used the right-side formula of logistic regression equation except the constant to calculate the risk score of MetS. The optimal cut point for MetS was 25.70. The accuracy, sensitivity, specificity, and AUC for the risk score were all the same as those for logistic regression model.

Figure 1

Click figure to enlarge

Figure 2

Click figure to enlarge

Figure 3

Click figure to enlarge

Prediction of MetS Using the ANN

The ANN analysis showed that the radial basis function neural network with 10 hidden nodes provided the optimal network architecture. Figure 1 shows the network architecture with clinical variables in descending order of importance—waist circumference, BMI, and DBP. All variables were identified in the univariate analysis. The optimal cut point for predicted values was 0.26. The ANN model had high accuracy (93.8%), sensitivity (93.3%), and specificity (94.0%) in the training data set. When we applied the optimal ANN model in the internal validation set of 128 patients, an accuracy rate of 88.3% was obtained for the total number of cases in which the correct classification rates of 93.1% and 86.9% were achieved for MetS and non-MetS cases, respectively (Table 4). Two patients (6.9%) with MetS were incorrectly classified as non-MetS and 13 (13.1%) of the non-MetS were incorrectly classified as MetS.

Comparison of Predictive Performance on Internal Validation Set

The AUC, overall accuracy, sensitivity, specificity, PPV, and NPV of the ANN and logistic regression models are shown in Table 4. The overall accuracy rates of ANN and logistic regression were 88.3% and 83.6%, respectively. Both mean±SD AUCs for the ANN and logistic regression models were high (0.934±0.033 versus 0.922±0.035, P=.63; Figure 2).

Because our final ANN and logistic regression models include 2 diagnostic criteria of MetS—waist circumference and DBP—one may raise a question as to whether using nonlaboratory portions of the gold standard is enough for clinical screening. Therefore, we calculated the sensitivity and specificity of MetS with the rule in the IDF criteria that patients with central obesity (considering waist circumference and gender) and raised DBP or SBP were regarded as positive cases. The results showed that the sensitivity and specificity for MetS were 34.5% and 98.0%, respectively. Although central obesity is the required criterion for MetS, of patients with central obesity, only 34.7% had high DBP or SBP.

Comparison of Predictive Performance on External Validation Set

Of the 69 patients at National Taiwan University Hospital, 29 (42.0%) were male. Mean±SD age was 39.8±12.0 years (range, 19–66). Twenty-seven patients (39.1%) fulfilled MetS criteria. The AUC, overall accuracy, sensitivity, specificity, PPV, and NPV of the ANN and logistic regression models (Table 5 and Figure 3) were similar to those on internal validation set. Of greater importance in terms of general applicability of predictive models was our observation that the sensitivity and NPV were especially high for both ANN and logistic regression models in an external, independent population.

Table 5

Click figure to enlarge

DISCUSSION

Atypical antipsychotics have become first-line medications, especially when there is a concern about drug compliance, cognitive deficits, and possible higher vulnerability to extrapyramidal side effects. However, the quality of life of these patients may be greatly affected by excessive weight gain, hyperglycemia, and dyslipidemia, which can result in increased comorbid medical illness, increased relapse rate associated with noncompliance, or social stigma associated with being obese.36 Most previous studies have investigated antipsychotic-induced weight gain or individual metabolic disturbance. Few have assessed all the criteria of MetS. Since MetS is prevalent in patients taking SGAs, it is important to identify these patients to decrease their comorbidity and enhance their quality of life.

Advances in computer processing speed and neural network theory have facilitated the application of neural networks to the nonlinear analysis of complex data in the psychopharmacologic domain. For example, ANN was used to forecast antidepressant treatment response in patients receiving sertraline treatment.37,38 Furthermore, multiple gene polymorphisms have been included in the neural network analysis of fluvoxamine response, with a sensitivity of 77.5% and a specificity of 51.2%.39 In our previous study, we demonstrated that ANN is also useful in predicting antipsychotic response.40 In our prediction of MetS in the present study, the overall accuracy, sensitivity, and specificity were high for both the ANN and logistic regression models. About 93% of MetS cases and 87% of non-MetS were successfully predicted by the ANN model and about 86% of MetS and 84% of non-MetS were successfully predicted by the logistic regression model. This finding implies clinically that most SGA-treated patients with MetS would be successfully identified by either ANN or logistic regression model prediction using only 3 easily and immediately available clinical variables—waist circumference, DBP, and female gender—for the logistic regression model and waist circumference, DBP, and BMI for the ANN model. The logistic regression model and the ANN model differed in one variable only: BMI in ANN model versus sex in logistic regression model. Currently, physicians rely mostly on laboratory data when diagnosing MetS. With their high sensitivity and NPV, our ANN and logistic regression models show promise for assisting physicians in the clinical screening of MetS.

A classification result may be overly optimistic if performance cannot be measured on a data set not used for model building. In the ideal case, testing on a separate data set will provide an unbiased estimate of generalization error.31 In our study, the ANN analysis was performed by training the networks with the training set and testing their performance with the internal validation set. To allow a direct comparison, the logistic regression model was constructed from the training set and its performance assessed in the case of the internal validation set. Because accuracy can be influenced by the class distribution in the data set, we randomly distributed patients with MetS to the training and internal validation sets proportionally. In addition to internal validation, a previous study41 has validated ANN models to predict acute lower gastrointestinal hemorrhage, with external validation data from a different institute. Our ANN and logistic regression models were also applied to an external validation set, and the results showed that the performance of our final models did not deteriorate. This implies that our models can be generalized to the same clinical population in a different clinical setting. Our models performed especially well in terms of sensitivity (85.2%–96.3%) and NPV (89.2%–97.7%) in both the internal and external validation sets. This feature suggests that our ANN and logistic regression models may have a role in screening of MetS for patients treated with SGAs in a routine or large-scale way. A low proportion of patients with MetS will be missed with our models. Another issue affecting the performance of a model is the determination of optimal cut point. Although our rule to determine the optimal cut point generalized well, there are other documented methods to determine the cut point.42

Although baseline BMI, cross-sectional BMI, baseline weight, and SBP were significantly associated with MetS in univariate analysis, their results became statistically insignificant after covariate adjustment in logistic regression analysis. This may be due to multicollinear effect. On the other hand, sex was statistically significant in logistic regression analysis despite a low significance in the univariate analysis. The differences of waist circumference and HDL in the IDF criteria for men and women may explain why sex remained in the final multivariate logistic regression model. Increased rates of obesity, diabetes, and MetS have been observed in female patients treated with antipsychotic medications.9,10,43 This gender difference was mainly explained by more frequent central obesity in female patients.9 Our result also showed more frequent central obesity in female patients, but the prevalence of MetS between men and women was similar. However, female gender was still a significant predictor for MetS in our logistic regression model. Although the CATIE trial demonstrated that clozapine and olanzapine treatment were associated with highest risk of metabolic disturbances and that risperidone treatment can produce intermediate changes,8,18 the use of different SGA agents did not have a significant influence on the prediction of MetS in our final ANN and logistic regression models. However, it would be interesting to see whether the optimal models would be different depending on different SGA agents if the sample size were adequate for each SGA agent in the future studies.

Metabolic syndrome is prevalent in patients taking SGAs. Therefore, the identification of MetS in patients with SGA medications can facilitate early intervention to prevent severe metabolic or cardiovascular morbidity or mortality. The treatment of hypertension has been associated with a 20%–25% reduction in myocardial infarction, a 50% reduction in heart failure, and a 35%–40% reduction in stroke incidence.44 It was demonstrated that decreasing lipids by 10% decreased heart disease by 20%–30%.45 Therefore, some metabolic disturbances in patients taking SGAs may be modifiable. The emergent diabetes in patients treated with clozapine,13 olanzapine,14 and risperidone15 resolved on discontinuation of the drugs. However, for patients with schizophrenia or schizoaffective disorder, this approach is unrealistic because long-term antipsychotic treatment is required for many of these patients.7 Switching from an SGA with a high risk of weight gain to a different SGA with a reduced propensity to cause weight gain has been suggested.7,8 Metformin46 or cognitive/behavioral group intervention47 has also been proved to be effective in reducing antipsychotic-associated weight gain or metabolic disturbances, but the preliminary results required evidences from more carefully designed studies.

There are some limitations to this study. First, our study design was cross-sectional. Some clinical variables at the start of SGA use may be lacking: for example, waist circumference at baseline. Nevertheless, our preliminary findings are encouraging. Since psychotic patients may be reluctant to undergo or reject blood drawing and laboratory tests may require more of a financial burden for disabled psychotic patients, our findings may provide a highly sensitive tool that requires only clinically available information, which is inexpensive and noninvasive. Second, our final ANN and logistic regression models include 2 diagnostic criteria of MetS—waist circumference and DBP. One may wonder whether using nonlaboratory portions of the gold standard is enough for clinical screening. The sensitivity and specificity using nonlaboratory criteria as used in the gold standard were 34.5% and 98.0%, respectively. This result means that 65.5% of patients with MetS were missed. Therefore, our ANN and logistic regression models can provide better models, with both high sensitivity and specificity to identify MetS. Third, although our models generalized well to a population in a different hospital, more cases from other sources are needed in the future for better generalizability.

To our knowledge, this is the first study to identify MetS using ANN and logistic regression models. Both ANN and logistic regression yielded satisfactory results. Logistic regression can be easily understood and implemented as a risk score by clinicians. However, the capability of ANN analysis may be continuously refined by retraining and testing the network when new data become available.48 Although frequent monitoring of glucose levels or other metabolic indexes before and after antipsychotic treatment was proposed, clear emergent diabetes after initiation of an antipsychotic may be rare,49 and frequent (monthly or weekly) monitoring can be impractical and costly.27 Our models can provide an inexpensive, fast, and auxiliary method to identify MetS. However, when a patient is identified as having MetS by our models, blood drawing to check and follow up on the severity of metabolic abnormality is still required. For countries with better developed economies where there is no barrier for routine laboratory procedures, laboratory parameters should be included in the detection of MetS. In the future, our models should be confirmed by more studies including patients who have been on SGA treatment for more than 6 months and have been seen in different clinics.

Drug names: clozapine (FazaClo, Clozaril, and others), fluvoxamine (Luvox and others), olanzapine (Zyprexa), metformin (Riomet, Fortamet, and others), risperidone (Risperdal and others), sertraline (Zoloft and others).

Author affiliations: Graduate Institute of Medical Sciences, College of Medicine (Dr Lin), and Graduate Institute of Biomedical Informatics (Drs Chiu and Li), Taipei Medical University; Department of Psychiatry, National Taiwan University Hospital and National Taiwan University College of Medicine, Taipei (Drs Lin and Hwang); the Department of Psychiatry, Taipei Veterans General Hospital (Dr Bai); Department of Psychiatry, College of Medicine, National Yang-Ming University, Taipei (Dr Bai); and the Department of Psychiatry, Yuli Veterans Hospital, Yuli (Drs J-Y Chen and T-T Chen), Taiwan.

Author contributions: Drs Chiu and Li contributed equally to this work and are to whom correspondence should be addressed.

Financial disclosure: None reported.

Funding/support: National Science Council, Taipei, Taiwan (NSC 96-2314-B-002-121).

REFERENCES

1. Bilder RM, Goldman RS, Volavka J, et al. Neurocognitive effects of clozapine, olanzapine, risperidone, and haloperidol in patients with chronic schizophrenia or schizoaffective disorder. Am J Psychiatry. 2002;159(6):1018–1028. PubMed doi:10.1176/appi.ajp.159.6.1018

2. Leucht S, Wahlbeck K, Hamann J, et al. New generation antipsychotics versus low-potency conventional antipsychotics: a systematic review and meta-analysis. Lancet. 2003;361(9369):1581–1589. PubMed doi:10.1016/S0140-6736(03)13306-5

3. Purdon SE, Jones BD, Stip E, et al. Neuropsychological change in early phase schizophrenia during 12 months of treatment with olanzapine, risperidone, or haloperidol. The Canadian Collaborative Group for research in schizophrenia. Arch Gen Psychiatry. 2000;57(3):249–258. PubMed doi:10.1001/archpsyc.57.3.249

4. Correll CU, Leucht S, Kane JM. Lower risk for tardive dyskinesia associated with second-generation antipsychotics: a systematic review of 1-year studies. Am J Psychiatry. 2004;161(3):414–425. PubMed doi:10.1176/appi.ajp.161.3.414

5. Schooler N, Rabinowitz J, Davidson M, et al. Risperidone and haloperidol in first-episode psychosis: a long-term randomized trial. Am J Psychiatry. 2005;162(5):947–953. PubMed doi:10.1176/appi.ajp.162.5.947

6. Rettenbacher MA. Disturbances of glucose and lipid metabolism during treatment with new generation antipsychotics. Curr Opin Psychiatry. 2005;18(2):175–179. PubMed doi:10.1097/00001504-200503000-00011

7. Weiden PJ, Buckley PF. Reducing the burden of side effects during long-term antipsychotic therapy: the role of “switching” medications. J Clin Psychiatry. 2007;68(suppl 6):14–23.

8. Newcomer JW. Metabolic considerations in the use of antipsychotic medications: a review of recent evidence. J Clin Psychiatry. 2007;68(suppl 1):20–27. PubMed

9. De Hert MA, van Winkel R, Van Eyck D, et al. Prevalence of the metabolic syndrome in patients with schizophrenia treated with antipsychotic medication. Schizophr Res. 2006;83(1):87–93. PubMed doi:10.1016/j.schres.2005.12.855

10. McEvoy JP, Meyer JM, Goff DC, et al. Prevalence of the metabolic syndrome in patients with schizophrenia: baseline results from the Clinical Antipsychotic Trials of Intervention Effectiveness (CATIE) schizophrenia trial and comparison with national estimates from NHANES III. Schizophr Res. 2005;80(1):19–32. PubMed doi:10.1016/j.schres.2005.07.014

11. Saari KM, Lindeman SM, Viilo KM, et al. A 4-fold risk of metabolic syndrome in patients with schizophrenia: the Northern Finland 1966 Birth Cohort study. J Clin Psychiatry. 2005;66(5):559–563. PubMed

12. Bai YM, Chen JY, Yang WS, et al. Adiponectin as a potential biomarker for the metabolic syndrome in Chinese patients taking clozapine for schizophrenia. J Clin Psychiatry. 2007;68(12):1834–1839. PubMed doi:10.4088/JCP.v68n1202

13. Koller E, Schneider B, Bennett K, et al. Clozapine-associated diabetes. Am J Med. 2001;111(9):716–723. PubMed doi:10.1016/S0002-9343(01)01000-2

14. Koller EA, Doraiswamy PM. Olanzapine-associated diabetes mellitus. Pharmacotherapy. 2002;22(7):841–852. PubMed doi:10.1592/phco.22.11.841.33629

15. Koller EA, Cross JT, Doraiswamy PM, et al. Risperidone-associated diabetes mellitus: a pharmacovigilance study. Pharmacotherapy. 2003;23(6):735–744. PubMed doi:10.1592/phco.23.6.735.32178

16. Peuskens J, De HM, Mortimer A. Metabolic control in patients with schizophrenia treated with amisulpride or olanzapine. Int Clin Psychopharmacol. 2007;22(3):145–152. PubMed doi:10.1097/YIC.0b013e3280148c29

17. Sernyak MJ, Leslie DL, Alarcon RD, et al. Association of diabetes mellitus with use of atypical neuroleptics in the treatment of schizophrenia. Am J Psychiatry. 2002;159(4):561–566. PubMed doi:10.1176/appi.ajp.159.4.561

18. Hosojima H, Togo T, Odawara T, et al. Early effects of olanzapine on serum levels of ghrelin, adiponectin and leptin in patients with schizophrenia. J Psychopharmacol. 2006;20(1):75–79. PubMed doi:10.1177/0269881105056647

19. Allison DB, Mackell JA, McDonnell DD. The impact of weight gain on quality of life among persons with schizophrenia. Psychiatr Serv. 2003;54(4):565–567. PubMed doi:10.1176/appi.ps.54.4.565

20. Werneke U, Taylor D, Sanders TA. Options for pharmacological management of obesity in patients treated with atypical antipsychotics. Int Clin Psychopharmacol. 2002;17(4):145–160. PubMed doi:10.1097/00004850-200207000-00001

21. Weiden PJ, Mackell JA, McDonnell DD. Obesity as a risk factor for antipsychotic noncompliance. Schizophr Res. 2004;66(1):51–57. PubMed doi:10.1016/S0920-9964(02)00498-X

22. Harris EC, Barraclough B. Excess mortality of mental disorder. Br J Psychiatry. 1998;173:11–53. PubMed doi:10.1192/bjp.173.1.11

23. Dalmau A, Bergman B, Brismar B. Somatic morbidity in schizophrenia–a case control study. Public Health. 1997;111:393–397. PubMed doi:10.1016/S0033-3506(97)00089-9

24. Kurzthaler I, Fleischhacker WW. The clinical implications of weight gain in schizophrenia. J Clin Psychiatry. 2001;62(suppl 7):32–37. PubMed

25. Allison DB, Casey DE. Antipsychotic-induced weight gain: a review of the literature. J Clin Psychiatry. 2001;62(suppl 7):22–31. PubMed

26. Marder SR, Essock SM, Miller AL, et al. Physical health monitoring of patients with schizophrenia. Am J Psychiatry. 2004;161(8):1334–1349. PubMed doi:10.1176/appi.ajp.161.8.1334

27. Cohn TA, Sernyak MJ. Metabolic monitoring for patients treated with antipsychotic medications. Can J Psychiatry. 2006;51(8):492–501. PubMed

28. Nasrallah HA, Meyer JM, Goff DC, et al. Low rates of treatment for hypertension, dyslipidemia and diabetes in schizophrenia: data from the CATIE schizophrenia trial sample at baseline. Schizophr Res. 2006;86(1-3):15–22. PubMed doi:10.1016/j.schres.2006.06.026

29. American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders. 4th ed. Washington, DC: American Psychiatric Association; 1994.

30. International Diabetes Federation. Worldwide definition of the metabolic syndrome. http://www.idf.org/webdata/docs/MetS_def_update2006.pdf. Accessed August 27, 2009

31. Dreiseitl S, Ohno-Machado L. Logistic regression and artificial neural network classification models: a methodology review. J Biomed Inform. 2002;35(5-6):352–359. PubMed doi:10.1016/S1532-0464(03)00034-0

32. Subasi A, Ercelebi E. Classification of EEG signals using neural network and logistic regression. Comput Methods Programs Biomed. 2005;78(2):87–99. PubMed doi:10.1016/j.cmpb.2004.10.009

33. Tourassi GD, Floyd CE. The effect of data sampling on the performance evaluation of artificial neural networks in medical diagnosis. Med Decis Making. 1997;17(2):186–192. PubMed doi:10.1177/0272989X9701700209

34. Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology. 1982;143(1):29–36. PubMed

35. Hanley JA, McNeil BJ. A method of comparing the areas under receiver operating characteristic curves derived from the same cases. Radiology. 1983;148(3):839–843. PubMed

36. Haddad PM, Sharma SG. Adverse effects of atypical antipsychotics: differential risk and clinical implications. CNS Drugs. 2007;21(11):911–936. PubMed doi:10.2165/00023210-200721110-00004

37. Politi E, Franchini L, Spagnolo C, et al. Supporting tools in psychiatric treatment decision-making: sertraline outcome investigation with artificial neural network method. Psychiatry Res. 2005;134(2):181–189. PubMed doi:10.1016/j.psychres.2004.07.011

38. Franchini L, Spagnolo C, Rossini D, et al. A neural network approach to the outcome definition on first treatment with sertraline in a psychiatric population. Artif Intell Med. 2001;23(3):239–248. PubMed doi:10.1016/S0933-3657(01)00088-4

39. Serretti A, Smeraldi E. Neural network analysis in pharmacogenetics of mood disorders. BMC Med Genet. 2004;5:27. PubMed doi:10.1186/1471-2350-5-27

40. Lin CC, Wang YC, Chen JY, et al. Artificial neural network prediction of clozapine response with combined pharmacogenetic and clinical data. Comput Methods Programs Biomed. 2008;91(2):91–99. PubMed doi:10.1016/j.cmpb.2008.02.004

41. Das A, Ben-Menachem T, Cooper GS, et al. Prediction of outcome in acute lower-gastrointestinal haemorrhage based on an artificial neural network: internal and external validation of a predictive model. Lancet. 2003;362(9392):1261–1266. PubMed doi:10.1016/S0140-6736(03)14568-0

42. McNeil BJ, Keller E, Adelstein SJ. Primer on certain elements of medical decision making. N Engl J Med. 1975;293(5):211–215. PubMed

43. Cohn T, Prud’homme D, Streiner D, et al. Characterizing coronary heart disease risk in chronic schizophrenia: high prevalence of the metabolic syndrome. Can J Psychiatry. 2004;49(11):753–760. PubMed

44. Neal B, MacMahon S, Chapman N. Effects of ACE inhibitors, calcium antagonists, and other blood-pressure-lowering drugs: results of prospectively designed overviews of randomised trials. Blood Pressure Lowering Treatment Trialists’ Collaboration. Lancet. 2000;356(9246):1955–1964. PubMed doi:10.1016/S0140-6736(00)03307-9

45. Redelmeier DA, Tan SH, Booth GL. The treatment of unrelated disorders in patients with chronic medical diseases. N Engl J Med. 1998;338(21):1516–1520. PubMed doi:10.1056/NEJM199805213382106

46. Chen CH, Chiu CC, Huang MC, et al. Metformin for metabolic dysregulation in schizophrenic patients treated with olanzapine. Prog Neuropsychopharmacol Biol Psychiatry. 2008;32(4):925–931. PubMed doi:10.1016/j.pnpbp.2007.11.013

47. Weber M, Wyne K. A cognitive/behavioral group intervention for weight loss in patients treated with atypical antipsychotics. Schizophr Res. 2006;83(1):95–101. PubMed doi:10.1016/j.schres.2006.01.008

48. Haydon GH, Jalan R, la-Korpela M, et al. Prediction of cirrhosis in patients with chronic hepatitis C infection by artificial neural network analysis of virus and clinical factors. J Viral Hepat. 1998;5(4):255–264. PubMed doi:10.1046/j.1365-2893.1998.00108.x

49. Holt RI, Peveler RC. Association between antipsychotic drugs and diabetes. Diabetes Obes Metab. 2006;8(2):125–135. PubMed doi:10.1111/j.1463-1326.2005.00495.x