Telemedicine for Anger Management Therapy in a Rural Population of Combat Veterans With Posttraumatic Stress Disorder: A Randomized Noninferiority Trial
Objective: To demonstrate the noninferiority of a telemedicine modality, videoteleconferencing, compared to traditional in-person service delivery of a group psychotherapy intervention for rural combat veterans with posttraumatic stress disorder (PTSD).
Method: A randomized controlled noninferiority trial of 125 male veterans with PTSD (according to DSM criteria on the Clinician-Administered PTSD Scale) and anger difficulties was conducted at 3 Veterans Affairs outpatient clinics. Participants were randomly assigned to receive anger management therapy delivered in a group setting with the therapist either in-person (n = 64) or via videoteleconferencing (n = 61). Participants were assessed at baseline, midtreatment (3 weeks), posttreatment (6 weeks), and 3 and 6 months posttreatment. The primary clinical outcome was reduction of anger difficulties, as measured by the anger expression and trait anger subscales of the State-Trait Anger Expression Inventory-2 (STAXI-2) and by the Novaco Anger Scale total score (NAS-T). Data were collected from August 2005 to October 2008.
Results: Participants in both groups showed significant and clinically meaningful reductions in anger symptoms, with posttreatment and 3 and 6 months posttreatment effect sizes ranging from .12 to .63. Using a noninferiority margin of 2 points for STAXI-2 subscales anger expression and trait anger and 4 points for NAS-T outcomes, participants in the videoteleconferencing condition demonstrated a reduction in anger symptoms similar (“non-inferior”) to symptom reductions in the in-person groups. Additionally, no significant between-group differences were found on process variables, including attrition, adherence, satisfaction, and treatment expectancy. Participants in the in-person condition reported significantly higher group therapy alliance.
Conclusions: Clinical and process outcomes indicate delivering cognitive-behavioral group treatment for PTSD-related anger problems via videoteleconferencing is an effective and feasible way to increase access to evidence-based care for veterans residing in rural or remote locations.
Trial Registration: clinicaltrials.gov Identifier: NCT00122109
J Clin Psychiatry 2010;71(7):855–863
© Copyright 2010 Physicians Postgraduate Press, Inc.
Submitted: August 11, 2009; accepted September 28, 2009.
Online ahead of print: January 26, 2010 (doi:10.4088/JCP.09m05604blu).
Corresponding author: Leslie A. Morland, PsyD, National Center for PTSD-Pacific Islands Division, Department of Veterans Affairs Pacific Islands Healthcare System, 3375 Koapaka St, Suite I-560, Honolulu, HI 96819 (Leslie.Morland@va.gov).
Telemedicine service delivery strategies, such as videoteleconferencing, hold the promise of improving access to care for underserved populations, including people living in rural communities1–3 and ethnic minorities who exhibit wide-ranging disparities in health status.4,5 Telemedicine offers potential solutions for large health care systems, such as the US Department of Veterans Affairs (VA) and the US Department of Defense, that provide comprehensive services to patients in rural and remote areas. While research on videoteleconferencing demonstrates feasibility for providing a range of services and good reliability for clinical assessments, there are few randomized controlled trials (RCTs) demonstrating clinical effectiveness of psychotherapy via videoteleconferencing.1–3 Moreover, prior RCTs of videoteleconferencing psychotherapy have not stringently tested whether these treatments produce outcomes comparable to those from in-person care, which requires use of a noninferiority trial design. Such a design has previously been used to compare videoteleconferencing versus in-person psychiatric pharmacotherapy6 but not psychotherapy.
The VA is making concerted efforts to expand access to evidence-based treatments for veterans with posttraumatic stress disorder (PTSD). Recent research on returning service members from Iraq and Afghanistan indicates high levels of combat-related PTSD (4%–17%) and other posttraumatic psychiatric conditions.7–9 One challenge in reaching this population is that 40% of service members leaving active duty return to rural or remote areas,10 where access to evidence-based mental health care is often limited.10,11 Thus, it is critical that researchers develop and implement effective strategies to increase access to efficacious treatments for these returning service members. Pilot studies have suggested the feasibility of providing psychotherapy to patients with PTSD by using videoteleconferencing, but they have not examined noninferiority relative to in-person care.12,13
Combat-related PTSD is associated with significant distress, functional impairment, and reduced quality of life.14 One aspect of PTSD that substantially impairs functioning is disregulated anger,15,16 which also has significant effects on marriages and families.17 DiGiuseppe and Tafrate18 have conducted a meta-analysis of experimental trials that demonstrates the efficacy of cognitive-behavioral therapy (CBT) for treatment of anger, and Deffenbacher19 has further summarized these findings. There is a compelling need to increase access to appropriate evidence-based anger treatment for military populations with PTSD, especially for rural veterans, given the close family networks and social interconnectedness of typical rural communities. Moreover, while there is a strong evidence base to support psychiatric interventions for treatment of PTSD in civilians, there is a paucity of research demonstrating similar treatment effectiveness in veterans.20,21
The current study compares the effectiveness of videoteleconferencing-delivered group anger management therapy (AMT) and in-person delivery of the same treatment in a sample of rural combat veterans with PTSD. This is, to our knowledge, the first RCT to investigate the use of videoteleconferencing for a manual-based CBT group psychotherapy. Moreover, this is a particularly challenging application of telemedicine, as we are remotely delivering a potentially evocative treatment to a population of complex and potentially volatile patients. Using a noninferiority trial design, we aimed to assess not only if videoteleconferencing psychotherapy is effective but also if it is as good as in-person delivery. We hypothesized that providing a group CBT intervention via videoteleconferencing would result in similar reductions in anger symptoms as obtained from traditional in-person care. Further, we hypothesized that key process indicators, including attrition, adherence, satisfaction, and therapeutic alliance, would not be significantly different between the videoteleconferencing and in-person conditions.
Design and Study Participants
A noninferiority-designed RCT was conducted with male combat veterans with PTSD to compare the clinical effectiveness of a cognitive-behavioral anger management intervention provided via videoteleconferencing relative to traditional in-person treatment. Recruitment, treatment, and follow-up assessment took place between August 2005 and October 2008. The VA Pacific Island Health Care System’s Institutional Review Board (IRB) approved the protocol, and all participants provided written informed consent prior to study enrollment.
Female veterans were not included due to their limited number (< 4%). Inclusion criteria were (1) PTSD (current or lifetime) determined by the Clinician-Administered PSTD Scale (CAPS)22; (2) score of 20 or higher on the 10-item trait anger (T-ANG) subscale of the State-Trait Anger Expression Inventory-2 (STAXI-2),23 indicating moderate to severe anger problems; and (3) stable medication regimen for a minimum of 2 months prior to study entry. Exclusion criteria were (1) active psychotic symptoms/disorder, (2) active homicidal or suicidal ideation, (3) significant cognitive impairment or history of organic mental disorder, and (4) current substance dependence or unwillingness to refrain from substance abuse during treatment.
Recruitment and Randomization
Attempts were made to recruit 16 participants for each of 9 cohorts, with 10 participants randomly assigned into each condition (videoteleconferencing vs in person) within a cohort. Participants were recruited across 3 VA clinical sites and 3 Vet Centers across the Hawaiian Islands of Hawaii, Maui, and Oahu. Each recruitment site had a designated project liaison as the primary local site contact for the project coordinator.
A total of 218 male veterans were referred to this study (Figure 1), of whom 160 consented and completed a comprehensive assessment battery. Of 160 veterans assessed, 35 (22%) were excluded or declined to participate before being randomly assigned, leaving an intent-to-treat (ITT) sample of 125. Following completion of initial assessment and informed consent, participants in each clinic were stratified by war era (Vietnam, Desert Storm, or other) and randomly assigned by an off-site statistician to 1 of 2 treatment conditions at their local VA clinic. In the ITT sample (64 receiving in-person condition and 61 receiving videoteleconferencing condition), 112 participants (57 receiving in-person condition and 55 receiving videoteleconferencing condition) attended at least 9 treatment sessions and were included in the completer sample.
Click figure to enlarge
The study coordinator was informed by the off-site statistician of the participants’ treatment assignment. Successful randomization was indicated by findings that participants in the 2 conditions did not differ by demographic variables, psychiatric comorbidity, severity of PTSD (Table 1), or severity of baseline anger scores (Table 2).
Click figure to enlarge
Click figure to enlarge
Participants were assessed at baseline, at midtreatment (3 weeks), immediately at posttreatment (6 weeks), and at 3 and 6 months posttreatment. At baseline, a structured clinical interview was administered to obtain demographic and health information. The Structured Clinical Interview for DSM-IV (SCID)24 was used to evaluate exclusionary diagnoses and comorbidities. The CAPS was used to assess current or lifetime PTSD, using the “1/2 rule,” which stipulates that symptoms occur at least monthly, with moderate intensity, and all diagnostic criteria are met. The PTSD Checklist-Military Version (PCL-M)25 provided an additional measure of PTSD severity at baseline and posttreatment. All assessments were conducted in person by a master’s- or doctoral-level assessor not involved with delivering the treatment and occurred at the same VA clinic where the treatment was delivered.
Primary clinical outcome variables of anger severity included anger expression, trait anger, and anger disposition. Anger expression was measured using the STAXI-2 32-item self-report anger expression subscale. Trait anger, or level of anger experienced over time, was assessed using the STAXI-2 10-item trait anger subscale. The STAXI-2 subscales have robust psychometric properties, including high internal consistency, external validity, and construct validity.23 Anger disposition was assessed using the Novaco Anger Scale (NAS), a well-validated self-report instrument designed to measure cognitive, arousal, and behavioral aspects of anger in both clinical and nonpatient populations.26,27 The NAS total score (NAS-T) has high internal consistency (.95) and test-retest reliability (.84) as well as concurrent and predictive validity.26 A secondary clinical outcome variable included examining PTSD symptom reduction posttreatment using the PCL-M.25
Process variables included attrition, treatment adherence, patient satisfaction, treatment expectancy, and group therapeutic alliance. Treatment attrition, adherence, and attendance were assessed using weekly compliance logs completed at each treatment session. Technical problems with videoteleconferencing equipment or transmission were systematically tracked and assessed throughout each group session for each cohort, using a standardized form identifying frequency and quality of technical difficulties and the impact on the treatment session. Satisfaction with services was assessed using the Charleston Psychiatric Outpatient Satisfaction Scale-VA (CPOSS-VA),28 a 16-item measure adapted specifically to evaluate satisfaction among veterans treated within VA clinics. It demonstrated good convergent validity and excellent reliability (α = .96) in this sample.
Treatment credibility scales29 were used to assess for differences in outcome expectancy at midtreatment assessment. Four questions were adapted for this study: specifically, questions regarding how logical the treatment appears, how confident veterans are about the treatment, how expectant they are of success, and how confident veterans would be to refer others to this treatment. To assess therapeutic alliance, we used an abbreviated version of the Group Therapy Alliance Scale (GTAS),30 which has demonstrated adequate psychometric properties. In this sample, internal consistency and split-half reliability coefficients for the GTAS are .88 and .82, respectively. Previous research supports the predictive validity of the GTAS in an RCT31 for borderline personality disorder.
Anger Management Intervention
Anger management therapy is a 12-session manual-based CBT intervention that was developed by the National Institute on Drug Abuse Behavioral Therapies Development Program.32,33 Anger management therapy has been found to be effective for anger management treatment among veterans with substance abuse problems32,33 and PTSD.34 The AMT intervention has 2 primary objectives: (1) to teach participants to monitor their anger using the “anger meter” and to identify the specific cues and triggers to their anger and (2) to help participants develop cognitive and behavioral coping strategies for controlling their anger and consolidate these strategies into a specific individualized control plan. Additional objectives of the treatment include examining the relationships between anger and violence, the contribution of self-statements to the escalation of anger, and the ways anger is used to cover up other emotions. Each group session is divided into 2 sections. In the first section, patients “check in” to report on their current level of anger, to practice anger management strategies, and to evaluate anger-provoking incidents that have occurred during the previous week. In the second section, therapists review the conceptual framework for understanding anger and teach CBT strategies and techniques to the veterans.
Both treatment conditions received the same manual-based 12-session AMT protocol, with 2 sessions per week over a 6-week period. All participants received an AMT workbook. There were a total of 5 doctoral-level therapists, all with prior experience in conducting CBT groups with veterans. Within each cohort, the same therapist delivered both interventions, usually at the same time of the day but on a different day of the week. The participants met, in a group, at their local VA in the same conference room for both conditions. The therapist traveled to the clinical site for the in-person condition and remained at the Honolulu VA for the videoteleconferencing condition. All treatment sessions were audiotaped and reviewed by the treatment supervisor for monitoring competence and protocol adherence using an adapted version of an adherence measure previously used with this treatment protocol. Therapist protocol adherence was excellent (94.6%) across the 12-session protocol for the 10 cohorts. There were no differences in adherence between the 2 conditions. A senior clinician independent of the treatment listened to a random 11% of the audiotapes for interrater reliability, yielding .96 interrater agreement.
A VA Information Resource Management Service staff member was available during scheduled videoteleconferencing groups in case of technical difficulties. The VA IRB required an observer to sit in the group room during the remote videoteleconferencing sessions. The observer was silent except for intervening in cases of technological or clinical emergencies. Over 120 group sessions, the observer intervened only twice: once to reestablish the videoteleconferencing connection and once to check on a veteran who briefly left the session for a “time-out.”
Noninferiority trial designs differ from conventional superiority trials, which test the hypothesis that one treatment is superior to another.35 If findings fail to show superiority of one treatment, they do not prove that two treatments are similar—it may be lack of sufficient power to detect a difference that actually exists. A noninferiority trial tests whether one treatment produces results that are clinically noninferior to another known treatment, first by positing the smallest difference in outcomes (the noninferiority margin) that would be clinically meaningful. On the basis of consultation with anger-treatment experts, we set conservative noninferiority margins at 2.0 points on the STAXI-2 anger expression subscale, 2.0 points on the STAXI-2 trait anger subscale, and 4.0 points on NAS-T. We set a noninferiority margin of 4 points for the PCL-M. A noninferiority analysis then requires estimating the 95% confidence interval (CI) of the difference between conditions (videoteleconferencing minus in person) in change scores on outcomes. A positive value indicates less reduction of symptoms in videoteleconferencing relative to in person. To conclude that videoteleconferencing is noninferior to in person, the upper limit of the 95% CI must be below the preset noninferiority margin.
The SAS MEANS procedure (SAS Institute Inc, Cary, North Carolina) was used to calculate means and standard deviations (SDs) for change from baseline to posttreatment and 3- and 6-month follow-ups for the videoteleconferencing group, the in-person group, and all participants in both groups.25 Both ITT and per-protocol (treatment completer) analyses were conducted using this procedure. The SAS procedures MI (multiple imputation using the Markov Chain Monte Carlo method) and MIANALYZE were used for imputation in the ITT analysis.36 Our use of SAS MEANS procedure was based on assumptions that cohort effects were minimal and that randomization adequately controlled for baseline differences. To test these assumptions, we conducted secondary analyses using mixed models that included cohort effects and baseline scores as covariates. The mixed-model estimates of changes from baseline to the posttreatment and 3- and 6-month follow-ups were almost identical to results calculated by SAS MEANS procedure, so means and SDs from the SAS MEANS procedure were used in the noninferiority analysis.
In a priori power analyses based on anticipated effect sizes and our preset margins for noninferiority, we estimated that a total sample size of 180 participants would give us 86%–97% power to detect the noninferiority of videoteleconferencing versus in person on primary outcomes. We also estimated that if we reached two-thirds of our recruitment goal (120 participants, or 60 for each arm) and two-thirds of these participants dropped out in the course of the study (leaving only 40 participants in each arm), we would still have 79%–91% power to detect inferiority. Thus, our final sample of 125 participants (including 114 completers) should provide adequate power.
Participant characteristics at baseline did not differ between groups (Table 1). Mean age of the sample was 54.7 (SD = 9.6) years, and 63% were married. The majority of the participants were of Pacific-Islander (33%), white (33%), or Asian-American (27%) descent; 90% of the participants reported combat exposure, and 75% served in Vietnam. Further demographic information is reported in Table 1. Data on psychiatric diagnoses, in addition to PTSD, revealed a significant rate of psychiatric comorbidity (54% current, 94% lifetime; see Table 1) and no significant between-group differences. Of the 125 veterans who completed treatment, 112 attended at least 9 of 12 treatment sessions and were included in our completer sample (in person, n = 57 and videoteleconferencing, n = 55) (see Figure 1). There were no statistically significant differences between baseline characteristics of the 112 completers and 23 noncompleters.
Participants in both conditions showed substantial improvement at posttreatment on mean anger scores (Table 2). The mixed-model ITT analysis shows there is a statistically significant improvement in scores on the STAXI-2 subscales anger expression and trait anger and on the NAS-T for both overall and for videoteleconferencing and in-person group.
Figure 2 shows the CIs for mean difference scores for 3 outcome measures (STAXI-2 subscales anger expression and trait anger and the NAS-T) between conditions for the completer sample (n = 112) and for the ITT sample (N = 125) at all assessment points. Baseline scores for the 2 conditions were not significantly different. The vertical axes are the differences in score between in-person and videoteleconferencing improvements from baseline; since there can be no change at baseline, these are necessarily zero. If in person is superior to videoteleconferencing, then the difference is positive. The 95% CIs are 2-sided, but the upper limits of the CI demonstrate the margins for noninferiority in the graph. The dotted lines show predetermined minimum clinically meaningful differences (margins of noninferiority, Δ). Both figures show that for all 3 anger scales at posttreatment and at 3 months, the upper limit of the CI is below the minimum clinically meaningful difference, indicating that videoteleconferencing was noninferior to in person. At 6 months posttreatment, we found noninferiority of videoteleconferencing relative to in person for both anger expression and trait anger. However, results on the NAS-T at 6 months posttreatment were inconclusive due to the large SD around the scores. Results from the completer analysis (not shown) were similar to those from the ITT analyses. Although our goal was to confirm only that videoteleconferencing was noninferior to in-person (ie, any difference was below a preset threshold), we found that mean improvements in the videoteleconferencing condition were actually slightly larger than in the in-person condition. Post hoc analyses of effect size differences between the conditions are shown in Table 3.
Click figure to enlarge
Click figure to enlarge
Secondary analyses examined change in PTSD symptoms posttreatment. There was significant reduction of PTSD symptoms on the PCL-M at posttreatment (d = 2.5, P = .05) across both conditions in the total ITT sample (N = 125). However, as shown in the last panel of Figure 2, we could not conclude that the videoteleconferencing condition was noninferior to the in-person condition on the PCL-M outcomes.
Overall, participants reported attending a mean of 10.3 (SD = 2.5) sessions and completing approximately 7.7 (SD = 3.5) homework assignments (of 11 assignments). One-way analyses of variance (ANOVAs) were conducted and revealed no significant differences between the 2 conditions on attendance (F1,123 = 1.14, P = .29) or homework completion (F1,111 = 0.00, P = .98). A total of 13 participants (10.4%) dropped out of treatment, and there were no significant differences between in-person and videoteleconferencing conditions on frequency of treatment dropout (12.5% vs 8.2%, respectively; χ2 = 0.62, P = .43). Participants reported a mean score of 31.4 (SD = 7.3) of a possible 40 on the treatment expectancy measure, a mean score of 62.5 (SD = 13.1) of a possible 80 on the CPOSS-VA, and a mean score of 128.1 (SD = 17.0) out of 150 on the GTAS. One-way ANOVAs revealed no differences between the in-person and videoteleconferencing participants on the CPOSS-VA (F1,105 = 0.01, P = .94) or treatment expectancy measure (F1,104 = 2.33, P = .13); however, compared with participants in the videoteleconferencing condition, participants in the in-person condition reported higher overall group therapeutic alliance on the GTAS (F1,95 = 5.79, P = .02).
This is the first RCT to demonstrate that providing evidence-based group psychotherapy via videoteleconferencing not only is feasible but also produces outcomes that are as good as in-person treatment. We found that a manual-based CBT anger management conducted by videoteleconferencing is as effective as in-person delivery of the same treatment in reducing anger problems among rural veterans with PTSD. Participants in both conditions tolerated and benefitted from AMT, making this one of the few large RCTs to show meaningful treatment benefits for reducing anger problems in veterans with PTSD. Findings are enhanced by the fact that there was a large percentage of racial minorities in this study, including Asian-Americans (26%) and Pacific Islanders (33%), for whom there is a dearth of empirical data regarding PTSD impairment and treatment outcomes.37
Our process outcomes confirm the acceptability and safety of implementing videoteleconferencing for this population. No adverse events were observed for any participant in either group. Participants in both conditions reported high levels of treatment credibility, satisfaction with care, homework adherence, and high alliance with the therapist and other group members. Treatment drop out (10%) was lower than the 20%–35% often reported in clinical trials with veterans or PTSD patients.21 Videoteleconferencing evidenced very few disruptions caused by technical difficulties, as no treatment sessions were cancelled or postponed due to technological difficulties. Finally, clinical intervention by the in-room observer in the videoteleconferencing condition was needed on only one occasion; to address minor situational distress. Together, these data indicate that videoteleconferencing can be a valuable service delivery strategy for reducing geographic disparities in access to evidence-based psychotherapy.
This study has several important and novel aspects. First, it is one of only a handful of methodologically rigorous noninferiority-designed RCTs of videoteleconferencing interventions and the first involving psychotherapy with a PTSD population. Second, process outcomes clearly establish the acceptability and feasibility of using videoteleconferencing to improve access to psychiatric care for rural veterans with severe mental illnesses. Third, clinical outcome results show promising anger outcomes from posttreatment to 6-month follow-up for a population (veterans with PTSD) for which there is a lack of evidence regarding effective interventions.21 Fourth, there is good reason to believe that study participants are representative of the broader population of rural veterans with PTSD, given inclusion/ exclusion criteria,38 which allowed for high rates of psychiatric comorbidity. Sixth, this study includes high rates of rural residents (100%) and racial minorities (69.3%), which are 2 groups that are often excluded from clinical research on PTSD. Finally, study implementation was rigorously controlled, including careful a priori noninferiority analyses and sample size calculations, use of an evidenced-based intervention, careful therapist fidelity monitoring, follow-up assessments up to 6 months, and high participant adherence and retention rates (89%) in a difficult-to-treat clinical sample.
Despite its merits, this study has important limitations. We did not evaluate changes in functional impairment or marital/family relationships, so it unclear how improvements on anger may have affected these other domains. It is unknown how easily patients who are accustomed to in-person group therapy would accept transition to videoteleconferencing treatment. The study focused on alleviating anger problems in patients with PTSD rather than treating PTSD symptoms per se. Due to the targeted nature of the anger intervention, we conservatively excluded participants with acute safety concerns (homicidal or suicidal) and current substance dependence. However, other telemental health literature suggests that both substance use and crisis management issues can be safely addressed via telemedicine.39,40 Finally, our primary anger outcome measures were self-report questionnaires and therefore subject to participant perception. However, these instruments are considered the current standard outcomes measures used in RCTs focused on anger outcomes.
Future research should rigorously assess potential noninferiority of videoteleconferencing delivery of specific evidence-based interventions (exposure therapy) that target core PTSD symptoms.20 Studies should also assess videoteleconferencing delivery into other environments. The current project provided videoteleconferencing to veterans at local community clinics; research is therefore needed to examine in-home and other individualized telemedicine strategies. Cost analyses are also necessary to understand the relative costs and cost-benefits of telemedicine, as well as other systemic and economic implications of increasing access to mental health care for rural populations. Finally, additional implementation research is needed on how to most effectively disseminate telemedicine for populations with PTSD and integrate telemedicine with existing models of care.41
Author affiliations: National Center for PTSD-Pacific Islands Division, Department of Veterans Affairs Pacific Islands Healthcare System (Drs Morland and Greene); Pacific Health Research Institute (Dr He), Honolulu, Hawaii; National Center for PTSD-Dissemination & Training Division, Department of Veterans Affairs Palo Alto Healthcare System (Drs Greene and Rosen); Department of Psychiatry and Behavioral Sciences, Stanford University School of Medicine, Palo Alto (Dr Rosen); Graduate School of Education and Psychology, Pepperdine University, Malibu (Dr Foy); San Francisco Veterans Affairs Medical Center, San Francisco (Dr Reilly), California; Centers for American Indian and Alaska Native Mental Health, University of Colorado Denver, and Veterans Rural Health Resource Center Western Region, Department of Veterans Affairs Office of Rural Health, Denver (Dr Shore); and The Menninger Clinic and Baylor College of Medicine, Houston, Texas (Dr Frueh).
Author contributions: Dr Morland, principal investigator of this study, had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Potential conflicts of interest: The authors have no affiliations with or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the article.
Funding/support: This work was partially supported by grant TEL 03-080-3 from the Veterans Affairs Health Services Research and Development. This work was also supported by the Office of Research and Development, Medical Research Service, Department of Veterans Affairs.
Disclaimer: All views and opinions expressed herein are those of the authors and do not necessarily reflect those of our respective institutions or the Department of Veterans Affairs.
Previous presentations: Presented at the Office of Mental Health Services Conference, July 2009, Baltimore, Maryland; the HSR&D VA Information Resource Center Clinical Informatics Seminar (cyber seminar), June 2009; and the Annual VA HSR&D Conference [poster], February 2009, Baltimore, Maryland.
Acknowledgment: The authors wish to acknowledge statistical, methodological, and project execution contributions by Valerie Durkalski, PhD (Department of Biostatistics, Bioinformatics and Epidemiology, Medical University of South Carolina, Charleston, South Carolina); Dan King, PhD (Boston University School of Medicine); Lynda King, PhD (Boston University School of Medicine); Alexandra Macdonald, PhD (National Crime Victims Research and Treatment Center, Medical University of South Carolina); and the National Center for PTSD-Pacific Island Health Care System study project–funded research staff, including Anneliese Andrews, MA; Kathleen Grubbs, MA; Karen Kloezeman, MA; Cynthia J’Anthony, MA; Chris Lu, MA; Tiare MacDonald, BA; Richard Reis, MA; Kristy Sakai, MA; and Thad Strom, PhD. The authors are deeply appreciative of the veterans and VA mental health providers who contributed to this research effort.
1. Frueh BC, Deitsch SE, Santos AB, et al. Procedural and methodological issues in telepsychiatry research and program development. Psychiatr Serv. 2000;51(12):1522–1527. PubMed doi:10.1176/appi.ps.51.12.1522
2. Monnier J, Knapp RG, Frueh BC. Recent advances in telepsychiatry: an updated review. Psychiatr Serv. 2003;54(12):1604–1609. PubMed doi:10.1176/appi.ps.54.12.1604
3. Morland LA, Frueh BC, Pierce K, et al. PTSD and tele-mental health: updates and future directions. National Center for PTSD Clinical Quarterly. 2003;12(1):1–5.
4. Kincade K. Top 10 telemedicine programs for 1999: experience pays off. Diagn Imaging (San Franc). 2000;22(3):63–67. PubMed
5. Field M. Telemental Health: A Guide to Assessing Telecommunications in Health Care. Washington, DC: National Academy Press; 1996.
6. O’Reilly R, Bishop J, Maddox K, et al. Is telepsychiatry equivalent to face-to-face psychiatry? Results from a randomized controlled equivalence trial. Psychiatr Serv. 2007;58(6):836–843. PubMed doi:10.1176/appi.ps.58.6.836
7. Grieger TA, Cozza SJ, Ursano RJ, et al. Posttraumatic stress disorder and depression in battle-injured soldiers. Am J Psychiatry. 2006;163(10):1777–1783, quiz 1860. PubMed doi:10.1176/appi.ajp.163.10.1777
8. Hoge CW, Auchterlonie JL, Milliken CS. Mental health problems, use of mental health services, and attrition from military service after returning from deployment to Iraq or Afghanistan. JAMA. 2006;295(9):1023–1032. PubMed doi:10.1001/jama.295.9.1023
9. Seal KH, Bertenthal D, Miner CR, et al. Bringing the war back home: mental health disorders among 103,788 US veterans returning from Iraq and Afghanistan seen at Department of Veterans Affairs facilities. Arch Intern Med. 2007;167(5):476–482. PubMed doi:10.1001/archinte.167.5.476
10. Tanielian T, Jaycox LH. Invisible Wounds of War: Psychosocial and Cognitive Injuries, Their Consequences, and Services to Assist Recovery. Santa Monica, CA: RAND Corporation; 2008.
11. Committee on the Future of Rural Health Care, Institute of Medicine. Quality Through Collaboration: The Future of Rural Health Care. Washington, DC: National Academies Press; 2004.
12. Frueh BC, Monnier J, Yim E, et al. A randomized trial of telepsychiatry for post-traumatic stress disorder. J Telemed Telecare. 2007;13(3):142–147. PubMed doi:10.1258/135763307780677604
13. Morland LA, Pierce K, Wong MY. Telemedicine and coping skills groups for Pacific Island veterans with post-traumatic stress disorder: a pilot study. J Telemed Telecare. 2004;10(5):286–289. PubMed doi:10.1258/1357633042026387
14. Dohrenwend BP, Turner JB, Turse NA, et al. The psychological risks of Vietnam for US veterans: a revisit with new data and methods. Science. 2006;313(5789):979–982. PubMed doi:10.1126/science.1128944
15. Chemtob CM, Hamada RS, Roitblat HL, et al. Anger, impulsivity, and anger control in combat-related posttraumatic stress disorder. J Consult Clin Psychol. 1994;62(4):827–832. PubMed doi:10.1037/0022-006X.62.4.827
16. McFall ME, Wright PW, Donovan DM, et al. Multidimensional assessment of anger in Vietnam veterans with posttraumatic stress disorder. Compr Psychiatry. 1999;40(3):216–220. PubMed doi:10.1016/S0010-440X(99)90006-8
17. Stanton MD, Figley CR. Treating the Vietnam veteran within the family system. In: Figley CR, ed. Stress Disorder Among Vietnam Veterans: Theory Research and Treatment. New York, NY: Bruner-Mazel; 1978:281–290.
18. DiGiuseppe R, Tafrate RC. Anger treatment for adults: a meta-analytic review. Clin Psychol Sci Pract. 2003;10(1):70–84. doi:10.1093/clipsy/10.1.70
19. Deffenbacher JL. Evidence for effective treatment of anger-related disorders. In: Feindler EL, ed. Anger Related Disorders: A Practitioner’s Guide to Comparative Treatments. New York, NY: Springer Publishing Co.; 2006:43–69.
20. Frueh BC, Grubaugh AL, Elhai JD, et al. US Department of Veterans Affairs disability policies for posttraumatic stress disorder: administrative trends and implications for treatment, rehabilitation, and research. Am J Public Health. 2007;97(12):2143–2145. PubMed doi:10.2105/AJPH.2007.115436
21. Institute of Medicine and National Research Council. Treatment of Posttraumatic Stress Disorder: An Assessment of the Evidence. Washington, DC: The National Academies Press; 2007.
22. Blake DD, Weathers FW, Nagy LM, et al. The development of a Clinician-Administered PTSD Scale. J Trauma Stress. 1995;8(1): 75–90. PubMed doi:10.1002/jts.24900801
23. Speilberger CD. Manual for the State Trait Anger Expression Inventory. Odessa, FL: Psychological Assessment Resources; 1988.
24. Spitzer RL, Williams JB, Gibbon M, et al. Structured Clinical Interview for DSM-IV. Washington, DC: American Psychiatric Press; 1997.
25. Weathers FW, Huska JA, Keane TM. PCL-M for DSM-IV. Boston, MA: National Center for PTSD–Behavioral Science Division; 1991.
26. Novaco RW. Anger as a risk factor for violence among the mentally disordered. In: Monahan J, Steadman H, eds. Violence and Mental Disorder. Chicago, IL: University of Chicago Press; 1994: 21–59.
27. Chemtob CM, Novaco RW, Hamada RS, et al. Anger regulation deficits in combat-related posttraumatic stress disorder. J Trauma Stress. 1997;10(1):17–36. PubMed doi:10.1002/jts.2490100104
28. Frueh BC, Pellegrin KL, Elhai JD, et al. Patient satisfaction among combat veterans receiving specialty PTSD treatment. J Psychiatr Pract. 2002;8(5):326–332. PubMed doi:10.1097/00131746-200209000-00010
29. Borkovec TD, Nau SD. Credibility of analogue therapy rationales. J Behav Ther Exp Psychiatry. 1972;3(4):257–260. doi:10.1016/0005-7916(72)90045-6
30. Pinsof WM, Catherall DR. The integrative psychotherapy alliance: family, couple and individual therapy scale. J Marital Fam Ther. 1986;12(2):137–151. doi:10.1111/j.1752-0606.1986.tb01631.x
31. Marziali E, Munroe-Blum H, McCleary L. The contribution of group cohesion and group alliance to the outcome of group psychotherapy. Int J Group Psychother. 1997;47(4):475–497. PubMed
32. Reilly PM, Shopshire MS. Anger Management for Substance Abuse and Mental Health Clients: A Cognitive Behavioral Therapy Manual. DHHS Pub. No. (SMA) 02-3756. Rockville, MD: Center for Substance Abuse Treatment, Substance Abuse and Mental Health Services Administration; 2002.
33. Reilly PM, Shopshire MS. Anger management group treatment for cocaine dependence: preliminary outcomes. Am J Drug Alcohol Abuse. 2000;26(2):161–177. PubMed doi:10.1081/ADA-100100598
34. Reilly PM, Clark HW, Shopshire MS, et al. Anger management, posttraumatic stress disorder, and substance abuse. NIDA Research Monograph: Problems of Drug Dependence, Proceedings of the 57th Annual Scientific Meeting of the College on Problems of Drug Dependence, Inc. 1996;62:322.
35. Greene CJ, Morland LA, Durkalski VL, et al. Noninferiority and equivalence designs: issues and implications for mental health research. J Trauma Stress. 2008;21(5):433–439. PubMed doi:10.1002/jts.20367
36. SAS Institute Inc. SAS 9.1.3 Java Metadata Interface: User’s Guide. Cary, NC: SAS Institute Inc.; 2004.
37. Pole N, Gone JP, Kulkarni M. Posttraumatic stress disorder among ethnoracial minorities in the United States. Clin Psychol Sci Pract. 2008;15(1):35–61. doi:10.1111/j.1468-2850.2008.00109.x
38. Stirman SW. The applicability of randomized controlled trials of psychosocial treatments for PTSD to a veteran population. J Psychiatr Pract. 2008;14(4):199–208. PubMed doi:10.1097/01.pra.0000327309.58411.e0
39. Shore JH, Hilty DM, Yellowlees P. Emergency management guidelines for telepsychiatry. Gen Hosp Psychiatry. 2007;29(3):199–206. PubMed doi:10.1016/j.genhosppsych.2007.01.013
40. Yellowlees P, Burke MM, Marks SL, et al. Emergency telepsychiatry. J Telemed Telecare. 2008;14(6):277–281. PubMed doi:10.1258/jtt.2008.080419
41. Fortney JC, Pyne JM, Edlund MJ, et al. A randomized trial of telemedicine-based collaborative care for depression. J Gen Intern Med. 2007;22(8):1086–1093. PubMed doi:10.1007/s11606-007-0201-9