Multidisciplinary Cognitive-Behavioral Therapy Training for the Veterans Affairs Primary Care Setting

Multidisciplinary Cognitive-Behavioral Therapy Training for the Veterans Affairs Primary Care Setting

Jeffrey A. Cully, PhD; Andra L. Teten, PhD; Jared F. Benge, PhD; Kristen H. Sorocco, PhD; and Michael R. Kauth, PhD

Objective: Current research highlights the need to embed evidence-based psychotherapies such as cognitive-behavioral therapy (CBT) within primary care settings. Few studies have examined CBT training procedures, and no studies have examined the impact of CBT training in the primary care setting. The current study sought to describe and assess the feasibility and effectiveness of a focused CBT training program for a diverse sample of primary care mental health providers in the Department of Veterans Affairs (VA).

Method: A multidisciplinary group of 28 mental health clinicians from 6 VA medical centers and 15 community-based outpatient clinics received an intensive 1½-day CBT workshop, held in Houston, Texas, in May 2008, including didactic presentations, expert modeling, and small-group role plays. CBT experts also provided biweekly follow-up group telephone consultation calls for participants over 12 weeks to aid in development of CBT skills. Participant program evaluation surveys and self-reported CBT knowledge, ability, and utilization were measured preworkshop, postworkshop, and 3 months postworkshop. Analyses compared mean change scores at baseline to those at 3-month follow-up. Wilcoxon signed rank tests were completed, and Cohen d effect-size calculations were also computed.

Results: Statistical analyses found that participant self-reported CBT knowledge (P < .01, effect size [ES] = 0.49) was significantly improved postworkshop and maintained at 3-month follow-up. Self-reported abilities were also improved (P = .07, ES = 0.40). The potency of the training experience appeared to be enhanced by the multimodal nature of the program.

Conclusion: Although challenges exist, focused and intensive training in CBT appears feasible for multidisciplinary mental health practitioners in the primary care setting.

Prim Care Companion J Clin Psychiatry 2010;12(3):e1-e8

© Copyright 2010 Physicians Postgraduate Press, Inc.

Submitted: May 15, 2009; accepted September 21, 2009.

Published online: June 24, 2010 (doi:10.4088/PCC.09m00838blu).

Corresponding author: Jeffrey A. Cully, PhD, Houston Center for Quality of Care & Utilization Studies, MEDVAMC (152), 2002 Holcombe Blvd, Houston, TX 77030 (jcully@bcm.tmc.edu).

Recent empirical reviews suggest that psychological interventions are effective for depression in the primary care setting.1,2 Cognitive-behavioral therapy (CBT), a specific type of psychological intervention, is considered a well-established and well-supported treatment for depression3 and holds strong potential for use in the primary care setting as a focused, time-limited treatment. Wolf and Hopko2 provide a comprehensive summary of current research efforts in psychotherapy for depression in primary care, including specific reviews for cognitive therapies and CBTs. Several important conclusions were generated from this review, including the need for additional studies of CBT to further validate its feasibility and efficacy in the primary care setting, the need to examine the ability of front-line practitioners to conduct CBT, and an implicit need for the establishment of CBT training for primary care practitioners.2

Successful implementation of CBT in the primary care setting is contingent upon the provision of a comprehensive, yet feasible, CBT training program. Evidence suggests that quality of treatment (provider expertise) is related to success or failure of depression treatments.4 Unfortunately, we are aware of no published data that address provider skill level or training needed to administer CBT in the primary care setting.

Some of the most rigorous CBT training occurs within clinical and counseling psychology programs, in which students receive both didactic and experiential CBT instruction. Experiential instruction in CBT, using 1-year practicum placements focusing on CBT, has been shown to be effective at increasing knowledge and utilization of CBT techniques.5 Psychiatry residency training programs use similar approaches for teaching CBT. Sudak et al6 state that most advanced training studies emphasize at least 6 months (24-30 hours) of didactic lecture, plus at least 2 hours a week of clinical work with patients and 2 hours a week of supervision for 6 to 12 months.

While such in-depth training experiences are ideal, these procedures are not feasible for on-the-job training of primary care-based providers and may include a level of detail or complexity unnecessary for a non-mental health specialty clinic. Several studies have examined the use of brief CBT training methods within specialty mental health settings. These studies suggest that CBT training of 1 to 3 days’ duration can result in increased provider knowledge and skill acquisition.7-10 Unfortunately, little is known about these brief CBT training models over time or their utility for the range of provider disciplines found within the primary care setting.

Although brief training programs have been shown to result in increased CBT knowledge and skill in settings other than primary care, the impact of these changes on clinical practice is not clear. For example, after a brief CBT workshop, medical-record reviews did not suggest that trained therapists increased their use of CBT techniques.11 Kavanaugh12 found a similar trend, with actual implementation of CBT techniques following a workshop being relatively limited, although perceived knowledge increased. However, the combination of intensive knowledge and skills acquisition followed by supervised clinical casework has been shown to increase not only perceptions of knowledge but also demonstrable CBT ability,10 suggesting that ongoing supervision or consultation might improve the potency of training.

The Department of Veterans Affairs (VA) currently has an emerging national initiative to embed mental health care within the primary care setting.13 Although significant resources have been allocated by the VA to incorporate mental health into primary care, limited attention has been given to the training needed to apply evidence-based psychotherapies. Given the size of the VA mental health in primary care initiative, opportunities exist for establishing evidence-based psychotherapy training procedures and monitoring the impact of this training on provider abilities.

The current study was paired with the VA mental health in primary care initiative to develop, implement, and examine a formalized, multimodal CBT training approach for a heterogeneous mix of VA mental health practitioners in primary care. Specifically, the current study sought to (1) describe and evaluate the feasibility and acceptability of a standardized brief CBT training program and (2) examine preliminary results related to the impact of the training program on practitioner self-reported CBT knowledge, abilities, and utilization 3 months postworkshop.

METHOD

Participants

Training candidates were drawn from VA medical centers (VAMCs) and community-based outpatient clinics (CBOCs) by mental health clinical leaders within the South Central Veterans Integrated Service Network (VISN), which includes parts of Texas, Oklahoma, Louisiana, Arkansas, Mississippi, Alabama, Missouri, and Florida. Selected clinics included 7 VAMCs and 3 CBOCs participating in the VISN mental health and primary care collaborative-care initiative and 12 additional CBOCs providing primarily mental health services. Clinical leaders from these sites nominated 30 therapists to participate in training. Study personnel interviewed all nominees to assess their motivation to conduct CBT. Twenty-eight therapists were selected to attend the training, representing 6 VAMCs and 15 CBOCs. Most of the 28 therapists were women (75%, n = 21) and social workers (61%, n = 17). The remaining included 8 psychologists, 1 psychiatrist, 1 nurse, and 1 nurse practitioner. Twenty-three therapists consented to participate in a larger research study to test an implementation strategy called external facilitation described elsewhere (M.R.K., G. Sullivan, J.A.C., et al, unpublished data, 2009). This research participant group (n = 23) reported an average of 5.9 years at the VA and 9.4 years as a therapist. Sixty-one percent reported having had some form of training in CBT in the past, although the type and depth of training was not solicited.

CBT Training

The CBT training workshop was held in Houston, Texas, in May 2008. The training, including participant travel and lodging, was provided at no cost to participants through a grant provided by the South Central Mental Illness Research, Education, and Clinical Center. Training consisted of a 1½-day brief CBT workshop and 6 postworkshop biweekly group consultation calls. Cognitive-behavioral therapy consultation calls were led by 6 CBT experts who were present at the workshop. These calls focused on extending the content and practices of the initial training using real-world patient examples encountered by participants following the workshop. As part of a larger research study, willing participants were randomly allocated to receive no additional services or a facilitation intervention. For those allocated to the facilitation group, individuals received additional guidance from a facilitation expert via telephone calls and e-mail. The facilitator, who did not have CBT expertise, focused on ways to improve implementation of CBT procedures by addressing practice barriers such as time and space constraints, as well as setting individual provider practice goals. When issues related to the provision of CBT techniques arose during a facilitation call, the facilitator encouraged the provider to discuss that issue with his/her CBT consultant. Notably, training participation was voluntary, and attendance at the consultation and facilitation calls was expected but not required.

Brief CBT workshop. The workshop focused on the provision of time-limited or brief CBT and consisted of 1½ days of intensive and structured didactic instruction, expert modeling, and experiential exercises, using small groups led by a CBT expert. As part of the workshop, each participant was provided with a brief CBT training manual14 and standardized patient profiles for expert modeling and small-group role plays. Specific content of the workshop was organized into discrete "modules," as detailed in the training manual and described in Table 1. The use of modules allows the therapist to apply specific skills in a more flexible manner (than that described in more rigid session-by-session manuals) and has been found to be an important factor associated with improving provider attitudes toward evidence-based practices.15 For the current training, each module was introduced and presented to the entire training group via projector and slides. All modules except "Using Supervision/Consultation" and "Ending Treatment and Maintaining Changes" also included an expert modeling session, participant small-group role play facilitated by a CBT expert, and a larger-group discussion of the facilitated role plays.

Click figure to enlarge

During the first phase of the workshop, participants were grouped into "clinician networks" of 4 to 5 participants and moderated by a CBT expert. Clinician networks were designed to improve the training process, continuity of the workshop, and small-group exercises but also served to establish rapport and training exchanges between participants and a dedicated CBT expert for follow-up telephone consultation calls.

Three patient vignettes detailing general medical, mental, vocational, and social history, along with presenting complaints, were used throughout the workshop to standardize expert modeling and small-group role plays. Vignettes were created by the workshop leaders (J.A.C., A.L.T.) to be gender and ethnically diverse and to address common issues encountered in the primary care setting (eg, mixture of physical and emotional health concerns). The use of the same 3 case vignettes allowed CBT trainers to efficiently use workshop time (avoiding introduction of additional cases) and expand upon the vignettes during role-play exercises to focus participants on specific skill sets (eg, Socratic questioning).

Consultation telephone sessions. Following the workshop but as a continuation of training, all participants were encouraged to participate in biweekly, 1-hour telephone consultation calls. As noted by Sholomskas et al,10 the use of ongoing consultation was viewed as a vital component to enhance the training program and increase the ability of providers to effectively carry out CBT procedures postworkshop. Consultation calls occurred for each clinician network and were designed to allow participants to interact with CBT experts and other participants around "real-world" clinical encounters.

Training Assessment and Outcomes

Preworkshop. All participants completed a series of survey questions immediately preceding the workshop. Baseline questions focused on participant use of psychotherapy and CBT over the prior month, frequency of use of specific CBT techniques and principles, and separate ratings for CBT knowledge and ability related to overall and specific CBT techniques and principles. Overall psychotherapy and CBT use over the past month was rated for frequency during "your regular VA practice," with responses ranging from 1 (rarely) to 4 (almost always). The assessment of CBT techniques and principles focused exclusively on the instructional modules presented during the workshop (see Table 1). Frequency of CBT principles and techniques was rated on a 5-point scale (never, sometimes, often, almost always, always). Cognitive-behavioral therapy knowledge and abilities were also rated on a 5-point scale (poor, fair, good, very good, excellent). A final question asked participants about CBT learning interests for the workshop and sought to elicit open-ended responses and feedback. The assessment schedule is shown in Table 2.

Click figure to enlarge

Postworkshop. Immediately following the workshop, participants completed a follow-up survey. The postworkshop evaluation focused on participants’ self-reported CBT knowledge and ability (in comparison with the evaluation completed at baseline). Participants also completed a 12-item workshop evaluation. This survey used a 5-point Likert format ranging from poor to excellent for 6 items related to general and specific experiences with the workshop (eg, workshop content/material, workshop activities, trainers). Three questions asked about the general appropriateness of the workshop in terms of duration and level of complexity (introductory, intermediate, advanced), and 3 additional open-ended questions elicited participants’ likes and dislikes, along with suggested areas for workshop improvements.

Postconsultation (12 weeks postworkshop). Following the conclusion of the biweekly consultation calls, participants completed a survey with identical items as the baseline evaluation, including frequency of psychotherapy and CBT utilization data. Six additional questions were added for this follow-up survey, which asked participants to: (1) rate and compare their CBT knowledge and ability to their preworkshop levels and (2) using 3 open-ended questions, describe any changes in CBT practice. Participants also completed a short rating form related to their experiences in the consultation process.

Data Analyses

Analyses focused on describing participant reactions to the training as well as preliminary training outcomes related to participant self-reported CBT knowledge, perceived abilities, and utilization. Descriptive and qualitative data were used to understand participant reactions to specific training experiences. Analyses of outcomes compared mean change scores for participant preworkshop scores to those at 3-month follow-up (posttraining and consultation). Statistical comparisons of means were completed using nonparametric Wilcoxon signed rank tests. Cohen d effect-size (ES) calculations were also computed. Effect-size calculations represent the size (rather than the statistical significance) of the clinical effect, in which scores of 0.2 (odds ratio [OR] = 1.2) are considered small, 0.5 (OR = 1.6) are considered medium, and 0.8 (OR = 2.2) are considered large.

RESULTS

Workshop Evaluation and Feedback

Participant ratings for the overall workshop, materials, activities, visual aids and handouts, and trainers ranged from 4.18 to 4.68 (on a 5-point scale, in which 4 = very good and 5 = excellent). Practicality of the workshop "for your needs and interests" was rated slightly lower at 3.86 (3 = good and 4 = very good). On a 3-point scale (1 = introductory, 2 = intermediate, and 3 = advanced), participants’ mean ratings were 1.50. Using a 3-point scale for duration (1 = too short, 2 = right length, 3 = too long), participant mean ratings were 1.89.

Qualitative feedback indicated that participants particularly liked the incorporation of a CBT training manual and small-group discussions, as well as the interactive nature of the workshop. In terms of participant dislikes, there was a bimodal distribution for comments related to the length and complexity of the workshop, with some individuals indicating a need for more time to solidify basic CBT skills and others requesting advanced CBT training and/or shortening of the "introductory" elements. There was also variance related to the role-play procedures. Many participants found the role plays helpful, but several indicated that the role plays were too advanced. Trainee suggestions for improvement included dividing participants according to skill level and providing separate basic and advanced workshops as well as using videotaped examples for role plays and/or having video materials available prior to the workshop.

Post hoc analyses did not show any added benefit for facilitated participants in terms of specific CBT knowledge, skill, or ability at 3 months postworkshop when compared with those who did not receive facilitation.

Consultation Evaluation and Feedback

Participants reported attending a mean of 3.2 (median = 3.0) consultation sessions (of 6 possible). Ratings of the consultation experience were generally high (mean = 4.08 on a 5-point scale, in which 4 = very good and 5 = excellent), as were ratings of the CBT consultants (mean = 4.52). Practicality of the consultation process "for your needs and interests" was rated lower at 3.44 (3 = good, 4 = very good). On a 3-point scale (1 = introductory, 2 = intermediate, and 3 = advanced), participants’ mean ratings were 2.04, suggesting a higher level of complexity compared with the workshop.

Qualitative feedback from the postconsultation survey suggested that participants appreciated the ability to obtain various clinical perspectives and feedback in an informal and flexible setting. For example, one participant said, "I enjoyed having a place to hear the struggles that other therapists were experiencing and possible solutions to these situations. I felt supported and encouraged to keep going and trying." However, participants reported that it was difficult to regularly participate in calls because of competing demands and time constraints.

Participant Self-Reported CBT Utilization

Baseline self-reported utilization of psychotherapy was high (mean = 3.21 on a 4-point scale), as was self-reported utilization of CBT (mean = 2.89). Utilization of specific CBT procedures at baseline varied, with highest ratings for rapport (mean = 4.59), problem solving (mean = 4.03), and identifying and responding to maladaptive thoughts (mean = 3.57 for both) and lowest for muscle relaxation (mean = 2.75), ending treatment (mean = 2.85), orienting patients to CBT (mean = 2.86), setting goals (mean = 2.86), and assigning homework (mean = 2.93).

Nonparametric Wilcoxon signed rank tests were used to compare baseline and 3-month follow-up survey data. No differences were found for overall psychotherapy or CBT utilization. An examination of specific CBT procedures suggested consistently higher utilization ratings at 3-month follow-up for all concepts except imagery. Statistical tests using the Wilcoxon and Cohen d procedures found robust P value and effect sizes for positive utilization of procedures: orienting patients to CBT (P = .003, ES = 0.53), setting goals (P = .002, ES = 0.78), and behavioral activation (P = .01, ES = 0.68).

Participant Self-Reported CBT Knowledge and Abilities

Click figure to enlarge

Mean baseline general CBT knowledge was 3.54, and general CBT ability was 3.39 (5-point scale) (see Table 3). Ratings for knowledge and ability for specific CBT procedures varied but were generally high. Knowledge and ability were higher for procedures such as rapport, deep breathing, problem solving, and identifying and responding to maladaptive thoughts, while ratings were generally lower for procedures such as case conceptualization, functional assessments, setting agendas, and ending treatment.

Postworkshop survey results indicated notable changes, including improved knowledge and ability ratings across all general and specific CBT procedures and techniques; these changes were generally maintained at 3-month follow-up. Nonparametric Wilcoxon signed rank tests were used to compare knowledge and ability ratings between baseline and 3-month follow-up. Participants reported a statistically significant increase in CBT knowledge (P < .01, ES = 0.49). Significant improvements were also observed in perceived CBT ability (ES = 0.40), but this finding was not statistically significant, likely because of the small sample size (P = .07). Specific CBT procedures with statistically significant and robust effect sizes included self-reported improvements in knowledge and ability for rapport (ES = 0.57 and 0.43), case conceptualization (ES = 0.67 and 0.48), goal setting (knowledge only, ES = 0.61), agenda setting (knowledge only, ES = 0.51), assigning homework (ES = 0.56 and 0.69), behavioral activation (knowledge only, ES = 0.76), muscle relaxation (ability only, ES = 0.40), orienting to CBT (knowledge only, ES = 0.54), and problem solving (ES = 0.45 and 0.39). Results for the remaining procedures did not reach statistical significance, although several procedures had similar levels of effect (according to Cohen d estimates).

Qualitative feedback from participants regarding changes in CBT knowledge and abilities suggested that participants felt more confident in their ability to use CBT procedures. They also reported greater competency and use of specific CBT skills, including presenting CBT to patients, increasing goal focus, setting agendas, using Socratic questioning, and expanding CBT skills to a wider range of patients.

Trainer/Consultant Viewpoint

The trainers and consultants noted that participants actively participated in the workshop. The use of standardized case vignettes and the creation of clinician networks served to streamline the training experience and differentiate and focus on specific CBT skill sets that helped to develop a rapport between participants and experts. Many attendees noted at the beginning of the workshop that they had previous CBT experience or were CBT therapists; however, these participants varied greatly on knowledge and experience with specific CBT techniques. This became especially clear during the role-play exercises. Trainers attempted to tailor role-play experiences to the level of the group and the specific needs of individual group members but observed wide variability in the needs of different attendees, even among those who reported experience with CBT. During the postworkshop consultation telephone calls, consultants noted the difficulty participants had attending calls on a regular basis. Consultants reported that, when they attended, participants generally provided valuable clinical information and group-based discussions around specific practice-based challenges in using CBT (eg, setting goals for a complex patient).

DISCUSSION

This study represents the first known published study to examine the effectiveness of CBT training for primary care-based mental health providers and outlines a novel, reproducible approach to teaching that incorporates traditional didactic presentations with expert modeling, expert-facilitated small-group discussions, and prolonged CBT consultation. The current demonstration project found that a focused CBT training program for mental health in primary care providers is feasible, was well-received by trainees, and generated preliminary findings that suggest positive improvements in the areas of provider self-reported CBT knowledge, abilities, and utilization of CBT skills. These positive provider changes were maintained over the 3-month training period.

Several training and methodological issues highlight the unique contributions of this work. First, the developed CBT training was standardized using a training manual14; didactic presentations; expert modeling; vignette-based, participant, small-group interactions moderated by CBT experts; and extended and ongoing CBT consultation-call availability for participants. Second, the CBT training not only was examined for its feasibility of implementation and acceptability to participants but also involved analytic comparisons for self-reported CBT use, knowledge, and abilities over a 3-month follow-up period. Outcome data from this study suggest statistically significant mean differences and clinically meaningful effect sizes for general CBT knowledge and general CBT ability at 3-month follow-up. Self-reported improvements and increases in utilization were noted across all CBT specific skills. Third, although data included only a modest analytic sample, participants included a heterogeneous mix of professionals from various geographic regions and professional disciplines.

Implications for Training and Practice

With the current press to improve depression care in the primary care setting, a clear and urgent need exists to identify and standardize effective training procedures, especially for complex skill sets such as those involved in the provision of evidence-based psychotherapies. Training for primary care-based mental health providers differs distinctly from the training of mental health-based providers in 2 important ways. First, mental health providers embedded within the primary care setting are likely to come from diverse professional backgrounds with varying exposure and competency providing psychotherapies such as CBT. Second, the primary care setting is a challenging, fast-paced environment with a large and diverse patient population. Mental health providers in the primary care setting are charged with providing evidence-based treatments in a brief, time-limited approach that is distinctly different from that of many specialty mental health clinics.

Our data suggest that a focused training in brief, time-limited CBT for primary care practitioners is both feasible and effective at increasing self-reported CBT knowledge, abilities, and skill use. Changes obtained immediately following the workshop were maintained at 3 months. It also appeared that, when used, consultation with CBT experts was highly valued by participants to consolidate and expand CBT skill development postworkshop. Participants generally valued the use of expert modeling and the group-based role plays. Anecdotally, we found that participants from rural or small clinics were particularly enthusiastic about the training program. These providers noted the limited availability of local CBT training programs and supervisors and the value of having a telephone-based method of interacting with a CBT expert over a sustained period of time. Similar procedures may therefore be highly useful for disseminating evidence-based training to rural providers, who are largely isolated from the training programs available within urban clinics and hospitals.

In terms of lessons learned, we want to highlight 2 areas for improvement of future CBT and/or psychotherapy training programs. First, although the program was positively received by almost all participants, complexity of the overall training program was the most frequently identified area of concern for participants and trainers. This issue entails a significant challenge for future primary care-based psychotherapy training initiatives in that some providers enter training with limited psychotherapeutic skills, while others have specific CBT-based experiences and/or training. The most logical response to this challenge is to customize the CBT training experience by organizing workshops into basic and advanced workshops. Other possibilities include a more active use of experienced CBT participants in the formal training program (eg, learners helping learners) or the utilization of video-based technologies to conduct preworkshop training for those without the necessary foundational skills.

Recent evidence from a large survey of psychotherapists in the United States and Canada found several influential factors associated with practice changes, including the role of mentors and respected colleagues, the comfort level of the therapist with the new technique, and the ability of the practice change to be easily integrated.16 Therefore, as a second area for improvement, future programs might leverage the use of expert consultation to support therapists’ comfort, suggest areas for successful CBT integration within existing practices, and, most importantly, fill the role of mentor and/or respected colleague. These interactions may improve the link between knowledge acquisition and changes in practice patterns (eg, CBT utilization).

In the current study, the consultation process was valued by participants and trainers; however, attendance during these CBT expert consultation calls was sporadic, decreasing the potency of this training technique. Consultation calls generally provided participants with the opportunity to enhance workshop training through discussion of "real-world" cases and difficulties applying CBT procedures in everyday practice settings. When poorly attended, group interaction was limited. Future training efforts would be wise to include this sustained period of learning but might benefit from any or all of the following: requiring participation in the consultation experience, taking additional steps to ensure member participation (eg, logistics related to scheduling, such as blocking clinics), providing continuing education credits, and/or using online/e-mail group-discussion formats.

Study Limitations

Important considerations when interpreting these data include the relatively small sample, our focus on VA practitioners (within the highly structured VA health care system), and our reliance on self-reported skill use. Additional non-VA training studies are needed, especially where practitioner skills are evaluated by self-report and expert ratings.

CONCLUSION

Training in brief, time-limited CBT is both feasible and effective at increasing primary care mental health providers’ self-reported knowledge, abilities, and use of CBT techniques. Continued efforts to formalize evidence-based CBT training for the primary care setting will likely result in greater provider satisfaction and comfort in providing CBT but will also likely improve the quality of care provided within the primary care setting.

Author affiliations: Michael E. DeBakey Veterans Affairs Medical Center, and Menninger Department of Psychiatry and Behavioral Sciences, Baylor College of Medicine, Houston, Texas (Drs Cully, Teten, Benge, and Kauth); Veterans Affairs South Central Mental Illness Research, Education, and Clinical Center, and Veterans Affairs Health Services Research and Development Center of Excellence, Houston, Texas (Drs Cully and Kauth); and Donald W. Reynolds Department of Geriatric Medicine, University of Oklahoma Health Science Center, Oklahoma City, and Oklahoma City Veterans Affairs Medical Center, Oklahoma (Dr Sorocco).

Potential conflicts of interest: The authors report no additional financial or other relationships relevant to the subject of this article.

Funding/support: Dr Cully is a VA Health Services Research and Development Career Development Awardee (CDA-2: 05-288). This work was supported in part by the South Central Mental Illness Research, Education, and Clinical Center and the Houston VA HSR&D Center of Excellence (HFP90-020).

Disclaimer: The views expressed herein are those of the authors and do not necessarily reflect those of the Department of Veterans Affairs or Baylor College of Medicine.

REFERENCES

1. Bortolotti B, Menchetti M, Bellini F, et al. Psychological interventions for major depression in primary care: a meta-analytic review of randomized controlled trials. Gen Hosp Psychiatry. 2008;30(4):293-302. PubMed doi:10.1016/j.genhosppsych.2008.04.001

2. Wolf NJ, Hopko DR. Psychosocial and pharmacological interventions for depressed adults in primary care: a critical review. Clin Psychol Rev. 2008;28(1):131-161. PubMed doi:10.1016/j.cpr.2007.04.004

3. DeRubeis RJ, Crits-Christoph P. Empirically supported individual and group psychological treatments for adult mental disorders. J Consult Clin Psychol. 1998;66(1):37-52. PubMed doi:10.1037/0022-006X.66.1.37

4. Hegel MT, Barrett JE, Cornell JE, et al. Predictors of response to problem-solving treatment of depression in primary care. Behav Ther. 2002;33(4):511-527. doi:10.1016/S0005-7894(02)80014-4

5. Freiheit SR, Overholser JC. Training issues in cognitive-behavioral psychotherapy. J Behav Ther Exp Psychiatry. 1997;28(2):79-86. PubMed doi:10.1016/S0005-7916(97)00001-3

6. Sudak DM, Beck JS, Wright J. Cognitive behavioral therapy: a blueprint for attaining and assessing psychiatry resident competency. Acad Psychiatry. 2003;27(3):154-159. PubMed doi:10.1176/appi.ap.27.3.154

7. Crits-Christoph P, Siqueland L, Chittams J, et al. Training in cognitive, supportive-expressive, and drug counseling therapies for cocaine dependence. J Consult Clin Psychol. 1998;66(3):484-492. PubMed doi:10.1037/0022-006X.66.3.484

8. Miller WR, Mount KA. A small study of training in motivational interviewing: does one workshop change clinician and client behavior? Behav Cogn Psychother. 2001;29(4):457-471. doi:10.1017/S1352465801004064

9. Miller WR, Yahne CE, Moyers TB, et al. A randomized trial of methods to help clinicians learn motivational interviewing. J Consult Clin Psychol. 2004;72(6):1050-1062. PubMed doi:10.1037/0022-006X.72.6.1050

10. Sholomskas D, Syracuse G, Rounsaville BJ, et al. We don’ t train in vain: a dissemination trial of three strategies of training clinicians in cognitive-behavioral therapy. J Consult Clin Psychol. 2005;73:106-115. PubMed doi:10.1037/0022-006X.73.1.106

11. Jensen-Doss A, Cusack KJ, de Arellano MA. Workshop-based training in trauma-focused CBT: an in-depth analysis of impact on provider practices. Community Ment Health J. 2008;44(4):227-244. PubMed doi:10.1007/s10597-007-9121-8

12. Kavanagh DJ. Issues in multidisciplinary training of cognitive-behavioural interventions. Behav Change. 1994;11:38-44.

13. Zeiss AM, Karlin BE. Integrating mental health and primary care services in the Department of Veterans Affairs health care system. J Clin Psychol Med Settings. 2008;15(1):73-78. PubMed doi:10.1007/s10880-008-9100-4

14. Cully JA, Teten AL. A Therapist’s Guide to Brief Cognitive Behavioral Therapy. Houston, TX: South Central Mental Illness Research, Education, and Clinical Center; 2008.

15. Borntrager CF, Chorpita BF, Higa-McMillan C, et al. Provider attitudes toward evidence-based practices: are the concerns with the evidence or with the manuals? Psychiatr Serv. 2009;60(5):677-681. PubMed doi:10.1176/appi.ps.60.5.677

16. Cook JM, Schnurr PP, Biyanova T, et al. Apples don’ t fall far from the tree: influences on psychotherapists’ adoption and sustained use of new therapies. Psychiatr Serv. 2009;60(5):671-676. PubMed doi:10.1176/appi.ps.60.5.671