Optimizing Implementation Coaching to Support Successful EBP Delivery

Optimizing Implementation Coaching to Improve Treatment Quality and Client Outcomes in Community Mental Health Centers

This project engages community mental health center (CMHC) clinical supervisors in the development and examination of an optimized coaching strategy for psychotherapists utilizing Cognitive Behavioral Therapy (CBT) in Washington State. The optimized coaching strategy has the potential to enhance the adoption and fidelity of evidence-based practice (EBP).

Study Overview

• Status: Completed
• Conditions: Anxiety Depression; Mental Health Issue
• Intervention / Treatment: Behavioral: Supervisor-led Implementation Coaching

Detailed Description

The Washington State EBP Initiative provides clinical training and consultation for Cognitive Behavioral Therapy (CBT) and some basic organizational support; however, consistent with the literature, community mental health centers (CMHCs) often need more to successfully implement CBT. To accelerate treatment quality and clinical outcomes for youth, CMHCs need practical strategies for "successfully implementing, sustaining, and improving" EBPs that can be tailored to fit their specific context. These strategies will have the greatest impact if designed and delivered by CMHC practice community members who have CMHC-specific EBP implementation experience and expertise.

Implementation facilitation, a multifaceted, dynamic strategy of interactive problem-solving and support, improves adoption and fidelity of interventions. Implementation facilitation may be a promising strategy for CMHCs, but the resources and demands of traditional facilitation may present challenges for CMHCs. Motivated by global mental health frugal innovations work, the Principal Investigator piloted a modified facilitation approach for CMHCs, Implementation Coaching (hereafter "Coaching"), delivered to CMHCs participating in the Initiative. Coaching involved 4, group-based, virtual meetings with CMHC supervisors, who are frontline leaders in CMHCs. The investigator led coaching, supporting supervisors in developing workplans for CBT implementation in their CMHCs across 3 implementation phases. Coaching recommends a limited set of implementation strategies in each phase (e.g., Implementation phase: local technical assistance) that address known, prioritized determinants. Supervisors are supported in tailoring strategies for their CMHCs. In the pilot, CMHCs that received Coaching had clinicians that engaged in CBT delivery earlier and with more youth. Qualitative interviews suggested that supervisors were engaged and found Coaching acceptable and feasible. Based on the pilots' success, Project 2 will engage CMHC supervisors in: a) co-designing an optimized version of Coaching, and b) leading Coaching for other supervisors to optimize CBT implementation in their CMHCs.

In this exploratory research project, the Investigators' goal is to extend a pilot study that included the development and implementation of Implementation Coaching (hereafter "Coaching") within the Washington State EBP Initiative. In the pilot, the Coaching involved 4, group-based, virtual meetings with CMHC supervisors, who are frontline leaders in CMHCs. The results of the pilot indicated that Coaching was acceptable and feasible. In the current study, in collaboration with CMHC supervisors, investigators aim to optimize and test the effectiveness of Coaching for improving CBT fidelity and youth outcomes for four common conditions in CMHCs. The hypothesized mechanism through which Coaching operates is implementation climate or the degree to which an organization expects, supports, and rewards an innovation. Investigators expect that Coaching can be most effective in supporting CMHCs' high-quality EBP implementation if optimized with the practice community (supervisors) for the goals of limited resource use, fit with CMHC workflow, and wider reach. Coaching reach can be improved by moving beyond delivery by University-based personnel. In the Initiative, peer CMHC supervisors with CBT experience and expertise already co-lead CBT clinical training.

The study includes the following aims: Aim 1a) Collaborating with CMHCs, investigators will identify and convene purposively sampled groups of supervisors for diverse perspectives and demographics (e.g., ethnically diverse supervisors/clientele; urban/rural) during 2 one-day, in-person Ideation workshops; Aim 1b) Up to 150 clinicians will be recruited through the Initiative listserv and CBT+ training (for clinicians who recently completed their training) to participate in an online study in which clinicians will be randomized to read up to 6 vignettes that describe different strategies developed in the ideation phase and complete a survey with relevant subscale/s of implementation climate (EBP use is expected, supported and rewarded); Aim 2) Clinicians will be randomly assigned to receive standard implementation support or standard implementation support plus Coaching by trained supervisors.

• Study Type: Interventional
• Enrollment (Actual): 213
• Phase: Not Applicable

Contacts and Locations

Study Locations

• United States
  • Washington
    • Seattle, Washington, United States, 981095
      • WA EBP Initiative

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: 18 years and older (Adult, Older Adult)
• Accepts Healthy Volunteers: No

Description

Inclusion Criteria:

• Supervisors in aim 1a and clinicians/supervisors in aim 2 must

  • be employed in CMHCs whose leadership has agreed to participate in the study
  • have participated or be participating in the Initiative.
  • Clinicians in aim 1b do not need to be employed in a CMHC whose leadership has agreed to participate in the study.

Exclusion Criteria:

• There is no exclusion criteria for supervisors in Aim 1a or clinicians in 1b.

• Supervisors who participate in Aim 1 will be excluded from participating in Aim 2 as recipients of Implementation Coaching.
• Supervisor coaches will be recruited from the supervisors that participated in Aim 2. There is no exclusion criteria for clinicians in Aim 2.

Study Plan

How is the study designed?

Design Details

• Primary Purpose: Health Services Research
• Allocation: Randomized
• Interventional Model: Parallel Assignment
• Masking: None (Open Label)

Number of Arms

2

Arms and Interventions

Participant Group / Arm	Intervention / Treatment
Active Comparator: Standard/As Usual CBT+/EBP Initiative Implementation; All CMHCs involved in the CBT+/EBP Initiative receive virtual clinician training, CBT-phone consultation options over 6-months, access to a supervisor monthly call, listservs (one supervisor-focused), and a yearly advanced clinical training and supervisor training. The Initiative is a program that has been ongoing since 2009.	Behavioral: Supervisor-led Implementation Coaching; Implementation coaching, support, or facilitation includes the support provided to supervisors and their organizations for successfully delivering an evidence-based practice (EBP). Implementation support can include consultation, supervision, and coaching.; Other Names: Facilitation; Implementation Support
Experimental: Supervisor-led Implementation Coaching; The structure of this arm will depend on Aim 1. However, the investigators imagine that peer supervisors leading implementation coaching, trained in Aim 1, will have 3-5 virtual meetings with supervisors and email communication. Meetings focus on orientation to the theory and rationale for Coaching and supervisors' role as frontline leaders to support EBP implementation. Meetings are likely over 6 months and focus on different implementation phases. Coaches will support supervisors in developing tailored workplans for implementation in their CMHCs. In each meeting, the coach will review the strategies in the workplan, provide examples of how the strategy could be specified, and then facilitate discussion among supervisors. The coach will support supervisors in building a tailored workplan for their CMHC. Supervisors in the Coaching condition receive Coaching plus standard Initiative Implementation support.	Behavioral: Supervisor-led Implementation Coaching; Implementation coaching, support, or facilitation includes the support provided to supervisors and their organizations for successfully delivering an evidence-based practice (EBP). Implementation support can include consultation, supervision, and coaching.; Other Names: Facilitation; Implementation Support

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
CBT Fidelity	Clinicians participating in the Initiative, and who participate in this study, routinely enter session and de-identified CBT client outcome data. The investigative team will code cases enrolled study clinicians enter into the de-identified Dashboard. Based on the CBT model the clinician specifies, investigators will code cases for delivery of "essential elements" of that CBT model for each of the mental health conditions of focus: Anxiety, Depression, Posttraumatic Stress, and Behavioral Problems. Essential element coding will produce a count and a "fidelity" ratio of sessions including essential CBT elements (vs. only other elements) for each client, specific to the CBT model they received. Higher scores represent higher CBT fidelity reflecting greater essential element delivery (content) and coverage (dose).	through study completion, an average of 7 months

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Externalizing Behavior Symptoms	Also to be entered in the de-identified online dashboard, the externalizing subscale of the Pediatric Symptom Checklist (PSC-17) will be used to assess externalizing symptoms for youth receiving CBT for behavior problems. The full PSC-17 consists of 17 items rated using a 3-point Likert scale, with three subscales: externalizing (7-items; items used for this outcome), internalizing (5-items), and attention problems (5-items). It was developed as a caregiver-report measure for children ages 4 to 17; a youth self-report version is also available for youth 11 and up (the Y-PSC-17).	through study completion, an average of 7 months
Depressive Symptoms	Also to be entered in the de-identified online dashboard, the Short Mood and Feelings Questionnaire (SMFQ) will be used to assess depression symptoms for de-identified clients receiving CBT for Depression. The SMFQ is a 13-item self-report questionnaire designed to measure core depressive symptomology in children and adolescents aged 6-17 years old. It assesses the presence of affective and cognitive symptoms of depression that have been experienced in the past 2 weeks. Items are rated on a 3-point Likert scale (not true = 0; sometimes true = 1; not true = 2).	through study completion, an average of 7 months
Anxiety Symptoms	Also to be entered into the de-identified online dashboard system, the Screen for Child Anxiety Related Emotional Disorders (SCARED) will be used to assess anxiety symptoms for de-identified clients receiving CBT for Anxiety. The SCARED is a 5-item, brief child-report measure that screens for common anxiety symptoms across anxiety disorders, for youth ages 8 to 17. Statements are rated on a 3-point Likert scale (0 = not true or hardly ever true, 2 = true or often true). scores 3 or higher suggested a positive screen for anxiety symptoms. The Initiative uses the brief version because it is more pragmatic.	through study completion, an average of 7 months
Posttraumatic Stress Symptoms	Also to be entered in in the de-identified online dashboard, the Child and Adolescent Trauma Screen (CATS) will be used to assess post-traumatic stress symptoms for de-identified youth receiving CBT for Posttraumatic Stress (Trauma-focused CBT). This self-report measure assesses the frequency of PTSD symptoms and was also designed to assess PTSD diagnosis. The CATS has 15 items measuring traumatic events, 20 items measuring DSM-5 PTSD symptoms, and 5 items measuring psychosocial functioning.	through study completion, an average of 7 months

Collaborators and Investigators

• Sponsor: University of Washington
• Collaborators: National Institute of Mental Health (NIMH)
• Investigators: Principal Investigator: Shannon Dorsey, PhD, University of Washington

Publications and helpful links

Helpful Links

• Initiative's online dashboard system

Study record dates

Study Major Dates

• Study Start (Actual): July 22, 2022
• Primary Completion (Actual): October 29, 2024
• Study Completion (Actual): October 29, 2024

Study Registration Dates

• First Submitted: January 12, 2023
• First Submitted That Met QC Criteria: March 22, 2023
• First Posted (Actual): April 5, 2023

Study Record Updates

• Last Update Posted (Actual): March 25, 2025
• Last Update Submitted That Met QC Criteria: February 14, 2025
• Last Verified: December 1, 2024

More Information

Terms related to this study

• Additional Relevant MeSH Terms: Behavioral Symptoms; Depression
• Other Study ID Numbers: STUDY00014969; 1P50MH126219 (U.S. NIH Grant/Contract)

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: NO

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No
• product manufactured in and exported from the U.S.: No