Novel Methods for Implementing Measurement-Based Care for Youth in Low-resource Environments (NIMBLE)

IMPACT Center Signature Project 1, "Novel Methods for Implementing Measurement-Based Care for Youth in Low-resource Environments", is Embedded Within IMPACT Center P50 Entitled: Optimizing Evidence-Based Practice Implementation for Clinical Impact: the IMPACT Center.

This study will visit 6 community mental health clinics in Washington state and work with clinic staff to find out if staff are regularly evaluating clinical outcomes using questionnaires (which is also called measurement-based care). The study team will partner with clinic staff to find out what makes using questionnaires difficult and then plan to improve the use of questionnaires. The study activities will include trainings, interviews, collaborative discussions, and interactive activities.

Study Overview

• Status: Recruiting
• Conditions: Mental Health
• Intervention / Treatment: Behavioral: IMPACT Methods to Identify and Prioritize Determinants (Challenge I), Challenge and Match strategies to Prioritized Determinants (Challenge II)

Detailed Description

Rapid Ethnographic Assessment (REA). REA will be conducted over 2-day site visits at each clinic, following a kickoff training on the principles of MBC. This team will conduct ethnographic observations on both days, using primarily unobtrusive techniques on Day 1, documenting observations of activities (e.g., clinician documentation), interactions (e.g., client check in), and events (e.g., staff meeting) using written and audio-recorded field notes. The investigators will interview staff for responses to the "example" question (for instance, "Could you show me how you use MBC in your work and describe what you are doing?") described above. The investigators expect up to 30 interviews per clinic: 30-45 minutes for therapists, 15 minutes for staff. The investigators will synthesize learnings from observations, interviews, and surveys in a debriefing session, using notes to generate a list of unique barriers. Day 2 will be focused observations of therapists shadowed by team members to document, track, and note the duration and location of activities, guided by barriers identified in the evidence review and Day 1 learnings. The investigators will conduct semi-structured, focused interviews based on Day 1 learnings, with therapists (therapists, supervisors, psychiatrists) and staff (up to 30). Day 2 interviews will be longer (15-30 minutes), audio recorded.

Design Kits. To complement REA and collect real-time data from therapists and youth, at the end of site visits, the investigators will leave design kit materials including disposable cameras, journals, pens and prompts to use to describe using MBC by taking pictures, make diary entries, draw maps, and record images. The investigators will ask therapists (5-8 per clinic) to participate. The investigators will strive to recruit at least one youth from each participating therapist to also complete design kits.

The investigators will recruit 5-8 youth clients to also complete design kits. This is the only youth-related activity in the study. Participants will have one week to document experiences before returning the toolkits to the investigators in self-addressed, postage-paid stamped envelopes. Within a week of receiving the data, team members will conduct follow-up 15-minute phone interviews, inviting participants to interpret and comment on their responses. The investigators will create memos documenting barriers and describing their salience, meaning, and importance to the participant.

Barrier Prioritization. The investigators will engage 5-8 therapists and/or staff in the prioritization activity for a 2-hour facilitated group activity. The activity will occur in-person (Site Visit 2) but could be delivered virtually if needed. Barriers identified through the rapid evidence review, rapid ethnographic assessment, and design kits (50+ expected) will be prioritized using a participant-engaged method.

Causal Pathway Diagramming. For each clinic, the investigators will develop Causal Pathway Diagrams CPDs for the top 3 prioritized barriers. The goal is to assess how well each implementation strategy is matched to a prioritized barrier, based on its mechanism of action, and to clarify the causal chain of events that must take place to achieve MBC fidelity.

Focus Groups to Improve IMPACT methods. The investigators will engage partners in a semi-structured focus group to capture their experience engaging in all IMPACT Center methods in the 3 site visits to inform further toolkit refinement. A facilitator not previously engaged in any of the activities will conduct a 60-minute focus group to solicit reactions to all aspects of IMPACT's methods, including the steps and personnel requirements of each method, and their output (e.g., prioritized barriers; utility of the information captured in the CPDs).

AIM 2: Compare MBC fidelity post IMPACT Center methods deployment versus historical controls.

To examine MBC fidelity (or frequency of outcome measure collection) in Aim 2, the investigators use routine session process data entered by therapists to ascertain whether the use (frequency) of outcome questionnaire increased after their participation in IMPACT methods. Dr. Shannon Dorsey is a co-director of the Washington State CBT+ Initiative and the MPI of the IMPACT center. The dashboard with de-identified cases from which the investigators are collecting data from is maintained as part of the CBT+ Initiative, and the CBT+ Initiative is the primary "laboratory" for IMPACT center methods. When data from CBT+ aligns with IMPACT aims, data can be curated by their team and sent to IMPACT NIMBLE co-investigator Ruben Martinez, via secure file transfer per CBT+ operational guidelines and procedures. Data will be shared back with clinics and the CBT+ initiative only in aggregate form for the purposes of quality/continuous improvement. CBT+ has routinely used data from the dashboard for quality improvement over the past 10 years.

Historical data has already been collected and is available to the investigators as part of their involvement in the CBT+ initiative and ongoing practice improvement work. As part of their clinic training, each therapist in these clinics participates in a CBT+ training. For the training, participants track MBC data for up to 5 cases. To log the data for these 5 cases, therapists log into an account set up for them by an administrator and enter the data under a pseudonym. Participants can choose any code (e.g., "Marty McFly") to identify their patient so long is it does not contain the name of their patient or any other identifying information. Thus, all client-level data are de-identified before participants are entered into the system. While administrators have access to website data that contains therapist names and other information, that information is not and will not be accessible to these investigators.

For post-IMPACT comparison data, therapists providing data will fall into two categories. The first category are therapists who have not gone through the CBT+ training yet. These therapists will follow the same procedures as those who entered historical data. The second category are therapists who have already completed their CBT+ training. The investigators will ask the second category therapists to track data for 5-8 new cases. Participant will go through the same procedure as the other therapists with one exception. Rather than creating codes to identify patients themselves, the investigators will have each therapist select a paper list of up to 8 identifying codes (e.g., AA1, AA2…AA8) to use to identify their patients. The investigators will not keep any document that links any specific codes to any therapist. In this way, the investigators will be able to isolate data identified by those codes and identify that the data came from a therapist within a specific clinic, but the investigators will not be able to identify therapists individually.

For statistical analysis, the investigators will use 3-level generalized mixed effects models (fidelity score nested within therapist nested within clinic), with only fixed effects at the clinic level as this is a nuisance cluster. Each predictor will be modeled separately, with appropriate link functions for distributional form and dichotomous or count variables. Significance of model fit and individual coefficients will be determined via deviation tests (likelihood ratio, Akaike Information Criterion, & Bayes Information Criterion). For statistical power: assuming the investigators will have data from 5 intervention therapists from each of 6 clinics with 5 clients each in the intervention year (a total of 30 therapists and 150 clients); 3 comparison therapists in each clinic in each of two prior years of the study, each with 3 clients (a total of 108 clients), the investigators have power of .80 to detect an effect size of d = .36 - .48.

• Study Type: Interventional
• Enrollment (Estimated): 228
• Phase: Not Applicable

Contacts and Locations

• Study Contact: Name: Carolyn E Bain, MPH; Phone Number: 206-287-4611; Email: Carolyn.E.Bain@kp.org
• Study Contact Backup: Name: Rene Hawkes, BS; Phone Number: 206-287-2073; Email: rene.j.hawkes@kp.org

Study Locations

• United States
  • Washington
    • Seattle, Washington, United States, 98101
      • Recruiting
      • KPWHRI
      • Contact: Carolyn E Bain, MPH; Phone Number: 206-287-4611; Email: Carolyn.E.Bain@kp.org
      • Principal Investigator: Ruben G Martinez, PhD

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: 13 years and older (Child, Adult, Older Adult)
• Accepts Healthy Volunteers: Yes

Study Population

Therapists and staff members at community mental health clinics in Washington state.

Description

Inclusion Criteria:

• Aim 1: Co-create plans for optimizing Measurement Based Care (MBC) implementation for youth with comorbidities. Clinic staff/Therapists (N=180) across 6 CMHCs in Washington state.
• Aged 21+
• Working at a CMHC that is participating in the WA State CBT (Cognitive Behavioral Therapy) + Initiative.
• Youth (N=48)
• Aged 13-17 years
• Receiving treatment at CMHC participating in the WA State CBT (Cognitive Behavioral Therapy) + Initiative.
• Aim 2: Compare MBC fidelity post IMPACT Center methods deployment versus historical controls. Clinic staff/Therapists (N=30) across 6 CMHCs in Washington.
• Must be aged 21+
• Working at a CMHC that is participating in the WA State CBT (Cognitive Behavioral Therapy) + Initiative.

Exclusion Criteria:

• Aim 1: Clinic staff/Therapists: Unwilling to participate in study activities.
• Youth:
• Suicidal or psychotic
• Not cognitively able to safely participate in MBC
• Not able to read and write in English.
• Aim 2: Clinic staff/Therapists:
• Unwilling to participate in study activities.

Study Plan

How is the study designed?

Design Details

• Primary Purpose: Health Services Research
• Allocation: N/A
• Interventional Model: Single Group Assignment
• Masking: None (Open Label)

Number of Arms

1

Arms and Interventions

Participant Group / Arm

Intervention / Treatment

Other: Measurement Based Care Implementation; Six community mental health clinics located in Washington state will receive trainings on how to implement Measurement Based Care (MBC) in their clinics. We are creating implementation plans to improve therapists' use of measurement-based care.

Behavioral: IMPACT Methods to Identify and Prioritize Determinants (Challenge I), Challenge and Match strategies to Prioritized Determinants (Challenge II); IMPACT Center developed two methods for identifying determinants that involved providers, staff, and youth at community mental health centers in site visit 1: Rapid Ethnographic Assessment, Design Probes. Then, practice partners engage in a facilitated group activity to rate the criticality, chronicity, and ubiquity of the list of determinants in Site Visit 2. Finally, practice partners engage in a third site visit where they create Causal Pathway Diagrams that allow them to match strategies to prioritize determinants and create Implementation Plans.

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Change in measurement-based care frequency	Therapists will report on the number of clinical outcome questionnaires administered to 5 pilot clients (the total number of outcome questionnaires per client used) before and after engaging in our researcher-driven implementation process.	We will assess at 6, 12 and 18 months after enrollment.

Collaborators and Investigators

• Sponsor: Kaiser Permanente
• Collaborators: National Institute of Mental Health (NIMH)
• Investigators: Study Director: Ruben G Martinez, PhD, Kaiser Permanente

Study record dates

Study Major Dates

• Study Start (Actual): October 25, 2022
• Primary Completion (Estimated): April 30, 2024
• Study Completion (Estimated): August 31, 2025

Study Registration Dates

• First Submitted: November 18, 2022
• First Submitted That Met QC Criteria: November 30, 2022
• First Posted (Actual): December 9, 2022

Study Record Updates

• Last Update Posted (Actual): June 27, 2023
• Last Update Submitted That Met QC Criteria: June 23, 2023
• Last Verified: June 1, 2023

More Information

Terms related to this study

• Keywords: Implementation Science; Youth Mental Health; Community Mental Health Centers; Improving Measurement-based Care
• Other Study ID Numbers: 1943932-2; P50MH126219 (U.S. NIH Grant/Contract)

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: NO

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No