Project FACTS (Fidelity Accuracy: Comparing Three Strategies)

April 20, 2023 updated by: Rinad Beidas, Northwestern University

Increasing Accuracy and Efficiency of Fidelity Measurement in CBT

The objective is to compare the accuracy, costs, and cost-effectiveness of three fidelity measurement methods to assess fidelity to cognitive-behavioral therapy for youth. The investigators will randomize 135 therapists, implementing cognitive-behavioral therapy, to 3 conditions: self-report, chart stimulated recall, and behavioral rehearsal (N = 45 for each group). To calculate the outcomes of interest, each condition will be compared to the gold-standard fidelity measurement method, direct observation.

Study Overview

Status

Completed

Conditions

Intervention / Treatment

Detailed Description

To achieve the primary objective, the investigators plan to (1) identify the most accurate fidelity measurement method; (2) estimate the economic costs and cost-effectiveness of the fidelity measurement methods; and (3) compare stakeholders' motivation to use each method, as well as identify their perceived barriers and facilitators to use of each method. This study will have a significant positive impact in two ways. First, it will validate fidelity measurement methods that can be used for research. Second, it will produce tools that can be used by community mental health clinics to monitor therapist fidelity, an indicator of therapy quality.

Study Type

Interventional

Enrollment (Actual)

126

Phase

  • Not Applicable

Participation Criteria

Researchers look for people who fit a certain description, called eligibility criteria. Some examples of these criteria are a person's general health condition or prior treatments.

Eligibility Criteria

Ages Eligible for Study

3 years and older (Child, Adult, Older Adult)

Accepts Healthy Volunteers

Yes

Description

Inclusion Criteria: Therapists

  • they provide mental health treatment services in community mental health agencies in the City of Philadelphia participating in this study and

    • have been trained in cognitive-behavioral therapy (CBT) through the City-sponsored CBT evidence-based practice initiatives, and/or
    • respond to a brief screening survey that they use CBT; and/or
    • are nominated by a supervisor as a therapist who uses CBT.

Exclusion Criteria: Therapists

  • n/a

Inclusion Criteria: Supervisors

  • they are supervisors or administrators (i.e., they hold leadership positions) in community mental health agencies in the City of Philadelphia participating in this study.

Exclusion Criteria: Supervisors

  • n/a

Inclusion Criteria: Youth and their Legal Guardians

  • Youth and their legal guardians will be eligible to participate in this study if the youth

    • is 3-24 years old and
    • has completed at least one session with a therapist who has enrolled to participate in this study.

Exclusion Criteria: Youth and their Legal Guardians

  • if the child does not have a legal guardian who is able to consent (e.g., the Department of Human Services (DHS) is the guardian).

Study Plan

This section provides details of the study plan, including how the study is designed and what the study is measuring.

How is the study designed?

Design Details

  • Primary Purpose: Health Services Research
  • Allocation: Randomized
  • Interventional Model: Parallel Assignment
  • Masking: None (Open Label)

Arms and Interventions

Participant Group / Arm
Intervention / Treatment
Experimental: Self-report
Therapists randomized to this condition will complete a brief self-report measure, the Therapy Process Observational Coding System for Child Psychotherapy Strategies Scale-Self Report version (TPOCS-SR) for each of the recorded clinical encounters with enrolled youth. The TPOCS-SR will be a self-report version of the Therapy Process Observational Coding System for Child Psychotherapy-Strategies Scale (TPOCS-S) and will be created in collaboration with the instrument developer (McLeod). In this condition, the investigators will (a) provide an operational definition for each item on the TPOCS-SR (e.g., cognitive education: teaches client the cognitive model (e.g., thoughts influence behavior)/identifies how the cognitive model applies to a specific aspects of the client's life), and (b) provide therapists with a 30-minute training session that includes sample vignettes of particular behaviors and information about how those vignettes should be rated.
Behavioral: Self-report
Experimental: Chart Stimulated Recall
Therapists randomized to this condition will be asked to bring the charts of three enrolled youth to the chart-stimulated recall interview. A trained interviewer will ask the therapists how well they recall the encounter (rating of memory quality) followed by an open-ended question ("Talk me through your last session with your client. Tell me what you did."). While the therapists are speaking, the interviewer will note any elements that represent a prescribed CBT strategy. The interviewer will go through a list of cognitive-behavioral strategies based upon the TPOCS-S and probe to determine if the therapists completed any of the strategies. Follow-up questions will be used to explore to what degree an element was used and how skillfully and responsively the strategies were used.
Behavioral: Chart-Stimulated Recall
Experimental: Behavioral Rehearsal
Therapists randomized to this condition will be asked to engage in role-plays demonstrating the CBT strategies used with the three enrolled youth. The investigators will provide therapists with a list of the TPOCS-S CBT strategies and ask them to identify the CBT strategies used in their recorded encounter. The investigators will randomly select one of the strategies they report for each role-play. The investigators will then tell them, "Please role-play how you used this strategy in session with your client, with the trained actor in front of you." Later, an independent rater will rate therapists' adherence and skill based on established scoring criteria.
Behavioral: Behavioral Rehearsal

What is the study measuring?

Primary Outcome Measures

Outcome Measure
Measure Description
Time Frame
Total Fidelity Scores for Each Study Condition Measured Via Direct Observation (TPOCS-RS) and Study Arm Fidelity Method Scores
Time Frame: through study completion, an average of 1 month

Client sessions for all 3 conditions were scored for CBT adherence via direct observation using the Therapy Process Observational Coding System-Revised Strategies (TPOCS-RS) Scale (McLeod et al., 2015) on a 7-point Likert Scale (1='not at all,' 7='extensively'). We analyzed the TPOCS-RS' Maximum CBT score (highest coded intervention technique across all 12 possible interventions in a given session).

We also measured CBT adherence during client sessions using the strategies from each study condition (described in 'Arms/Groups' below). All 3 conditions used parallel 7-point scales to the TPOCS-RS.

Higher scores for the 3 study condition strategies indicate greater reported use, and higher scores on the TPOCS-RS indicate greater observed use of CBT interventions. Our goal was to compare the study condition methods of assessing CBT adherence to direct observation (TPOCS-RS), the gold standard.
through study completion, an average of 1 month

Collaborators and Investigators

This is where you will find people and organizations involved with this study.

Sponsor

Northwestern University

Collaborators

University of Pennsylvania

Investigators

  • Principal Investigator: Rinad S Beidas, PhD, Northwestern University
  • Principal Investigator: Emily M Becker-Haimes, PhD, University of Pennsylvania

Publications and helpful links

The person responsible for entering information about the study voluntarily provides these publications. These may be about anything related to the study.

General Publications

Study record dates

These dates track the progress of study record and summary results submissions to ClinicalTrials.gov. Study records and reported results are reviewed by the National Library of Medicine (NLM) to make sure they meet specific quality control standards before being posted on the public website.

Study Major Dates

Study Start

November 1, 2016

Primary Completion (Actual)

June 30, 2020

Study Completion (Actual)

June 30, 2020

Study Registration Dates

First Submitted

June 3, 2016

First Submitted That Met QC Criteria

June 30, 2016

First Posted (Estimate)

July 1, 2016

Study Record Updates

Last Update Posted (Actual)

April 21, 2023

Last Update Submitted That Met QC Criteria

April 20, 2023

Last Verified

April 1, 2023

More Information

Terms related to this study

Keywords

Other Study ID Numbers

  • R01MH108551 (U.S. NIH Grant/Contract)

Plan for Individual participant data (IPD)

Plan to Share Individual Participant Data (IPD)?

YES

This information was retrieved directly from the website clinicaltrials.gov without any changes. If you have any requests to change, remove or update your study details, please contact register@clinicaltrials.gov. As soon as a change is implemented on clinicaltrials.gov, this will be updated automatically on our website as well.

Clinical Trials on Assess Fidelity to Cognitive-behavioral Therapy for Youth

Clinical Trials on Self-report

Search Similar Trials

Sponsors and Collaborators

Medical Conditions

Drug Interventions

CROs by country

CROs in Malawi

Conditions

Rare Diseases

Drug Interventions

Dietary Supplements

Sponsor/Collaborators

Locations

3
Subscribe