Leveraging Telepsychology and Behavioral Economics to Increase Fidelity to CBT (BENEFITS)

The goal of this study is to develop and test a telehealth tool intended to support community mental health clinicians to deliver cognitive behavioral therapy (CBT).

The main aims of the study are to:

1. Collaborate with clinicians and supervisors to design an accessible prototype to increase clinician use of CBT in community mental health settings through focus groups, interviews, and surveys.
2. Adapt the initial model through feedback from clinicians and their patients who will be asked to test the tool then complete interviews and surveys.
3. Test the model with clinicians and their patients who will be asked to use the tool for several weeks.
4. Evaluate the acceptability and feasibility of using the tool in a pilot trial through surveys and interviews.

Study Overview

• Status: Enrolling by invitation
• Conditions: Implementation Science
• Intervention / Treatment: Device: Tele-BE

Detailed Description

Our primary goal in this exploratory project is to develop, iteratively, refine and evaluate the preliminary effectiveness, acceptability, and feasibility of "Tele-BE," a novel telehealth platform that integrates behavioral economics (BE) strategies into the clinician user-interface of standard telehealth technology, to increase clinician use of structural components of cognitive behavioral therapy (CBT). Primary endpoints for this study are the refined Tele-BE platform implementation strategy and evaluation of Tele-BE's preliminary effectiveness to increase clinician fidelity and acceptability, and feasibility. Secondary outcomes include target implementation mechanisms (clinician intentions and their determinants: attitudes, norms and self-efficacy) consistent with our conceptual model. Our primary hypotheses are that Tele-BE will increase clinician fidelity to CBT structural components during the observation period and that Tele-BE will be an acceptable and feasible implementation strategy. Secondary hypotheses are that Tele-BE will engage our target implementation mechanisms of interest, as delineated in Table 1 of the Research Strategy. Of note, as this is a pilot trial, it is not intended to be fully powered to detect effects and we will rely on qualitative and mixed methods analyses to examine study hypotheses.

Our goal is to develop and pilot-test Tele-BE to nudge clinicians to deliver six CBT structural components of interest: (1) symptom monitoring, (2) agenda setting guided by shared decision-making, (3) out-of-session practice ("homework") review, (4) skill instruction, (5) skill practice, and (6) homework planning. Tele-BE will leverage the infrastructure of standard telehealth platforms to alter the clinician interface and incorporate behavioral economics (BE) strategies to make it easier and more motivating to deliver structural components of interest to patients. We will partner with BlueJeans by Verizon, who currently partners with Penn Medicine to support HIPAA compliant telehealth technology throughout the system, to create the Tele-BE prototype that will be tested in this study.

In Aim 1, we will adapt and refine Tele-BE in collaboration with a Stakeholder Advisory Board, BlueJeans, and Transmogrify, a web development company with an established partnership with Penn Medicine. Aim 1 is a largely preparatory aim during which we will refine our Tele-BE prototype in collaboration with an initial group of 3 community clinicians as our target end-users as well as 3 clinical supervisors. We will recruit three additional clinicians in Aim 1 to test out early elements of Tele-BE. These additional clinicians will engage in laboratory-based usability testing with a basic functional version of Tele-BE. Participants will meet with the study team and interact with the Tele-BE platform for 30 minutes; participants also will conduct a mock therapy session with a graduate student clinician. Participants will be recorded through the BlueJeans platform while interacting with Tele-BE. We will use a "think aloud" framework to have clinicians verbalize their thoughts when using the program. Following the think aloud protocol, participants will complete brief questionnaires and a brief interview to inform Tele-BE refinement.

In Aim 2, we will work closely with our web development team to field test and iteratively refine Tele-BE using rapid cycle prototyping to optimize the user experience and refine our specific BE strategies. We anticipate recruiting 5 community clinicians and 5 of their patients to pilot iterations of Tele-BE in their clinical practice for 4-5 weeks. These participants will provide qualitative feedback on Tele-BE's ability to engage target implementation mechanisms. if saturation is not reached with an initial sample of participants, we may recruit up to 5 more clinician and patient dyads. The primary outcome of Aim 2 will be a refined version of Tele-BE to be tested in the pilot trial.

Aim 3 is a pilot 16-week open trial to test preliminary effectiveness of the refined Tele-BE to enhance clinician CBT structural component fidelity. We will recruit 30 community mental health clinicians and randomize them to Tele-BE or telehealth as usual (Tele-AU). The trial will include 2 patients per clinician (60 patients total). Note that the exact features included in Tele-BE will be dependent on the outcomes of Aims 1 and 2 and are not designed to change clinician session content (i.e., the specific intervention techniques a clinician employs) in any way. However, we anticipate that it will include some or all of the following design features: (1) Reminder message prompting clinician to identify a skill to review, introduce, or practice, prior to the start of a clinical encounter, (2) Completion of progress measures (in this case, measures of therapeutic alliance) in the telehealth "waiting room," (3) Template agenda of structural components with an option to select CBT interventions that will be delivered in session, (4) Checkmarks indicating the CBT component is completed, with a display that adjusts as goals are reached, (5) Gamification features (e.g., badges) for goals met (e.g., agenda completed >90% of the time), (6) Feedback messages about elements completed from clinical supervisor (e.g., messages offering praise for hitting certain benchmarks, and (7) Downloadable summary of session activities for the clinical progress note. All therapy sessions will be recorded and subsequently coded for fidelity (Aim 3 primary outcomes). Clinicians will complete measures of intentions and their determinants at the week of sessions 1, 5, 9, and 12 (Aim 3 secondary outcomes).

Using data from the open trial, Aim 4 will examine acceptability and feasibility of Tele-BE from patient and clinician perspectives and potential ethical issues with Tele-BE. We will measure Tele-BE acceptability and feasibility via web analytics, surveys, interviews, and recruitment and retention statistics, and patient outcomes. Patients will complete self-report measures of therapeutic alliance and symptoms on the same schedule as clinician report; they additionally will complete measures of patient centered care at pre and post. Select patients will also complete qualitative interviews.

To recruit and enroll clinicians, we first will identify a minimum of 4 Philadelphia agencies that are already using telehealth and are trained in transdiagnostic CBT through the Penn Collaborative for CBT and Implementation Science (directed by MPI Dr. Creed). While the telehealth landscape will likely shift as the COVID-19 pandemic ebbs, Philadelphia's Medicaid behavioral health payor has committed to continuing to reimburse for telehealth after the crisis concludes. We then will employ a cascaded recruitment strategy to recruit 30 clinicians who then will identify their patients aged 14 or older (the age of consent for mental health services in Pennsylvania) with whom they are providing psychotherapy via telehealth. We will work with clinicians to identify two patients that they will approach for study participation. For each clinician-patient dyad, participation will occur over a 16-week trial period. Clinicians and patients will complete study measures as outlined in Table 3 of the Research Strategy.

• Study Type: Interventional
• Enrollment (Estimated): 100
• Phase: Not Applicable

Contacts and Locations

Study Locations

• United States
  • Pennsylvania
    • Philadelphia, Pennsylvania, United States, 19104
      • University of Pennsylvania

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: Child; Adult; Older Adult
• Accepts Healthy Volunteers: Yes

Description

Inclusion Criteria:

Prototyping open trial phase:

Clinician Participants

• Clinician is an outpatient mental health clinician in the City of Philadelphia trained in transdiagnostic CBT through the Penn Collaborative for CBT and Implementation Science
• Clinician has completed the 22-hour intensive CBT workshop through the BCI or equivalent CBT training
• Clinician provides direct mental health services to a treatment-seeking population via telehealth
• Proficient in English language
• Has access to a computer with internet connectivity

Patient Participants

• Patient is 14 years or older
• Patient is receiving CBT from a participating clinician
• Patient is able to provide informed consent (or if the client is between the ages of 14-17, the client is able to assent and has a legal guardian who is able to provide consent)
• Proficient in English language
• Patient has had at least one prior telehealth therapy session with enrolled therapist
• Has access to a computer with internet connectivity or smartphone through which they regularly attend telehealth therapy sessions

Pilot randomized controlled trial phase:

Clinician Participants

• Clinician is an outpatient mental health clinician in the City of Philadelphia trained in transdiagnostic CBT through the Penn Collaborative for CBT and Implementation Science
• Clinician has completed the 22-hour intensive CBT workshop through the BCI or equivalent CBT training
• Clinician is a practicing mental health clinician in the Philadelphia region in one of our partner local community agencies delivering telehealth services
• Clinician has at least 2 clients on their caseload over the age of 14 with whom they are delivering CBT telehealth services
• Proficient in English language
• Has access to a computer with internet connectivity

Patient Participants

• Patient is 14 years or older
• Patient is able to provide informed consent (or if the client is between the ages of 14 -17, the client is able to assent and has a legal guardian who is able to provide consent)
• Proficient in English language
• Patient has had at least one prior telehealth therapy session with enrolled therapist
• Has access to a computer with internet connectivity or smartphone through which they regularly attend telehealth therapy sessions

Exclusion Criteria:

Prototyping open trial phase:

Clinician Participants - Clinician does not provide any telehealth services.

Patient Participants

• Patients will be excluded if they are unable to adequately consent to study procedures. For example, clients may be excluded if they are at imminent risk for suicide or homicide requiring hospitalization.
• Youth under the age of 18 will be excluded from the study if they do not have an available legal guardian who is able to provide consent for their study participation (e.g., the state has legal custody of the youth).
• Patients under the age of 14 will be excluded

Pilot randomized controlled trial phase: Participants will not be excluded based on sex, demographics, and or past experience.

Clinician Participants

- Clinician does not provide telehealth services to at least 2 individuals on their caseload.

Patient Participants

• Patients will be excluded if they are unable to adequately consent to study procedures. For example, clients may be excluded if they are at imminent risk for suicide or homicide requiring hospitalization or are experiencing any cognitive impairment that prevents them from participating in therapy on the day their session is to be recorded.
• Youth under the age of 18 will be excluded from the study if they do not have an available legal guardian who is able to provide consent for their study participation (e.g., the state has legal custody of the youth).
• Patients under the age of 14 will be excluded

Study Plan

How is the study designed?

Design Details

• Primary Purpose: Health Services Research
• Allocation: Randomized
• Interventional Model: Parallel Assignment
• Masking: None (Open Label)

Number of Arms

2

Arms and Interventions

Participant Group / Arm	Intervention / Treatment
No Intervention: Treatment as usual; Clinicians will deliver cognitive behavioral therapy as usual, as consistent with routine clinic practices, to patients receiving telehealth services
Experimental: Tele-BE; Clinicians will deliver cognitive behavioral therapy as usual, as consistent with routine clinic practices, with the support of the Tele-BE platform, to patients receiving telehealth services	Device: Tele-BE; Clinicians and their patients use the newly developed telehealth tool to support their use of CBT during session

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Acceptability of Tele-BE	Acceptability of Intervention Measure: A 4-item, psychometrically-validated measure that indexes the extent to which stakeholders believe an implementation strategy is acceptable Qualitative Interview	Week of sessions 1, 5, 9, and 12
Feasibility of Tele-BE	Feasibility of Intervention Measure; A 4-tem, psychometrically-validated measure that indexes the extent to which stakeholders perceive an implementation strategy is feasible Web Analytics (i.e., clinician use of Tele-BE features)	Week of sessions 1, 5, 9, and 12
Usability of Tele-BE	- System Usability Scale (SUS): A 10-item questionnaire on a 5-point scale from 1=Strongly disagree to 5=Strongly agree which measures the usability of a system (in this case, Tele-BE).	Week of sessions 1, 5, 9, and 12
Clinician Fidelity to CBT Structural Components	Cognitive Therapy Rating Scale (CTRS): Clinician fidelity to structural components will be captured via direct observation to each of the 6 components of interest: ((1) progress monitoring with evidence-based assessment tools, (2) agenda setting guided by shared decision-making, (3) out-of-session practice ("homework") review, (4) skill instruction, (5) skill practice, and (6) homework planning. Items will be coded using adapted items of the CTRS, a gold-standard CBT fidelity measure that has a subscale designed to measure structural components	Weekly over the trial period (approximately 12 sessions, up to 16 weeks)

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Clinician intentions to use the 6 structural components of EBPs of interest	Established question stems from behavioral science using two items on a 7-point scale asking how willing and how likely one is to use each structural component	Week of sessions 1, 5, 9, and 12
Clinician attitudes, norms, and self-efficacy (i.e., determinants of intentions) to use the 6 structural components of EBPs of interest	Established question stems from behavioral science, all measured on a 7-point scale. The total scale score will be computed by averaging all z-standardized items within each structural component. Attitudes are measured on the extent to which clinicians agree/disagree with each statement (higher score indicates more favorable attitude). Norms are measured on a scale of degree of agreement regarding the perception that "other people like me" will use the component, and that other people who are important to them (e.g., supervisors) will approve of them using it. Self-efficacy is measured on degree of agreement about clinicians' perceptions of themselves as having the ability to perform each component.	Week of sessions 1, 5, 9, and 12
Patient clinical symptoms	Patient symptoms will be measured with the Strengths and Difficulties Questionnaire, a broadband questionnaire of self-reported mental health symptoms	Week of sessions 1, 5, 9, and 12
Therapeutic alliance	The Working Alliance Inventory-Short Form (WAI-SF) is a validated measure that asks patients to rate 12 items regarding their perceptions of their relationship with their therapist using a 5-point Likert scale	Week of sessions 1, 5, 9, and 12

Other Outcome Measures

Outcome Measure	Measure Description	Time Frame
contextual factors	measured through interview and quantitatively in background questionnaire	after the trial period (approximately 12 sessions, up to 16 weeks)
Edward's Social Desirability Scale	A 17-item measure which asks true/false questions which measures the tendency to give a socially desirable response.	Clinician Baseline (week of session 1)
Modified Cognitive Therapy Knowledge Quiz	A 19-item measure adapted from Creed et al. (2016) that assesses a clinician's general knowledge of CBT.	Clinician Baseline (week of session 1)
Self-reported Therapist Intervention Fidelity for Youth Supervision (SeRTIFY) - Adapted	A 14-item questionnaire adapted to be a clinician-reported measure of supervision practices. This measure was revised to be inclusive of clinicians who see adult clients and to prompt specifically about supervision practices as they relate to our CBT constructs of interest. The first part of the measure includes 2 questions about hours spent attending individual or group supervision. The second part of the measure includes 12 questions on a 7-point scale (1=not at all; 7=extensively) on the content of supervision (general, CBT-related, and overall rating).	Clinician Baseline and post intervention (week of session 1 and after session 12)

Collaborators and Investigators

• Sponsor: University of Pennsylvania

Publications and helpful links

General Publications

• Becker-Haimes EM, Mandell DS, Kuo PB, Lynch KG, Brady M, Young S, Creed TA. Behavioral Nudges to Enhance Fidelity in Telehealth Sessions (BENEFITS): Protocol for Developing and Pilot Testing a Telehealth Tool to Improve Cognitive Behavioral Therapy Implementation. JMIR Res Protoc. 2025 Sep 18;14:e76035. doi: 10.2196/76035.

Study record dates

Study Major Dates

• Study Start (Actual): March 4, 2025
• Primary Completion (Estimated): February 1, 2027
• Study Completion (Estimated): April 30, 2027

Study Registration Dates

• First Submitted: August 15, 2024
• First Submitted That Met QC Criteria: September 16, 2024
• First Posted (Actual): September 19, 2024

Study Record Updates

• Last Update Posted (Actual): May 5, 2026
• Last Update Submitted That Met QC Criteria: April 29, 2026
• Last Verified: April 1, 2026

More Information

Terms related to this study

• Other Study ID Numbers: R34MH135965 (U.S. NIH Grant/Contract)

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: YES

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No