Development of a Training Intervention to Improve Mental Health Treatment Effectiveness and Engagement for Youth With Documented Mental Health Disparities

The overall goal of the larger 3-aim study is to develop and pilot test a training intervention to increase mental health providers' use of evidence-based practices with youth patients. Aim 3 (registered here) of the study is an open trial pilot study at a multi-clinic mental health agency, aimed at examining the feasibility and acceptability of conducting a future randomized controlled trial (RCT).

Study Overview

• Status: Completed
• Conditions: Knowledge, Attitudes, Practice
• Intervention / Treatment: Behavioral: Training Intervention

Detailed Description

The overall goal of the larger 3-aim study is to develop and pilot test a training intervention to increase mental health providers' use of evidence-based practices with youth patients. The Boston College Institutional Review Board (IRB) approved protocol #21.247.01-15. Aim 1 involved designing the training intervention through an extensive literature review and community-engaged methods that centered on collaborations with providers, youth patients, and parents. Aim 2 (in progress) entails refining the training intervention via human centered design methods. Aim 3 (registered here) of the study is an open trial pilot study at a multi-clinic mental health agency, aimed at examining the feasibility and acceptability of conducting a future randomized controlled trial (RCT). Effectiveness and implementation data from both providers (n=49) and patients (i.e., youth and their parents; n=50 GMY and n=30 caregivers) will be collected and analyzed. Primary outcomes are measured at the provider-level and secondary outcomes are measured at the patient-level (i.e., patients and their parents).

• Study Type: Interventional
• Enrollment (Actual): 286
• Phase: Not Applicable

Contacts and Locations

Study Locations

• United States
  • Massachusetts
    • Chestnut Hill, Massachusetts, United States, 02467
      • Boston College

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: 12 years and older (Child, Adult, Older Adult)
• Accepts Healthy Volunteers: Yes

Description

Inclusion Criteria:

For provider participants:

• Mental health care providers (e.g., psychologists, social workers) employed by the pilot site at the time of the pilot test and who work with clients ages 12-25
• Age 18 or older
• Fluent in English

For GMY participants:

• Identify as a gender-minority.
• Age 12-25
• Currently receiving mental health services from the pilot site
• Fluent in English

For parents of youth participants:

• Parents of youth currently receiving mental health services at the pilot site
• 18 or older
• Fluent in English

Exclusion Criteria:

• Provided that participants meet the aforementioned inclusion criteria, there are no explicit criteria for exclusion.

Study Plan

How is the study designed?

Design Details

• Primary Purpose: Health Services Research
• Allocation: N/A
• Interventional Model: Single Group Assignment
• Masking: None (Open Label)

Number of Arms

1

Arms and Interventions

Participant Group / Arm

Intervention / Treatment

Experimental: Training Intervention; The single-arm intervention study will test an online asynchronous evidence-based practices training intervention.

Behavioral: Training Intervention; The training intervention is a modular online asynchronous training intervention designed to increase mental health providers' use of evidence-based practices with youth patients (ages 12-25). It was developed using community-engaged and human-centered design methods with key stakeholders (youth, their parents, mental health providers). In this open trial, the training will be offered to mental health providers at a multisite mental health clinic in the U.S. (referred to as "pilot site" herein).

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Provider Participant Recruitment Rates (Feasibility of Conducting a Larger Trial)	Percent of providers at pilot site who consented to participate in the study relative to number of employed providers at pilot site.	At time of training intervention
Change in Provider Participant Assessment Completion Rates at Post-training (Feasibility of Conducting a Larger Trial)	Change in the percent of providers who complete pre- and post- assessments of those who consented to participate. These assessment completion rates will also be compared to the 6- and 12-month assessment completion rates.	Pre-training to post-training (up to 4 weeks after the training)
Change in Provider Participant Assessment Completion Rates at 6-month Followup (Feasibility of Conducting a Larger Trial)	Change in the percent of providers who complete post- and 6-month followup assessments of those who consented to participate. These assessment completion rates will also be compared to the pre-training and 12-month assessment completion rates.	Post-training (up to 4 weeks after the training) to 6-month followup
Change in Provider Participant Assessment Completion Rates at 12-month Followup (Feasibility of Conducting a Larger Trial)	Change in the percent of providers who complete 6- and 12-month followup assessments of those who consented to participate. These assessment completion rates will also be compared to the pre-training and post-training assessment completion rates.	6-month followup to 12-month followup
Change in the Adoption of Practices Learned in the Training Intervention at 12-month Followup	A self-report survey measuring whether, and the extent to which, provider participants used the practices they learned in the training intervention. The survey is currently being developed and will be adapted based on the Pachankis et al. (2022) measure Providers' Familiarity With and Use of LGBTQ-affirmative CBT Skills. The survey will be administered at the 6- and 12 month followups to see the change over time. Scores are calculated using mean scores for values 0-4 with higher numbers indicating more of the use of the practice (i.e., (0) Not at all, (4) Very much)	Baseline, Post-training, 6-month to 12-month followup
Electronic Monitoring of Training Intervention Completion (Adoption of Training Intervention)	Electronic monitoring of training intervention completion by providers (i.e., whether all modules were completed).	up to the 12-month followup
Primary Outcome: Electronic Monitoring of Training Intervention Engagement (Adoption of Training Intervention)	Electronic monitoring of training intervention engagement (e.g., days spent completing the training)	up to the 12-month followup

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Client Behavioral Engagement by Session Participation	Available chart data from clients of providers who completed the intervention will be monitored for their behavioral engagement. Session participation will be measured by electronic health record monitoring of treatment measures completion (e.g., answering all questions and submitting measures related to client treatment, such as the TAQs).	Weekly from pre-intervention to 12 months after training intervention completion
Client Behavioral Engagement by Session Attendance	Available chart data from clients of providers who completed the intervention will be monitored for their behavioral engagement. Session attendance will be measured as an aspect of behavioral engagement by noting attended and canceled sessions. Outcomes report the total number of GMY participants who attended therapy during the timeframe	Weekly from pre-intervention to 12 months after training intervention completion
Therapeutic Alliance Quality Scale (Client Satisfaction & Attitudinal Engagement)	The Therapeutic Alliance Quality Scale (TAQS) is a 5-item questionnaire that asks clients and their caregivers to reflect on their last session with their therapist. Items are rated on a 5-point scale with higher responses corresponding to higher quality alliance (e.g., 1= "Not at all," 2= "Only a little," 3= "Somewhat," 4= "Quite a bit," and 5= "Totally"). The TAQS will be administered to clients aged 12-25 and caregivers. Outcomes report pre-intervention TAQs score (i.e., first instance of participant completing TAQs prior-to intervention) and 12-month follow-up TAQs score.	Weekly from pre-intervention to 12-month follow up
Service Satisfaction Scale (Client Satisfaction & Attitudinal Engagement)	The Service Satisfaction Scale (SSS) is a 5-item questionnaire that asks clients and their caregivers to reflect on how satisfied they are with the mental health services that they receive. Items are rated on a 4-point scale with higher responses corresponding to higher service satisfaction (e.g., 1= "No, definitely not," 2= "No, not really," 3= "Yes, generally," and 4= "Yes, definitely"). The SSS will be administered to clients aged 12-25 and caregivers. Outcomes report pre-intervention SSS score (i.e., first instance of participant completing SSS prior-to intervention) and 12-month follow-up SSS score.	Every other week from pre-intervention to 12-month follow up
Treatment Outcomes Expectation Scale (Client Satisfaction & Attitudinal Engagement)	The Treatment Outcomes Expectation Scale (TOES) is a 8-item questionnaire that asks clients and their caregivers about their expectations from counseling. Items are rated on a 3-point scale with higher responses corresponding to more expectations (e.g., 1= "I do not expect this," 2= "I am not sure", and 3= "I do expect this"). The TOES will be administered to clients aged 12-25 and caregivers.	Pre-intervention
Treatment Process Expectations Index (Client Satisfaction & Attitudinal Engagement)	The Treatment Process Expectations Index is a 9-item questionnaire that asks clients and their caregivers about their expectations about counseling. Items are rated on a 3-point scale (e.g., 1= "I do not expect this," 2= "I am not sure", and 3= "I do expect this"). There is no right or wrong answer. The reported scores should be interpreted at the item level to identify clinical suggestions for providers based on the clients reported expectations at baseline. The TPEI will be administered to clients aged 12-25 and caregivers.	Pre-intervention
Ohio Functioning Scale (Client Functioning)	The Ohio Functioning Scale is a 20-item questionnaire that asks clients and their caregivers to answer questions regarding how the client's problems may get in the way of everyday activities. Items are rated on a 5-point scale with higher responses corresponding to better ability to do everyday activities (e.g., 0= "Extreme troubles," 1= "Quite a few troubles," 2= "Some troubles", 3= "Ok," and 4= "Doing very well"). A total functioning score ranges from 20-100 and is calculated by summing the ratings for all 20 items. Higher scores are indicative of better functioning. The Ohio Functioning Scale will be administered to clients aged 12-25 and caregivers. Outcomes report pre-intervention Ohio score (i.e., first instance of participant completing Ohio prior-to intervention) and 12-month follow-up Ohio score.	Monthly from pre-intervention to 12-month follow up
Symptoms and Functionality Severity Scale (Client Symptomatology & Functioning)	The Symptoms and Functionality Severity Scale which is a 13-item questionnaire that asks clients and their caregivers to reflect on behaviors, thoughts, and feelings that the client had in the last 2 weeks. Items are rated on a 5-point scale with higher responses corresponding to more frequent experiences of negative behaviors, thoughts, and feelings (e.g., 1= "Never," 2= "Hardly Ever," 3= "Sometimes," 4= "Often," and 5= "Very Often"). Scores can range from 33 to 86 for youths, where a high score represents high severity and a low score indicates low severity. Total scores are calculated by adding the rating and then divide the sum by the number of items to calculate the average score.The SFSS will be administered to clients aged 12-25 and caregivers. Outcomes report pre-intervention SFSS score (i.e., first instance of participant completing SFSS prior-to intervention) and 12-month follow-up SFSS score.	Monthly from pre-intervention to 12-month follow up

Collaborators and Investigators

• Sponsor: Boston College
• Collaborators: National Institute of Mental Health (NIMH)
• Investigators: Principal Investigator: Maggi Price, PhD, Boston College School of Social Work

Publications and helpful links

General Publications

• Bidell MP. The Sexual Orientation Counselor Competency Scale: Assessing attitudes, skills, and knowledge of counselors working with lesbian, gay, and bisexual clients. Counselor Education and Supervision. 2011; 44(4): 267-279.
• Pachankis JE, Soulliard ZA, Seager van Dyk I, Layland EK, Clark KA, Levine DS, Jackson SD. Training in LGBTQ-affirmative cognitive behavioral therapy: A randomized controlled trial across LGBTQ community centers. J Consult Clin Psychol. 2022 Jul;90(7):582-599. doi: 10.1037/ccp0000745.
• Athay MM, Bickman L. Development and psychometric evaluation of the youth and caregiver Service Satisfaction Scale. Adm Policy Ment Health. 2012 Mar;39(1-2):71-7. doi: 10.1007/s10488-012-0407-y.
• Duppong Hurley K, Lambert MC, Van Ryzin M, Sullivan J, Stevens A. Therapeutic Alliance Between Youth and Staff in Residential Group Care: Psychometrics of the Therapeutic Alliance Quality Scale. Child Youth Serv Rev. 2013 Jan 1;35(1):56-64. doi: 10.1016/j.childyouth.2012.10.009. Epub 2012 Nov 8.
• Bickman L, Kelley SD, Breda C, de Andrade AR, Riemer M. Effects of routine feedback to clinicians on mental health outcomes of youths: results of a randomized trial. Psychiatr Serv. 2011 Dec;62(12):1423-9. doi: 10.1176/appi.ps.002052011.
• Bickman L, Douglas SR, De Andrade AR, Tomlinson M, Gleacher A, Olin S, Hoagwood K. Implementing a Measurement Feedback System: A Tale of Two Sites. Adm Policy Ment Health. 2016 May;43(3):410-25. doi: 10.1007/s10488-015-0647-8.
• Price MA, Mulkern PJ, Condon M, Rakhilin M, Johansen K, Lyon AR, Saldana L, Pachankis J, Woodward SA, Roeder KM, Moran LR, Jerskey BA. Leveraging Community Engagement and Human-Centered Design to Develop Multilevel Implementation Strategies to Enhance Adoption of a Health Equity Intervention. Res Sq [Preprint]. 2025 Mar 28:rs.3.rs-5702080. doi: 10.21203/rs.3.rs-5702080/v1.

Study record dates

Study Major Dates

• Study Start (Actual): September 1, 2022
• Primary Completion (Actual): July 31, 2024
• Study Completion (Actual): August 31, 2025

Study Registration Dates

• First Submitted: November 1, 2022
• First Submitted That Met QC Criteria: November 16, 2022
• First Posted (Actual): November 23, 2022

Study Record Updates

• Last Update Posted (Estimated): September 25, 2025
• Last Update Submitted That Met QC Criteria: September 5, 2025
• Last Verified: September 1, 2025

More Information

Terms related to this study

• Keywords: adolescent; young adult; child; youth; teen; psychologist; social worker; psychiatrist; counselor; provider; therapist; clinician
• Additional Relevant MeSH Terms: Behavior
• Other Study ID Numbers: Training Development Study; K23MH124670 (U.S. NIH Grant/Contract)

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: NO

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No