Remote Training in Rural Schools

Development and Pilot Testing of a Remote Training Strategy for the Implementation of Mental Health Evidence-Based Practices in Rural Schools

Rural areas have fewer, and less well trained, health care providers than non-rural areas. Schools have become more involved in the delivery of mental health services and hold great potential for increasing access to children and adolescents. Innovations in training and service delivery are needed to improve mental health care quality and availability in rural schools. Evidence-based practices (EBPs) can be incorporated into school-wide multi-tiered systems that are currently used to improve school climate and safety. School-wide Positive Behavioral Interventions and Supports (PBIS), a service-delivery strategy based on the public health model is one example. Investigators will use an iterative process (Rapid Prototyping) to develop and evaluate the appropriateness, feasibility, acceptability, and preliminary efficacy of a remote training strategy that provides resources to support use of Tier 2 EBPs and effective support for care coordination practices in rural schools.

Study Overview

• Status: Completed
• Conditions: Behavior Disorders
• Intervention / Treatment: Behavioral: Coping Power Program (CPP); Behavioral: Check-in/Check-out (CI/CO); Behavioral: Cognitive behavioral therapy (CBT) for Anxiety Treatment in Schools (CATS)

• Study Type: Interventional
• Enrollment (Actual): 174
• Phase: Not Applicable

Contacts and Locations

Study Locations

• United States
  • Pennsylvania
    • Philadelphia, Pennsylvania, United States, 19104
      • Children's Hospital of Philadelphia

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: 18 years to 99 years (Adult, Older Adult)
• Accepts Healthy Volunteers: Yes

Description

Phase 1 (semi-structured interviews and rapid prototyping) Inclusion Criteria: Any BHS (e.g., school counselor, school social worker or teacher who is under employment from a school district) based at a school implementing PBIS and who may be nominated by the principal as a potential participant is eligible to be included in the study. Additionally, the research staff will reach out to school behavioral health staff concurrently with the school principal. BHS staff will provide consent to participate.

Exclusion Criteria: Staff from schools not implementing PBIS.

Phase 2 (pilot RCT)

Inclusion Criteria:

Administrator: Any school principal or assistant principal from participating schools implementing PBIS.

Behavioral Health Staff: Any counselor, social worker, or teacher from participating schools implementing PBIS who work with students in grades 4-8.

Students:

• Attending one of the participating schools
• Being in grades 4-8
• Identified by the Tier 2 team as not responding to Tier 1 intervention, thus needing Tier 2 support
• Scoring ≥ 1 SD above the mean on the Emotional Symptoms or Conduct Problems scales of the Strength and Difficulties Questionnaire (SDQ) 54 completed by a parent or a teacher (determined based on existing literature and demographic for which EBP was designed)

Exclusion Criteria:

Administrator: School staff who are not principals or assistant principals.

Behavioral Health Staff: School staff who are not part of the PBIS team and who do not work with students in grades 4-8.

Students: Students who do not meet screening or group participation criteria. Students with a history of intellectual disability or serious developmental delays according to school records will not be included because they would be unlikely to benefit from the interventions used in the study. Students with a history of psychotic or autistic spectrum disorders as reported by parents will not be included because these interventions are not appropriate for that demographic.

Study Plan

How is the study designed?

Design Details

• Primary Purpose: Health Services Research
• Allocation: Randomized
• Interventional Model: Parallel Assignment
• Masking: None (Open Label)

Number of Arms

3

Arms and Interventions

Participant Group / Arm	Intervention / Treatment
Active Comparator: Video (TV); Behavioral health staff (BHS, intervention implementers) received an initial synchronous and asynchronous training and ongoing access to asynchronous videos of intervention implementation.	Behavioral: Coping Power Program (CPP); CPP is an evidence-based intervention designed for students with externalizing behavior disorder. CPP consists of twelve 45-minute sessions. This EBP has been found to be effective at reducing aggressive behavior, covert delinquent behavior and substance abuse among aggressive boys, with gains maintained at one-year follow-up. Growth curve analyses showed that CPP had linear effects for three years after intervention on reductions in aggressive behavior and academic behavior problems.; Behavioral: Check-in/Check-out (CI/CO); CI/CO is a targeted, Tier 2 intervention for students at risk of developing externalizing and internalizing mental health disorders. CI/CO is designed to provide immediate feedback (i.e., at the end of each class period) to students, based on the use of a daily report card. This feedback is developmentally sensitive. CI/CO implementers meet individually with students for a brief 'check-in' in the morning and a brief 'check-out' in the afternoon. Research on the use of CI/CO has shown it to be effective in reducing externalizing and internalizing problems with elementary school students. CI/CO will be offered to individual students for a three-month period of time, which is the same time-frame needed for the implementation of CPP.; Behavioral: Cognitive behavioral therapy (CBT) for Anxiety Treatment in Schools (CATS); CATS is an adaptation of Friends for Life (FRIENDS). The adapted protocol retains the core elements of evidence-based CBT for anxiety and the FRIENDS group format. Investigators implemented planned adaptations to the protocol based on collective experience. Changes were made to the language, cultural methods, number of sessions, and activities while maintaining the 5 essential components of the treatment. This resulted in a briefer (8-session) and more feasible, engaging and culturally appropriate protocol for urban under-resourced schools than the original FRIENDS.
Active Comparator: Video Plus Consultation (TVC); BHS (intervention implementers) received an initial synchronous and asynchronous training, ongoing access to asynchronous videos of intervention implementation, and regular synchronous consultation from project staff.	Behavioral: Coping Power Program (CPP); CPP is an evidence-based intervention designed for students with externalizing behavior disorder. CPP consists of twelve 45-minute sessions. This EBP has been found to be effective at reducing aggressive behavior, covert delinquent behavior and substance abuse among aggressive boys, with gains maintained at one-year follow-up. Growth curve analyses showed that CPP had linear effects for three years after intervention on reductions in aggressive behavior and academic behavior problems.; Behavioral: Check-in/Check-out (CI/CO); CI/CO is a targeted, Tier 2 intervention for students at risk of developing externalizing and internalizing mental health disorders. CI/CO is designed to provide immediate feedback (i.e., at the end of each class period) to students, based on the use of a daily report card. This feedback is developmentally sensitive. CI/CO implementers meet individually with students for a brief 'check-in' in the morning and a brief 'check-out' in the afternoon. Research on the use of CI/CO has shown it to be effective in reducing externalizing and internalizing problems with elementary school students. CI/CO will be offered to individual students for a three-month period of time, which is the same time-frame needed for the implementation of CPP.; Behavioral: Cognitive behavioral therapy (CBT) for Anxiety Treatment in Schools (CATS); CATS is an adaptation of Friends for Life (FRIENDS). The adapted protocol retains the core elements of evidence-based CBT for anxiety and the FRIENDS group format. Investigators implemented planned adaptations to the protocol based on collective experience. Changes were made to the language, cultural methods, number of sessions, and activities while maintaining the 5 essential components of the treatment. This resulted in a briefer (8-session) and more feasible, engaging and culturally appropriate protocol for urban under-resourced schools than the original FRIENDS.
Active Comparator: Control Condition (T); BHS (intervention implementers) received an initial synchronous and asynchronous training only.	Behavioral: Coping Power Program (CPP); CPP is an evidence-based intervention designed for students with externalizing behavior disorder. CPP consists of twelve 45-minute sessions. This EBP has been found to be effective at reducing aggressive behavior, covert delinquent behavior and substance abuse among aggressive boys, with gains maintained at one-year follow-up. Growth curve analyses showed that CPP had linear effects for three years after intervention on reductions in aggressive behavior and academic behavior problems.; Behavioral: Check-in/Check-out (CI/CO); CI/CO is a targeted, Tier 2 intervention for students at risk of developing externalizing and internalizing mental health disorders. CI/CO is designed to provide immediate feedback (i.e., at the end of each class period) to students, based on the use of a daily report card. This feedback is developmentally sensitive. CI/CO implementers meet individually with students for a brief 'check-in' in the morning and a brief 'check-out' in the afternoon. Research on the use of CI/CO has shown it to be effective in reducing externalizing and internalizing problems with elementary school students. CI/CO will be offered to individual students for a three-month period of time, which is the same time-frame needed for the implementation of CPP.; Behavioral: Cognitive behavioral therapy (CBT) for Anxiety Treatment in Schools (CATS); CATS is an adaptation of Friends for Life (FRIENDS). The adapted protocol retains the core elements of evidence-based CBT for anxiety and the FRIENDS group format. Investigators implemented planned adaptations to the protocol based on collective experience. Changes were made to the language, cultural methods, number of sessions, and activities while maintaining the 5 essential components of the treatment. This resulted in a briefer (8-session) and more feasible, engaging and culturally appropriate protocol for urban under-resourced schools than the original FRIENDS.

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Content Fidelity of Cognitive Behavioral Therapy (CBT) for Anxiety Treatment in Schools (CATS) Implementation	The Cognitive Behavioral Therapy (CBT) for Anxiety Treatment in Schools (CATS) Content Fidelity Checklist (CFC) measures implementer adherence to required components of each CATS group session based on audio-recorded sessions. A separate 4-item checklist is completed for each of the eight CATS sessions, with session-specific items scored using a yes/no format to indicate whether required content was delivered. Fidelity is calculated as the percentage of items scored "Yes" out of the total items on the checklist/expected, with scores ranging from 0% to 100%. Higher scores indicate greater fidelity of implementation.	Across the 8-session CATS intervention period (approximately 8 weeks)
Content Fidelity of Coping Power Program (CPP) Implementation	The Coping Power Program (CPP) Content Fidelity Checklist (CFC) measures implementer adherence to required components of each CPP group session based on audio-recorded sessions. A separate checklist is completed for each of the 12 CPP sessions, with session-specific items (ranging from 4 to 8 items depending on the session) scored using a yes/no format to indicate whether required content was delivered. Fidelity is calculated as the percentage of items scored "Yes" out of the total items on the checklist, with scores ranging from 0% to 100%. Higher scores indicate greater fidelity of implementation.	Across the 12-session CPP intervention period (approximately 12 weeks)
Process Fidelity of Cognitive Behavioral Therapy (CBT) for Anxiety Treatment in Schools (CATS) and Coping Power Program (CPP) Implementation	The Process Fidelity Checklist (PFC) measures the quality of intervention delivery during the Cognitive Behavioral Therapy (CBT) for Anxiety Treatment in Schools (CATS) and Coping Power Program (CPP) group sessions, including organization, use of active learning strategies, clarity of presentation, engagement of students, and relevance of examples. The checklist includes 10 items rated on a 0 to 5 scale (0=Not at all to 5=Very Often). Scores are calculated as the mean rating across items and sessions, with possible scores ranging from 0 to 5. Higher scores indicate greater fidelity of implementation.	Across intervention session periods (approximately 8 to 12 weeks of intervention delivery)
Content and Process Fidelity of Check-In/Check-Out (CICO) Implementation	The Implementation Guide Fidelity Checklist (IGF) - Content and Process measure assesses both content and process fidelity of implementation of the Check-In/Check-Out (CICO) intervention. Content fidelity is assessed using 40 items scored as "Yes," "No," or "Not Applicable" across key components (e.g., roles, logistics, motivation system, training, and data monitoring). Content fidelity scores are calculated as the percentage of items scored "Yes" out of applicable items (excluding "Not Applicable"), ranging from 0% to 100%, with higher percentages indicating greater adherence. Within the same checklist, process fidelity is assessed using quality indicator items rated 0 to 3 reflecting the comprehensiveness, clarity, and feasibility of implementation procedures. Process fidelity scores are calculated as the average quality rating across items, ranging from 0 to 3. Higher scores indicate greater fidelity of implementation.	Once, at the completion of Check-In/Check-Out (CICO) implementation guide development (within the first year of implementation)
Penetration of Cognitive Behavioral Therapy (CBT) for Anxiety Treatment in Schools (CATS) and Coping Power Program (CPP) Implementation	The Penetration Inventory (PI) is an Excel tracking tool used to document teacher referrals for Cognitive Behavioral Therapy (CBT) for Anxiety Treatment in Schools (CATS) and Coping Power Program (CPP) interventions among students in grades 4-8 and the number of students who received one of the evidence-based practices (EBPs). Penetration is calculated as the proportion of referred students who received an EBP, expressed as a percentage, with higher scores indicating greater penetration of Tier 2 services.	Throughout the intervention implementation period (up to approximately 5 years from study start)

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Change in Behavioral and Emotional Functioning (The Behavior Assessment System for Children, Third Edition [BASC-3])	The Behavior Assessment System for Children, Third Edition (BASC-3) parent report measures children's and adolescents' behavioral and emotional functioning. Scores are reported as standardized T-scores, with higher scores indicating greater behavioral or emotional problems. Parents/caregivers completed the BASC-3 at pre-intervention and post-intervention to assess change over time.	From pre-intervention to post-intervention (approximately 8-12 weeks)
Change in Behavioral and Emotional Symptoms (Behavior and Feelings Scale - Youth Self-Report)	The Behavior and Feelings Survey (BFS) - Youth Self-Report measures youths' behavioral and emotional symptoms. Possible scores range from 0 to 48 on the total score, with higher scores indicating greater symptom severity. Students completed the BFS at pre-intervention and post-intervention to assess change over time.	From pre-intervention to post-intervention (approximately 8-12 weeks)
Change in Student Academic Engagement (Engagement Versus Disaffection with Learning - Teacher Report [EvsD-TR])	The Engagement Versus Disaffection with Learning - Teacher Report (EvsD-TR) measures students' academic engagement and disaffection in the classroom. The instrument includes four subscales: Behavioral Engagement, Emotional Engagement, Behavioral Disaffection, and Emotional Disaffection. Each subscale score ranges from 5 to 20, with higher scores indicating greater levels of the respective construct (engagement or disaffection). Teachers completed the EvsD-TR at pre-intervention and post-intervention to assess change in academic engagement over time.	From pre-intervention to post-intervention (approximately 8-12 weeks)

Other Outcome Measures

Outcome Measure	Measure Description	Time Frame
Acceptability of Remote Training Strategy (Acceptability of Intervention Measure [AIM])	The Acceptability of Intervention Measure (AIM) assesses perceived acceptability of the remote training strategy. The AIM includes 4 items rated on a 1 to 5 scale, with total scores ranging from 4 to 20, and higher scores indicating greater acceptability of the training strategy.	Immediately after review of training modules during the pre-trial prototype evaluation phase (approximately 4 months after study start)
Appropriateness of Remote Training Strategy (Intervention Appropriateness Measure [IAM])	The Intervention Appropriateness Measure (IAM) assesses perceived appropriateness of the remote training strategy. The IAM includes 4 items rated on a 1 to 5 scale, with total scores ranging from 4 to 20, and higher scores indicating greater perceived appropriateness.	Immediately after review of training modules during the pre-trial prototype evaluation phase (approximately 4 months after study start)
Feasibility of Remote Training Strategy (Feasibility of Intervention Measure [FIM])	The Feasibility of Intervention Measure (FIM) assesses perceived feasibility of the remote training strategy. The FIM includes 4 items rated on a 1 to 5 scale, with total scores ranging from 4 to 20, and higher scores indicating greater perceived feasibility.	Immediately after review of training modules during the pre-trial prototype evaluation phase (approximately 4 months after study start)
Usability of Training Modules (Telehealth Usability Questionnaire - Usability Subscale [TUQ-US])	The Usability Subscale of the Telehealth Usability Questionnaire (TUQ-US) assesses usability of the training modules. The subscale includes items rated on a 1 to 7 scale, with higher scores indicating greater perceived usability of the training modules.	Immediately after review of training modules during the pre-trial prototype evaluation phase (approximately 4 months after study start)

Collaborators and Investigators

• Sponsor: Children's Hospital of Philadelphia
• Collaborators: Agency for Healthcare Research and Quality (AHRQ); Devereux Center for Effective Schools
• Investigators: Principal Investigator: Ricardo Eiraldi, PhD, Children's Hospital of Philadelphia

Publications and helpful links

General Publications

• Eiraldi R, McCurdy BL, Khanna MS, Goldstein J, Comly R, Francisco J, Rutherford LE, Wilson T, Henson K, Farmer T, Jawad AF. Development and evaluation of a remote training strategy for the implementation of mental health evidence-based practices in rural schools: pilot study protocol. Pilot Feasibility Stud. 2022 Jun 17;8(1):128. doi: 10.1186/s40814-022-01082-4.

Study record dates

Study Major Dates

• Study Start (Actual): September 30, 2020
• Primary Completion (Actual): July 1, 2025
• Study Completion (Actual): July 1, 2025

Study Registration Dates

• First Submitted: September 1, 2021
• First Submitted That Met QC Criteria: September 1, 2021
• First Posted (Actual): September 5, 2021

Study Record Updates

• Last Update Posted (Actual): February 19, 2026
• Last Update Submitted That Met QC Criteria: February 16, 2026
• Last Verified: February 1, 2026

More Information

Terms related to this study

• Keywords: Mental Health; Evidence-based Practices; Positive-behavior Interventions and Supports; Tier 2; Rural Schools
• Additional Relevant MeSH Terms: Behavior; Personal Satisfaction; Mental Disorders; Psychological Well-Being; Behavior Therapy; Psychotherapy; Behavioral Disciplines and Activities; Non-Medical Public and Private Facilities; Cognitive Behavioral Therapy; Schools
• Other Study ID Numbers: 20-017895; R18HS027755 (U.S. AHRQ Grant/Contract)

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: NO

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No