Iterative Redesign of a Behavioral Skills Training Program for Use in Educational Settings

This study proposes to redesign the RUBI Parent Training program, a low-intensity intervention for youth with autism spectrum disorder and disruptive behavior, for use by school personnel in the classroom. Using a mixed-methods approach, 40 school staff members from 20 elementary schools will be recruited to inform current classroom behavior management practices and RUBI redesign needs.

Study Overview

• Status: Completed
• Conditions: Autism Spectrum Disorder; Disruptive Behavior
• Intervention / Treatment: Behavioral: RUBI; Behavioral: RUBIES

Detailed Description

Approximately 50% of children with autism spectrum disorder (ASD) exhibit disruptive behaviors such as tantrums, aggression, and noncompliance that significantly impact social, adaptive, and academic functioning. The RUBI program is an evidence-based parent-mediated intervention that improves disruptive behavior in children with ASD. Considering schools serve as the primary intervention setting for children with ASD, and teachers and school staff often struggle to address challenging behavior, there is an opportunity to meaningfully improve care by adapting RUBI for delivery in schools. To address these needs, the Discover, Design/Build, Test (DDBT) framework,which leverages user-centered design and implementation science, will be used to engage 40 end users from 20 elementary schools to inform current classroom behavior management practices and redesign needs for the RUBI curriculum to ensure that the modified curriculum (RUBI in Educational Settings; RUBIES) is useful and useable for school personnel who work with children with ASD. Specific aims include:

Aim 1: Identify the contextual constraints and end users (teachers, school psychologists, behavior technicians, classroom aides) relevant to the management of disruptive behavior in the classroom. Through in-class behavioral observations and interviews, Aim 1 will allow us to understand the values and priorities of the school context and end users directly involved with children with ASD (e.g. resources, work flow, policies).

Aim 2: Identify targets for RUBI redesign. Using in-depth intervention demonstration of RUBI paired with behavioral rehearsal, prospective think-aloud, and structured assessment methods, Aim 2 will allow us to determine which RUBI components should be retained, eliminated, or modified to address the needs of the context and end-users identified in Aim 1.

Aim 3: Iteratively adapt RUBI content and procedures based on identified targets for redesign and continual user testing to improve the hypothesized mechanisms of redesign (usability) and perceptual implementation outcomes (feasibility, acceptability, and appropriateness). Aim 3 will allow us to determine the extent to which redesigned RUBI, or RUBIES, improves usability and implementation outcomes.

• Study Type: Observational
• Enrollment (Actual): 40

Contacts and Locations

Study Locations

• United States
  • Washington
    • Seattle, Washington, United States, 98105
      • Seattle Public Schools

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: Child; Adult; Older Adult
• Accepts Healthy Volunteers: No
• Genders Eligible for Study: All

• Sampling Method: Non-Probability Sample

Study Population

Elementary school personnel who work with students with autism spectrum disorder

Description

Inclusion Criteria:

• Elementary school (kindergarten through 5th grade) personnel (i.e. general and special education teachers, school psychologist, paraprofessionals)
• Works with at least one student with ASD for part of the day

Exclusion Criteria:

-School personnel who do not have contact with youth with ASD during the work day

Study Plan

How is the study designed?

Design Details

• Observational Models: Cohort
• Time Perspectives: Retrospective

Number of groups / cohorts

4

Cohorts and Interventions

Group / Cohort	Intervention / Treatment
Classroom Observation; The investigators will use day-long observational and interview procedures with eight staff from four schools. The classroom observation will focus on documenting episodes of classroom disruptive behavior, including antecedents and consequences to the behaviors. The interview will involve discussing with staff decision-making processes and current needs around classroom behavioral management.
RUBI Redesign; Two separate demonstration studies comprising 6 staff members each will focus on informing adaptation or pruning needs related to RUBI content and structure to ensure the redesigned curriculum (RUBIES) is contextually appropriate for schools.	Behavioral: RUBI; Parent-mediated behavioral program targeting disruptive behavior utilizing principle of applied behavior analysis
RUBIES Collaborative Design; Eight staff from 4 schools will attend one of four 2-hour in-person feedback sessions to support collaborative feedback around RUBI redesign, including feasibility and appropriateness and methods supporting implementation.	Behavioral: RUBIES; School-staff mediated behavioral program targeting disruptive behavior utilizing principle of applied behavior analysis
RUBIES Redesign; Two separate demonstration studies comprising 6 staff members each will focus on informing final RUBIES adaptation or pruning needs.	Behavioral: RUBIES; School-staff mediated behavioral program targeting disruptive behavior utilizing principle of applied behavior analysis

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Acceptability of Intervention Measure (AIM): 5-point scale	The 4-item AIM assesses participants' acceptability of RUBI(ES). Raters score items on a 5-point scale ranging from (1) "Completely Disagree" to (5) "Completely Agree". Internal consistency (α = .89) and test-retest reliability were good (α = .83).	Completed by participants during the 4-hour RUBI Redesign meeting (Aim 1)
Acceptability of Intervention Measure (AIM): 5-point scale	The 4-item AIM assesses participants' acceptability of RUBI(ES). Raters score items on a 5-point scale ranging from (1) "Completely Disagree" to (5) "Completely Agree". Internal consistency (α = .89) and test-retest reliability were good (α = .83).	Completed by participants during the 4-hour RUBIES Redesign meeting (Aim 3)
Intervention Appropriateness Measure (IAM): 5-point scale	The 4-item IAM assesses the appropriateness of RUBI(ES). Raters score each item on a 5-point scale ranging from (1) "Completely Disagree" to (5) "Completely Agree". Internal consistency (α = .87) and test-retest reliability were good (α = .87).	Completed by participants during the 4-hour RUBI Redesign meeting (Aim 1)
Intervention Appropriateness Measure (IAM): 5-point scale	The 4-item IAM assesses the appropriateness of RUBI(ES). Raters score each item on a 5-point scale ranging from (1) "Completely Disagree" to (5) "Completely Agree". Internal consistency (α = .87) and test-retest reliability were good (α = .87).	Completed by participants during the 4-hour RUBIES Redesign meeting (Aim 3)
Feasibility of Intervention Measure (FIM): 5-point scale	The 4-item FIM assesses the feasibility of RUBI(ES). Raters score each item on a 5-point scale ranging from (1) "Completely Disagree" to (5) "Completely Agree". Internal consistency (α = .89) and test-retest reliability were good (α = .88).	Completed by participants during the 4-hour RUBI Redesign meeting (Aim 1)
Feasibility of Intervention Measure (FIM): 5-point scale	The 4-item FIM assesses the feasibility of RUBI(ES). Raters score each item on a 5-point scale ranging from (1) "Completely Disagree" to (5) "Completely Agree". Internal consistency (α = .89) and test-retest reliability were good (α = .88).	Completed by participants during the 4-hour RUBIES Redesign meeting (Aim 3)

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Intervention Usability Scale (IUS)	The 10-item IUS is adapted from the System Usability Scale and assesses intervention usability on a 5-point scale ranging from (1) "Strongly Disagree" to (5) "Strongly Agree." A score ≥70 on the IUS demonstrates "good" or better usability.	Completed by participants during the 4-hour RUBI Redesign meeting (Aim 1)
Intervention Usability Scale (IUS)	The 10-item IUS is adapted from the System Usability Scale and assesses intervention usability on a 5-point scale ranging from (1) "Strongly Disagree" to (5) "Strongly Agree." A score ≥70 on the IUS demonstrates "good" or better usability.	Completed by participants during the 4-hour RUBIES Redesign meeting (Aim 3)

Other Outcome Measures

Outcome Measure	Measure Description	Time Frame
Teacher Demographics	Measures individual characteristics (age, sex, gender identity, race/ethnicity, years' experience, etc.).	Completed by participants at the in-class behavior observation, 4-hour RUBI Redesign meeting, 2 hour RUBIES Collaborative design meeting, or 4 hour RUBIES Redesign meeting
School Demographics	School Demographics (free/reduced price lunch rate, school size, number of students with disabilities, resources, services, etc.) will be collected through the Office of the Superintendent of Public Instruction and school records.	Collected over the course of the 12 month study through public record review of schools where participants are employed

Collaborators and Investigators

• Sponsor: University of Washington
• Collaborators: National Institute of Mental Health (NIMH)
• Investigators: Principal Investigator: Karen Bearss, PhD, University of Washington

Study record dates

Study Major Dates

• Study Start (Actual): September 1, 2019
• Primary Completion (Actual): June 30, 2021
• Study Completion (Actual): September 30, 2021

Study Registration Dates

• First Submitted: October 9, 2019
• First Submitted That Met QC Criteria: December 13, 2019
• First Posted (Actual): December 16, 2019

Study Record Updates

• Last Update Posted (Actual): May 20, 2022
• Last Update Submitted That Met QC Criteria: May 18, 2022
• Last Verified: May 1, 2022

More Information

Terms related to this study

• Additional Relevant MeSH Terms: Behavioral Symptoms; Mental Disorders; Neurodevelopmental Disorders; Child Development Disorders, Pervasive; Problem Behavior; Autism Spectrum Disorder
• Other Study ID Numbers: SCH-STUDY00001890; 1P50MH115837 (U.S. NIH Grant/Contract)

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No