Organizational Skills Training

Evaluation of Organizational Skills Training (OST) Program for Upper Elementary Students

The purpose of this study is to evaluate the Organizational Skills Training Program - School based version (OST-S) in a cluster randomized trial. Participating schools will be randomly assigned to one of two groups: 1) an OST-S intervention group and 2) a treatment as usual control group. Participating students (3rd through 5th grade) will be from at least 20 schools in Pennsylvania and New Jersey and include both urban and suburban schools serving a diverse population. OST-S is a small-group, skills training intervention for children, which includes parent and teacher consultation to support student use of new skills. The program is delivered by school staff who receive training and consultation from intervention experts.

Study Overview

• Status: Completed
• Conditions: Executive Function; Schools; Behavior Therapy; Behavioral Research
• Intervention / Treatment: Behavioral: Organizational Skills Training - School Based (OST-S)

Detailed Description

Context: Although multiple factors influence school functioning, executive function (EF) deficits have been found to be a key predictor of academic achievement. EF is a higher order cognitive ability associated with persistent goal-directed behavior. Organization, time management, and planning (OTMP) skills are aspects of EF that are particularly associated with children's academic performance. Organizational demands increase over the course of early schooling and are relatively high by 3rd through 5th grade. Poor OTMP skills during this period adversely impact academic functioning. In the late elementary school grades, as students are expected to become more organized, some students have difficulty learning these skills in spite of classroom supports provided by teachers, placing them at increased risk for academic failure.

Objectives: The purpose of this Goal 3 efficacy study is to conduct an evaluation of the Organizational Skills Training Program - School version (OST-S), a fully developed intervention for students in general education. The proposed study builds upon research demonstrating the efficacy of a clinic-based version of the OST intervention (OST-C) in remediating OTMP skills deficits and improving academic functioning for 3rd, 4th and 5th graders with attention-deficit hyperactivity disorder (ADHD), and a recent pilot research demonstrating the feasibility and potential effectiveness of OST-S provided by end users ("school partners") for 3rd through 5th graders. It also builds upon research training school staff to implement evidence-based interventions with high fidelity.

Study Design: This is a cluster-randomized trial with a treatment as usual (TAU) control group.

Setting/Participants: Schools are located in Pennsylvania and New Jersey and include at least 20 urban and suburban schools serving a diverse population. Students (3rd to 5th grade) who are struggling the most with OTMP deficits and whose academic performance is negatively impacted by their OTMP deficits will be referred to the study team by their general education teachers.

Study Interventions and Measures: OST-S is a small group skills training intervention, with parents and teachers supporting children's use of new skills. The program manual includes strategies for training and coaching school staff, referred to as school partners, to effectively implement OST-S and guidelines to modify the program for implementation in diverse schools with diverse students. Each student session includes: (a) homework review to assess completion of between-session skills implementation; (b) skill-building activities, which include the use of modeling, shaping, guided practice, and reinforcement for organized behavior; and (c) activities to promote generalization of skills. Sessions address four organizational challenges: (a) tracking assignments, (b) managing materials, (c) managing time, and (d) planning for long term assignments.

Investigators will measure the following: intervention fidelity, stakeholder engagement, student OTMP skills, student academic self-efficacy, student academic outcomes, student characteristics, feasibility, usability, and acceptability of OST-S. In addition, investigators will track the interventions that are offered as treatment-as-usual in TAU schools. A cost analysis related to the implementation of OST-S will also be completed.

• Study Type: Interventional
• Enrollment (Actual): 381
• Phase: Not Applicable

Contacts and Locations

Study Locations

• United States
  • New York
    • New York, New York, United States, 10016
      • New York University
  • Pennsylvania
    • Philadelphia, Pennsylvania, United States, 19104
      • University of Pennsylvania
    • Philadelphia, Pennsylvania, United States, 19146
      • Childrens Hospital of Philadelphia

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: Child; Adult; Older Adult
• Accepts Healthy Volunteers: No

Description

Inclusion Criteria:

Any student enrolled in grades 3 thru 5 in one of the participating schools who meets the following criteria.

1. First, students will be recommended by their general education teacher if the following are true: (a) students who are struggling the most with OTMP skills, (b) whose OTMP skill deficits are the students' primary concern (c) whose academic performance is negatively impacted by their OTMP deficits, and (d) who have at least one parent who speaks English.
2. Second, students must have or be at risk for developing OTMP deficits.

Exclusion Criteria:

1. Students will be excluded if they are in a pull-out special education classroom for more than 50% of the day as the organizational demands for these students may differ from those students placed mostly in general education.
2. Students with a one-to-one aide will be excluded because the presence of an aide substantially alters how an organizational intervention is implemented.
3. Students from families in which both caregivers do not speak English will be excluded because the program has not yet been developed for non-English speakers.
4. Only one student per family will be included in the study.

Study Plan

How is the study designed?

Design Details

• Primary Purpose: Treatment
• Allocation: Randomized
• Interventional Model: Parallel Assignment
• Masking: None (Open Label)

Number of Arms

2

Arms and Interventions

Participant Group / Arm	Intervention / Treatment
Experimental: OST-S Intervention group	Behavioral: Organizational Skills Training - School Based (OST-S); OST-S is a small group skills training intervention, with parents and teachers supporting children's use of new skills. The program manual includes strategies for training and coaching school staff, referred to as school partners, to effectively implement OST-S and guidelines to modify the program for implementation in diverse schools with diverse students. Each student session includes: (a) homework review to assess completion of between-session skills implementation; (b) skill-building activities, which include the use of modeling, shaping, guided practice, and reinforcement for organized behavior; and (c) activities to promote generalization of skills. Sessions address four organizational challenges: (a) tracking assignments, (b) managing materials, (c) managing time, and (d) planning for long term assignments.
No Intervention: Treatment as usual control group

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Changes in Students' Organizational Functioning	The Children's Organizational Skills Scales (COSS) will be used to assess changes in OTMP functioning at home and school over several time points. COSS total scores have good discriminant validity and are sensitive to treatment effects. Each COSS version uses a 4-point rating scale (1=Hardly ever or never to 4=Just about all of the time). Although the COSS yields three subscale scores, only the total score will be used to reduce the number of measures in the analyses.	Baseline, 10-weeks (post-treatment), 5-months and 12-months
Changes in Students' Patterns of Adaptive Learning	Patterns of Adaptive Learning Scales (PALS) will be used to evaluate changes in students' perceptions of their competence in completing their classwork. Items are rated on a 5-item scale from 1 = not at all true to 5 = very true.	Baseline, 10-weeks (post-treatment), 5-months and 12-months

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Changes in Student's Academic Progress	The Academic Progress Report (APR) is a teacher-report measure that assesses (changes in) proficiency in seven academic subjects relative to standard expectations (1=Well below standard expected at this time of year; 3=At standard; 5=Well above standard). The sum of ratings across seven academic subjects is the unit of analysis. Reliability is acceptable (alpha = .84), and this measure is sensitive to OST treatment effects.	Baseline, 10-weeks (post-treatment), 5-months and 12-months
Students' Homework Performance	The Homework Performance Questionnaire (HPQ-T)-Teacher version will be used to assess changes in students' homework behavior during the past 4 weeks. Each item is rated on a five-point scale. The 9-item Student Self-Regulation factor will be used in the analyses. This factor has been demonstrated to have strong psychometric properties.	Baseline, 10-weeks (post-treatment), 5-months and 12-months
Changes in Academic Competence of Students	The Academic Competence Evaluation Scales (ACES) is a teacher-report scale that assesses changes in the academic competence of students in kindergarten through grade 12. The Reading/Language Arts and Math subscales of this measure will be used. Alpha coefficients and test-retest correlations for these subscales have been shown to be above .90. The average of these subscale scores will be used in the analyses.	Baseline, 10-weeks (post-treatment), 5-months and 12-months
Academic Grades of Students	Student report card grades for the marking period prior to students' participation in OST-S/TAU and for the marking period immediately after the intervention and follow-up periods. For students enrolled before the first marking period is over, baseline grades will be those obtained at the end of the previous school year.	Baseline, 10-weeks (post-treatment), 5-months and 12-months
Parent's Report of Changes in their Child's Homework Performance	The Homework Problem Checklist (HPC) is a 20-item parent-report measure that will be used to assess changes in student homework performance. The psychometric properties of this instrument have been shown to be acceptable, and the HPC was sensitive to treatment effects in the previous OST-C study. The total HPC score will be used in the analyses.	Baseline, 10-weeks (post-treatment), 5-months and 12-months
Changes in Participants' views of Treatment Acceptability	Measures of Acceptability of OST-S will be assessed using the following measure: The Treatment Evaluation Inventory - Short Form (TEI-SF), a 9-item measure, will be used to assess parents' views of treatment acceptability.	up to 10-weeks (post treatment)
Changes in Participants' views of Treatment Usability	The Usage Rating Profile-Intervention-Revised (URP-IR), a 29-item measure, will be adapted (with permission) to assess factors that may influence the usage of the OST-S intervention in a school-based setting.	up to 10-weeks (post treatment)
Changes in Participants' Views of Treatment Feasibility	The Usage Rating Profile-Intervention-Revised (URP-IR), a 29-item measure, will be adapted (with permission) to assess feasibility of usage of the OST-S intervention in a school-based setting.	up to 10-weeks (post treatment)
Changes in Participants' Views of Treatment Acceptability	The Children's Intervention Rating Profile (CIRP), a 7-item measure, will be adapted to assess student's perceptions of acceptability.	up to 10-weeks (post treatment)

Collaborators and Investigators

• Sponsor: Children's Hospital of Philadelphia
• Collaborators: University of Pennsylvania; New York University; U.S. Department of Education
• Investigators: Principal Investigator: Thomas Power, PhD, Children's Hospital of Philadelphia; Principal Investigator: Jenelle Nissley-Tsiopoinis, PhD, Children's Hospital of Philadelphia; Principal Investigator: Jennifer Mautone, PhD, Childrens Hospital of Philadelphia

Publications and helpful links

General Publications

• Abikoff H, Gallagher R, Wells KC, Murray DW, Huang L, Lu F, Petkova E. Remediating organizational functioning in children with ADHD: immediate and long-term effects from a randomized controlled trial. J Consult Clin Psychol. 2013 Feb;81(1):113-28. doi: 10.1037/a0029648. Epub 2012 Aug 13.
• Best JR, Miller PH, Naglieri JA. Relations between Executive Function and Academic Achievement from Ages 5 to 17 in a Large, Representative National Sample. Learn Individ Differ. 2011 Aug;21(4):327-336. doi: 10.1016/j.lindif.2011.01.007.
• Best JR, Miller PH, Jones LL. Executive Functions after Age 5: Changes and Correlates. Dev Rev. 2009 Sep 1;29(3):180-200. doi: 10.1016/j.dr.2009.05.002.
• Abikoff, H., & Gallagher, R. (2009). Children's organizational skills scales (COSS), technical manual. North Tonawanda, NY: Multi-Health Systems.
• Eiraldi R, McCurdy B, Khanna M, Mautone J, Jawad AF, Power T, Cidav Z, Cacia J, Sugai G. A cluster randomized trial to evaluate external support for the implementation of positive behavioral interventions and supports by school personnel. Implement Sci. 2014 Jan 15;9:12. doi: 10.1186/1748-5908-9-12.
• Langberg JM, Dvorsky MR, Evans SW. What specific facets of executive function are associated with academic functioning in youth with attention-deficit/hyperactivity disorder? J Abnorm Child Psychol. 2013 Oct;41(7):1145-59. doi: 10.1007/s10802-013-9750-z.

Helpful Links

• Behavioral Research - Childrens Hospital of Philadelphia Research Institute

Study record dates

Study Major Dates

• Study Start (Actual): November 8, 2017
• Primary Completion (Actual): March 31, 2023
• Study Completion (Actual): March 31, 2023

Study Registration Dates

• First Submitted: November 28, 2017
• First Submitted That Met QC Criteria: February 21, 2018
• First Posted (Actual): February 23, 2018

Study Record Updates

• Last Update Posted (Actual): June 8, 2023
• Last Update Submitted That Met QC Criteria: June 7, 2023
• Last Verified: June 1, 2023

More Information

Terms related to this study

• Keywords: Randomized Controlled Trial; Executive Function; Schools; Behavior Therapy; Students; Cost Analysis; Behavioral Research
• Other Study ID Numbers: 17-014227; R305A170052-20 (Other Grant/Funding Number: US Dept. of Education)

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: YES
• IPD Plan Description: Per Institute for Educational Sciences (IES) requirements, the timeframe for accessibility of data will be reviewed during each annual progress review and revised as necessary. The final dataset will be exported from REDCap to Microsoft Excel and commonly used statistical software packages such as SPSS and SAS, as requested by the individuals seeking access to the data. The research team will prepare a codebook to share with the requesting individuals. The codebook will provide information about all of the variables and how the composite variables (e.g. factor scores) were derived.
• IPD Sharing Time Frame: Data will become accessible after the study is completed and primary study findings are published in a peer-reviewed journal. Data will be available for 10 years from that time. Per Institute for Educational Sciences (IES) requirements, the timeframe for accessibility of data will be reviewed during each annual progress review and revised as necessary.
• IPD Sharing Access Criteria: Individuals interested in using the data should contact Dr. Thomas Power (power@chop.edu). When our research team receives a request for access to the data a data file and supporting documentation will be prepared by the research team. Only de-identified data will be made accessible in order to prevent identification of students, parents, and school staff who participated in this project. Therefore, per Children's Hospital of Philadelphia (CHOP) policies, a data licensing agreement is anticipated to not be necessary.
• IPD Sharing Supporting Information Type: STUDY_PROTOCOL; ICF

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No