Remediating Academic Skill Deficits Among Disadvantaged Youth

This research aims to continue to study the effectiveness of a promising academic intervention (implemented by SAGA Innovations) that has previously been shown to significantly improve academic outcomes for disadvantaged youth. In addition, this study will begin to investigate the effects of scaling up this promising strategy by exploring variation in tutor effectiveness and the optimal instructor-student and student-student pairings for improving academic outcomes.

Study Overview

• Status: Unknown
• Conditions: Educational Achievement
• Intervention / Treatment: Other: SAGA Innovations

Detailed Description

The University of Chicago Education Lab research team is carrying out a randomized controlled trial of a promising academic intervention during the 2015-16 academic year in partnership with the Chicago Public Schools (CPS) and SAGA Innovations. Male and female CPS students in grades 9 and 10 will be randomly assigned either to receive what investigators believe to be a best-practice intensive academic support, or to a control group receiving status quo CPS and community services, for one academic year (AY2015-16). The intervention is high-dosage math tutoring provided by SAGA Innovations (previously Match Education of Boston). A previous randomized controlled trial conducted by the University of Chicago research team found that one year of this intervention, delivered in AY2013-14, generated between one and two extra years of academic growth in math, over and above what the normal U.S. high school student learns in one year. The estimated effects for math achievement are on the order of 0.19 to 0.30 SD, depending on the exact test and norming used. The intervention also improved student grades in math, by 0.58 points on a 1-4 grade scale, compared to a control mean of 1.77. These gains are particularly important because math success versus failure is a strong predictor of high school graduation.

This current study aims to replicate the investigators' previous findings, and to that end the research team will again look at the academic, behavioral, and long-term effects of this high-dosage math tutoring program on youth. This study is also designed to explore issues that will be central to efforts to scale-up this promising strategy, including variation in tutor quality and whether there are optimal tutor-student and student-student pairings in terms of gender and race.

The SAGA Innovations program expands on the nationally recognized innovation of high-dosage, in-school-day tutoring developed in Match Education's charter school in Boston. The tutoring program meets as a scheduled course, Math Lab, once a day during the normal school day, and is provided in addition to a student's regular math class. Students taking the course receive an elective credit upon completion. Every student works with the same full-time, professional tutor for the entirety of the school year. The content of the tutoring sessions is aligned with what students are learning in their regular math courses, but is also targeted to address individual gaps in math knowledge. Also following the original model developed by Match Education, SAGA tutors use frequent internal formative assessments of student progress to individualize instruction.

In addition to replicating previous studies that suggest the promise of this high-dosage tutoring model for improving the academic outcomes of at-risk youth, this study also aims to provide insight into the ability of this program to serve youth at a much larger scale. Despite the great need for programs that can affect the national dropout crisis and improve youth outcomes, little is known about how to take promising education interventions to scale. This study will begin to explore whether there is a trade-off between effectiveness and scale by randomly assigning students to pairings and randomly assigning pairings to tutors. Tutors will be separately ranked from highest to lowest quality by SAGA leadership, and by randomly assigning tutors to students, the investigators will be able to explore what effect, if any, tutor quality has on student outcomes. In addition, this study will look at whether gender and race composition of student-tutor pairings and student-student pairings has an effect on outcomes. This work will enable the investigators to begin to learn about variation in tutor effectiveness and the optimal way to match kids to tutors. The research team hopes this work will have important implications for how to scale this promising strategy both within Chicago and beyond.

• Study Type: Interventional
• Enrollment (Anticipated): 1848
• Phase: Not Applicable

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: Child; Adult; Older Adult
• Accepts Healthy Volunteers: No
• Genders Eligible for Study: All

Description

Inclusion Criteria:

• Low-performing Chicago Public School high schools, chosen in collaboration with the Chicago Public Schools based on criteria such as dropout rate, test scores, scores on academic rating scale, etc.
• School administrators are enthusiastic about the program and agree to terms and conditions of the experimental design
• Male and female youth within these schools who are rising 9th and 10th graders in academic year (AY) 2015-16

Exclusion Criteria:

• Youth who have missed >60% of days during AY2014-15 (through March), and so would not be expected to show up in school enough during intervention year (AY2015-16) to benefit from school-based programming
• Youth who have failed >75% of classes during AY2014-15 (through March)
• Youth who have Individualized Education Program (IEP) designations for autism, "educable mentally handicapped," and/or traumatic brain injury

Study Plan

How is the study designed?

Design Details

• Primary Purpose: Prevention
• Allocation: Randomized
• Interventional Model: Parallel Assignment
• Masking: None (Open Label)

Number of Arms

2

Arms and Interventions

Participant Group / Arm	Intervention / Treatment
No Intervention: Control group; These youth will receive standard mathematics instruction and support, but not the intensive tutoring offered through the intervention.
Experimental: SAGA Innovations; These youth will receive the intensive mathematics tutoring by SAGA Innovations, with students randomized to tutors.	Other: SAGA Innovations; An intensive math tutoring program.

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Math achievement	Performance on math standardized achievement test scores	1-year
Absentee rate	Number of school absences, obtained from Chicago Public Schools (CPS) administrative database	1-year
Student misconduct	Number of school misconduct infractions, obtained from Chicago Public Schools administrative database	1-year
Total courses failed	Number of total school courses failed, obtained from Chicago Public Schools administrative database	1-year
Math courses failed	Number of math courses failed, obtained from Chicago Public Schools administrative database	1-year
Non-math courses failed	Number of non-math courses failed, obtained from Chicago Public Schools administrative database	1-year
Math course grades	Math course grades, obtained from Chicago Public Schools administrative database	1-year
School persistence	Measure from CPS student records of school persistence (enrollment or graduation status by end of academic year)	1-year

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Violent crime arrests	Number of violent crime arrests, obtained from Chicago Police Department and Illinois State Police administrative databases	1-year
Other arrests (property, drug, and other crimes)	Number of non-violent crime arrests, including property crimes, drug crimes, and other crimes, obtained from Chicago Police Department and Illinois State Police administrative databases	1-year
Quarterly earnings data	Quarterly earnings collected by the Illinois Department of Employment Security, maintained for the state unemployment insurance system	1-year

Collaborators and Investigators

• Sponsor: University of Chicago
• Collaborators: Northwestern University; Chicago Public Schools; Crown Family Philanthropies; Paul M. Angell Family Foundation; Polk Bros. Foundation; SAGA Innovations; Lloyd A. Fry Foundation
• Investigators: Principal Investigator: Jonathan Guryan, PhD, Northwestern University; Principal Investigator: Kelly Hallberg, PhD, University of Chicago

Publications and helpful links

General Publications

• Cook P, Dodge K, Farkas G, Fryer RG, Guryan J, Ludwig J, Mayer S, Pollack H, Steinberg L. Not Too Late: Improving Academic Outcomes for Disadvantaged Youth. Northwestern Institute for Policy Research Working Paper, February 2015.
• Cook P, Dodge K, Farkas G, Fryer RG, Guryan J, Ludwig J, Mayer S, Pollack H, Steinberg L. The (Surprising) Efficacy of Academic and Behavioral Intervention with Disadvantaged Youth: Results from a Randomized Experiment in Chicago. Cambridge, MA: National Bureau of Economic Research, Working Paper No. 19862, 2014.
• Fryer RG. Injecting Charter School Best Practices into Traditional Public Schools: Evidence from Field Experiments. The Quarterly Journal of Economics 129(3): 1355-1407, 2014.

Study record dates

Study Major Dates

• Study Start: August 1, 2015
• Primary Completion (Anticipated): June 1, 2018
• Study Completion (Anticipated): June 1, 2019

Study Registration Dates

• First Submitted: January 28, 2016
• First Submitted That Met QC Criteria: February 1, 2016
• First Posted (Estimate): February 4, 2016

Study Record Updates

• Last Update Posted (Actual): July 11, 2017
• Last Update Submitted That Met QC Criteria: July 7, 2017
• Last Verified: July 1, 2017

More Information

Terms related to this study

• Keywords: Educational Achievement
• Other Study ID Numbers: SBS IRB15-0711