Predict the Best Level of Care Placement for Each Child's Behavioral Health Needs - Efficacy Study

Placement Success Predictor: Using Site-Customized Machine Learning Models to Predict the Best Level of Care Placement for Each Child's Behavioral Health Needs

The purpose of this randomized clinical trial is to test the efficacy of a new clinical decision support tool, Placement Success Predictor (PSP). PSP will provide placement-specific predictions about the likelihood of a youth having a good outcome in each placement type using machine learning algorithms.

The primary hypothesis is that if clinical team members have access to PSP results for youth in the experimental group, these youth will have better outcomes at the 3-month follow-up compared to youth in the control group.

Study Overview

• Status: Enrolling by invitation
• Conditions: Mental Health Wellness; Adolescent Well-Being
• Intervention / Treatment: Other: Clinical team access to Placement Success Predictor (PSP) results

Detailed Description

In 2017, a total of 669,799 children were confirmed victims of maltreatment in the United States; of the 442,733 children in foster care, 34% have been in more than one placement and 11% are in a group home or institution. Stakes are extremely high for making the best out-of-home placement choice per child because some placement types and multiple placements are associated with poor outcomes. In the past few years, legislation has been created to guide placement decisions for children. Federal law 42 U.S. Code 675 requires that children in the care of the state are placed "in a safe setting that is the least restrictive (most family like)." In addition, the Family First Prevention Services Act signed into law by the U.S. Congress in 2018 includes measures to reduce the number of children in long-term residential settings. This study is an effort to develop and test a science-based clinical decision support tool using behavioral health data collected through standard clinical practice.

A randomized controlled trial (RCT) design will be used to assess efficacy of clinical team access to Placement Success Predictor (PSP) on child welfare clients' well-being outcomes and healthcare costs.

Sample. Clients at the State of Iowa Department of Health and Human Services (Iowa HHS) are the sample for this efficacy study.

Randomization. The outcome of a single coin toss was applied to an undisclosed algorithm for the client's record number to determine who gets assigned to the experimental group (i.e., client has PSP results).

Methods. The Treatment Outcome Package (TOP), a behavioral health assessment, is a standard part of care delivered in Iowa and its completion is required by the state. Iowa HHS clinical teams will be provided PSP results for clients in the experimental condition. A request for a waiver of consent for this study was approved by the WCG IRB.

• Study Type: Interventional
• Enrollment (Estimated): 300
• Phase: Not Applicable

Contacts and Locations

Study Locations

• United States
  • Massachusetts
    • Framingham, Massachusetts, United States, 01701
      • Outcome Referrals, Inc.

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: Child; Adult; Older Adult
• Accepts Healthy Volunteers: No

Description

Inclusion Criteria:

• Completed TOP CS assessment

Exclusion Criteria:

• None

Study Plan

How is the study designed?

Design Details

• Primary Purpose: Health Services Research
• Allocation: Randomized
• Interventional Model: Parallel Assignment
• Masking: None (Open Label)

Number of Arms

2

Arms and Interventions

Participant Group / Arm	Intervention / Treatment
Experimental: Access to PSP site-specific placement prediction scores for that youth; The PSP system will provide site-specific placement success prediction scores [i.e., client's likelihood of success per placement based on machine learning models] for each youth randomized to this condition in the efficacy study.	Other: Clinical team access to Placement Success Predictor (PSP) results; PSP is a machine-learning based clinical decision support tool that is designed to assist clinical team members in making placement decisions for youth.
No Intervention: No PSP site-specific placement prediction scores for that youth

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Mean difference on average z-scores across raters within two weeks on the Clinical Scale of the Treatment Outcome Package (TOP-CS) between a) the beginning of the study (between February and October 2025) and b) approximately 3 months later.	TOP is a comprehensive well-being assessment that is used in behavioral health and child welfare settings. The Child TOP Clinical Scale (TOP-CS) is a 58-item scale for children (ages 3 - 18) that assesses 13 domains. The Adolescent TOP Clinical Scale (TOP-CS) is a 48-item scale for adolescents (ages 11 - 21) that assesses 12 domains. TOP-CS assesses the client's past 2-week experience on domains including Depression, Violence, and Suicidality (scores are risk-adjusted for case mix variables assessed via 37 items on the companion TOP-Case Mix form regarding stressful life events, comorbidity). Participants answer "All" to "None of the Time" for each item on a 6-point Likert scale. The z-scores (standard deviation units relative to the general population mean for each domain) will be averaged together to create one summary score. Higher scores suggest higher severity/lower behavioral well-being.	At baseline and in 3 months

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Total cost of treatment	Estimated treatment cost per placement per participant will be assessed by calculating the number of days in treatment times cost per day	3 months
Mean difference on average z-scores across raters within two weeks on the TOP-CS between a) the PREDICTED [i.e., risk-adjusted] scores at beginning of the study and b) the ACTUAL follow up scores approximately 3 months later.	See Outcome description 1.	At baseline and in 3 months

Collaborators and Investigators

• Sponsor: Outcome Referrals, Inc.
• Collaborators: National Institute of Mental Health (NIMH); State of Iowa Department of Health and Human Services

Publications and helpful links

General Publications

• Kraus DR, Seligman DA, Jordan JR. Validation of a behavioral health treatment outcome and assessment tool designed for naturalistic settings: The Treatment Outcome Package. J Clin Psychol. 2005 Mar;61(3):285-314. doi: 10.1002/jclp.20084.
• Trudeau KJ, Yang J, Di J, Lu Y, Kraus DR. Predicting Successful Placements for Youth in Child Welfare with Machine Learning. Child Youth Serv Rev. 2023 Oct;153:107117. doi: 10.1016/j.childyouth.2023.107117. Epub 2023 Aug 4.

Study record dates

Study Major Dates

• Study Start (Actual): February 3, 2025
• Primary Completion (Estimated): December 31, 2025
• Study Completion (Estimated): December 31, 2025

Study Registration Dates

• First Submitted: January 28, 2025
• First Submitted That Met QC Criteria: February 3, 2025
• First Posted (Actual): March 25, 2025

Study Record Updates

• Last Update Posted (Actual): March 25, 2025
• Last Update Submitted That Met QC Criteria: February 13, 2025
• Last Verified: February 1, 2025

More Information

Terms related to this study

• Other Study ID Numbers: 2R44MH125486-02A1-Aim 2A; 2R44MH125486-02A1 (U.S. NIH Grant/Contract)

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: NO

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No
• product manufactured in and exported from the U.S.: No