Predict the Best Level of Care Placement for Each Child's Behavioral Health Needs - Effectiveness Study

Placement Success Predictor: Using Site-Customized Machine Learning Models to Predict the Best Level of Care Placement for Each Child's Behavioral Health Needs

The purpose of this study is to test the effectiveness of a new clinical decision support tool, Placement Success Predictor (PSP), in a naturalistic setting. PSP will provide placement-specific predictions about the likelihood of a youth having a good outcome in each placement type at a behavioral health center using machine learning algorithms.

The primary hypothesis is that clients in at least one placement within one standard deviation of the placement with the highest predicted likelihood of success will have better outcomes than the clients who were not.

The secondary hypothesis is that clients' level of improvement over time will be positively correlated with the number of days they are in at least one placement within one standard deviation of the placement with the highest predicted likelihood of success.

Study Overview

• Status: Enrolling by invitation
• Conditions: Mental Health Wellness; Adolescent Well-Being
• Intervention / Treatment: Other: Clinical team access to Placement Success Predictor (PSP) results

Detailed Description

In 2017, a total of 669,799 children were confirmed victims of maltreatment in the United States; of the 442,733 children in foster care, 34% have been in more than one placement and 11% are in a group home or institution. Stakes are extremely high for making the best out-of-home placement choice per child because some placement types and multiple placements are associated with poor outcomes. In the past few years, legislation has been created to guide placement decisions for children. Federal law 42 U.S. Code 675 requires that children in the care of the state are placed "in a safe setting that is the least restrictive (most family like)." In addition, the Family First Prevention Services Act signed into law by the U.S. Congress in 2018 includes measures to reduce the number of children in long-term residential settings. This effectiveness study is to assess and improve the usage of PSP in a behavioral health setting.

Sample. Clients at Children's Hope Alliance (CHA) who completed the TOP, CHA's standard behavioral health assessment. The target recruitment goal is 700 clients.

Methods. PSP results will be available for all clients with recent behavioral health assessment data.

• Study Type: Observational
• Enrollment (Estimated): 700

Contacts and Locations

Study Locations

• United States
  • Massachusetts
    • Framingham, Massachusetts, United States, 01701
      • Outcome Referrals, Inc.

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: Child; Adult; Older Adult
• Accepts Healthy Volunteers: No

• Sampling Method: Non-Probability Sample

Study Population

Clients at Children's Hope Alliance

Description

Inclusion Criteria:

• Completed TOP CS assessment

Exclusion Criteria:

• None

Study Plan

How is the study designed?

Number of groups / cohorts

2

Cohorts and Interventions

Group / Cohort	Intervention / Treatment
In PSP-Recommended Placement; Clients in placement with PSP results within one standard deviation of the highest predicted likelihood of success for that client at follow up	Other: Clinical team access to Placement Success Predictor (PSP) results; PSP is a machine-learning based clinical decision support tool that is designed to assist clinical team members in making placement decisions for youth. PSP provides site-specific placement success prediction scores [i.e., client's likelihood of success per placement based on machine learning models] for each youth.
Not In PSP-Recommended Placement; Clients not in placement with PSP result within one standard deviation of the highest predicted likelihood of success for that client at follow up	Other: Clinical team access to Placement Success Predictor (PSP) results; PSP is a machine-learning based clinical decision support tool that is designed to assist clinical team members in making placement decisions for youth. PSP provides site-specific placement success prediction scores [i.e., client's likelihood of success per placement based on machine learning models] for each youth.

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Mean difference on average z-scores across raters within two weeks on the Clinical Scale of the Treatment Outcome Package (TOP-CS) between a) the beginning of the study (between February and October 2025) and b) approximately 3 months later.	TOP is a comprehensive well-being assessment that is used in behavioral health and child welfare settings. The Child TOP Clinical Scale (TOP-CS) is a 58-item scale for children (ages 3 - 18) that assesses 13 domains. The Adolescent TOP Clinical Scale (TOP-CS) is a 48-item scale for adolescents (ages 11 - 21) that assesses 12 domains. TOP-CS assesses the client's past 2-week experience on domains including Depression, Violence, and Suicidality (scores are risk-adjusted for case mix variables assessed via 37 items on the companion TOP-Case Mix form regarding stressful life events, comorbidity). Participants answer "All" to "None of the Time" for each item on a 6-point Likert scale. The z-scores (standard deviation units relative to the general population mean for each domain) will be averaged together to create one summary score. Higher scores suggest higher severity/lower behavioral well-being.	At baseline and every 30 days up to approximately 3 months

Collaborators and Investigators

• Sponsor: Outcome Referrals, Inc.
• Collaborators: National Institutes of Health (NIH); Children's Hope Alliance

Publications and helpful links

General Publications

• Kraus DR, Seligman DA, Jordan JR. Validation of a behavioral health treatment outcome and assessment tool designed for naturalistic settings: The Treatment Outcome Package. J Clin Psychol. 2005 Mar;61(3):285-314. doi: 10.1002/jclp.20084.
• Trudeau KJ, Yang J, Di J, Lu Y, Kraus DR. Predicting Successful Placements for Youth in Child Welfare with Machine Learning. Child Youth Serv Rev. 2023 Oct;153:107117. doi: 10.1016/j.childyouth.2023.107117. Epub 2023 Aug 4.

Study record dates

Study Major Dates

• Study Start (Actual): February 3, 2025
• Primary Completion (Estimated): December 31, 2025
• Study Completion (Estimated): December 31, 2025

Study Registration Dates

• First Submitted: February 13, 2025
• First Submitted That Met QC Criteria: February 13, 2025
• First Posted (Actual): March 25, 2025

Study Record Updates

• Last Update Posted (Actual): March 25, 2025
• Last Update Submitted That Met QC Criteria: February 13, 2025
• Last Verified: February 1, 2025

More Information

Terms related to this study

• Other Study ID Numbers: 2R44MH125486-02A1-Aim 2B; 2R44MH125486-02A1 (U.S. NIH Grant/Contract)

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: NO

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No
• product manufactured in and exported from the U.S.: No