Treatment Planning for ABA Employing Auxiliary Tools v2+ (TREAAT2+) (TREAAT2+)

The Efficacy of Technology-Based Tools in Applied Behavior Analysis Treatment Planning for Autism Spectrum Disorder

Technology enhancement in Applied Behavior Analysis (ABA) treatment planning may increase confidence, efficiency, consistency, and satisfaction for Board Certified Behavior Analysts (BCBAs) which, in turn, can provide for better clinical outcomes for patients on the autism spectrum. To this end, the investigators will examine the use of >3 technology-based tools that will be implemented in the BCBAs' clinical workflow to aid with treatment planning. The study will initially involve two aims that are non-interventional (these processes will occur in the background and will have no impact on any cohorts), followed by an interventional aim that includes two arms (i.e., two BCBA cohorts). BCBAs within both arms will observe and practice the standard of care for ABA, and thus patient care will not be impacted. The outcome measures are primarily focused on the BCBAs as follows: Arm 1: An experimental group (BCBA Tech cohort) will receive the full tech package (TREAAT2+) from the start. Arm 2: The control group (BCBA non-Tech cohort) will not have access to any tools from the tech package for the first 6 months. In the subsequent 18 months, they will receive one tool every 6 months until gaining access to the entire tech package.

Study Overview

• Status: Not yet recruiting
• Conditions: Autism Spectrum Disorder
• Intervention / Treatment: Other: Tech-enabled ABA treatment planning

Detailed Description

Autism spectrum disorder (ASD) is a complex, heterogeneous neurodevelopmental disorder that accounts for >$250 billion in direct and indirect annual expenditures and is estimated to occur in 1 out of 36 children <8 years of age in the US. Validated ASD treatments, such as Applied Behavior Analysis (ABA), rely upon Board Certified Behavior Analysts (BCBAs) to develop highly individualized treatment plans via a complex and labor-intensive process. Integrating technology-based approaches for treatment planning (largely unexplored in the context of ABA), such as data-driven clinical decision support (CDS) systems and assistive technology, can modernize and optimize the BCBA workflow, which, in turn, can enhance patient care. There has been a high demand for BCBAs in recent years, which has led to shortages straining care providers, with about 72% of BCBAs experiencing significant burnout, increasing BCBA turnover. One contributing factor to the BCBA burnout rate is the workload, which can fuel exhaustion and disengagement.The BCBA workflow includes significant time spent on developing effective individualized treatment plans via a tedious, non-automated, and heterogeneous process lacking standardized tools. Thus, there is a need to automate existing workflows to increase BCBA efficiency, confidence, consistency, and satisfaction, in order to mitigate burnout and consequently improve the patient management process, and by extension patient clinical outcomes.

In this SBIR project, the investigators propose to integrate a data-driven technological package (TREAAT2+) into the BCBA's workflow to assist with streamlined and consistent ABA treatment planning. TREAAT2+ consists of (1) a machine learning algorithm (MLA)-based CDS tool that analyzes data from electronic health records (EHRs) and recommends treatment dosage in terms of hours, where the MLA is integrated into a proprietary application ("app"); (2) a treatment planning software tool integrated with the proprietary app to facilitate highly accessible treatment oversight; and (3) individual patient progress reports pushed onto the proprietary app from Autism Analytica (AA). The predecessor of the app-integrated MLA, TREAAT, was validated and achieved excellent performance (AUROC of 0.895) for a binary treatment dosage recommendation (<20 or >20 hours/week). The investigators will enhance the capacity of the MLA for more granular treatment dosage recommendations, deploy a treatment planning software tool in the app for BCBA use, and provide pushed AA patient data assessments in the app. This will improve BCBA efficiency and confidence within their workflow, and thereby significantly reduce the burden related to the manual and subjective nature of the treatment planning process. TREAAT was validated with proprietary data from patients of Montera Health TX LLC ("Montera"), and the investigators will use a larger number of patients to fine-tune and validate the app-integrated MLA to improve generalizability. The investigators expect that the MLA will perform as well as or better than the original TREAAT in this expanded patient population and that the MLA, in conjunction with the treatment planning software tool and the pushed AA data, will significantly improve the BCBA workflow. The lack of current workflow automation coexists with significant BCBA burnout rates, and TREAAT2+ provides the solution of modernizing time-consuming tasks within treatment planning. By bridging the technological gap in the BCBA workflow, TREAAT2+ will mitigate BCBA burnout, and by extension improve patient care.

Study Aim 1: Retraining and upgrading the MLA of TREAAT2+. The investigators will use a larger set of retrospective and prospective Montera patient data than was employed for the original TREAAT training, and will design the MLA output as increments of treatment dosage recommendation (in 10 hr increments). The investigators will additionally integrate the MLA into our proprietary app. The investigators expect MLA performance metrics to be comparable to or better than retrospective benchmarks from the pilot study (AUROC: 0.895; 95% CI: 0.811 - 0.962).

Study Aim 2: Test the robustness of the MLA of TREAAT2+ in treatment plan development. The investigators will conduct a non-interventional prospective evaluation of the app-integrated MLA. The agreement between the incremental treatment dosage suggested by the MLA and the dosage prescribed by the BCBA will be assessed. MLA performance will be evaluated on demographic subpopulations to ensure bias minimization. The investigators expect that there will be substantial agreement between the treatment dosage prescribed by the BCBA and the dosage suggested by MLA, as measured by minimum inter-rater reliability (e.g., Cohen's Kappa) of greater than 0.6, which indicates substantial agreement according to the Landis and Koch's classification system.

Study Aim 3: Evaluate the impact of deploying TREAAT2+ within the BCBA workflow. The investigators will utilize prospective data from two BCBA cohorts. An experimental group (BCBA Tech cohort) will receive the full tech package (TREAAT2+) from the start. The control group (BCBA non-Tech cohort) will not have access to any tools from the tech package for the first 6 months. In the subsequent 18 months, they will receive one tool every 6 months until gaining access to the entire tech package. The investigators expect to demonstrate the efficacy of TREAAT2+ with statistically significant improvement (p < 0.05) over baseline in qualitative endpoints (efficiency, confidence, consistency, and satisfaction; as measured by BCBA self-reported Likert questionnaire scales) and quantitative endpoints (time allocated to treatment plan development).

• Study Type: Interventional
• Enrollment (Estimated): 20
• Phase: Phase 2; Phase 1

Contacts and Locations

• Study Contact: Name: Qingqing Mao, PhD; Phone Number: 4158051725; Email: qmao@fortahealth.com

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: Child; Adult; Older Adult
• Accepts Healthy Volunteers: Yes

Description

Inclusion Criteria:

• BCBAs will be eligible for enrollment if they are actively employed by Montera and are actively providing ABA treatment to Montera patients.

Exclusion Criteria:

• BCBAs will be excluded from the study for one or more of the following reasons:
• BCBA requests that their data is not used in the study;
• BCBA does not complete the required assessments;
• BCBA does not have the aforementioned data required for inclusion.

Study Plan

How is the study designed?

Design Details

• Primary Purpose: Health Services Research
• Allocation: Randomized
• Interventional Model: Parallel Assignment
• Masking: Single

Number of Arms

2

Arms and Interventions

Participant Group / Arm	Intervention / Treatment
Experimental: technology integrated care planning cohort; This cohort will consist of 15 BCBAs that will receive a minimum of 3 technology-based tools (2 proprietary tools and at least 1 non-proprietary tool) for use in conjunction with the standard of care to develop and manage ABA treatment plans for active patients.	Other: Tech-enabled ABA treatment planning; The tech-enabled ABA treatment planning will involve the use of a proprietary MLA integrated within a proprietary software application for use on a tablet, phone, or other smart device by BCBAs (one proprietary software application, one non-proprietary software application). The proprietary application provides functionality for treatment plan development, including templates and centralized resource availability. This tech will be used as adjunct or auxiliary tools to develop and manage ABA treatment plans. The proposed period of time for this intervention (the latter portion of the study subsequent to the non-interventional phases) will be 24 months.
No Intervention: non-technology integrated care cohort; This cohort will consist of 5 BCBAs that will follow the standard of care that does not involve the use of the tech-based tools to develop and manage ABA treatment plans for active patients receiving ABA.

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Retraining and upgrading the MLA (non-interventional)	Performance of MLA as measured by AUROC, sensitivity, and specificity	6 months
Prospective evaluation of the app-integrated MLA (non-interventional)	Attain substantial agreement between the treatment dosage prescribed by the BCBA and the dosage suggested by MLA, as measured by minimum inter-rater reliability (e.g., Cohen's Kappa) of greater than 0.6, which indicates substantial agreement according to the Landis and Koch's classification system.	6 months
Evaluate the impact of deploying the tech package within the BCBA workflow	Endpoint: Efficiency (Qualitative). Demonstrate the efficacy of the tech package with statistically significant improvement (p < 0.05) in efficiency at 6 month intervals. Likert questionnaires will be used to analyze answers to questions individually and the sum of the entire assessment measure will be the endpoint. The Likert Scale will be applied as (1) Strongly Disagree, (2) Disagree, (3) Neither Agree Nor Disagree (4) Agree, and (5) Strongly Agree. The minimum value of 1 indicates worse outcomes and the maximum value of 5 indicates best outcomes.	24 months
Evaluate the impact of deploying the tech package within the BCBA workflow	Endpoint: Efficiency (Quantitative). Demonstrate the efficacy of the tech package with statistically significant improvement (p < 0.05) in efficiency at 6 month intervals.The time allocated to treatment plan development by individual BCBAs for individual patients will be measured. The minimum and maximum values are represented by number of hours, with fewer hours (minimum value: 4 hours) associated with better outcomes (i.e., increased efficiency) and a higher number of hours (maximum value: 40 hours) associated with worse outcomes (i.e., no change or decreased efficiency).	24 months
Evaluate the impact of deploying the tech package within the BCBA workflow	Endpoint: Confidence (Qualitative). Demonstrate the efficacy of the tech package with statistically significant improvement (p < 0.05) in confidence at 6 month intervals. Likert questionnaires will be used to analyze answers to questions individually and the sum of the entire assessment measure will be the endpoint. The Likert Scale will be applied as (1) Strongly Disagree, (2) Disagree, (3) Neither Agree Nor Disagree (4) Agree, and (5) Strongly Agree. The minimum value of 1 indicates worse outcomes and the maximum value of 5 indicates best outcomes.	24 months
Evaluate the impact of deploying the tech package within the BCBA workflow	Endpoint: Consistency (Qualitative). Demonstrate the efficacy of the tech package with statistically significant improvement (p < 0.05) in consistency at 6 month intervals. Likert questionnaires will be used to analyze answers to questions individually and the sum of the entire assessment measure will be the endpoint. The Likert Scale will be applied as (1) Strongly Disagree, (2) Disagree, (3) Neither Agree Nor Disagree (4) Agree, and (5) Strongly Agree. The minimum value of 1 indicates worse outcomes and the maximum value of 5 indicates best outcomes.	24 months
Evaluate the impact of deploying the tech package within the BCBA workflow	Endpoint: Satisfaction. Demonstrate the efficacy of the tech package with statistically significant improvement (p < 0.05) in satisfaction at 6 month intervals. Likert questionnaires will be used to analyze answers to questions individually and the sum of the entire assessment measure will be the endpoint. The Likert Scale will be applied as (1) Strongly Disagree, (2) Disagree, (3) Neither Agree Nor Disagree (4) Agree, and (5) Strongly Agree. The minimum value of 1 indicates worse outcomes and the maximum value of 5 indicates best outcomes.	24 months

Collaborators and Investigators

• Sponsor: Montera Health Texas LLC

Publications and helpful links

General Publications

• Maharjan J, Garikipati A, Dinenno FA, Ciobanu M, Barnes G, Browning E, DeCurzio J, Mao Q, Das R. Machine learning determination of applied behavioral analysis treatment plan type. Brain Inform. 2023 Mar 2;10(1):7. doi: 10.1186/s40708-023-00186-8.
• Garikipati A, Ciobanu M, Singh NP, Barnes G, Decurzio J, Mao Q, Das R. Clinical Outcomes of a Hybrid Model Approach to Applied Behavioral Analysis Treatment. Cureus. 2023 Mar 27;15(3):e36727. doi: 10.7759/cureus.36727. eCollection 2023 Mar.

Study record dates

Study Major Dates

• Study Start (Estimated): July 1, 2024
• Primary Completion (Estimated): June 30, 2027
• Study Completion (Estimated): June 30, 2027

Study Registration Dates

• First Submitted: December 12, 2023
• First Submitted That Met QC Criteria: January 11, 2024
• First Posted (Actual): January 12, 2024

Study Record Updates

• Last Update Posted (Actual): January 12, 2024
• Last Update Submitted That Met QC Criteria: January 11, 2024
• Last Verified: January 1, 2024

More Information

Terms related to this study

• Keywords: Autism Spectrum Disorder; Autistic Disorder; Decision Support Systems, Clinical; Applied Behavior Analysis
• Additional Relevant MeSH Terms: Mental Disorders; Neurodevelopmental Disorders; Autistic Disorder; Autism Spectrum Disorder; Child Development Disorders, Pervasive
• Other Study ID Numbers: Mont_2023

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: NO

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No