Randomized Controlled Trial of a Behavioral Training App

The investigators propose addressing and evaluating the efficacy of a behavioral, contingency management app by conducting a randomized controlled trial (RCT) that will evaluate the app compared to a placebo app control group. Participants will be randomly assigned to immediate intervention using the treatment condition or to a placebo app condition. Participants will not know which condition they are assigned to until after the study completion; they will also not know that they are using the treatment app until after the study completion. The sample will consist of 100 families of school-age children (ages 6-12) who display significant behavioral problems. The treatment app is a task management platform containing a self-directed rewards system through which caregivers can assign token rewards to their children for completing routine tasks (e.g., cleaning their room, completing homework). Participants in the placebo app condition will have a similar user experience, however, tokens are assigned non-contingently (dispensed daily, rather than based on task completion). While participants will not be required to complete any tasks or behaviors in the app as part of their participation in the study, they will be asked to use the app at least once per day for 8 weeks. Given the self-directed nature of the treatment app (parents decide how frequently they access the app-based rewards system and how many tasks they assign to their child), parent involvement in the intervention will vary slightly in duration. However, based on the literature and routine practice, we anticipate parents will log in at least once a day to assign tasks to their child and follow up on completed rewards. Assessments (see details below) will occur at baseline, weekly during the intervention, and at the end of the study. The study will take about 8 weeks in total to complete once participants have been enrolled in the app.

Study Overview

• Status: Recruiting
• Conditions: Depression; Stress; Anxiety; Executive Dysfunction; Behavior, Child; Disruptive Behavior; Attention-deficit
• Intervention / Treatment: Behavioral: Sham app; Behavioral: Treatment App

Detailed Description

Over the past several decades, much of the literature on treating both children with elevated levels of behavioral challenges as well as those children with clinical disorders such as attention-deficit/hyperactivity disorder (ADHD) and other disruptive behavior disorders (DBDs; i.e., conduct disorder (CD) and oppositional defiant disorder/ODD) has focused on evaluating the effectiveness of different treatment approaches. Behavioral interventions have been recommended as first-line psychosocial interventions for children with these challenges. Despite this recommendation, implementing behavioral interventions is often difficult for parents as there needs to be ongoing monitoring and support to refine this intervention to meet the needs of children. Reward systems are one prime example. Implementing a reward system for children facing behavioral challenges can yield numerous benefits. Firstly, it provides a positive reinforcement mechanism, encouraging desirable behavior by associating it with tangible rewards. This not only motivates children but also helps them develop a sense of accomplishment and self-control. A well-designed reward system can also enhance communication and understanding between parents, teachers, and the child, as clear expectations are established and acknowledged. Additionally, it fosters a supportive and nurturing environment, promoting a sense of trust and confidence in the child. Over time, the consistent application of a reward system can contribute to the development of long-term behavioral improvements and the acquisition of essential life skills, ultimately empowering the child to navigate challenges more effectively. However, parents often find it difficult to start and maintain a reward system.

Compounding these challenges, systemic issues such as costs of mental health services, policy, and legal restraints, limited numbers, and availability of mental health providers, regulatory restrictions, mental health stigma, and cultural and ethnic attitudes/beliefs affect access to adequate mental health services. Logistical barriers include insufficient time, lack of transportation and child care, conflicting commitments, and extensive agency waiting lists. Furthermore, there is an anticipated net decrease in mental health care professionals over the next ten years. It is of great concern that the children most at risk for untoward outcomes - those from lower socioeconomic strata - are significantly less likely to receive evidence-based treatments - particularly non-pharmacological evidence-based behavioral interventions. Both logistical and perceptual barriers pose significant barriers to service utilization and access to psychotherapy. Collectively, the limited availability of and challenges in accessing therapy are notable and impose significant limitations to the potential benefit of behavioral interventions for treating common childhood behavioral problems and clinical disorders.

Technology-based approaches to delivering evidence-based interventions have the potential to revolutionize mental health service access and delivery across multiple mental health disorders. Online (web-based and/or mobile applications) self-directed behavioral interventions are potentially more feasible, affordable, and acceptable, can have significant reach to include traditionally underserved populations, and are readily scalable and sustainable. Over 13 studies have recently been conducted demonstrating that online (web-based and mobile application) behavioral interventions can improve child behavioral outcomes.

In light of these findings, our research team has collaborated with Joon, Inc (a leading technology-based company) to test their technology-based solution to an effective, evidence-based behavioral intervention, namely, a reward system. This program evaluation aims to evaluate the already developed and commercially available Joon reward system app for its effectiveness relative to a placebo app in improving child behavioral and academic problems, impairment, and parent mental health. The evaluation will also collect feedback on participant's experience using the app.

The investigators propose addressing and evaluating the efficacy of a behavioral contingency management app by conducting a randomized controlled trial (RCT) that will evaluate the treatment app compared to a placebo app control group. Participants will be randomly assigned to immediate intervention using the treatment app condition or to a placebo app condition. Participants will not know which condition they are assigned to until after the study completion; they will also not know that they are using the treatment app until after the study completion. The sample will consist of 100 families of school-age children (ages 6-12) who display significant behavioral problems. The treatment app is a task management platform containing a self-directed rewards system through which caregivers can assign token rewards to their children for completing routine tasks (e.g., cleaning their room, completing homework). Participants in the placebo app condition will have a similar user experience, however, tokens are assigned non-contingently (dispensed daily, rather than based on task completion). While participants will not be required to complete any tasks or behaviors in the app as part of their participation in the study, they will be asked to use the app at least once per day for 8 weeks. Researchers will contact the parents via phone call to provide a 5 to 10-minute support call only if the parents do not log into the app over the course of a week (Monday through Friday). Additional support may be provided to participants based on parent requests via the study email. Given the self-directed nature of the treatment app (parents decide how frequently they access the app-based rewards system and how many tasks they assign to their child), parent involvement in the intervention will vary slightly in duration. However, based on the literature and routine practice, we anticipate parents will log in at least once a day to assign tasks to their child and follow up on completed rewards. Assessments (see details below) will occur at baseline, weekly during the intervention, and at the end of the study. The study will take about 8 weeks in total to complete once participants have been enrolled in the app.

• Study Type: Interventional
• Enrollment (Estimated): 100
• Phase: Not Applicable

Contacts and Locations

• Study Contact: Name: Anil Chacko, PhD; Phone Number: (212) 998-5749; Email: ac5489@nyu.edu

Study Locations

• United States
  • New York
    • New York, New York, United States, 10003
      • Recruiting
      • New York University
      • Contact: Anil Chacko, PhD; Phone Number: 212-998-5749; Email: ac5489@nyu.edu

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: Child
• Accepts Healthy Volunteers: No

Description

Inclusion Criteria:

• the child must be between the ages of 6-12 at the time of baseline/pre-intervention data collection
• the child must have an elevated score based on the strengths and Difficulties Questionnaire (SDQ)
• the parent/caregiver must be a fluent English speaker
• the parent/caregiver must be the legal guardians of and have full-time custody/child care responsibilities for their child (parents/caregiver with limited contact with their child will not have the opportunity to sufficiently use the app with their child)
• the parent/caregiver must have reliable and stable mobile internet access
• the parent/caregiver must use an Apple iOS operating system (the research version of the app will only be available in iOS) on their phone
• the parent/caregiver and their child must reside in the United States.

Exclusion Criteria:

• the child or parent/caregiver presents with emergency psychiatric needs that require services beyond those that can be managed within this intervention format (e.g., hospitalization, specialized placement outside the home); In this case, an active intervention by research staff to secure what is needed will be made
• if the child has an elevated score for Autism Spectrum Disorder (ASD) on the Developmental Behavior Checklist-Autism Screening Algorithm and the parent indicates that their child was diagnosed with autism or another pervasive developmental disorder;
• if the parent/caregiver is currently enrolled or has recently enrolled in any type of behavioral therapy program (i.e., within the past six months)
• the caregiver has used the Joon app in the past. If the child is currently taking medication, families that participate in the study will be asked to keep their child's medication status stable and report changes immediately to the project director.
• Additionally, participants may be removed from the study if their child's symptoms worsen considerably such that the child requires more immediate professional intervention.

Study Plan

How is the study designed?

Design Details

• Primary Purpose: Treatment
• Allocation: Randomized
• Interventional Model: Parallel Assignment
• Masking: Single

Number of Arms

2

Arms and Interventions

Participant Group / Arm	Intervention / Treatment
Placebo Comparator: Placebo app; Behavioral interventions are a well-established psychosocial intervention for the treatment of disruptive behavior disorders and related behavioral difficulties (e.g., ODD, ADHD). Behavioral training is based on social learning and operant conditioning principles in which parents are instructed to utilize positive reinforcement and reward systems to facilitate positive behaviors (e.g., compliance) in their child and reduce challenging behaviors (e.g., opposition). The intervention involves the use of an app to deliver a behavioral intervention using contingency reward systems. It translates established content and processes into a digital format to be delivered through a mobile app. For both conditions, there will be a parent and a child version of the app which are linked. The placebo arm will receive access to an app that does not include the contingency reward system.	Behavioral: Sham app; As described above, the child will access a virtual pet that they can feed, clean and buy various accessories by redeeming virtual coins. They will not be able to explore different lands with that pet. Coins in the placebo version will be rewarded non-contingently. Additionally, children will not be able to redeem coins for real-life rewards as above in the treatment app. Caregivers will not need to review and approve tasks in order to redeem coins. The caregiver-version of the app will not contain the Assigned or Review tabs as assigned above, since the child will earn coins regardless of whether or not they complete tasks, so caregivers will not be able to review nor assign tasks. Instead, caregivers in this group will only have access to the guide tab which will contain helpful articles the parent can read about disruptive behavior problems in children.
Experimental: Treatment app; Behavioral interventions are a well-established psychosocial intervention for the treatment of disruptive behavior disorders and related behavioral difficulties (e.g., ODD, ADHD). Behavioral training is based on social learning and operant conditioning principles in which parents are instructed to utilize positive reinforcement and reward systems to facilitate positive behaviors (e.g., compliance) in their child and reduce challenging behaviors (e.g., opposition). The intervention involves the use of an app to deliver a behavioral intervention using contingency reward systems. It translates established content and processes into a digital format to be delivered through a mobile app. For both conditions, there will be a parent and a child version of the app which are linked. The treatment app involves the use of contingency reward systems to shape targeted behaviors in children. The treatment app also includes psychoeducational guides for parents to review daily.	Behavioral: Treatment App; The behavioral treatment app content consists of tasks that caregivers can assign to their children in the app (e.g., finish homework, make bed, etc.). These tasks are completed offline, and then caregivers "approve" the child's completion of those tasks through the app, administering a token reward (coins) in-app. In the child version of the app, children take care of a virtual pet by feeding, cleaning, buying various accessories (using in-game coins), and exploring different lands with the virtual pet. The caregiver-assigned token coins can be used to redeem items needed to raise their pet, and access virtual lands. Caregivers can also choose to have their child redeem virtual tokens for real-life rewards. Ultimately, the app combines a video game with evidence-based principles to help children practice important tasks and behaviors as administered by caregivers.

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
IOWA Conners Rating Scale	The IOWA-CRS (Waschbusch & Willoughby, 2008) is a widely used brief measure of attention-deficit/hyperactivity disorder and oppositional-defiant behavior in children completed by parents. The IOWA-CRS consists of 10 items evaluated using a four-point Likert scale with the following anchors: not at all (0); just a little (1); pretty much (2); and very much (3).	before starting the intervention, throughout the 8 week intervention
The Disruptive Behavior Disorders (DBD) Rating Scale (Pelham et al., 1992)	The DBD scale consists of 23 items from the full 45-item measure that asks parents to rate symptoms of ADHD, ODD, and CD on a 4-point Likert scale (i.e., "not at all", "just a little", "pretty much", or "very much"), with higher scores indicating more problems. For this study, the average scores for ADHD, ODD and CD will be used. Support for the scale's reliability and validity have been reported in past samples, with acceptable levels of internal consistency (.82) and concurrent correlations with other measures used for diagnosis of ADHD and ODD being reported (Pelham et al., 2005).	before starting the intervention, 4 weeks after starting the intervention, and 8 weeks after starting the intervention
Impairment Rating Scale - Parent/Teacher Version (IRS) (Pelham, Fabiano, & Massetti, 2005)	The IRS is a multidimensional measure that assesses functioning across domains. Specifically, the IRS qualifies and quantifies impairment present in a child's life, both in school and non-school settings. The scale has parent and teacher versions that ask about the degree to which the child has problems that warrant treatment, intervention, or special services in specific areas of functioning. For children ages 4 through 12, the IRS has shown good psychometric properties and has empirically derived cutoff points. The IRS asks the informant to respond using a 7-point scale that ranges from "No problem; definitely does not need treatment or special services'' to "Extreme problem; definitely needs treatment or special services." The IRS exhibits concurrent, discriminant, and convergent validity, and acceptable levels of temporal stability.	before starting the intervention, 4 weeks after starting the intervention, and 8 weeks after starting the intervention

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Revised Child Anxiety and Depression Scale - Parent version (R-CADS-P; Chorpita & Spence, 2003)	The R-CADS is a 47-item, parent-report questionnaire with subscales including separation anxiety disorder, social phobia, generalized anxiety disorder, panic disorder, obsessive-compulsive disorder, and low mood (major depressive disorder). It also yields a Total Anxiety Scale (sum of the 5 anxiety subscales) and a Total Internalizing Scale (sum of all 6 subscales). The questionnaire takes between five and ten minutes to administer.	before starting the intervention, 4 weeks after starting the intervention, and 8 weeks after starting the intervention
The Parenting Stress Index-Short Form (PSI-SF) (Abidin, 1995)	PSI-SF measures the level of stress in the parent-child relationship and is appropriate for use with children aged 1 month to 12 years. It was derived from the longer Parenting Stress Index and consists of 36 statements, each rated on a 1 to 5 scale (1=strongly disagree, 5=strongly agree), which produces a Total Stress score. The items also cluster to produce three subscale scores (Parental Distress, Difficult Child, and Parent-child Dysfunctional Interaction) each derived from ratings of 12 items.	before starting the intervention, 4 weeks after starting the intervention, and 8 weeks after starting the intervention
Usability Survey	The Usability Survey is a short measure of participants' perceptions of the app and their experiences. The Usability Survey was developed by the Joon team for the purposes of internal review and has been piloted in an internal study. The study consists of 13 items, rated on a scale from 0 to 4 (0 = do not agree at all; 4 = definitely agree).	8 weeks after starting the intervention

Collaborators and Investigators

• Sponsor: New York University
• Collaborators: Joon, Inc.
• Investigators: Principal Investigator: Anil Chacko, PhD, New York University

Publications and helpful links

General Publications

• Breider S, de Bildt A, Nauta MH, Hoekstra PJ, van den Hoofdakker BJ. Self-directed or therapist-led parent training for children with attention deficit hyperactivity disorder? A randomized controlled non-inferiority pilot trial. Internet Interv. 2019 Aug 8;18:100262. doi: 10.1016/j.invent.2019.100262. eCollection 2019 Dec.
• Wolraich ML, Hagan JF Jr, Allan C, Chan E, Davison D, Earls M, Evans SW, Flinn SK, Froehlich T, Frost J, Holbrook JR, Lehmann CU, Lessin HR, Okechukwu K, Pierce KL, Winner JD, Zurhellen W; SUBCOMMITTEE ON CHILDREN AND ADOLESCENTS WITH ATTENTION-DEFICIT/HYPERACTIVE DISORDER. Clinical Practice Guideline for the Diagnosis, Evaluation, and Treatment of Attention-Deficit/Hyperactivity Disorder in Children and Adolescents. Pediatrics. 2019 Oct;144(4):e20192528. doi: 10.1542/peds.2019-2528. Erratum In: Pediatrics. 2020 Mar;145(3):
• Tate DF, Finkelstein EA, Khavjou O, Gustafson A. Cost effectiveness of internet interventions: review and recommendations. Ann Behav Med. 2009 Aug;38(1):40-5. doi: 10.1007/s12160-009-9131-6.
• McKay MM, Bannon WM Jr. Engaging families in child mental health services. Child Adolesc Psychiatr Clin N Am. 2004 Oct;13(4):905-21, vii. doi: 10.1016/j.chc.2004.04.001.
• Day JJ, Sanders MR. Do Parents Benefit From Help When Completing a Self-Guided Parenting Program Online? A Randomized Controlled Trial Comparing Triple P Online With and Without Telephone Support. Behav Ther. 2018 Nov;49(6):1020-1038. doi: 10.1016/j.beth.2018.03.002. Epub 2018 Mar 9.
• DuPaul GJ. Promoting Success Across School Years for Children With Attention-Deficit/Hyperactivity Disorder: Collaborative School-Home Intervention. J Am Acad Child Adolesc Psychiatry. 2018 Apr;57(4):231-232. doi: 10.1016/j.jaac.2018.02.001.
• Nieuwboer CC, Fukkink RG, Hermanns JM. Peer and professional parenting support on the Internet: a systematic review. Cyberpsychol Behav Soc Netw. 2013 Jul;16(7):518-28. doi: 10.1089/cyber.2012.0547. Epub 2013 May 9.
• Thongseiratch T, Leijten P, Melendez-Torres GJ. Online parent programs for children's behavioral problems: a meta-analytic review. Eur Child Adolesc Psychiatry. 2020 Nov;29(11):1555-1568. doi: 10.1007/s00787-020-01472-0. Epub 2020 Jan 10.
• Chacko A, Jensen SA, Lowry LS, Cornwell M, Chimklis A, Chan E, Lee D, Pulgarin B. Engagement in Behavioral Parent Training: Review of the Literature and Implications for Practice. Clin Child Fam Psychol Rev. 2016 Sep;19(3):204-15. doi: 10.1007/s10567-016-0205-2.
• Evans SW, Owens JS, Wymbs BT, Ray AR. Evidence-Based Psychosocial Treatments for Children and Adolescents With Attention Deficit/Hyperactivity Disorder. J Clin Child Adolesc Psychol. 2018 Mar-Apr;47(2):157-198. doi: 10.1080/15374416.2017.1390757. Epub 2017 Dec 19.
• Barbaresi WJ, Campbell L, Diekroger EA, Froehlich TE, Liu YH, O'Malley E, Pelham WE Jr, Power TJ, Zinner SH, Chan E. Society for Developmental and Behavioral Pediatrics Clinical Practice Guideline for the Assessment and Treatment of Children and Adolescents with Complex Attention-Deficit/Hyperactivity Disorder. J Dev Behav Pediatr. 2020 Feb/Mar;41 Suppl 2S:S35-S57. doi: 10.1097/DBP.0000000000000770.
• Weisz JR, Kuppens S, Eckshtain D, Ugueto AM, Hawley KM, Jensen-Doss A. Performance of evidence-based youth psychotherapies compared with usual clinical care: a multilevel meta-analysis. JAMA Psychiatry. 2013 Jul;70(7):750-61. doi: 10.1001/jamapsychiatry.2013.1176.
• Cuffe SP, Moore CG, McKeown R. ADHD and health services utilization in the national health interview survey. J Atten Disord. 2009 Jan;12(4):330-40. doi: 10.1177/1087054708323248.

Study record dates

Study Major Dates

• Study Start (Actual): March 27, 2024
• Primary Completion (Estimated): August 30, 2024
• Study Completion (Estimated): October 30, 2024

Study Registration Dates

• First Submitted: May 28, 2024
• First Submitted That Met QC Criteria: June 3, 2024
• First Posted (Actual): June 7, 2024

Study Record Updates

• Last Update Posted (Actual): June 7, 2024
• Last Update Submitted That Met QC Criteria: June 3, 2024
• Last Verified: April 1, 2024

More Information

Terms related to this study

• Additional Relevant MeSH Terms: Behavioral Symptoms; Problem Behavior
• Other Study ID Numbers: IRB-FY2024-8463

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: NO

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No