Development and Pilot Evaluation of a Tablet-Based Application to Improve Quality of Care in Child Mental Health Treatment

Kenneth J Ruggiero, Brian E Bunnell, Arthur R Andrews Iii, Tatiana M Davidson, Rochelle F Hanson, Carla Kmett Danielson, Benjamin E Saunders, Kathryn Soltis, Caleb Yarian, Brian Chu, Zachary W Adams, Kenneth J Ruggiero, Brian E Bunnell, Arthur R Andrews Iii, Tatiana M Davidson, Rochelle F Hanson, Carla Kmett Danielson, Benjamin E Saunders, Kathryn Soltis, Caleb Yarian, Brian Chu, Zachary W Adams

Abstract

Background: Children need access to high quality mental health care. Effective treatments now exist for a wide range of mental health conditions. However, these interventions are delivered with variable effectiveness in traditional mental health service settings. Innovative solutions are needed to improve treatment delivery quality and effectiveness.

Objective: The aim of this study was to develop a scalable, sustainable technology-based approach to improve the quality of care in child mental health treatment.

Methods: A tablet-based resource was developed with input from mental health training experts, mental health providers, and patients. A series of qualitative data collection phases (ie, expert interviews, patient and provider focus groups, usability testing) guided the initial concept and design of the resource, and then its refinement. The result was an iPad-based "e-workbook" designed to improve child engagement and provider fidelity in implementation of a best-practice treatment. We are currently conducting a small scale randomized controlled trial to evaluate the feasibility of e-workbook facilitated child mental health treatment with 10 providers and 20 families recruited from 4 local community-based mental health clinics.

Results: Usability and focus group testing yielded a number of strong, favorable reactions from providers and families. Recommendations for refining the e-workbook also were provided, and these guided several improvements to the resource prior to initiating the feasibility trial, which is currently underway.

Conclusions: This study aimed to develop and preliminarily evaluate a tablet-based application to improve provider fidelity and child engagement in child mental health treatment. If successful, this approach may serve as a key step toward making best-practice treatment more accessible to children and families. As various technologies continue to increase in popularity worldwide and within the health care field more specifically, it is essential to rigorously test the usability, feasibility, acceptability, and effectiveness of novel health technology solutions. It is also essential to ensure that patients and providers drive decision making that supports the development of these resources to ensure that they can be seamlessly integrated into practice.

Trial registration: Clinicaltrials.gov NCT01915160; https://ichgcp.net/clinical-trials-registry/NCT01915160 (Archived by WebCite at http://www.webcitation.org/6cPIiQDpu).

Keywords: child mental health treatment; feasibility test; fidelity; mobile health; patient engagement; technology; traumatic stress.

Conflict of interest statement

Conflicts of Interest: None declared.

Figures

Figure 1
Figure 1
Screenshot of the eight “decks” of virtual cards in the “What Do You Know?” psychoeducation activity. Clinicians and families select the most pertinent decks for each child. The cards within each deck are customizable by the clinician to enhance relevance to each family’s needs.
Figure 2
Figure 2
An example card from the Domestic Violence deck. Users can swipe left or right through the deck to view additional cards. A scoreboard is presented at the bottom to increase engagement.
Figure 3
Figure 3
A Provider tab is located at the top of the screen and contains specific notes for each card. These notes may be used as discussion points during session.
Figure 4
Figure 4
At the end of each activity, progress is tracked graphically via color-coding (ie, green=complete). On this page, module-specific homework assignments may be assigned. The Menu button at the top of each screen allows users to access the Home screen or end a session and assign homework.
Figure 5
Figure 5
One activity in the Affective Regulation module involves directing the child to list as many different feeling words in 90 seconds as he or she is able to. This activity uses a free-write function. The Provider tab at the bottom of the screen provides tips for clinicians on strategies for using this activity effectively in session.
Figure 6
Figure 6
Another Affective Regulation activity is Feelings Charades. Users “spin” a virtual wheel by dragging down on the feeling words. They are then instructed to act out the feeling on which the wheel lands. A scoreboard is available to increase engagement with the activity.
Figure 7
Figure 7
An activity in the Cognitive Coping module has children practice identifying and categorizing thoughts, feelings, and behaviors by dragging the words and phrases on the left of the screen (shown in gray) to the appropriate icon on the right of the screen. Feedback is provided in response to each user interaction.

References

    1. Costello EJ, Egger H, Angold A. 10-year research update review: the epidemiology of child and adolescent psychiatric disorders: I. Methods and public health burden. J Am Acad Child Adolesc Psychiatry. 2005 Oct;44(10):972–86. doi: 10.1097/01.chi.0000172552.41596.6f.
    1. Kessler RC, Avenevoli S, Costello EJ, Georgiades K, Green JG, Gruber MJ, He J, Koretz D, McLaughlin KA, Petukhova M, Sampson NA, Zaslavsky AM, Merikangas KR. Prevalence, persistence, and sociodemographic correlates of DSM-IV disorders in the National Comorbidity Survey Replication Adolescent Supplement. Arch Gen Psychiatry. 2012 Apr;69(4):372–80. doi: 10.1001/archgenpsychiatry.2011.160.
    1. Merikangas KR, He J, Burstein M, Swanson SA, Avenevoli S, Cui L, Benjet C, Georgiades K, Swendsen J. Lifetime prevalence of mental disorders in U.S. adolescents: results from the National Comorbidity Survey Replication--Adolescent Supplement (NCS-A) J Am Acad Child Adolesc Psychiatry. 2010 Oct;49(10):980–9. doi: 10.1016/j.jaac.2010.05.017.
    1. Chorpita B, Daleiden E, Ebesutani C. Evidence-based treatments for children and adolescents: An updated review of indicators of efficacy and effectiveness. Clin Psychol Sci Prac. 2011;18:154–72.
    1. Drake RE, Torrey WC, McHugo GJ. Strategies for implementing evidence-based practices in routine mental health settings. Evid Based Ment Health. 2003 Feb;6(1):6–7.
    1. Garland AF, Brookman-Frazee L, Hurlburt MS, Accurso EC, Zoffness RJ, Haine-Schlagel R, Ganger W. Mental health care for children with disruptive behavior problems: a view inside therapists' offices. Psychiatr Serv. 2010 Aug;61(8):788–95. doi: 10.1176/appi.ps.61.8.788.
    1. Kazak AE, Hoagwood K, Weisz JR, Hood K, Kratochwill TR, Vargas LA, Banez GA. A meta-systems approach to evidence-based practice for children and adolescents. Am Psychol. 2010;65(2):85–97. doi: 10.1037/a0017784.
    1. McHugo GJ, Drake RE, Whitley R, Bond GR, Campbell K, Rapp CA, Goldman HH, Lutz WJ, Finnerty MT. Fidelity outcomes in the National Implementing Evidence-Based Practices Project. Psychiatr Serv. 2007 Oct;58(10):1279–84. doi: 10.1176/appi.ps.58.10.1279.
    1. Raghavan R, Inoue M, Ettner SL, Hamilton BH, Landsverk J. A preliminary analysis of the receipt of mental health services consistent with national standards among children in the child welfare system. Am J Public Health. 2010 Apr;100(4):742–9. doi: 10.2105/AJPH.2008.151472.
    1. McCabe O. Crossing the Quality Chasm in Behavioral Health Care: The Role of Evidence-Based Practice. Professional Psychology: Research and Practice. 2004;35(6):571–579. doi: 10.1037/0735-7028.35.6.571.
    1. Cross WF, West JC. Examining implementer fidelity: Conceptualizing and measuring adherence and competence. J Child Serv. 2011;6(1):18–33. doi: 10.5042/jcs.2011.0123.
    1. Bond G, Becker D, Drake R. Measurement of fidelity of implementation of evidence based-practices: Case examples of the IPS fidelity scale. Clin Psychol Sci Prac. 2011;18:126–41. doi: 10.1111/j.1468-2850.2011.01244.x.
    1. McHugo GJ, Drake RE, Teague GB, Xie H. Fidelity to assertive community treatment and client outcomes in the New Hampshire dual disorders study. Psychiatr Serv. 1999 Jun;50(6):818–24.
    1. Schoenwald SK, Sheidow AJ, Letourneau EJ. Toward effective quality assurance in evidence-based practice: links between expert consultation, therapist fidelity, and child outcomes. J Clin Child Adolesc Psychol. 2004 Mar;33(1):94–104. doi: 10.1207/S15374424JCCP3301_10.
    1. Karlin BE, Ruzek JI, Chard KM, Eftekhari A, Monson CM, Hembree EA, Resick PA, Foa EB. Dissemination of evidence-based psychological treatments for posttraumatic stress disorder in the Veterans Health Administration. J Trauma Stress. 2010 Dec;23(6):663–73. doi: 10.1002/jts.20588.
    1. Saunders B. Bringing in evidence-based practice: How do we make EBP TAU?. Leadership Symposium on Evidence-Based Practice in Human Services; January 30, 2009; San Diego, CA. 2009.
    1. Markiewicz J, Ebert L, Ling D, Amaya-Jackson L, Kisiel C. Learning Collaborative Toolkit. Raising the standard of care for traumatized children and their families. Los Angeles, CA and Durham, NC: National Center for Child Traumatic Stress; 2006. [2015-07-23]. .
    1. Kubben PL, van SH, Cornips E M J. Vaccaro AR, Dvorak MF, van Rhijn L W. Scherpbier A J J A. Hoogland H. An evidence-based mobile decision support system for subaxial cervical spine injury treatment. Surg Neurol Int. 2011;2:32. doi: 10.4103/2152-7806.78238.
    1. Roy P, Durieux P, Gillaizeau F, Legall C, Armand-Perroux A, Martino L, Hachelaf M, Dubart A, Schmidt J, Cristiano M, Chretien J, Perrier A, Meyer G. A computerized handheld decision-support system to improve pulmonary embolism diagnosis: a randomized trial. Ann Intern Med. 2009 Nov 17;151(10):677–86. doi: 10.7326/0003-4819-151-10-200911170-00003.
    1. Kang B, Tan S. Impact of digital games on intrinsicextrinsic motivation, achievement,satisfaction. McFerrin K. eds. Proceedings of Society for Information Technology & Teacher Education International Conference; Society for Information Technology & Teacher Education International Conference; 03/03/2008; Las Vegas, NV. Chesapeake, VA: Association for the Advancement of Computing in Education; 2008. Mar 03, pp. 1825–32.
    1. Papastergiou M. Digital Game-Based Learning in high school Computer Science education: Impact on educational effectiveness and student motivation. Computers & Education. 2009 Jan;52(1):1–12. doi: 10.1016/j.compedu.2008.06.004.
    1. McKay M, Hibbert R, Hoagwood K. Integrating evidence based engagement interventions into "real world" child mental health settings. Brief Treatment and Crisis Intervention. 2004;4:177–86. doi: 10.1093/brief-treatment/mhh014.
    1. Kazdin AE, Blase SL. Rebooting Psychotherapy Research and Practice to Reduce the Burden of Mental Illness. Perspect Psychol Sci. 2011 Jan;6(1):21–37. doi: 10.1177/1745691610393527.
    1. Luxton D, McCann R, Bush N, Mishkind M, Reger G. mHealth for mental health: Integrating smartphone technology in behavioral healthcare. Prof Psychol Res Pr. 2011;42:505. doi: 10.1037/a0024485.
    1. Amstadter AB, Broman-Fulks J, Zinzow H, Ruggiero KJ, Cercone J. Internet-based interventions for traumatic stress-related mental health problems: a review and suggestion for future research. Clin Psychol Rev. 2009 Jul;29(5):410–20. doi: 10.1016/j.cpr.2009.04.001.
    1. Smith A. Pew Research Center. 2015. Apr 01, [2015-10-27]. U.S. Smartphone Use in 2015 .
    1. Carlbring P, Andersson Internet and psychological treatment: How well can they be combined? Comput Hum Behav. 2006;22:545–53. doi: 10.1016/j.chb.2004.10.009.
    1. Cuijpers P, Marks IM, van SA, Cavanagh K, Gega L, Andersson G. Computer-aided psychotherapy for anxiety disorders: a meta-analytic review. Cogn Behav Ther. 2009;38(2):66–82. doi: 10.1080/16506070802694776.
    1. Clough BA, Casey LM. Technological adjuncts to enhance current psychotherapy practices: a review. Clin Psychol Rev. 2011 Apr;31(3):279–92. doi: 10.1016/j.cpr.2010.12.008.
    1. Luxton DD, June JD, Kinn JT. Technology-based suicide prevention: current applications and future directions. Telemed J E Health. 2011;17(1):50–4. doi: 10.1089/tmj.2010.0091.
    1. Morris ME, Kathawala Q, Leen TK, Gorenstein EE, Guilak F, Labhard M, Deleeuw W. Mobile therapy: case study evaluations of a cell phone application for emotional self-awareness. J Med Internet Res. 2010;12(2):e10. doi: 10.2196/jmir.1371.
    1. Sandler E. Psychol Today. 2010. [2015-10-27]. New mobile apps provide 24-7 check-ins .
    1. Deblinger E, Mannarino AP, Cohen JA, Steer RA. A follow-up study of a multisite, randomized, controlled trial for children with sexual abuse-related PTSD symptoms. J Am Acad Child Adolesc Psychiatry. 2006 Dec;45(12):1474–84. doi: 10.1097/.
    1. Cohen JA, Deblinger E, Mannarino AP, Steer RA. A multisite, randomized controlled trial for children with sexual abuse-related PTSD symptoms. J Am Acad Child Adolesc Psychiatry. 2004 Apr;43(4):393–402. doi: 10.1097/00004583-200404000-00005.
    1. Cohen JA, Mannarino AP, Iyengar S. Community treatment of posttraumatic stress disorder for children exposed to intimate partner violence: a randomized controlled trial. Arch Pediatr Adolesc Med. 2011 Jan;165(1):16–21. doi: 10.1001/archpediatrics.2010.247.
    1. Deblinger E, Mannarino AP, Cohen JA, Runyon MK, Steer RA. Trauma-focused cognitive behavioral therapy for children: impact of the trauma narrative and treatment length. Depress Anxiety. 2011 Jan;28(1):67–75. doi: 10.1002/da.20744.
    1. Silverman WK, Ortiz CD, Viswesvaran C, Burns BJ, Kolko DJ, Putnam FW, Amaya-Jackson L. Evidence-based psychosocial treatments for children and adolescents exposed to traumatic events. J Clin Child Adolesc Psychol. 2008 Jan;37(1):156–83. doi: 10.1080/15374410701818293.
    1. Wethington HR, Hahn RA, Fuqua-Whitley DS, Sipe TA, Crosby AE, Johnson RL, Liberman AM, Mościcki E, Price LN, Tuma FK, Kalra G, Chattopadhyay SK, Task Force on Community Preventive Services The effectiveness of interventions to reduce psychological harm from traumatic events among children and adolescents: a systematic review. Am J Prev Med. 2008 Sep;35(3):287–313. doi: 10.1016/j.amepre.2008.06.024.
    1. Hanson RF, Gros KS, Davidson TM, Barr S, Cohen J, Deblinger E, Mannarino AP, Ruggiero KJ. National trainers' perspectives on challenges to implementation of an empirically-supported mental health treatment. Adm Policy Ment Health. 2014 Jul;41(4):522–34. doi: 10.1007/s10488-013-0492-6.
    1. Hanson R, Self-Brown S, Begle A. Factors related to therapist enrollment and retention in Implementation Research: Lessons learned from the BRidGE project. 2011
    1. Davidson TM, Soltis K, Albia CM, de AM, Ruggiero KJ. Providers' perspectives regarding the development of a web-based depression intervention for Latina/o youth. Psychol Serv. 2015 Feb;12(1):37–48. doi: 10.1037/a0037686.
    1. Davidson TM, Lopez CM, Saulson R, Borkman AL, Soltis K, Ruggiero KJ, de AM, Wingood GM, Diclemente RJ, Danielson CK. Development and preliminary evaluation of a behavioural HIV-prevention programme for teenage girls of Latino descent in the USA. Cult Health Sex. 2014;16(5):533–46. doi: 10.1080/13691058.2014.891049.
    1. Grubaugh A, Gros K, Davidson T, Frueh Bc, Ruggiero K. Providers’ perspectives regarding the feasibility and utility of an Internet-based mental health intervention for veterans. Psychological Trauma: Theory, Research, Practice, and Policy. 2014;6(6):624–631. doi: 10.1037/a0035772.
    1. Leon A, Davis L, Kraemer H. The role and interpretation of pilot studies in clinical research. Journal of Psychiatric Research. 2011 May;45(5):626–629. doi: 10.1016/j.jpsychires.2010.10.008.
    1. Hanson R, Deblinger E, Cohen J, Ruggiero K, Suffoletta-Maierle S. Unpublished manuscript. Charleston, SC: Unpublished manuscript; 2004. The Treatment Adherence Checklist-Revised.
    1. Chu B, Kendall P. Child Involvement Rating Scale (CIRS): Scoring manual. Piscataway, NJ Department of Clinical Psychology, GSAPP, Rutgers University: Unpublished scoring manual; 1999.
    1. Chu BC, Kendall PC. Positive association of child involvement and treatment outcome within a manual-based cognitive-behavioral treatment for children with anxiety. J Consult Clin Psychol. 2004 Oct;72(5):821–9. doi: 10.1037/0022-006X.72.5.821.
    1. Chu BC, Kendall PC. Therapist responsiveness to child engagement: flexibility within manual-based CBT for anxious youth. J Clin Psychol. 2009 Jul;65(7):736–54. doi: 10.1002/jclp.20582.
    1. Kaufman J, Birmaher B, Brent D, Rao U, Flynn C, Moreci P, Williamson D, Ryan N. Schedule for Affective Disorders and Schizophrenia for School-Age Children-Present and Lifetime Version (K-SADS-PL): initial reliability and validity data. J Am Acad Child Adolesc Psychiatry. 1997 Jul;36(7):980–8. doi: 10.1097/00004583-199707000-00021.
    1. Weissman M, Orvaschel H, Padian N. Children's symptom and social functioning self-report scales. Comparison of mothers' and children's reports. J Nerv Ment Dis. 1980 Dec;168(12):736–40.
    1. Faulstich M, Carey M, Ruggiero L, Enyart P, Gresham F. Assessment of depression in childhood and adolescence: an evaluation of the Center for Epidemiological Studies Depression Scale for Children (CES-DC) Am J Psychiatry. 1986 Aug;143(8):1024–7.
    1. Pynoos R, Rodriguez N, Steinberg A, Stuber M, Frederick C. UCLA PTSD Index for DSM IV (Adolescent version) Los Angeles, CA: UCLA Trauma Psychiatry Service; 1998.
    1. Pynoos R, Rodriguez N, Steinberg A, Stuber M, Frederick C. UCLA PTSD Index for DSM IV (Child version) Los Angeles, CA: UCLA Trauma Psychiatry Service; 1998.
    1. Rodriguez N, Steinberg A, Pynoos R. PTSD Index: Preliminary psychometric analysis of the child and parent versions. The Annual Meeting of the International Society for Traumatic Stress Studies; 2001; New Orleans, Louisiana. 2001.
    1. Shirk S, Russell R. Change Processes in Child Psychotherapy: Revitalizing Treatment and Research. NY: The Guilford Press; 1996.
    1. Shirk S, Saiz Cc. Clinical, empirical, and developmental perspectives on the therapeutic relationship in child psychotherapy. Develop. Psychopathol. 2008 Oct 31;4(04):713–28. doi: 10.1017/S0954579400004946.
    1. Elvins R, Green J. The conceptualization and measurement of therapeutic alliance: an empirical review. Clin Psychol Rev. 2008 Oct;28(7):1167–87. doi: 10.1016/j.cpr.2008.04.002.
    1. Lang J, Franks R. Child/Adolescent Satisfaction Questionnaire. Farmington, CT: Connecticut Center for Effective Practice; 2007.
    1. Pynoos R, Rodriguez N, Steinberg A, Stuber M, Frederick C. UCLA PTSD Index for DSM IV (Parent version) Los Angeles, CA: UCLA Trauma Psychiatry Service; 1998.
    1. Achenbach T. Manual for the child behavior checklist/4-18 and 1991 profile. Burlington, VT: Dept. of Psychiatry, University of Vermont; 1991.
    1. Beck AT, Ward CH, Mendelson M, Mock J, Erbaugh J. An inventory for measuring depression. Arch Gen Psychiatry. 1961 Jun;4:561–71.
    1. Beck A, Steer R, Carbin Mg. Psychometric properties of the Beck Depression Inventory: Twenty-five years of evaluation. Clinical Psychology Review. 1988 Jan;8(1):77–100. doi: 10.1016/0272-7358(88)90050-5.
    1. Rizvi SL, Dimeff LA, Skutch J, Carroll D, Linehan MM. A pilot study of the DBT coach: an interactive mobile phone application for individuals with borderline personality disorder and substance use disorder. Behav Ther. 2011 Dec;42(4):589–600. doi: 10.1016/j.beth.2011.01.003.
    1. Horvath AO, Greenberg LS. Development and validation of the Working Alliance Inventory. J Couns Psychol. 1989;36:223–33.

Source: PubMed

Sponsors and Collaborators

Medical Conditions

Drug Interventions

3
Subscribe