Development and evaluation of a remote training strategy for the implementation of mental health evidence-based practices in rural schools: pilot study protocol

Ricardo Eiraldi, Barry L McCurdy, Muniya S Khanna, Jessica Goldstein, Rachel Comly, Jennifer Francisco, Laura E Rutherford, Tara Wilson, Kathryn Henson, Thomas Farmer, Abbas F Jawad, Ricardo Eiraldi, Barry L McCurdy, Muniya S Khanna, Jessica Goldstein, Rachel Comly, Jennifer Francisco, Laura E Rutherford, Tara Wilson, Kathryn Henson, Thomas Farmer, Abbas F Jawad

Abstract

Background: An increasing number of schools in rural settings are implementing multi-tier positive behavioral interventions and supports (PBIS) to address school-climate problems. PBIS can be used to provide the framework for the implementation of evidence-based practices (EBPs) to address children's mental health concerns. Given the large service disparities for children in rural areas, offering EBPs through PBIS can improve access and lead to better long-term outcomes. A key challenge is that school personnel need technical assistance in order to implement EBPs with fidelity and clinical effectiveness. Providing ongoing on-site support is not feasible or sustainable in the majority of rural schools, due to their remote physical location. For this reason, remote training technology has been recommended for providing technical assistance to behavioral health staff (BHS) in under-served rural communities.

Objectives: The purpose of this study is to use the user-centered design, guided by an iterative process (rapid prototyping), to develop and evaluate the appropriateness, feasibility, acceptability, usability, and preliminary student outcomes of two online training strategies for the implementation of EBPs at PBIS Tier 2.

Methods: The study will employ a pragmatic design comprised of a mixed-methods approach for the development of the training platform, and a hybrid type 2, pilot randomized controlled trial to examine the implementation and student outcomes of two training strategies: Remote Video vs. Remote Video plus Coaching.

Discussion: There is a clear need for well-designed remote training studies focused on training in non-traditional settings. Given the lack of well-trained mental health professionals in rural settings and the stark disparities in access to services, the development and pilot-testing of a remote training strategy for BHS in under-served rural schools could have a significant public health impact.

Ethics and dissemination: The project was reviewed and approved by the institutional review board. Results will be submitted to ClinicalTrials.gov and disseminated to community partners and participants, peer-reviewed journals, and academic conferences.

Trial registration: ClinicialTrials.gov, NCT05034198 and NCT05039164.

Keywords: Mental health evidence-based practices; Positive behavioral interventions and supports; Rural schools; Tier 2; User-centered design.

Conflict of interest statement

The authors declare that they have no competing interests.

© 2022. The Author(s).

Figures

Fig. 1
Fig. 1
Study flowchart
Fig. 2
Fig. 2
Development and evaluation of remote training platform
Fig. 3
Fig. 3
Interactive systems framework for dissemination and implementation

References

    1. Smalley KB, Yancey CT, Warren JC, Naufel K, Ryan R, Pugh JL. Rural mental health and psychological treatment: a review for practitioners. J Clin Psychol. 2010;66(5):479–489. doi: 10.1002/jclp.20688.
    1. Wagenfeld MO. A snapshot of rural and frontier America. In: Stamm BH, editor. Rural behavioral health care: an interdisciplinary guide. Am Psychol Assoc; 2003. pp. 33–40.
    1. Anderson NJ, Neuwirth SJ, Lenardson JD, Hartley D. Patterns of care for rural and urban children with mental health problems. 2013.
    1. Owens JS, Watabe Y, Michael KD. Culturally responsive school mental health in rural communities. In: Clauss-Ehlers CS, Serpell Z, Weist MD, editors. Handbook of culturally responsive school mental health: advancing research, training, practice, and policy. Springer; 2013. pp. 31–42.
    1. Sugai G, Horner R. The evolution of discipline practices: school-wide positive behavior supports. Child Fam Beh Therapy. 2002;24:23–50. doi: 10.1300/J019v24n01_03.
    1. Horner RH, Sugai G, Smolkowski K, et al. A randomized, wait-list controlled effectiveness trial assessing school-wide positive behavior support in elementary schools. J Posit Behav Interv. 2009;11(3):133–144. doi: 10.1177/1098300709332067.
    1. Cavanaugh B, Swan M. Building SWPBIS capacity in rural schools through building-based coaching: early findings from a district-based model. Rural Sp Edu Quart. 2015;34(4):29–39. doi: 10.1177/875687051503400404.
    1. Fitzgerald CB, Geraci LM, Swanson M. Scaling up in rural schools using positive behavioral interventions and supports. Rural Sp Edu Quart. 2014;33(1). 10.1177/875687051403300104.
    1. McCrary D, Lechtenberger D, Wang E. The effect of schoolwide positive behavioral supports on children in impoverished rural community schools. Prev Sch Fail. 2012;56(1):1–7. doi: 10.1080/1045988X.2010.548417.
    1. Steed EA, Pomerleau T, Muscott H, Rohde L. Program-wide positive behavioral interventions and supports in rural preschools. Rural Sp Edu Quart. 2013;32(1):38–46. doi: 10.1177/875687051303200106.
    1. Elias MJ, Zins JE, Graczyk PA, Weissberg RP. Implementation, sustainability, and scaling up of social-emotional and academic innovations in public schools. Sch Psychol Rev. 2003;32(3):303–319. doi: 10.1080/02796015.2003.12086200.
    1. Eiraldi R, Mautone JA, Khanna MS, et al. Group CBT for externalizing disorders in urban schools: effect of training strategy on treatment fidelity and patient outcomes. Behav Ther. 2018. 10.1016/j.beth.2018.01.001.
    1. Eiraldi R, McCurdy B, Khanna M, et al. A cluster randomized trial to evaluate external support for the implementation of positive behavioral interventions and supports by school personnel. Implement Sci. 2014;9(12). 10.1186/1748-5908-9-12.
    1. Garbacz AS, Watkins ND, Diaz Y, Barnabas ERJ, Schwartz B, Eiraldi R. Using conjoint behavioral consultation to implement evidence-based practices for students in low-income urban schools. J Educ Psychol Consult. 2016;61(3):198–210. doi: 10.1080/1045988X.2016.1261078.
    1. Eiraldi R, Power TJ, Schwartz BS, et al. Examining effectiveness of group cognitive-behavioral therapy for externalizing and internalizing disorders in urban schools. Behav Modif. 2016. 10.1177/0145445516631093.
    1. Eiraldi R, McCurdy B, Schwartz B, et al. Pilot study for the fidelity, acceptability and effectiveness of a PBIS program plus mental health supports in under-resourced urban schools. Psychol Sch. 2019:1–16. 10.1002/pits.22272.
    1. Khanna MS, Kendall PC. Bringing technology to training: Web-based therapist training to promote the development of competent cognitive-behavioral therapists. Cogn Behav Pract. 2015;22(3):291–301. doi: 10.1016/j.cbpra.2015.02.002.
    1. Riding-Malon R, Werth JL. Psychological practice in rural settings: at the cutting edge. Prof Psychol. 2014;45(2):85–91. doi: 10.1037/a0036172.
    1. Lyon AR, Koerner K. User-centered design for psychosocial intervention development and implementation. Clin Psychol (New York) 2016;23(2):180–200. doi: 10.1111/cpsp.12154.
    1. Larman C, Basili VR. Iterative and incremental development: a brief history. Computer. 2003;36:47–56. doi: 10.1109/MC.2003.1204375.
    1. Wilson J, Rosenberg D. Rapid prototyping for user interface design. In: Helander M, editor. Handbook of human-computer interaction. North-Holland: Elsevier; 1988. p. 859–75.
    1. Maguire M. Methods to support human-centred design. Inter J Human-Comp St. 2001;55:587–634. doi: 10.1006/ijhc.2001.0503.
    1. Mohr DC, Lyon AR, Lattie EG, et al. Accelerating digital mental health research from early design and creation to successful implementation and sustainment. J Med Internet Res. 2017;19(5):e153. doi: 10.2196/jmir.7725.
    1. Kobak KA, Craske MG, Rose RD, et al. Web-based therapist training on cognitive behavior therapy for anxiety disorders: a pilot study. Psychother. 2013;50(2):235–247. doi: 10.1037/a0030568.
    1. Barnett JE. Utilizing technological innovations to enhance psychotherapy supervision, training, and outcomes. Psychother. 2011;48:103–108. doi: 10.1037/a0023381.
    1. Abbass A, Arthey S, Elliott J, et al. Web-conference supervision for advanced psychotherapy training: a practical guide. Psychother. 2011;48(2):109–118. doi: 10.1037/a0022427.
    1. Weisz JR. Agenda for child and adolescent psychotherapy research: on the need to put science into practice. Arch Gen Psychiatry. 2000;57(9):837–838. doi: 10.1001/archpsyc.57.9.837.
    1. Cohen J, Mannarino AP. Disseminating and implementing trauma-focused CBT in community settings. Trauma Viol Ab. 2008;9(4):214–226. doi: 10.1177/1524838008324336.
    1. Cully JA, Curry AD, Ryan SR, et al. Development of a computer-aided training program for brief cognitive-behavioral therapy in primary care. Acad Psychiatry. 2013;37(2):120–124. doi: 10.1176/appi.ap.11040078.
    1. Fischer AJ, Dart EH, Leblanc H, et al. An investigation of the acceptability of videoconferencing within a school-based behavioral consultation framework. Psychol Sch. 2017;53(3):240–252. doi: 10.1002/pits.21.
    1. Bice-Urbach BJ, Kratochwill TR. Teleconsultation: The use of technology to improve evidence-based practices in rural communities. J Sch Psychol. 2016;56:27–43. 10.1016/j.jsp.2016.02.001.
    1. King HC, Bloomfield BS, Wu S, et al. A systematic review of school teleconsultation: implications for research and practice. Sch Psychol Rev. 2021. 10.1080/2372966X.2021.1894478.
    1. Rakovshik SG, McManus F, Westbrook D, Kholmogorova AB, et al. Randomized trial comparing internet-based training in cognitive behavioural therapy theory, assessment and formulation to delayed-training control. Behav Res Ther. 2013;51:231–239. doi: 10.1016/j.brat.2013.01.009.
    1. Herschell AD, Kolko DJ, Baumann BL, Davis AC. The role of therapist training in the implementation of psychosocial treatments: a review and critique with recommendations. Clin Psychol Rev. 2010;30(4):448–466. doi: 10.1016/j.cpr.2010.02.005.
    1. Beidas RS, Kendall PC. Training therapists in evidence-based practice: a critical review of studies from a systems-contextual perspective. Clin Psychol Sci Pract. 2010;17(1):1–30. doi: 10.1111/j.1468-2850.2009.01187.x3.
    1. Webster-Stratton CH, Reid MJ, Marsenich L. Improving therapist fidelity during implementation of evidence-based practices: incredible years program. Psychiatr Serv. 2014;65(6):789–795. doi: 10.1176/appi.ps.201200177.
    1. Proctor E, Silmere H, Raghavan R, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Admin Pol Ment Health. 2011;38(2):65–76. doi: 10.1007/s10488-010-0319-7.
    1. Lewis CC, Proctor EK, Brownson RC. Measurement issues in dissemination and implementation research. In: Brownson RC, Colditz GA, Proctor EK, editors. Dissemination and implementation research in health. 2. Oxford University Press; 2018. pp. 229–244.
    1. Lewis CC, Fischer S, Weiner BJ, Stanick C, Kim M, Martinez RG. Outcomes for implementation science: an enhanced systematic review of instruments using evidence-based rating criteria. Implement Sci. 2015;10:155. doi: 10.1186/s13012-015-0342-x.
    1. Weiner BJ, Lewis CC, Stanick C, et al. Psychometric assessment of three newly developed implementation outcome measures. Implement Sci. 2017;12:108. doi: 10.1186/s13012-017-0635-3.
    1. Brooke J. SUS: a quick and dirty usability scale. Usab Eval Indust. 1995;189(194):4–7.
    1. Lyon AR, Munson SA, Renn BN, et al. Use of human-centered design to improve implementation of evidence-based psychotherapies in low-resource communities: protocol for studies applying a framework to assess usability. JMIR Res Protoc. 2019;8:e14990. doi: 10.2196/14990.
    1. Bowen DJ, Kreuter M, Spring B, et al. How we design feasibility studies. Am J Prev Med. 2009;36(5):452–457. doi: 10.1016/j.amepre.2009.02.002.
    1. Curran GM, Bauer M, Mittman B, et al. Effectiveness-implementation hybrid designs: combining elements of clinical effectiveness and implementation research to enhance public health impact. Med Care. 2012;50(3):217–226. doi: 10.1097/MLR.0b013e3182408812.
    1. Hawken LS, Adolphson SL, Macleod S, et al. et al. Secondary-tier interventions and supports. In: Saylor W, et al., editors. Handbook of positive behavior support. Springer; 2009. pp. 395–420.
    1. Goodman R, Ford T, Simmons H, et al. Using the Strengths and Difficulties Questionnaire (SDQ) to screen for child psychiatric disorders in a community sample. Br J Psychiatry. 2000;177:534–539. doi: 10.1192/bjp.177.6.534.
    1. Stone LL, Otten R, Engels RCME, et al. Psychometric properties of the parent and teacher versions of the Strengths and Difficulties Questionnaire for 4- to 12-year-olds: a review. Clin Child Fam Psychol Rev. 2010;13(3):254–274. doi: 10.1007/s10567-010-0071-2.
    1. Eiraldi R, Khanna M, Jawad AF, et al. Implementation of targeted mental health interventions in urban schools: preliminary findings on the impact of training strategy on program fidelity. Evid Based Pract Child Adolesc Ment Health. 2020;5:437–451. doi: 10.1080/23794925.2020.1784056.
    1. Todd AW, Campbell AL, Meyer GB, et al. The effects of a targeted intervention to reduce problem behaviors. J Pos Behav Interv. 2008;10(1):46–55. doi: 10.1177/1098300707311369.
    1. Hawken LH, Homer RH. Evaluation of a targeted intervention within a schoolwide system of behavior support. J Behav Educ. 2003;12(3):225–240. doi: 10.1023/A:102551241.
    1. Filter KJ, McKenna MK, Benedict EA, et al. Check in/Check out: a post-hoc evaluation of an efficient, secondary-level targeted intervention for reducing problem behaviors in schools. Educ Treat Child. 2007;30(1):69–84. doi: 10.1353/etc.2007.0000.
    1. Reynolds CR, Kamphaus RW. Manual for the behavior assessment system for children - third Ed (BASC-3) Pearson Psych Corp; 2015.
    1. Skinner EA, Kindermann TA, Furrer CJ. A Motivational perspective on engagement and disaffection: conceptualization and assessment of children’s behavioral and emotional participation in academic activities in the classroom. Educ Psychol Meas. 2009;69(3):493–525. doi: 10.1177/0013164408323233.
    1. Skinner E, Furrer C, Marchand G, et al. Engagement and disaffection in the classroom: part of a larger motivational dynamic? J Educ Psychol. 2008;100(4):765–781. doi: 10.1037/a0012840.
    1. Eiraldi R, McCurdy BL, Khanna MS, et al. Study protocol: cluster randomized trial of consultation strategies for the sustainment of mental health interventions in under-resourced urban schools: rationale, design, and methods. BMC Psychol. 2022;10(1):24. doi: 10.1186/s40359-022-00733-8.
    1. Foster S, Rollefson M, Doksum T, et al. School mental health services in the United States, 2002-2003. Rockville: Center for Mental Health Services, Substance Abuse and Mental Health Administration; 2005.
    1. Lochman JE, Wells KC, Lenhart L. Coping power child group program: facilitator guide. New York: Oxford University Press; 2008.
    1. Khanna MS, Eiraldi R, Schwartz B, et al. CBT for anxiety treatment in schools. 2016.
    1. Crone DA, Hawken LS, Horner RH. Responding to problem behavior in schools: the behavior education program. 2. Guilford; 2010.
    1. Lochman JE, Wells KC. The coping power program for preadolescent aggressive boys and their parents: outcome effects at the 1-year follow-up. J Consult Clin Psychol. 2004;72(4):571–578. doi: 10.1037/0022-006X.72.4.571.
    1. Ellis ML, Lindsey MA, Barker ED, et al. Predictors of engagement in a school-based family preventive intervention for youth experiencing behavioral difficulties. Prev Sci. 2013;14(5):457–467. doi: 10.1007/s11121-012-0319-9.
    1. Barrett P. Friends for life: group leaders’ manual for children. Australia: Pathways Health and Research Centre; 2008.
    1. Hawken LS, MacLeod KS, Rawlings L. Effects of the behavior education program (BEP) on office discipline referrals of elementary school students. J Posit Behav Interv. 2007;9(2):94–101. doi: 10.1177/10983007070090020601.
    1. McCurdy BL, Kunsch C, Reibstein S. Secondary prevention in the urban school: implementing the behavior education program. Prev Sch Fail. 2007;51(3):12–19. doi: 10.3200/PSFL.51.3.12-19.
    1. Kirchner JE, Waltz TJ, Powell BJ, et al. Implementation strategies. In: Browson RC, Colditz GA, Proctor EK, et al., editors. Dissemination and implementation research in health: translating science to practice. 2. Oxford University Press; 2018.
    1. Kern L, Evans SW, Lewis TJ. Description of an iterative process for intervention development. Educ Treat Child. 2011;34(4):593–617. doi: 10.1353/etc.2011.0037.
    1. Kolb DA. Experiential learning: experience as the source of learning and development. Upper Saddle River: Prentice-Hall; 1984.
    1. Barrett S, Eber L, Weist M. Advancing education effectiveness: interconnecting school mental health and school-wide positive behavior support. 2013.
    1. Wandersman A, Duffy J, Flashpohler P, et al. Bridging the gap between prevention research and practice: the interactive systems framework for dissemination and implementation. Am J Community Psychol. 2008;41(3-4):171–181. doi: 10.1007/s10464-008-9174-z.
    1. Wandersman A, Chien VH, Katz J. Toward an evidence-based system for innovation support for implementing innovations with quality: tools, training, technical assistance, and quality assurance/quality improvement. Am J Community Psychol. 2012;50(3-4). 10.1007/s10464-012-9509-7.
    1. American Psychological Association Guidelines for clinical supervision in health service psychology. Am Psychol. 2015;70(1):33–46. doi: 10.1037/a0038112.
    1. Lewis TJ, Barrett S, Sugai G, et al. Blueprint for school-wide positive behavior support training and professional development (Ver 3). Eugene: University of Oregon, Center for Positive Behavioral Interventions and Supports; 2010.
    1. Sburlati ES, Scheniering CA, Lyneham HJ, et al. A model of therapist competencies for the empirically supported cognitive behavioral treatment of child and adolescent anxiety and depressive disorders. Clin Child Fam Psychol Rev. 2011;14(1):89–109. doi: 10.1007/s10567-011-0083-6.
    1. Beidas RS, Kendall PC. Training therapists in evidence-based practice: a critical review of studies from a systems-contextual perspective. Clin Psychol Sci Pract. 2010;17(1):1–30. doi: 10.1111/j.1468-2850.2009.01187.x.
    1. Merriam S. The changing landscape of adult learning theory. In: Comings J, Garner B, Smith C, editors. Review of adult learning and literacy: connecting research, policy and practice. Mahwah: Lawrence Erlbaum Associates; 2004. pp. 199–220.
    1. Locke EA, Latham GP. Building a practically useful theory of goal setting and task motivation: a 35-year odyssey. Am Psychol. 2002;57(9):705. doi: 10.1037/0003-066X.57.9.705.
    1. Denton CA, Hasbrouck J. A description of instructional coaching and its relationship to consultation. J Educ Psychol Consult. 2009;19:150–175. doi: 10.1080/10474410802463296.
    1. Kluger AN, DeNisi A. The effects of feedback interventions on performance: a historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychol Bull. 1996;119(2):254–284. doi: 10.1037/0033-2909.119.2.254.
    1. Stiles PG, Boothroyd RA, Snyder K, et al. Service penetration by persons with severe mental illness. How should it be measured? J Behav Health Serv Res. 2002;29(2):198. doi: 10.1007/BF02287706.
    1. Weisz JR, Vaughn-Coaxum RA, Evans SC, et al. Efficient monitoring of treatment response during youth psychotherapy: the Behavior and Feelings Survey. J Clin Child Adolesc Psychol. 2020;49(6):737–751. doi: 10.1080/15374416.2018.1547973.
    1. SAS/STAT® 9.2 User’s guide. SAS Institute Inc.; 2002-2008.
    1. Bradley EH, Curry LA, Devers KJ. Qualitative data analysis for health services research: developing taxonomy, themes, and theory. Health Serv Res. 2007;42(4):1758–1772. doi: 10.1111/j.1475-6773.2006.00684.x.
    1. Palinkas LA, Aarons GA, Horwitz S, et al. Mixed method designs in implementation research. Admin Pol Ment Health. 2011;38(1):44–53. doi: 10.1007/s10488-010-0314-z.
    1. Palinkas LA, Horwitz SM, Chamberlain P, et al. Mixed-methods designs in mental health services research: a review. Psychiatr Serv. 2011;62(3):255–263. doi: 10.1176/appi.ps.62.3.255.
    1. Rye CB, Kimberly JR. The adoption of innovations by provider organizations in health care. Med Care Res Rev. 2007;64(3):235–278. doi: 10.1177/1077558707299865.
    1. Schoenwald SK, Garland AF, Chapman JE, et al. Toward the effective and efficient measurement of implementation fidelity. Admin Pol Ment Health. 2011;38(1):32–43. doi: 10.1007/s10488-010-0321-0.
    1. Zeger SL, Liang KY. Longitudinal data analysis for discrete and continuous outcomes. Biometrics. 1986;42(1):121–130. doi: 10.2307/2531248.
    1. Liang KW, Zeger SL. Longitudinal data analyses using generalized linear models. Biometrika. 1986;73:13–22. doi: 10.1093/biomet/73.1.13.
    1. Power Analysis and Sample Size Software. NCSS, LLC; 2014. .
    1. Damschroder LJ, Aron DC, Keith RE, et al. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4:50. doi: 10.1186/1748-5908-4-50.
    1. Davidson TM, Ruggiero KJ, Egede LE. Promoting reach, dissemination, and engagement of technologies for addressing mental health care disparities among underserved populations. Clin Psychol. 2019;26(1):1–3.
    1. Caulfield A, Vatansever D, Lambert G, et al. WHO guidance on mental health training: a systematic review of the progress for non-specialist health workers. BMJ Open. 2019;9:e024059. doi: 10.1136/bmjopen-2018-024059.
    1. Wade V, Gray L, Carati C. Theoretical frameworks in telemedicine research. J Telemed Telecare. 2017;23(1):181–187. doi: 10.1177/1357633X15626650.
    1. Whitten P, Johannessen LK, Soerensen T, et al. A systematic review of research methodology in telemedicine studies. J Telemed Telecare. 2007;13(5):230–235. doi: 10.1258/135763307781458976.
    1. Collins FS, Tabak LA. Policy: NIH plans to enhance reproducibility. Nature. 2014;505(7485):612–613. doi: 10.1038/505612a.
    1. National Institutes of Health . Reproducibility standards. 2014.

Source: PubMed

Sponsorit ja yhteistyökumppanit

Sairaudet

Huumeiden interventiot

3
Tilaa