Objective coding of content and techniques in workplace-based supervision of an EBT in public mental health

Shannon Dorsey, Suzanne E U Kerns, Leah Lucid, Michael D Pullmann, Julie P Harrison, Lucy Berliner, Kelly Thompson, Esther Deblinger, Shannon Dorsey, Suzanne E U Kerns, Leah Lucid, Michael D Pullmann, Julie P Harrison, Lucy Berliner, Kelly Thompson, Esther Deblinger

Abstract

Background: Workplace-based clinical supervision as an implementation strategy to support evidence-based treatment (EBT) in public mental health has received limited research attention. A commonly provided infrastructure support, it may offer a relatively cost-neutral implementation strategy for organizations. However, research has not objectively examined workplace-based supervision of EBT and specifically how it might differ from EBT supervision provided in efficacy and effectiveness trials.

Methods: Data come from a descriptive study of supervision in the context of a state-funded EBT implementation effort. Verbal interactions from audio recordings of 438 supervision sessions between 28 supervisors and 70 clinicians from 17 public mental health organizations (in 23 offices) were objectively coded for presence and intensity coverage of 29 supervision strategies (16 content and 13 technique items), duration, and temporal focus. Random effects mixed models estimated proportion of variance in content and techniques attributable to the supervisor and clinician levels.

Results: Interrater reliability among coders was excellent. EBT cases averaged 12.4 min of supervision per session. Intensity of coverage for EBT content varied, with some discussed frequently at medium or high intensity (exposure) and others infrequently discussed or discussed only at low intensity (behavior management; assigning/reviewing client homework). Other than fidelity assessment, supervision techniques common in treatment trials (e.g., reviewing actual practice, behavioral rehearsal) were used rarely or primarily at low intensity. In general, EBT content clustered more at the clinician level; different techniques clustered at either the clinician or supervisor level.

Conclusions: Workplace-based clinical supervision may be a feasible implementation strategy for supporting EBT implementation, yet it differs from supervision in treatment trials. Time allotted per case is limited, compressing time for EBT coverage. Techniques that involve observation of clinician skills are rarely used. Workplace-based supervision content appears to be tailored to individual clinicians and driven to some degree by the individual supervisor. Our findings point to areas for intervention to enhance the potential of workplace-based supervision for implementation effectiveness.

Trial registration: NCT01800266 , Clinical Trials, Retrospectively Registered (for this descriptive study; registration prior to any intervention [part of phase II RCT, this manuscript is only phase I descriptive results]).

Keywords: Behavioral health; Children’s mental health; Clinical supervision; Evidence-based practice; Evidence-based treatment; Objective coding; Public mental health.

Conflict of interest statement

Ethics approval and consent to participate

All procedures were approved by the Washington State Institutional Review Board (reference number D-022412-U). Supervisors and clinicians provided consent prior to participation.

Consent for publication

Not applicable

Competing interests

Authors SD, LB, and ED have received honorariums for providing TF-CBT training. All other authors declare no conflicts of interest.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Figures

Fig. 1
Fig. 1
Content of EBT Supervision
Fig. 2
Fig. 2
Techniques of EBT Supervision

References

    1. Powell BJ, Waltz TJ, Chinman MJ, Damschroder LJ, Smith JL, Matthieu MM, et al. A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implement Sci. 2015;10:21. doi: 10.1186/s13012-015-0209-1.
    1. Beidas RS, Kendall PC. Training therapists in evidence-based practice: a critical review of studies from a systems-contextual perspective. Clin Psychol Sci Pract. 2010;17:1–30. doi: 10.1111/j.1468-2850.2009.01187.x.
    1. Herschell AD, Kolko DJ, Baumann BL, Davis AC. The role of therapist training in the implementation of psychosocial treatments: a review and critique with recommendations. Clin Psychol Rev. 2010;30:448–466. doi: 10.1016/j.cpr.2010.02.005.
    1. Beidas RS, Edmunds JM, Marcus SC, Kendall PC. Training and consultation to promote implementation of an empirically supported treatment: a randomized trial. Psychiatr Serv. 2012;63:660–665. doi: 10.1176/appi.ps.201100401.
    1. Sholomskas DE, Syracuse-Siewert G, Rounsaville BJ, Ball SA, Nuro KF, Carroll KM. We don’t train in vain: a dissemination trial of three strategies of training clinicians in cognitive–behavioral therapy. J Consult Clin Psychol. 2005;73:106–115. doi: 10.1037/0022-006X.73.1.106.
    1. Rakovshik SG, McManus F, Vazquez-Montes M, Muse K, Ougrin D. Is supervision necessary? Examining the effects of internet-based CBT training with and without supervision. J Consult Clin Psychol. 2016;84:191–199. doi: 10.1037/ccp0000079.
    1. Chorpita BF, Regan J. Dissemination of effective mental health treatment procedures: maximizing the return on a significant investment. Behav Res Ther. 2009;47:990–993. doi: 10.1016/j.brat.2009.07.002.
    1. Nadeem E, Gleacher A, Beidas RS. Consultation as an implementation strategy for evidence-based practices across multiple contexts: unpacking the black box. Adm Policy Ment Health Ment Health Serv Res. 2013;40:439–450. doi: 10.1007/s10488-013-0502-8.
    1. Edmunds JM, Beidas RS, Kendall PC. Dissemination and implementation of evidence–based practices: Training and consultation as implementation strategies. Clin Psychol. Sci Pract 2013;2:152–65.
    1. Miller WR, Yahne CE, Moyers TB, Martinez J, Pirritano M. A randomized trial of methods to help clinicians learn motivational interviewing. J Consult Clin Psychol. 2004;72:1050–1062. doi: 10.1037/0022-006X.72.6.1050.
    1. Edmunds JM, Kendall PC, Ringle VA, Read KL, Brodman DM, Pimentel SS, et al. An examination of behavioral rehearsal during consultation as a predictor of training outcomes. Adm Policy Ment Health Ment Health Serv Res. 2013;40:456–466. doi: 10.1007/s10488-013-0490-8.
    1. Stirman SW, Pontoski K, Creed T, Xhezo R, Evans AC, Beck AT, et al. A non-randomized comparison of strategies for consultation in a community-academic training program to implement an evidence-based psychotherapy. Adm Policy Ment Health Ment Health Serv Res. 2015;44:55–66. doi: 10.1007/s10488-015-0700-7.
    1. Funderburk B, Chaffin M, Bard E, Shanley J, Bard D, Berliner L. Comparing client outcomes for two evidence-based treatment consultation strategies. J Clin Child Adolesc Psychol. 2014;44:730–741. doi: 10.1080/15374416.2014.910790.
    1. Bearman SK, Weisz JR, Chorpita BF, Hoagwood K, Ward A, Ugueto AM, et al. More practice, less preach? The role of supervision processes and therapist characteristics in EBP implementation. Admin Pol Ment Health. 2013;40:518–529. doi: 10.1007/s10488-013-0485-5.
    1. Bearman SK, Schneiderman RL, Zoloth E. Building an evidence base for effective supervision practices: an analogue experiment of supervision to increase EBT fidelity. Adm Policy Ment Health Ment Health Serv Res. 2017;44:293–307. doi: 10.1007/s10488-016-0723-8.
    1. Schoenwald SK, Chapman JE, Kelleher K, Hoagwood KE, Landsverk J, Stevens J, et al. A survey of the infrastructure for children’s mental health services: implications for the implementation of empirically supported treatments (ESTs) Adm Policy Ment Health Ment Health Serv Res. 2008;35:84–97. doi: 10.1007/s10488-007-0147-6.
    1. Dorsey S, Pullmann MD, Deblinger E, Berliner L, Kerns SE, Thompson K, et al. Improving practice in community-based settings: a randomized trial of supervision—study protocol. Implement Sci. 2013;8:89. doi: 10.1186/1748-5908-8-89.
    1. Dorsey S, Pullmann MD, Kerns SEU, Jungbluth N, Meza R, Thompson K, et al. The juggling act of supervision in community mental health: implications for supporting evidence-based treatment. Admin Pol Ment Health. 2017;44:838–52.
    1. Stewart RE, Adams DR, Mandell DS, Hadley TR, Evans AC, Rubin R, et al. The perfect storm: collision of the business of mental health and the implementation of evidence-based practices. Psychiatr Serv Wash DC. 2016;67:159–161. doi: 10.1176/appi.ps.201500392.
    1. Massatti R, Sweeney H, Panzano P, Roth D. The de-adoption of innovative mental health practices (IMHP): why organizations choose not to sustain an IMHP. Adm Policy Ment Health Ment Health Serv Res. 2008;35:50–65. doi: 10.1007/s10488-007-0141-z.
    1. Stirman SW. The sustainability of new programs and innovations: a review of the empirical literature and recommendations for future research. Implement Sci. 2012;7:1–19. doi: 10.1186/1748-5908-7-1.
    1. Cohen JA, Mannarino AP. Disseminating and implementing trauma-focused CBT in community settings. Trauma Violence Abuse. 2008;9:214–226. doi: 10.1177/1524838008324336.
    1. Dorsey S, Berliner L, Lyon AR, Pullmann MD, Murray LK. A statewide common elements initiative for children’s mental health. J Behav Health Serv Res. 2016;43:246–261. doi: 10.1007/s11414-014-9430-y.
    1. Sigel BA, Benton AH, Lynch CE, Kramer TL. Characteristics of 17 statewide initiatives to disseminate trauma-focused cognitive-behavioral therapy (TF-CBT) Psychol Trauma Theory Res Pract Policy. 2013;5:323–333. doi: 10.1037/a0029095.
    1. Hoge MA, Migdole S, Farkas MS, Ponce AN, Hunnicutt C. Supervision in public sector behavioral health: a review. Clin Superv. 2011;30:183–203. doi: 10.1080/07325223.2011.604276.
    1. Accurso EC, Taylor RM, Garland AF. Evidence-based practices addressed in community-based children’s mental health clinical supervision. Train Educ Prof Psychol. 2011;5:88–96.
    1. Roth AD, Pilling S, Turner J. Therapist training and supervision in clinical trials: implications for clinical practice. Behav Cogn Psychother. 2010;38:291–302. doi: 10.1017/S1352465810000068.
    1. Henggeler S, Schoenwald S, Borduin C, Rowland M, Cunningham P. Multisystemic treatment of antisocial behavior in children and adolescents. New York: Guilford Press; 1998.
    1. Ivers N, Jamtvedt G, Flottorp S, Young JM, Odgaard-Jensen J, French SD, et al. Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Dat Sys Rev. 2012;6:1–227.
    1. Henggeler SW, Schoenwald SK, Liao JG, Letourneau EJ, Edwards DL. Transporting efficacious treatments to field settings: the link between supervisory practices and therapist fidelity in MST programs. J Clin Child Adolesc Psychol. 2002;31:155–167. doi: 10.1207/S15374424JCCP3102_02.
    1. Schoenwald SK, Sheidow AJ, Chapman JE. Clinical supervision in treatment transport: effects on adherence and outcomes. J Consult Clin Psychol. 2009;77:410–421. doi: 10.1037/a0013788.
    1. Bradshaw T, Butterworth A, Mairs H. Does structured clinical supervision during psychosocial intervention education enhance outcome for mental health nurses and the service users they work with? J Psychiatr Ment Health Nurs. 2007;14:4–12. doi: 10.1111/j.1365-2850.2007.01021.x.
    1. Schoenwald SK, Mehta TG, Frazier SL, Shernoff ES. Clinical supervision in effectiveness and implementation research. Clin Psychol Sci Pract. 2013;20:44–59. doi: 10.1111/cpsp.12022.
    1. Bickman L. Our quality-assurance methods aren’t so sure. Behav Healthc Tomorrow. 2000;9:41–42.
    1. Beidas RS, Marcus S, Aarons GA, Hoagwood KE, Schoenwald S, Evans AC, et al. Predictors of community therapists’ use of therapy techniques in a large public mental health system. JAMA Peds. 2015;169:374–382. doi: 10.1001/jamapediatrics.2014.3736.
    1. Brookman-Frazee L, Haine RA, Baker-Ericzén M, Zoffness R, Garland AF. Factors associated with use of evidence-based practice strategies in usual care youth psychotherapy. Adm Policy Ment Health Ment Health Serv Res. 2010;37:254–269. doi: 10.1007/s10488-009-0244-9.
    1. Cohen JA, Mannarino AP, Deblinger E. Treating trauma and traumatic grief in children and adolescents. New York City: Guilford Press; 2006.
    1. McLeod BD. The therapy process observational coding system for child psychotherapy. Los Angeles: University of California, Los Angeles; 2001.
    1. McLeod BD, Weisz JR. The therapy process observational coding system for child psychotherapy strategies scale. J Clin Child Adolesc Psychol. 2010;39:436–443. doi: 10.1080/15374411003691750.
    1. Garland AF, Brookman-Frazee L, Hurlburt MS, Accurso EC, Zoffness RJ, Haine-Schlagel R, et al. Mental health care for children with disruptive behavior problems: a view inside therapists’ offices. Psychiatr Serv. 2010;61:788–795. doi: 10.1176/ps.2010.61.8.788.
    1. Hanson RF, Gros KS, Davidson TM, Barr S, Cohen J, Deblinger E, et al. National trainers’ perspectives on challenges to implementation of an empirically-supported mental health treatment. Admin Pol Ment Health. 2014;41:522–534. doi: 10.1007/s10488-013-0492-6.
    1. Milne DL, & Reiser RP. A manual for evidence-based CBT supervision. Wiley; 2017.
    1. Milne D, Aylott H, Fitzpatrick H, Ellis MV. How does clinical supervision work? Using a “best evidence synthesis” approach to construct a basic model of supervision. Clin Superv. 2008;27:170–190. doi: 10.1080/07325220802487915.
    1. Milne DL, Reiser RP, Cliffe T, Breese L, Boon A, Raine R, et al. A qualitative comparison of cognitive-behavioural and evidence-based clinical supervision. Cogn Behav Ther. 2011;4:152–166.
    1. Nakamura BJ, Selbo-Bruns A, Okamura K, Chang J, Slavin L, Shimabukuro S. Developing a systematic evaluation approach for training programs within a train-the-trainer model for youth cognitive behavior therapy. Behav Res Ther. 2014;53:10–19. doi: 10.1016/j.brat.2013.12.001.
    1. Cicchetti DV. Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychol Assess. 1994;6:284–290. doi: 10.1037/1040-3590.6.4.284.
    1. Cicchetti DV, Feinstein AR. High agreement but low kappa: II. Resolving the paradoxes. J Clin Epidemiol. 1990;43:551–558. doi: 10.1016/0895-4356(90)90159-M.
    1. Hutchinson TP. Kappa muddles together two sources of disagreement: tetrachoric correlation is preferable. Res Nurs Health. 1993;16:313–316. doi: 10.1002/nur.4770160410.
    1. Banerjee M, Capozzoli M, Mcsweeney L, Sinha D. Beyond kappa: a review of interrater agreement measures. Can J Stat. 1999;27:3–23. doi: 10.2307/3315487.
    1. Kolko DJ, Cohen JA, Mannarino AP, Baumann BL, Knudsen K. Community treatment of child sexual abuse: a survey of practitioners in the National Child Traumatic Stress Network. Adm Policy Ment Health Ment Health Serv Res. 2009;36:37–49. doi: 10.1007/s10488-008-0180-0.
    1. Borntrager C, Chorpita BF, Higa-McMillan CK, Daleiden EL, Starace N. Usual care for trauma-exposed youth: are clinician-reported therapy techniques evidence-based? Child Youth Serv Rev. 2013;35:133–141. doi: 10.1016/j.childyouth.2012.09.018.
    1. Forgatch M, Patterson G, Gewirtz A. Looking forward: the promise of widespread implementation of parent training programs. Perspect Psychol Sci. 2013;8:682–694. doi: 10.1177/1745691613503478.
    1. Southam-Gerow MA, Weisz JR, Kendall PC. Youth with anxiety disorders in research and service clinics: examining client differences and similarities. J Clin Child Adolesc Psychol. 2003;32:375–385. doi: 10.1207/S15374424JCCP3203_06.
    1. Lyon AR, Dorsey S, Pullmann M, Silbaugh-Cowdin J, Berliner L. Clinician use of standardized assessments following a common elements psychotherapy training and consultation program. Adm Policy Ment Health Ment Health Serv Res. 2014;42:47–60. doi: 10.1007/s10488-014-0543-7.
    1. Beidas RS, Maclean JC, Fishman J, Dorsey S, Schoenwald SK, Mandell DS, Marcus SC. A randomized trial to identify accurate and cost-effective fidelity measurement methods for cognitivebehavioral therapy: Project FACTS study protocol. BMC Psychiatry. 2016;16(1):323. .
    1. Schoenwald SK. It’s a bird, it’s a plane, it’s... fidelity measurement in the real world. Clin Psychol Sci Pract. 2011;18:142–147. doi: 10.1111/j.1468-2850.2011.01245.x.
    1. Schoenwald SK, Garland AF, Chapman JE, Frazier SL, Sheidow AJ, Southam-Gerow MA. Toward the effective and efficient measurement of implementation fidelity. Adm Policy Ment Health Ment Health Serv Res. 2011;38:32–43. doi: 10.1007/s10488-010-0321-0.
    1. Wiltsey Stirman S, Gutner CA, Gamarra J, Vogt D, Suvak M, Watchen JS, et al. Leveraging routine clinical materials to assess fidelity to an evidence-based psychotherapy. In: Lewis CC, editor. 3rd Bienn. Conf. Soc. Implement. Res. Collab. SIRC 2015 Adv. Effic. Methodol. Community Partnersh. Team Sci. A12. Seattle, WA; 2016.
    1. Beidas RS, Cross W, Dorsey S. Show me, don’t tell me: behavioral rehearsal as a training and analogue fidelity tool. Cogn Behav Pract. 2014;21:1–11. doi: 10.1016/j.cbpra.2013.04.002.
    1. Kavanagh DJ, Spence S, Sturk H, Strong J, Wilson J, Worrall L, et al. Outcomes of training in supervision: randomised controlled trial. Aust Psychol. 2008;43:96–104. doi: 10.1080/00050060802056534.
    1. Beidas RS, Marcus S, Wolk CB, Powell B, Aarons GA, Evans AC, et al. A prospective examination of clinician and supervisor turnover within the context of implementation of evidence-based practices in a publicly-funded mental health system. Adm Policy Ment Health Ment Health Serv Res. 2016;43:640–649. doi: 10.1007/s10488-015-0673-6.

Source: PubMed

Sponsorer og samarbejdspartnere

Medicinske tilstande

Narkotikainterventioner

3
Abonner