Improving practice in community-based settings: a randomized trial of supervision - study protocol

Shannon Dorsey, Michael D Pullmann, Esther Deblinger, Lucy Berliner, Suzanne E Kerns, Kelly Thompson, Jürgen Unützer, John R Weisz, Ann F Garland, Shannon Dorsey, Michael D Pullmann, Esther Deblinger, Lucy Berliner, Suzanne E Kerns, Kelly Thompson, Jürgen Unützer, John R Weisz, Ann F Garland

Abstract

Background: Evidence-based treatments for child mental health problems are not consistently available in public mental health settings. Expanding availability requires workforce training. However, research has demonstrated that training alone is not sufficient for changing provider behavior, suggesting that ongoing intervention-specific supervision or consultation is required. Supervision is notably under-investigated, particularly as provided in public mental health. The degree to which supervision in this setting includes 'gold standard' supervision elements from efficacy trials (e.g., session review, model fidelity, outcome monitoring, skill-building) is unknown. The current federally-funded investigation leverages the Washington State Trauma-focused Cognitive Behavioral Therapy Initiative to describe usual supervision practices and test the impact of systematic implementation of gold standard supervision strategies on treatment fidelity and clinical outcomes.

Methods/design: The study has two phases. We will conduct an initial descriptive study (Phase I) of supervision practices within public mental health in Washington State followed by a randomized controlled trial of gold standard supervision strategies (Phase II), with randomization at the clinician level (i.e., supervisors provide both conditions). Study participants will be 35 supervisors and 130 clinicians in community mental health centers. We will enroll one child per clinician in Phase I (N = 130) and three children per clinician in Phase II (N = 390). We use a multi-level mixed within- and between-subjects longitudinal design. Audio recordings of supervision and therapy sessions will be collected and coded throughout both phases. Child outcome data will be collected at the beginning of treatment and at three and six months into treatment.

Discussion: This study will provide insight into how supervisors can optimally support clinicians delivering evidence-based treatments. Phase I will provide descriptive information, currently unavailable in the literature, about commonly used supervision strategies in community mental health. The Phase II randomized controlled trial of gold standard supervision strategies is, to our knowledge, the first experimental study of gold standard supervision strategies in community mental health and will yield needed information about how to leverage supervision to improve clinician fidelity and client outcomes.

Trial registration: ClinicalTrials.gov NCT01800266.

References

    1. Weisz JR, Jensen-Doss JR, Hawley KM. Youth psychotherapy outcome research: a review and critique of the evidence base. Annual Rev Psychol. 2005;56:337–363. doi: 10.1146/annurev.psych.55.090902.141449.
    1. Chambless DL, Ollendick TH. Empirically supported psychological interventions: controversies and evidence. Annual Rev Psychol. 2001;52:685–716. doi: 10.1146/annurev.psych.52.1.685.
    1. Weisz JR, McCarty CA, Valeri SM. Effects of psychotherapy for depression in children and adolescents: a meta-analysis. Psychol Bull. 2006;132:132–149.
    1. Weisz JR, Doss AJ, Haley KM. Evidence-based youth psychotherapies versus usual clinical care: a meta-analysis of direct comparisons. Am Psychol. 2006;61:671–689.
    1. Weisz JR, Kuppens S, Eckshtain D, Ugueto AM, Hawley KM, Jensen-Doss A. Do evidence-based youth psychotherapies outperform usual clinical care? a multilevel meta-analysis. JAMA Psychiatry. in press.
    1. Fixsen DL, Naoom SF, Blase KA, Friedman RM, Wallace F. Implementation research: a synthesis of the literature (FMHI publication No. 231) Tampa, FL: The National Implementation Research Network; 2005. (University of South Florida, Louis de la Parte Florida mental health institute).
    1. Mitchell PF. Evidence-based practice in real-world services for young people with complex needs: new opportunities suggested by recent implementation science. Child Youth Serv Rev. 2011;33:207–216. doi: 10.1016/j.childyouth.2010.10.003.
    1. Beidas RS, Kendall PC. Training therapists in evidence-based practice: a critical review of studies from a systems-contextual perspective. Clin Psychol Sci Pract. 2010;17:1–30. doi: 10.1111/j.1468-2850.2009.01187.x.
    1. Herschell AD, Kolko DJ, Baumann BL, Davis AC. The role of therapist training in the implementation of psychosocial treatments: a review and critique with recommendations. Clin Psychol Rev. 2010;30:448–466. doi: 10.1016/j.cpr.2010.02.005.
    1. Ellis M, Krengel M, Ladany N, Schult D. Clinical supervision research from 1981–1993: a methodological critique. J Counsel Psychol. 1996;43:35–50.
    1. Kilminster SM, Jolly BC. Effective supervision in clinical practice settings: a literature review. Med Ed. 2000;34:827–840. doi: 10.1046/j.1365-2923.2000.00758.x.
    1. Callahan JL, Almstrom CM, Swift JK, Borja SE, Heath CJ. Exploring the contribution of supervisors to intervention outcomes. Train Ed Prof Psychol. 2009;3:72–77.
    1. Schoenwald S, Carter RE, Chapman JE, Sheidow AJ. Therapist adherence and organizational effects on change in youth behavior problems one year after Multisystemic Therapy. Admin Policy Mental Health Mental Health Serv Res. 2008;35:379–394. doi: 10.1007/s10488-008-0181-z.
    1. Schoenwald SK, Mehta TG, Frazier SL, Shernoff ES. Clinical supervision in effectiveness and implementation research. Clin Psychol Sci Pract. 2013;20:44–59. doi: 10.1111/cpsp.12022.
    1. Miller C, Gruskin S, Subramanian S, Rajaraman D, Heymann S. Orphan care in Botswana’s working households: growing responsibilities in the absence of adequate support. Am J Public Health. 2006;96:1429–1435. doi: 10.2105/AJPH.2005.072280.
    1. Schoenwald SK, Sheidow AJ, Chapman JE. Clinical supervision in treatment transport: effects on adherence and outcomes. J Consul Clin Psychol. 2009;77:410–421.
    1. Accurso EC, Taylor RM, Garland AF. Evidence-based practices addressed in community-based children’s mental health clinical supervision. Train Ed Prof Psychol. 2011;5:88–96.
    1. Carroll KM, Rounsaville BJ. A vision of the next generation of behavioral therapies research in the addictions. Addiction. 2007;102:850–862. doi: 10.1111/j.1360-0443.2007.01798.x.
    1. Gonsalvez CJ, Milne DL. Clinical supervisor training in Australia: a review of current problems and possible solutions. Australian Psychol. 2010;45:233–242. doi: 10.1080/00050067.2010.512612.
    1. Milne DL, Sheikh AI, Pattison S, Wilkinson A. Evidence-based training for clinical supervisors: a systematic review of 11 controlled studies. Clin Sup. 2011;30:53–71. doi: 10.1080/07325223.2011.564955.
    1. Eby LT, Burk H, Maher CP. How serious of a problem is staff turnover in substance abuse treatment? A longitudinal study of actual turnover. J Subs Abuse Treat. 2010;39:264–271. doi: 10.1016/j.jsat.2010.06.009.
    1. Chorpita BF, Regan J. Dissemination of effective mental health treatment procedures: maximizing the return on a significant investment. Behav Res Therapy. 2009;47:990–993. doi: 10.1016/j.brat.2009.07.002.
    1. Beidas RS, Edmunds JM, Marcus SC, Kendall PC. Training and consultation to promote implementation of an empirically supported treatment: a randomized trial. Psychiatric Serv. 2012;63:660–665. doi: 10.1176/appi.ps.201100401.
    1. Milne D. In: Casebook for clinical supervision: a competency-based approach. Falender CA, Shafranske EP, editor. Washington, DC US: American Psychological Association; 2008. Evaluating and enhancing supervision: an experiential model; pp. 211–233.
    1. Padesky CA. In: Frontiers of cognitive therapy. Salkovskis PM, editor. New York, NY US: Guilford Press; 1996. Developing cognitive therapist competency: teaching and supervision models; pp. 266–292.
    1. Sheidow AJ, Donahue BC, Hill HH, Heneggler SW, Ford JD. Development of an audio tape review system for supporting adherence to an evidence based treatment. Prof Psychol Res Prac. 2008;39:553–560.
    1. Bearman SK, Weisz JR, Chorpita BF, Hoagwood K, Ward A, Ugueto AM, Bernstein A. More practice, less preach? The role of supervision processes and therapist characteristics in EBP implementation. Admin Policy Mental Health Mental Health Serv Res. in press.
    1. Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, Griffey R, Hensley M. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Admin Policy Mental Health. 2010;38:65–76.
    1. Schoenwald SK, Garland AF, Chapman JE, Frazier SL, Sheidow AJ, Southam-Gerow MA. Toward the effective and efficient measurement of implementation fidelity. Admin Policy Mental Health Mental Health Serv Res. 2011;38:32–43. doi: 10.1007/s10488-010-0321-0.
    1. Garland AF, Bickman L, Chorpita BF. Change what? Identifying quality improvement targets by investigating usual mental health care. Admin Policy Mental Health Mental Health Serv Res. 2010;37:15–26. doi: 10.1007/s10488-010-0279-y.
    1. Hogue A, Ozechowski TJ, Robbins MS, Barrett WH. Making fidelity an intramural game: localizing quality assurance procedures to promote sustainability of evidence-based practices in usual care. Clin Psychol Sci Pract. 2013;20:60–77. doi: 10.1111/cpsp.12023.
    1. Hurlburt MS, Garland AF, Nguyen K, Brookman-Frazee L. Child and family therapy process: concordance of therapist and observational perspectives. Admin Policy Mental Health Mental Health Serv Res. 2010;37:230–244. doi: 10.1007/s10488-009-0251-x.
    1. Hrisos S, Eccles MP, Francis JJ, Dickinson HO, Kaner EFS, Beyer F, Johnston M. Are there valid proxy measures of clinical behavior? a systematic review. Implement Sci. 2009;4:37–56. doi: 10.1186/1748-5908-4-37.
    1. Perepletchikova F, Treat TA, Kazdin AE. Treatment integrity in psychotherapy research: analysis of the studies and examination of the associated factors. J Consul Clin Psychol. 2007;72:829–841.
    1. Cross W, Matthieu M, Cerel J, Knox KL. Proximate outcomes of gatekeeper training for suicide prevention in the workplace. Suicide Life-Threat Behav. 2007;37:659–670. doi: 10.1521/suli.2007.37.6.659.
    1. Beidas RS, Cross WF, Dorsey S. Show me don’t tell me: behavioral rehearsal as a training and fidelity tool. Cogn Behav Pract. in press.
    1. Freeston M. Clinical supervision: art or science? Boston, MA US: World Congress of Behavioral and Cognitive Therapies; 2010.
    1. Matthieu M, Cross W, Batres AR, Flora CM, Knox KL. Evaluation of gatekeeper training for suicide prevention in veterans. Archives Suicide Res. 2008;12:148–154. doi: 10.1080/13811110701857491.
    1. Peabody JW, Luck J, Glassman P, Dresselhaus TR, Lee M. Comparison of vignettes, standardized patients, and chart abstraction: a prospective validation study of 3 methods for measuring quality. JAMA J Am Med Assoc. 2000;283:1715–1722. doi: 10.1001/jama.283.13.1715.
    1. Unützer J, Chan YF, Hafer E, Knaster J, Shields A, Powers D, Veith RC. Quality improvement with pay-for-performance incentives in integrated behavioral health care. Am J Public Health. 2012;106:e41–e45.
    1. Worthen VE, Lambert MJ. Outcome oriented supervision: advantages of adding systematic client tracking to supportive consultations. Counsel Psychotherapy Res. 2007;7:48–53. doi: 10.1080/14733140601140873.
    1. Hunkeler E, Katon W, Tang L, Williams JW Jr, Kroenke K, Lin EB, Unützer J. Long term outcomes from the IMPACT randomised trial for depressed elderly patients in primary care. BMJ British Med J. 2006;332:259–263. doi: 10.1136/.
    1. Unützer J, Katon W, Callahan CM, Williams JW Jr, Hunkeler E, Harpole L, Hoffing M, Della Penna RD, Hitchock Noël P, Lin EHB. et al.Collaborative care management of late-life depression in the primary care setting: a randomized controlled trial. JAMA J Am Med Assoc. 2002;288:2836–2845. doi: 10.1001/jama.288.22.2836.
    1. Garland A, Landsverk J, Lau A. Racial/ethnic disparities in mental health service use among children in foster care. Child Youth Serv Rev. 2003;25:491–507. doi: 10.1016/S0190-7409(03)00032-X.
    1. Palmiter DJ Jr. A survey of the assessment practices of child and adolescent clinicians. Am J Orthopsych. 2004;74:122–128.
    1. Barber JP, Crits-Christoph P, Luborksy L. Effects of therapist adherence and competence on patient outcome in brief dynamic therapy. J Consul Clin Psychol. 1996;64:619–622.
    1. Schoenwald S, Carter RE, Chapman JE, Sheidow AJ. Therapist adherence and organizational effects on change in youth behavior problems one year after multisystemic therapy. Admin Policy Mental Health. 2008;35:379–394. doi: 10.1007/s10488-008-0181-z.
    1. Webb CA, DeRubeis RJ, Barber JP. Therapist adherence/competence and treatment outcome: a meta-analytic review. J Consul Clin Psychol. 2010;78:200–211.
    1. McHugh R, Murray HW, Barlow DH. Balancing fidelity and adaptation in the dissemination of empirically-supported treatments: the promise of transdiagnostic interventions. Behav Res Therapy. 2009;47:946–953. doi: 10.1016/j.brat.2009.07.005.
    1. Ellis MV, Ladany N. In: Handbook of psychotherapy supervision. Watkins CEJr, editor. New York, NY US: Wiley; 1997. Inferences concerning supervisees and clients in clinical supervision: an integrative review; pp. 447–507.
    1. Schoenwald SK, Sheidow AJ, Letourneau EJ, Liao JG. Transportability of multisystemic therapy: evidence for multi-level influences. Mental Health Serv Res. 2003;5:223–239. doi: 10.1023/A:1026229102151.
    1. Chaffin M, Hecht D, Bard D, Silovsky JF, Beasley WH. A statewide trial of the SafeCare home-based services model with parents in child protective services. Pediatrics. 2012;129:509–515. doi: 10.1542/peds.2011-1840.
    1. Aarons GA, Sommerfeld DH, Hecht DB, Silovsky JF, Chaffin MJ. The impact of evidence-based practice implementation and fidelity monitoring on staff turnover: evidence for a protective effect. J Consul Clin Psychol. 2009;77:270–280.
    1. Knudsen HK, Ducharme LJ, Roman PM. Clinical supervision, emotional exhaustion, and turnover intention: a study of substance abuse treatment counselors in the clinical trials network of the national institute on drug abuse. J Subs Abuse Treat. 2008;35:387–395. doi: 10.1016/j.jsat.2008.02.003.
    1. Cohen JA, Mannarino A, Deblinger E. Treating trauma and traumatic grief in Children and adolescents: a clinician’s guide. New York: Guilford Press; 2006.
    1. Cohen JA, Mannarino AP. Disseminating and implementing trauma-focused CBT in community settings. Trauma Violence Abuse. 2008;9:214–226. doi: 10.1177/1524838008324336.
    1. Sigel BA, Benton AH, Lynchm CE, Kramer TL. Preliminary findings for statewide dissemination programs using trauma-focused cognitive-behavioral therapy (TF-CBT) with children. Philedelphia, PA US: Presented at the University of Arkansas for Medical Sciences, Annual National Colloquium; 2010.
    1. Sigel BA, Kramer TL, Conners-Burrow NA, Church JK, Worley KB, Mitrani NA. Statewide dissemination of trauma-focused cognitive-behavioral therapy (TF-CBT) Child Youth Serv Rev. 2013;35:1023–1029. doi: 10.1016/j.childyouth.2013.03.012.
    1. Gleacher AA, Nadeem E, Moy AJ, Whited AL, Albano AM, Radigan M, Wang R, Chassman J, Myrhol-Clarke B, Hoagwood KE. Statewide CBT training for clinicians and supervisors treating youth: the New York State evidence based treatment dissemination center. J Emot Behav Disorders. 2010;19:182–192.
    1. Milne D, Aylott H, Fitzpatrick H, Ellis MV. How does clinical supervision work? Using a ‘best evidence synthesis’ approach to construct a basic model of supervision. Clin Sup. 2008;27:170–190. doi: 10.1080/07325220802487915.
    1. McLeod BD, Weisz JR. Therapy process observational coding system for child psychotherapy strategies scale. J Clin Child Adol Psychol. 2010;39:436–443. doi: 10.1080/15374411003691750.
    1. Steinberg AM, Brymer M, Decker K, Pynoos RS. The UCLA PTSD reaction index. Current Psychiatry Reports. 2004;6:96–100. doi: 10.1007/s11920-004-0048-2.
    1. Steinberg AM, Brymer MJ, Kim S, Briggs EC, Ippen CG, Ostrowsky SA, Gully KJ, Pynoos RS. Psychometric properties of the UCLA PTSD reaction index: part I. J Traumatic Stress. 2013;26:1–9. doi: 10.1002/jts.21780.
    1. Goodman R. The strengths and difficulties questionnaire: a research note. J Child Psychol Psychiatry. 1997;38:581–586. doi: 10.1111/j.1469-7610.1997.tb01545.x.
    1. Graen GB, Uhl-Bien M. Relationship-based approach to leadership: development of leader-member exchange (lmx) theory of leadership over 25 years: applying a multi-level multi-domain perspective. Leadership Quarter. 1995;6:219–247. doi: 10.1016/1048-9843(95)90036-5.
    1. Efstation JF, Patton MJ, Kardish CM. Measuring the working alliance in counsellor supervision. J Counsel Psychol. 1990;37:322–329.
    1. Horvath AO, Greenberg LS. In: The psychotherapeutic process: a research handbook. Greenberg LS, Pinsoff WM, editor. New York, NY US: Guildford Press; 1986. Development of the working alliance inventory; pp. 529–556.
    1. Rahim MA. The development of a leader power inventory. Multi Behav Res. 1988;23:491–503. doi: 10.1207/s15327906mbr2304_6.
    1. Lehman WEK, Greener JM, Simpson DD. Assessing organizational readiness for change. J Subs Abuse Treat. 2002;22:197–209. doi: 10.1016/S0740-5472(02)00233-7.
    1. Borntrager C, Chorpita BF, McMillan-Higa C, Weisz J. Provider attitudes towards evidence based practices: are the concerns with the evidence or the manuals? Psychiatric Serv. 2009;60:677–681. doi: 10.1176/appi.ps.60.5.677.
    1. Maslach C, Jackson S, Leiter M. Maslach burnout inventory manual. Mountain View, California: Consulting Psychologists Press; 1996.
    1. Walsh JP, Ashford SJ, Hill TE. Feedback obstruction: the influence of the information environment on employee turnover intentions. Human Relat. 1985;38:23–46. doi: 10.1177/001872678503800102.
    1. Birmaher B, Khetarpal S, Brent D, Cully M. The screen for child anxiety related emotional disorders (SCARED): scale construction and psychometric characteristics. J Am Acad Child Adol Psychiatry. 1997;36:545–553. doi: 10.1097/00004583-199704000-00018.
    1. Muris P, Merckelbach H, Korver P, Meesters C. Screening for trauma in childresn and adolescents: the validity of the traumatic stress disorder scale of the screen for Child anxiety related emotional disorders. J Clin Child Psychol. 2000;29:406–413. doi: 10.1207/S15374424JCCP2903_11.
    1. Gardner W, Lucas A, Kolko DJ, Campo JV. Comparison of the PSC-17 and alternative mental health screens in an at-risk primary care sample. J Am Acad Child Adol Psychiatry. 2007;46:611–618. doi: 10.1097/chi.0b013e318032384b.
    1. Kerns SEU, Dorsey S, Trupin EW, Berliner L. Project Focus: promoting emotional health and wellbeing for youth in foster care through connections to evidence-based practices. Report Emot Behav Disorders Youth. 2010;10:30–38.
    1. Baron RM, Kenny DA. The moderator-mediator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations. J Personal Soc Psychol. 1986;51:1173–1182.
    1. Cole DA, Maxwell SE. Testing meditational models with longitudinal data: questions and tips in the use of structural equation modeling. J Abnormal Psychol. 2003;112:558–577.
    1. MacKinnon DP. Introduction to statistical mediation analysis. New York, NY US: Lawrence Erlbaum Associates; 2008.
    1. Stice E, Presnell K, Gau J, Shaw H. Testing mediators of intervention effects in randomized controlled trials: an evaluation of two eating disorder prevention programs. J Consul Clin Psychol. 2007;75:20–32.
    1. QSR International. NVivo 9: qualitative analysis software program. 2010.
    1. Patton MQ. Qualitative evaluations and research methods. Newbury Park, CA: Sage; 1990.
    1. Strauss A, Corbin J. Basics of qualitative research. Newbury Park, CA: Sage; 1990.
    1. Pringle B, Chambers D, Wang PS. Toward enough of the best for all: research to transform the efficacy, quality, and reach of mental health care for youth. Admin Policy Mental Health Mental Health Serv Res. 2010;37:191–196. doi: 10.1007/s10488-010-0266-3.
    1. National Institute of Mental Health. The national institute of mental health strategic plan. Bethesda, MD US: U. S. Department of Health and Human Services, National Institutes of Health, National Institute of Mental Health; 2008. (NIH publication No. 08–6368 edition).
    1. Bruns EJ, Hoagwood KE. State implementation of evidence-based practice for youths, part I: responses to the state of the evidence. J Am Acad Child Adol Psychiatry. 2008;47:369–373. doi: 10.1097/CHI.0b013e31816485f4.
    1. Jensen-Doss A, Hawley KH, Lopez M, Osterberg LD. Using evidence-based treatments: the experiences of youth providers working under a mandate. Prof Psychol Res Pract. 2009;40:417–424.

Source: PubMed

Sponsorer og samarbejdspartnere

Medicinske tilstande

Narkotikainterventioner

3
Abonner