Mediators of measurement-based care implementation in community mental health settings: results from a mixed-methods evaluation

Cara C Lewis, Meredith R Boyd, C Nathan Marti, Karen Albright, Cara C Lewis, Meredith R Boyd, C Nathan Marti, Karen Albright

Abstract

Background: Tailored implementation approaches are touted as superior to standardized ones with the reasoning that tailored approaches afford opportunities to select strategies to resolve determinants of the local context. However, results from implementation trials on this topic are equivocal. Therefore, it is important to explore relevant contextual factors that function as determinants to evaluate if they are improved by tailoring and subsequently associated with changes in implementation outcomes (i.e., via statistical mediation) to better understand how tailoring achieves (or does not achieve) its effects. The present study examined the association between a tailored and standardized implementation approach, contextual factors that might mediate change, and a target implementation outcome in an initiative to implement measurement-based care (specifically the clinical integration of the Patient Health Questionnaire [PHQ-9] for depression) in a community mental health organization.

Methods: Using a cluster randomized control design, twelve community-based mental health clinics were assigned to a tailored or standardized implementation group. Clinicians completed a self-report battery assessing contextual factors that served as candidate mediators informed by the Framework for Dissemination at three time points: baseline, 5 months after active implementation support, and 10 months after sustainment monitoring. A subset of clinicians also participated in focus groups at 5 months. The routine use of the PHQ-9 (implementation outcome) was monitored during the 10-month sustainment period. Multi-level mediation analyses assessed the association between the implementation group and contextual factors and the association between contextual factors and PHQ-9 completion. Quantitative results were then elaborated by analyzing qualitative data from exemplar sites.

Results: Although tailored clinics outperformed standard clinics in terms of PHQ-9 completion at the end of active implementation, these group differences disappeared post sustainment monitoring. Perhaps related to this, no significant mediators emerged from our quantitative analyses. Exploratory qualitative analyses of focus group content emphasized the importance of support from colleagues, supervisors, and leadership when implementing clinical innovations in practice.

Conclusions: Although rates of PHQ-9 completion improved across the study, their sustained levels were roughly equivalent across groups and low overall. No mediators were established using quantitative methods; however, several partial quantitative pathways, as well as themes from the qualitative data, reveal fruitful areas for future research.

Trial registration: Standardized versus tailored implementation of measurement-based care for depression.

Clinicaltrials: gov NCT02266134, first posted on October 16, 2014.

Keywords: Community mental health; Depression; Implementation; Measurement-based care; Mediator; Standardized; Sustainment; Tailored.

Conflict of interest statement

The authors have no competing interests to report.

© 2022. The Author(s).

References

    1. Powell BJ, Beidas RS, Lewis CC, Aarons GA, McMillen JC, Proctor EK, et al. Methods to improve the selection and tailoring of implementation strategies. J Behav Health Serv Res. 2017;44(2):177–194.
    1. Goodman D, Ogrinc G, Davies L, Baker GR, Barnsteiner J, Foster TC, et al. Explanation and elaboration of the SQUIRE (Standards for Quality Improvement Reporting Excellence) Guidelines, V. 2.0: examples of SQUIRE elements in the healthcare improvement literature. BMJ qual saf. 2016;25(12):e7-e.
    1. Krause J, Van Lieshout J, Klomp R, Huntink E, Aakhus E, Flottorp S, et al. Identifying determinants of care for tailoring implementation in chronic diseases: an evaluation of different methods. Implement Sci. 2014;9(1):1–12.
    1. Baker R, Camosso‐Stefinovic J, Gillies C, Shaw EJ, Cheater F, Flottorp S, Robertson N, Wensing M, Fiander M, Eccles MP, Godycki‐Cwirko M. Tailored interventions to address determinants of practice. Cochrane Database Syst Rev. 2015(4).
    1. Wensing M. The tailored implementation in chronic diseases (TICD) project: introduction and main findings. Implement Sci. 2017;12(1):5.
    1. Lewis CC, Scott K, Marriott BR. A methodology for generating a tailored implementation blueprint: an exemplar from a youth residential setting. Implement Sci. 2018;13(1):1–13.
    1. Lewis CC, Boyd M, Puspitasari A, Navarro E, Howard J, Kassab H, et al. Implementing measurement-based care in behavioral health: a review. JAMA Psychiat. 2019;76(3):324–335.
    1. Lambert MJ, Whipple JL, Hawkins EJ, Vermeersch DA, Nielsen SL, Smart DW. Is it time for clinicians to routinely track patient outcome? a meta-analysis. Clin Psychol Sci Pract. 2003;10(3):288.
    1. Lambert MJ, Whipple JL, Kleinstäuber M. Collecting and delivering progress feedback: a meta-analysis of routine outcome monitoring. Psychotherapy. 2018;55(4):520.
    1. Brattland H, Koksvik JM, Burkeland O, Gråwe RW, Klöckner C, Linaker OM, et al. The effects of routine outcome monitoring (ROM) on therapy outcomes in the course of an implementation process: a randomized clinical trial. J Couns Psychol. 2018;65(5):641.
    1. Peterson AP, Fagan C. Training the next generation in routine outcome monitoring: current practices in psychology training clinics. Train Educ Prof Psychol. 2017;11(3):182.
    1. Lewis CC, Marti CN, Scott K, Walker MR, Boyd M, Puspitasari A, et al. Standardized versus tailored implementation of measurement-based care for depression in community mental health clinics. Psychiatric Services. 2022:appi. ps. 202100284.
    1. Kazdin AE. Mediators and mechanisms of change in psychotherapy research. Annu Rev Clin Psychol. 2007;3:1–27.
    1. Fetters MD, Curry LA, Creswell JW. Achieving integration in mixed methods designs—principles and practices. Health Serv Res. 2013;48(6pt2):2134–56.
    1. Löwe B, Kroenke K, Fau - Herzog W, Herzog W, Fau - Gräfe K, Gräfe K. Measuring depression outcome with a brief self-report instrument: sensitivity to change of the Patient Health Questionnaire (PHQ-9) J Affect Disord. 2004;81(1):61–6.
    1. Levis B, Benedetti A, Thombs BD. Accuracy of Patient Health Questionnaire-9 (PHQ-9) for screening to detect major depression: individual participant data meta-analysis. BMJ. 2019;365:l1476.
    1. Kroenke K, Spitzer RL, Williams JB, Löwe B. The patient health questionnaire somatic, anxiety, and depressive symptom scales: a systematic review. Gen Hosp Psychiatry. 2010;32(4):345–359.
    1. Schoonenboom J, Johnson RB. How to construct a mixed methods research design. KZfSS Kölner Zeitschrift für Soz Sozialpsychol. 2017;69(2):107–131.
    1. Chamberlain P, Brown CH, Saldana L, Reid J, Wang W, Marsenich L, et al. Engaging and recruiting counties in an experiment on implementing evidence-based practice in California. Admin Policy Ment Health Ment Health Serv Res. 2008;35(4):250–260.
    1. Lewis CC, Scott K, Marti CN, Marriott BR, Kroenke K, Putz JW, et al. Implementing measurement-based care (iMBC) for depression in community mental health: a dynamic cluster randomized trial study protocol. Implement Sci. 2015;10(1):1–14.
    1. Lewis CC, Puspitasari A, Boyd MR, Scott K, Marriott BR, Hoffman M, et al. Implementing measurement based care in community mental health: a description of tailored and standardized methods. BMC Health Serv Res. 2018;11(1):76.
    1. Valente TW, Chou CP, Pentz MA. Community coalitions as a system: effects of network change on adoption of evidence-based substance abuse prevention. Am J Public Health. 2007;97(5):880–886.
    1. Jensen-Doss A, Haimes EM, Smith AM, Lyon AR, Lewis CC, Stanick CF, Hawley KM. Monitoring treatment progress and providing feedback is viewed favorably but rarely used in practice. Adm Policy Mental Health Mental Health Serv Res. 2018;45(1):48–61.
    1. Mendel P, Meredith L, Schoenbaum M, Sherbourne C, Wells K. Interventions in organizational and community context: a framework for building evidence on dissemination and implementation in health services research. Admin Policy Ment Health Men Health Serv Res. 2008;35(1–2):21–37.
    1. Singer JD, Willet JB. Applied longitudinal data analysis: modeling change and event occurrence. New York, NY: Oxford University Press; 2003.
    1. RStudio Team. RStudio: integrated development environment for R. RStudio, PBC. 2021.
    1. MacKinnon D. Introduction to statistical mediation analysis. New York, NY: Routledge Academic; 2008.
    1. Bates DM, Sarkar D. lme4: linear mixed-effects models using S4 classes. Package 0.99875–6 ed: Comprehensive R Archive Network; 2007.
    1. Knowles JE, Frederick C. merTools: tools for analyzing mixed effect regression models. 2019.
    1. Rubin M. When to adjust alpha during multiple testing: a consideration of disjunction, conjunction, and individual testing. Synthese. 2021;199(3):10969–11000.
    1. Gelman A, Hill J. Data analysis using regression and multilevel/hierarchical models (final version: 5 July 2006) Please do not reproduce in any form without permission.
    1. Palinkas LA, Horwitz SM, Green CA, Wisdom JP, Duan N, Hoagwood K. Purposeful sampling for qualitative data collection and analysis in mixed method implementation research. Admin Policy Ment Health Ment Health Serv Res. 2015;42(5):533–544.
    1. Graneheim UH, Lundman B. Qualitative content analysis in nursing research: concepts, procedures and measures to achieve trustworthiness. Nurse Educ Today. 2004;24(2):105–112.
    1. Hsieh H-F, Shannon SE. Three approaches to qualitative content analysis. Qual Health Res. 2005;15(9):1277–1288.
    1. Charmaz K. Constructing grounded theory: a practical guide through qualitative analysis. London: Sage Publications; 2006.
    1. Hair J, Anderson R, Black B, Babin B. Multivariate data analysis: pearson education; 2016.
    1. Broome KM, Flynn PM, Knight DK, Simpson DD. Program structure, staff perceptions, and client engagement in treatment. J Subst Abuse Treat. 2007;33(2):149–158.
    1. Salyers MP, Rollins AL, McGuire AB, Gearhart T. Barriers and facilitators in implementing illness management and recovery for consumers with severe mental illness: trainee perspectives. Adm Policy Ment Health. 2009;36(2):102–111.
    1. Boyd MR, Powell BJ, Endicott D, Lewis CC. A method for tracking implementation strategies: an exemplar implementing measurement-based care in community behavioral health clinics. Behav Ther. 2018;49(4):525–537.
    1. Franco GE. Productivity standards: do they result in less productive and satisfied therapists? US: Educational Publishing Foundation; 2016. pp. 91–106.
    1. Burke LA, Saks AM. Accountability in training transfer: adapting schlenker’s model of responsibility to a persistent but solvable problem. Hum Resour Dev Rev. 2009;8(3):382–402.
    1. Westerlund A, Nilsen P, Sundberg L. Implementation of implementation science knowledge: the research practice gap paradox. Worldviews Evid-Based Nurs. 2019;16(5):332.
    1. Aarons GA, Ehrhart MG, Farahnak LR, Hurlburt MS. Leadership and organizational change for implementation (LOCI): a randomized mixed method pilot study of a leadership and organization development intervention for evidence-based practice implementation. Implement Sci. 2015;10:11.
    1. Skar AS, Braathu N, Peters N, Bækkelund H, Endsjø M, Babaii A, et al. A stepped-wedge randomized trial investigating the effect of the Leadership and Organizational Change for Implementation (LOCI) intervention on implementation and transformational leadership, and implementation climate. BMC Health Serv Res. 2022;22(1):298.
    1. Lyon AR, Liu FF, Connors EH, King KM, Coifman JI, Cook H, McRee E, Ludwig K, Law A, Dorsey S, McCauley E. How low can you go? Examining the effects of brief online training and post-training consultation dose on implementation mechanisms and outcomes for measurement-based care. Implement Sci Commun. 2022;3(1):1–5.
    1. Aarons GA, Sommerfeld DH, Willging CE. The soft underbelly of system change: the role of leadership and organizational climate in turnover during statewide behavioral health reform. Psychol Serv. 2011;8(4):269–281.
    1. Bunger AC, Birken SA, Hoffman JA, MacDowell H, Choy-Brown M, Magier E. Elucidating the influence of supervisors’ roles on implementation climate. Implement Sci. 2019;14(1):93.
    1. Brimhall KC, Fenwick K, Farahnak LR, Hurlburt MS, Roesch SC, Aarons GA. Leadership, organizational climate, and perceived burden of evidence-based practice in mental health services. Adm Policy Ment Health. 2016;43(5):629–639.
    1. Tang MY, Rhodes S, Powell R, McGowan L, Howarth E, Brown B, et al. How effective are social norms interventions in changing the clinical behaviours of healthcare workers? A systematic review and meta-analysis. Implement Sci. 2021;16(1):8.
    1. Schneider B, Ehrhart MG, Macey WH. Organizational climate and culture. Annu Rev Psychol. 2013;64:361–388.
    1. Reding ME, Chorpita BF, Lau AS, Innes-Gomberg D. Providers’ attitudes toward evidence-based practices: is it just about providers, or do practices matter, too? Adm Policy Ment Health. 2014;41(6):767–76.
    1. Jones SM, Gaffney A, Unger JM. Using patient-reported outcomes in measurement-based care: perceived barriers and benefits in oncologists and mental health providers. J Public Health. 2021:1–7.
    1. Lewis CC, Hannon PA, Klasnja P, Baldwin L-M, Hawkes R, Blackmer J, et al. Optimizing implementation in cancer control (OPTICC): protocol for an implementation science center. Implement Sci Commun. 2021;2(1):44.
    1. Beidas RS, Dorsey S, Lewis CC, Lyon AR, Powell BJ, Purtle J, et al. Promises and pitfalls in implementation science from the perspective of US-based researchers: learning from a pre-mortem. Implement Sci. 2022;17(1):55.
    1. Weiner BJ, Lewis CC, Sherr K. Practical implementation science: moving evidence into action: Springer Publishing Company; 2022.
    1. Rogers EM, Singhal A, Quinlan MM. Diffusion of innovations. An integrated approach to communication theory and research: Routledge; 2014. pp. 432–448.
    1. Kislov R, Pope C, Martin GP, Wilson PM. Harnessing the power of theorising in implementation science. Implement Sci. 2019;14(1):103.
    1. Lewis CC, Klasnja P, Powell BJ, Lyon AR, Tuzzio L, Jones S, et al. From classification to causality: advancing understanding of mechanisms of change in implementation science. Front Public Health. 2018;6:136.

Source: PubMed

스폰서 및 공동 작업자

건강 상태

약물 개입

3
구독하다