Evaluation of two implementation strategies in 51 child county public service systems in two states: results of a cluster randomized head-to-head implementation trial

C Hendricks Brown, Patricia Chamberlain, Lisa Saldana, Courtenay Padgett, Wei Wang, Gracelyn Cruden, C Hendricks Brown, Patricia Chamberlain, Lisa Saldana, Courtenay Padgett, Wei Wang, Gracelyn Cruden

Abstract

Background: Much is to be learned about what implementation strategies are the most beneficial to communities attempting to adopt evidence-based practices. This paper presents outcomes from a randomized implementation trial of Multidimensional Treatment Foster Care (MTFC) in child public service systems in California and Ohio, including child welfare, juvenile justice, and mental health.

Methods: Fifty-one counties were assigned randomly to one of two different implementation strategies (Community Development Teams (CDT) or independent county implementation strategy (IND)) across four cohorts after being matched on county characteristics. We compared these two strategies on implementation process, quality, and milestone achievements using the Stages of Implementation Completion (SIC) (Implement Sci 6(1):1-8, 2011).

Results: A composite score for each county, combining the final implementation stage attained, the number of families served, and quality of implementation, was used as the primary outcome. No significant difference between CDT and IND was found for the composite measure. Additional analyses showed that there was no evidence that CDT increased the proportion of counties that started-up programs (i.e., placed at least one family in MTFC). For counties that did implement MTFC, those in the CDT condition served over twice as many youth during the study period as did IND. Of the counties that successfully achieved program start-up, those in the CDT condition completed the implementation process more thoroughly, as measured by the SIC. We found no significant differences by implementation condition on the time it took for first placement, achieving competency, or number of stages completed.

Conclusions: This trial did not lead to higher rates of implementation or faster implementation but did provide evidence for more robust implementation in the CDT condition compared to IND implementation once the first family received MTFC services. This trial was successful from a design perspective in that no counties dropped out, even though this study took place during an economic recession. We believe that this methodologic approach of measurement utilizing the SIC, which is comprised of the three dimensions of quality, quantity, and timing, is appropriate for a wide range of implementation and translational studies.

Trial registration: Trial ID: NCT00880126 (ClinicalTrials.gov).

Figures

Figure 1
Figure 1
Consort diagram for California counties.
Figure 2
Figure 2
Consort diagram for Ohio counties.
Figure 3
Figure 3
Final stage of implementation for community development team (CDT) and independently administered (IND).
Figure 4
Figure 4
Comparison of placement quantiles for CDT and IND counties.

References

    1. Glasgow RE, Magid DJ, Beck A, Ritzwoller D, Estabrooks PA. Practical clinical trials for translating research to practice: design and measurement recommendations. Med Care. 2005;43(6):551–557. doi: 10.1097/01.mlr.0000163645.41407.09.
    1. Glasgow RE, Davidson KW, Dobkin PL, Ockene J, Spring B. Practical behavioral trials to advance evidence-based behavioral medicine. Ann Behav Med. 2006;31(1):5–13. doi: 10.1207/s15324796abm3101_3.
    1. National Research Council and Institute of Medicine . Preventing mental, emotional, and behavioral disorders among young people: progress and possibilities. In: O’Connell ME, Boat T, Warner KE, Board on Children, Youth, and Families, Division of Behavioral and Social Sciences and Education, editors. Committee on the Prevention of Mental Disorders and Substance Abuse Among Children, Youth, and Young Adults: Research Advances and Promising Interventions. Washington, DC: The National Academies Press; 2009.
    1. Brown CH, Ten Have TR, Jo B, Dagne G, Wyman PA, Muthen B, Gibbons RD. Adaptive designs for randomized trials in public health. Annu Rev Public Health. 2009;30:1–25. doi: 10.1146/annurev.publhealth.031308.100223.
    1. Spoth R, Rohrbach LA, Greenberg M, Leaf P, Brown CH, Fagan A, Catalano RF, Pentz MA, Sloboda Z, Hawkins JD, Society for Prevention Research Type 2 Translational Task Force Members and Contributing Authors Addressing core challenges for the next generation of type 2 translation research and systems: the translation science to population impact (TSci Impact) framework. Prev Sci. 2013;14:319–351. doi: 10.1007/s11121-012-0362-6.
    1. Landsverk J, Brown CH, Chamberlain P, Palinkas L, Rolls Reutz J, Horwitz SM. Design and analysis in dissemination and implementation research. In: Brownson RC, Colditz GA, Proctor EK, editors. Dissemination and Implementation Research in Health: Translating Science to Practice. London: Oxford University Press; 2012. pp. 225–260.
    1. Chamberlain P, Brown C, Saldana L, Reid J, Wang W, Marsenich L, Sosna T, Padgett C, Bouwman G. Engaging and recruiting counties in an experiment on implementing evidence-based practice in California. Adm Policy Ment Health Ment Health Serv Res. 2008;35(4):250–260. doi: 10.1007/s10488-008-0167-x.
    1. Brown CH, Wang W, Kellam SG, Muthén BO, Petras H, Toyinbo P, Poduska J, Ialongo N, Wyman PA, Chamberlain P, Sloboda Z, MacKinnon DP, Windham A, The Prevention Science Methodology Group Methods for testing theory and evaluating impact in randomized field trials: intent-to-treat analyses for integrating the perspectives of person, place, and time. Drug Alcohol Depend. 2008;95(Suppl 1):S74–S104. doi: 10.1016/j.drugalcdep.2007.11.013.
    1. Chamberlain P. Treating Chronic Juvenile Offenders: Advances Made Through the Oregon Multidimensional Treatment Foster Care Model. Washington, DC: American Psychological Association; 2003.
    1. Chamberlain P, Brown CH, Saldana L. Observational measure of implementation progress in community based settings: the Stages of Implementation Completion (SIC) Implement Sci. 2011;6(1):1–8. doi: 10.1186/1748-5908-6-116.
    1. University of Colorado Boulder: Blueprints programs..
    1. Leve LD, Harold GT, Chamberlain P, Landsverk JA, Fisher PA, Vostanis P. Practitioner review: children in foster care–vulnerabilities and evidence‐based interventions that promote resilience processes. J Child Psychol Psychiatry. 2012;53(12):1197–1211. doi: 10.1111/j.1469-7610.2012.02594.x.
    1. Chamberlain P, Roberts R, Jones H, Marsenich L, Sosna T, Price JM. Three collaborative models for scaling up evidence-based practices. Adm Policy Ment Health Ment Health Serv Res. 2012;39(4):278–290. doi: 10.1007/s10488-011-0349-9.
    1. Saldana L, Chamberlain P. Supporting implementation: the role of community development teams to build infrastructure. Am J Community Psychol. 2012;50(3–4):334–346. doi: 10.1007/s10464-012-9503-0.
    1. Saldana L, Chamberlain P, Wang W, Hendricks Brown C. Predicting program start-up using the stages of implementation measure. Adm Policy Ment Health Ment Health Serv Res. 2012;39(6):419–425. doi: 10.1007/s10488-011-0363-y.
    1. Saldana L. The stages of implementation completion for evidence-based practice: protocol for a mixed methods study. Implement Sci. 2014;9(1):43. doi: 10.1186/1748-5908-9-43.
    1. Chamberlain P, Saldana L, Brown CH, Leve LD. Implementation of multidimensional treatment foster care in California: a randomized trial of an evidence-based practice. In: Roberts-DeGennaro M, Fogel S, editors. Empirically Supported Interventions for Community and Organizational Change. Chicago, IL: Lyceum Books, Inc; 2010. pp. 218–234.
    1. Sosna T, Marsenich L. Community Development Team Model: Supporting the Model Adherent Implementation of Programs and Practices. California Institute of Mental Health: Sacramento, CA; 2006.
    1. Berwick DM. Continuous improvement as an ideal in health care. N Engl J Med. 1989;320(1):53–56. doi: 10.1056/NEJM198901053200110.
    1. Berwick DM. Quality comes home. Ann Intern Med. 1996;125(10):839–843. doi: 10.7326/0003-4819-125-10-199611150-00009.
    1. Laffel G, Blumenthal D. The case for using industrial quality management science in health care organizations. JAMA. 1989;262(20):2869–2873. doi: 10.1001/jama.1989.03430200113036.
    1. Nadeem E, Olin SS, Campbell L, Hoagwood KE, Horwitz SM. Understanding the components of quality improvement collaboratives: a systematic literature review. Milbank Q. 2013;91(2):354–394. doi: 10.1111/milq.12016.
    1. Pals SL, Wiegand RE, Murray DM. Ignoring the group in group-level HIV/AIDS intervention trials: a review of reported design and analytic methods. AIDS. 2011;25:989–996.
    1. Murray DM, Varnell SP, Blitstein JL. Design and analysis of group-randomized trials: a review of recent methodological developments. Am J Public Health. 2004;94(3):423–432. doi: 10.2105/AJPH.94.3.423.
    1. Gibbons RD, Hedeker D, Elkin I, Waternaux C, Kraemer HC, Greenhouse JB, Shea MT, Imber SD, Sotsky SM, Watkins JT. Some conceptual and statistical issues in analysis of longitudinal psychiatric data: application to the NIMH Treatment of Depression Collaborative Research Program dataset. Arch Gen Psychiatry. 1993;50(9):739–750. doi: 10.1001/archpsyc.1993.01820210073009.
    1. Schwarz G. Estimating the dimension of a model. Ann Stat. 1978;6:461–464. doi: 10.1214/aos/1176344136.
    1. Larsen K. The Cox proportional hazards model with a continuous latent variable measured by multiple binary indicators. Biometrics. 2005;61:1049–1055. doi: 10.1111/j.1541-0420.2005.00374.x.
    1. Agresti A. An Introduction to Categorical Data Analysis. New York: Wiley; 1996.
    1. Bates D, Maechler M: Linear mixed-effects models using S4 classes (Ime4 package). 2013. .
    1. Christensen RHB: Package ‘ordinal’, CRAN; 2013. .
    1. Box GEP, Cox DR. An analysis of transformations revisited, rebutted. J Am Stat Assoc. 1982;77(377):209. doi: 10.1080/01621459.1982.10477788.
    1. Fixsen DL, Naoom SF, Blase KA, Friedman RM, Wallace F. Implementation Research: A Synthesis of the Literature. Tampa, FL: FMHI Publication #231; 2005.
    1. Saldana L, Chamberlain P, Bradford WD, Campbell M, Landsverk J: The cost of implementing new strategies (COINS): a method for mapping implementation resources using the Stages of Implementation Completion.Children Youth Services Rev In Press.
    1. Rogers E. Diffusion of Innovations. 4. New York: The Free Press; 1995.
    1. Collins C, Harshbarger C, Sawyer R, Hamdallah M. The diffusion of effective behavioral interventions project: development, implementation, and lessons learned. AIDS Educ Prevent. 2006;18(supp):5–20. doi: 10.1521/aeap.2006.18.supp.5.
    1. Elliott DS, Mihalic S. Issues in disseminating and replicating effective prevention programs. Prev Sci. 2004;5(1):47–53. doi: 10.1023/B:PREV.0000013981.28071.52.
    1. Chinman M, Hunter SB, Ebener P, Paddock SM, Stillman L, Imm P, Wandersman A. The getting to outcomes demonstration and evaluation: an illustration of the prevention support system. Am J Community Psychol. 2008;41(3–4):206–224. doi: 10.1007/s10464-008-9163-2.
    1. Chinman M, Imm P, Wandersman A. Getting to Outcomes 2004: Promoting Accountability Through Methods and Tools for Planning, Implementation, and Evaluation. Santa Monica, CA: Rand Corporation; 2004.
    1. Chinman M, Early D, Ebener PA, Hunter S, Imm P, Jenkins P, Sheldon J, Wandersman A. Getting to outcomes: a community-based participatory approach to preventive interventions. J Interprof Care. 2004;18(4):441–443. doi: 10.1080/13561820400011727.
    1. Chinman M, Hannah G, Wandersman A, Ebener P, Hunter SB, Imm P, Sheldon J. Developing a community science research agenda for building community capacity for effective preventive interventions. Am J Community Psychol. 2005;35(3–4):143–157. doi: 10.1007/s10464-005-3390-6.
    1. Chinman M, Tremain B, Imm P, Wandersman A. Strengthening prevention performance using technology: a formative evaluation of interactive Getting To Outcomes®. Am J Orthopsychiatry. 2009;79(4):469–481. doi: 10.1037/a0016705.
    1. Wandersman A. Getting To Outcomes: an evaluation capacity building example of rationale, science, and practice. Am J Eval. 2013;35(1):100–106. doi: 10.1177/1098214013500705.
    1. Palinkas LA, Holloway IW, Rice E, Brown CH, Valente T, Chamberlain P. Influence network linkages across treatment conditions in randomized controlled trial of two strategies for scaling up evidence-based practices in public youth-serving systems. Implement Sci. 2013;8(1):133. doi: 10.1186/1748-5908-8-133.
    1. Trochim W, Kane C, Graham MJ, Pincus HA. Evaluating translational research: a process marker model. Clin Transl Sci. 2011;4(3):153–162. doi: 10.1111/j.1752-8062.2011.00291.x.

Source: PubMed

Sponsorzy i współpracownicy

Warunki medyczne

Interwencje lekowe

3
Subskrybuj