An intervention to improve program implementation: findings from a two-year cluster randomized trial of Assets-Getting To Outcomes

Joie Acosta, Matthew Chinman, Patricia Ebener, Patrick S Malone, Susan Paddock, Andrea Phillips, Peter Scales, Mary Ellen Slaughter, Joie Acosta, Matthew Chinman, Patricia Ebener, Patrick S Malone, Susan Paddock, Andrea Phillips, Peter Scales, Mary Ellen Slaughter

Abstract

Background: Studies have shown that communities have not always been able to implement evidence-based prevention programs with quality and achieve outcomes demonstrated by prevention science. Implementation support interventions are needed to bridge this gap between science and practice. The purpose of this article is to present two-year outcomes from an evaluation of the Assets Getting To Outcomes (AGTO) intervention in 12 Maine communities engaged in promoting Developmental Assets, a positive youth development approach to prevention. AGTO is an implementation support intervention that consists of: a manual of text and tools; face-to-face training, and onsite technical assistance, focused on activities shown to be associated with obtaining positive results across any prevention program.

Methods: This study uses a nested and cross-sectional, cluster randomized controlled design. Participants were coalition members and program staff from 12 communities in Maine. Each coalition nominated up to five prevention programs to participate. At random, six coalitions and their respective 30 programs received the two-year AGTO intervention and the other six maintained routine operations. The study assessed prevention practitioner capacity (efficacy and behaviors), practitioner exposure to and use of AGTO, practitioner perceptions of AGTO, and prevention program performance. Capacity of coalition members and performance of their programs were compared between the two groups across the baseline, one-, and two-year time points.

Results: We found no significant differences between AGTO and control group's prevention capacity. However, within the AGTO group, significant differences were found between those with greater exposure to and use of AGTO. Programs that received the highest number of technical assistance hours showed the most program improvement.

Conclusions: This study is the first of its kind to show that use of an implementation support intervention-AGTO -yielded improvements in practitioner capacity and consequently in program performance on a large sample of practitioners and programs using a randomized controlled design. CLINICALTRIALS.GOV IDENTIFIER: NCT00780338.

Figures

Figure 1
Figure 1
GTO 10 steps.
Figure 2
Figure 2
Plot of the any AGTO use vs. no AGTO use among participants assigned to the AGTO group: adjusted means on 0-100 capacity scales.

References

    1. Miller T, Hendrie D. Substance abuse prevention dollars and cents: A cost-benefit analysis (DHHS Pub. No. (SMA) 07–4298) Rockville, MD: Center for Substance Abuse Prevention, Substance Abuse and Mental Health Services Administration; 2009.
    1. Backer TE. Finding the balance: program fidelity and adaptation in substance abuse prevention: CSAP. 2001.
    1. Ennett ST, Ringwalt CL, Thorne J, Rohrbach LA, Vincus A, Simons-Rudolph A. et al.A comparison of current practice in school-based substance abuse prevention programs with meta-analysis findings. Prev Sci. 2003;4:1–14. doi: 10.1023/A:1021777109369.
    1. Cantor D, Crosse S, Hagen CA, Mason MJ, Siler AJ, Von Glatz A. Wide Scope, Questionable Quality: Drug and Violence Prevention Efforts in American Schools. Report on the Study on School Violence and Prevention. In. Washington, DC: U.S. Department of Education, Planning and Evaluation Service; 2001.
    1. Crosse S, Burr M, Cantor D, Hagen CA, Hantman I. Wide Scope, Questionable Quality: Drug and Violence Prevention Efforts in American Schools. Report on the Study on School Violence and Prevention. In. Washington, DC: U.S. Department of Education, Planning and Evaluation Service; 2001.
    1. Reuter P, Timpane JM. Assessing options for the safe and drug free schools and communities act. Santa Monica, CA: RAND; 2001. No. MR-1328.
    1. Silvia ES, Thorne J. School-based drug prevention programs: A longitudinal study in selected school districts. In. North Carolina: Research Triangle Institute; 1997.
    1. U.S. Department of Education OoP, Evaluation and Policy Development, Policy and Program Studies Service. Prevalence and Implementation Fidelity of Research-Based Prevention Programs in Public Schools: Final Report. Washington, D.C: ; 2011.
    1. Wandersman A, Florin P. Community interventions and effective prevention: bringing researchers/evaluators, funders and practitioners together for accountability. Am Psychol. 2003;58(6/7):441–448.
    1. Green L. From research to 'Best Practices' in other settings and populations. Am J Health Behav. 2001;25(3):165–178. doi: 10.5993/AJHB.25.3.2.
    1. Livet M, Wandersman A. In: Empowerment evaluation in practice. Fetterman DM, Wandersman A, editor. New York, NY: Guilford; 2005. Organizational functioning: facilitating effective interventions and increasing the odds of programming success; pp. 123–154.
    1. Chinman M, Hunter S, Ebener P, Paddock S, Stillman L, Imm P, Wandersman A. The Getting To Outcomes demonstration and evaluation: an illustration of the prevention support system. Am J Community Psychol. 2008;41:206–224. doi: 10.1007/s10464-008-9163-2.
    1. Chinman M, Tremain B, Imm P, Wandersman A. Strengthening prevention performance using technology: a formative evaluation of interactive Getting To OutcomesTM. Am J Orthopsychiatry. 2009;79:469–481.
    1. Fishbein M, Ajzen A. Beliefs, attitudes, intentions, and behavior: An introduction to theory and research. Reading, MA: Addison-Wesley; 1975.
    1. Ajzen I, Fishbein M. Attitude-behavior relations: a theoretical analysis and review of empirical research. Psychol Bull. 1977;84:888–918.
    1. Bandura A. Health promotion by social cognitive means. Health Educ Behav. 2004;31(2):143–164. doi: 10.1177/1090198104263660.
    1. Fishbein M, Ajzen I. Atitudes toward objects as predictive of single and multiple behavioral criteria. Psychol Rev. 1974;81:59–74.
    1. Chinman M, Acosta J, Ebener P, Burkhart Q, Clifford M, Corsello M, Duffy T, Hunter S, Jones M, Lahti M, Establishing and evaluating the key functions of an interactive systems framework based on Assets-Getting To Outcomes. Am J Commun Psychol. 2012. .
    1. Benson PL, Scales PC, Leffert N, Roehlkepartain ER. A fragile foundation: The state of developmental assets among American youth. Minneapolis, MN: Search Institute; 1999.
    1. Leffert N, Benson PL, Scales PC, Sharma AR, Drake DR, Blyth DA. Developmental assets: measurement and prediction of risk behaviors among adolescents. Appl Dev Sci. 1998;2:209–230. doi: 10.1207/s1532480xads0204_4.
    1. Scales PC, Benson PL, Leffert N, Blyth DA. Contribution of developmental assets to the prediction of thriving among adolescents. Appl Dev Sci. 2000;4:27–46. doi: 10.1207/S1532480XADS0401_3.
    1. Fisher D, Imm PS, Chinman M, Wandersman A. Getting To Outcomes with Developmental Assets: Ten steps to measuring success in youth programs and communities. Minneapolis, MN: Search Institute; 2006.
    1. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4:50–65. doi: 10.1186/1748-5908-4-50.
    1. Rohrbach LA, D’Onofrio CN, Backer TE, Montgomery SB. Diffusion of school-based substance abuse prevention programs. Am Behav Sci. 1996;39:919–934. doi: 10.1177/0002764296039007012.
    1. Rohrbach LA, Graham JW, Hansen WB. Diffusion of a school-based substance abuse prevention program: predictors of program implementation. Prev Med. 1993;22(2):237–260. doi: 10.1006/pmed.1993.1020.
    1. McCormick LK, Steckler AB, McLeroy KR. Diffusion of innovations in schools: a study of adoption and implementation of school-based tobacco prevention curricula. Am J Health Promot. 1995;9:210–219. doi: 10.4278/0890-1171-9.3.210.
    1. Steckler AB, McLeroy KR, Goodman RM, McCormick L, Bird ST. Toward integrating qualitative and quantitative methods: an introduction. Health Educ Q. 1992;19:1–8. doi: 10.1177/109019819201900101.
    1. Parcel GS, Erikson MP, Lovato CY, Gottlieb NH, Brink SG, Green LW. The diffusion of school-based tobacco-use prevention program: project description and baseline data. Health Educ Res. 1989;4:111–124. doi: 10.1093/her/4.1.111.
    1. Cividin TM, Ottoson JM. Linking reasons for continuing professional education participation with post-program application. J Contin Educ Health Professions. 1997;17:46–55. doi: 10.1002/chp.4750170107.
    1. Ottoson JM. After the applause: exploring multiple influences on application following an adult education program. Adult Educ Quart. 1997;47:92–107. doi: 10.1177/074171369704700203.
    1. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4:1. doi: 10.1186/1748-5908-4-1.
    1. Murray DM. Design and analysis of group-randomized trials. Oxford, UK: Press OU; 1998.
    1. Battelle. Annual findings report: Drug-free communities support program national evaluation. Washington, DC: Executive Office of the President Office of Administration Office of National Drug Control Policy; 2008.
    1. Butterfoss FD, Goodman RM, Wandersman A. Community coalitions for prevention and health promotion. Health Educ Res. 1993;8:315–330. doi: 10.1093/her/8.3.315.
    1. Ellickson PL, Bell RM. Drug prevention in junior high: a multi-site longitudinal test. Science. 1990;247:1265–1372.
    1. Botvin GJ, Baker E, Dusenbury L, Botvin EM, Diaz T. Long-term follow-up results of a randomized drug abuse prevention trial in a white middle-class population. JAMA. 1995;273(14):1106–1112. doi: 10.1001/jama.1995.03520380042033.
    1. Jemmott JB, Jemmott L, Spears H, Hewitt N, Cruz-Collins M. Self-efficacy, hedonistic expectancies, and condom-use intentions among inner-city Black adolescent women: a social cognitive approach to AIDS risk behavior. J Adolescent Health. 1992;13:512–519. doi: 10.1016/1054-139X(92)90016-5.
    1. Jemmott JB, Jemmott LS, Fong GT. Reductions in HIV risk-associated sexual behaviors among Black male adolescents: effects of an AIDS prevention intervention. Am J Public Health. 1992;82:372–377. doi: 10.2105/AJPH.82.3.372.
    1. Jemmott JB, Jemmott LS, Fong GT, McCaffree K. Reducing HIV risk-associated sexual behavior among African American adolescents: Testing the generality of intervention effects. Am J Community Psychol. 1999;27:161–187. doi: 10.1007/BF02503158.
    1. Chinman M, Hannah G, Wandersman A, Ebener P, Hunter S, Imm P, Sheldon J. Developing a community science research agenda for building community capacity for effective preventive interventions. Am J Community Psychol. 2005;3/4:143–157.
    1. Mitchell R, Stone-Wiggins B, Stevenson JF, Florin P. Cultivating capacity: outcomes of a statewide support system for prevention coalitions. J Prev Interv Community. 2004;27:67–87. doi: 10.1300/J005v27n02_05.
    1. O’Donnell LO, Scattergood P, Adler M, San Doval A, Barker M, Kelly JA. et al.The role of technical assistance in the replication of effective HIV interventions. AIDS Educ Prev. 2000;12:99–111.
    1. Stevenson JF, Florin P, Mills DS, Andrade M. Building evaluation capacity in human service organizations: a case study. Eval Program Plann. 2002;25(3):233–243. doi: 10.1016/S0149-7189(02)00018-6.
    1. Kitson A, Harvey G, McCormack B. Enabling the implementation of evidence based practice: a conceptual framework. Qual Health Care. 1998;7:149–158. doi: 10.1136/qshc.7.3.149.
    1. Rycroft-Malone J, Harvey G, Seers K, Kitson A, McCormack B, Titchen A. An exploration of the factors that influence the implementation of evidence into practice. J Clin Nurs. 2002;13:913–924.
    1. Stetler CB, Legro MW, Rycroft-Malone J, Bowman C, Curran G, Guihan M. et al.Role of 'external facilitation' in implementation of research findings: a qualitative evaluation of facilitation experiences in the Veterans Health Administration. Implement Sci. 2006;23
    1. Weiss E, Miller-Anderson R, Lasker R. Making the most of collaboration: exploring the relationship between partnership synergy and partnership functioning. Health Educ Behav. 2002;29:683–698. doi: 10.1177/109019802237938.
    1. Lehman WEK, Greener JM, Simpson DD. Assessing organizational readiness for change. J Subst Abuse Treat. 2002;22(4):197–209. doi: 10.1016/S0740-5472(02)00233-7.
    1. Hall GE, Hord SM. Implementing change. Boston, MA: Allyn and Bacon; 2001.
    1. Hall GE, Loucks SF, Rutherford WL, Newlove BW. Levels of use of the innovation: a framework for analyzing innovation adoption. J Teach Educ. 1975;26:52. doi: 10.1177/002248717502600114.
    1. Hall GE, Loucks SF. A developmental model for determining whether the treatment is actually implemented. Am Educ Res J. 1977;14(3):263–276. doi: 10.3102/00028312014003263.
    1. Chinman M, Acosta J, Ebener P, Burkhart Q, Clifford M, Corsello M, Duffy T, Hunter S, Jones M, Lahti M, Intervening with practitioners to improve the quality of prevention: One year findings from a randomized trial of Assets-Getting To Outcomes. J Prim Prev. In press.
    1. Chinman M, Tremain B, Imm P, Wandersman A. Strengthening prevention performance using technology: a formative evaluation of interactive Getting To Outcomes. Am J Orthopsychiatry. 2009;79:469–481. PMC2859836.
    1. Bryt T, Bishop J, Carlin JB. Bias, prevalence and kappa. J Clin Epidemiol. 1993;46(5):423–429. doi: 10.1016/0895-4356(93)90018-V.
    1. Lincoln YS, Guba EG. Naturalistic inquiry. In. Beverly Hills, CA: Sage; 1985.
    1. Graham ID, Logan J. Innovations in knowledge transfer and continuity of care. Can J Nurs Res. 2004;36(2):89–103.
    1. Hunter SB, Chinman M, Ebener P, Imm P, Wandersman A, Ryan G. Technical assistance as a prevention capacity-building tool: a demonstration using the Getting To Outcomes framework. Health Educ Behav. 2009;36:810–828. doi: 10.1177/1090198108329999.
    1. Keener DC. Toward a science of capacity building: An examination of technical assistance following a training program for prevention professionals. In. Columbia, SC, USA: University of South Carolina; 2007.
    1. Feinberg ME, Ridenour TA, Greenberg MT. The longitudinal effect of technical assistance dosage on the functioning of communities that care prevention boards in Pennsylvania. J Prim Prev. 2008;29(2):145–165. doi: 10.1007/s10935-008-0130-3.
    1. Hawkins JD, Oesterle S, Brown EC, Arthur MW, Abbott RD, Fagan AA, Catalano RF. Results of a type 2 translational research trial to prevent adolescent drug use and delinquency: a test of communities that care. Arch Pediatr Adolesc Med. 2009 September;163(9):789–798. doi: 10.1001/archpediatrics.2009.141.
    1. Spoth R, Redmond C, Shin C, Greenberg M, Clair S, Feinberg M. Substance-use outcomes at 18 months past baseline: the PROSPER Community-University partnership trial. Am J Prev Med. 2007 May;32(5):395–402. doi: 10.1016/j.amepre.2007.01.014.
    1. Wandersman A. Four keys to success (theory, implementation, evaluation, and resource/system support): high hopes and challenges in participation. Am J Community Psychol. 2009;43:3–21. doi: 10.1007/s10464-008-9212-x.
    1. Wandersman A, Chien VH, Katz J. In: Cultivating communities of practice. Wenger E, McDermott R, Snyder WM, editor. Boston, MA: Harvard Business School Press; 2002. Toward an evidence-based system for innovation support for implementing innovations with quality: tools, training, technical assistance, and quality assurance/quality improvement. In press.

Source: PubMed

Sponsoren und Mitarbeiter

Krankheiten

Drogeninterventionen

3
Abonnieren