Can implementation support help community-based settings better deliver evidence-based sexual health promotion programs? A randomized trial of Getting To Outcomes®

Matthew Chinman, Joie Acosta, Patricia Ebener, Patrick S Malone, Mary E Slaughter, Matthew Chinman, Joie Acosta, Patricia Ebener, Patrick S Malone, Mary E Slaughter

Abstract

Background: Research is needed to evaluate the impact of implementation support interventions over and above typical efforts by community settings to deploy evidence-based prevention programs.

Methods: Enhancing Quality Interventions Promoting Healthy Sexuality is a randomized controlled trial testing Getting To Outcomes (GTO), a 2-year implementation support intervention. It compares 16 Boys and Girls Club sites implementing Making Proud Choices (MPC, control group), a structured teen pregnancy prevention evidence-based program with 16 similar sites implementing MPC augmented with GTO (intervention group). All sites received training and manuals typical for MPC. GTO has its own manuals, training, and onsite technical assistance (TA) to help practitioners complete key programming practices specified by GTO. During the first year, TA providers helped the intervention group adopt, plan, and deliver MPC. This group then received training on the evaluation and quality improvement steps of GTO, including feedback reports summarizing their data, which were used in a TA-facilitated quality improvement process that yielded revised plans for the second MPC implementation. This paper presents results regarding GTO's impact on performance of the sites (i.e., how well key programming practices were carried out), fidelity of MPC implementation, and the relationship between amount of TA support, performance, and fidelity. Performance was measured using ratings made from a standardized, structured interview conducted with participating staff at all 32 Boys and Girls Clubs sites after the first and second years of MPC implementation. Multiple elements of fidelity (adherence, classroom delivery, dosage) were assessed at all sites by observer ratings and attendance logs.

Results: After 2 years, the intervention sites had higher ratings of performance, adherence, and classroom delivery (dosage remained similar). Higher performance predicted greater adherence in both years.

Conclusions: These findings suggest that in typical community-based settings, manuals and training common to structured EBPs may be sufficient to yield low levels of performance and moderate levels of fidelity but that systematic implementation support is needed to achieve high levels of performance and fidelity.

Trial registration: ClinicalTrials.gov, NCT01818791.

Keywords: Community-based; Evidence-based prevention; Fidelity; Implementation support.

Figures

Fig. 1
Fig. 1
Implementation support logic model
Fig. 2
Fig. 2
Getting To Outcomes flow

References

    1. Wandersman A, Florin P. Community interventions and effective prevention: bringing researchers/evaluators, funders and practitioners together for accountability. Am Psychol. 2003;58(6/7):441–8. doi: 10.1037/0003-066X.58.6-7.441.
    1. Glasgow R, Vinson C, Chambers D, Khoury M, Kaplan R, Hunter C. National Institutes of Health approaches to dissemination and implementation science: current and future directions. Am J Public Health. 2012;102(7):1274–81. doi: 10.2105/AJPH.2012.300755.
    1. Chinman M, Acosta J, Hunter S, Ebener P. Getting To Outcomes®: evidence of empowerment evaluation and evaluation capacity building at work. In: Fetterman DM, Kaftarian S, Wandersman A, editors. Empowerment Evaluation: Knowledge and Tools for Self-assessment, Evaluation Capacity Building and Accountability. 2. Thousand Oaks: Sage; 2015. pp. 316–36.
    1. Hamilton BE, Martin JA, Osterman MJK, Curtin SC. Births: preliminary data for 2013. Hyattsville: National Center for Health Statistics; 2014.
    1. Terry-Human E, Manlove J, Moore KA. Playing catch-up: how the children of teen mothers fare. Hyattsville: National Center for Health Statistics; 2005.
    1. Abma JC, Martinez GM, Mosher WD, Dawson BS. Teenagers in the United States: sexual activity, contraceptive use, and childbearing, 2002. Hyattsville: National Center for Health Statistics; 2004.
    1. Perper K, Peterson K, Manlove J. Diploma attachment among teen mothers (Fact Sheet) Washington: Child Trends; 2010.
    1. Nations U. Demographic yearbook 2009-2010: live births by age of mother. New York: NY United Nations; 2011.
    1. Goesling B, Colman S, Trenholm C, Terzian M, Moore K. Programs to reduce teen pregnancy, sexually transmitted infections, and associated sexual risk behaviors: a systematic review. J Adolesc Health. 2014;54(5):499–507. doi: 10.1016/j.jadohealth.2013.12.004.
    1. Chervin DD, Philliber S, Brindis CD, Chadwick AE, Revels ML, Kamin SL, et al. Community capacity building in CDC’s Community Coalition Partnership Programs for the Prevention of Teen Pregnancy. J Adolesc Health. 2005;37:S11–9. doi: 10.1016/j.jadohealth.2005.06.001.
    1. Kramer JS, Philliber S, Brindis CD, Kamin SL, Chadwick AE, Revels ML, et al. Coalition models: lessons learned from the CDC’s Community Coalition Partnership Programs for the Prevention of Teen Pregnancy. J Adolesc Health. 2005;37:S20–30. doi: 10.1016/j.jadohealth.2005.05.010.
    1. Green L. From research to ‘best practices’ in other settings and populations. Am J Health Behav. 2001;25(3):165–78. doi: 10.5993/AJHB.25.3.2.
    1. Livet M, Wandersman A. Organizational functioning: facilitating effective interventions and increasing the odds of programming success. In: Fetterman DM, Wandersman A, editors. Empowerment evaluation in practice. New York: Guilford; 2005. pp. 123–54.
    1. Chinman M, Imm P, Wandersman A. Getting to Outcomes 2004: promoting accountability through methods and tools for planning, implementation, and evaluation. Santa Monica: RAND Corporation; 2004.
    1. Lewis KM, Lesesne CA, Zahniser SC, Wilson MM, Desiderio G, Wandersman A, et al. Developing a prevention synthesis and translation system to promote science-based approaches to teen pregnancy, HIV and STI prevention. Am J Community Psychol. 2012;30(3-4):553–71. doi: 10.1007/s10464-012-9510-1.
    1. Imm P, Chinman M, Wandersman A, Rosenbloom D, Guckenburg S, Leis R. Using the “Getting to Outcomes” approach to help communities prevent underage drinking. Santa Monica: RAND Corporation; 2007.
    1. Fisher D, Imm P, Chinman M, Wandersman A. Getting To Outcomes with developmental assets: ten steps to measuring success in youth programs and communities. Minneapolis: Search Institute; 2006.
    1. Chinman M, Hannah G, Wandersman A, Ebener P, Hunter S, Imm P, et al. Developing a community science research agenda for building community capacity for effective preventive interventions. Am J Community Psychol. 2005;3/4:143–57. doi: 10.1007/s10464-005-3390-6.
    1. Goodman RM, Wandersman A, Chinman M, Imm P, Morrissey E. An ecological assessment of community-based interventions for prevention and health promotion: approaches to measuring community coalitions. Am J Community Psychol. 1996;24(1):33–61. doi: 10.1007/BF02511882.
    1. Ajzen I, Fishbein M. Attitude-behavior relations: a theoretical analysis and review of empirical research. Psychol Bull. 1977;84:888–918. doi: 10.1037/0033-2909.84.5.888.
    1. Bandura A. Health promotion by social cognitive means. Health Educ Behav. 2004;31(2):143–64. doi: 10.1177/1090198104263660.
    1. Fishbein M, Ajzen I. Atitudes toward objects as predictive of single and multiple behavioral criteria. Psychol Rev. 1974;81:59–74. doi: 10.1037/h0035872.
    1. Fishbein M, Ajzen I. Belief, attitude, intention, and behavior: an introduction to theory and research. Reading: Addison-Wesley; 1975.
    1. Acosta J, Chinman M, Ebener P, Malone PS, Paddock S, Phillips A, et al. An intervention to improve program implementation: findings from a two-year cluster randomized trial of Assets—Getting To Outcomes. Implement Sci. 2013;8:87. doi: 10.1186/1748-5908-8-87.
    1. Smelson DA, Chinman M, McCarthy S, Hannah G, Sawh L, Glickman M. A cluster randomized hybrid type III trial testing an implementation support strategy to facilitate the use of an evidence-based practice in VA homeless programs. Implement Sci. 2015;10:79. doi: 10.1186/s13012-015-0267-4.
    1. Durlak JA, DuPre EP. Implementation matters: a review of research on the influence of implementation on program outcomes and the factors affecting implementation. Am J Community Psychol. 2008;41:327–50. doi: 10.1007/s10464-008-9165-0.
    1. Zief S, Shapiro R, Strong D. How states are implementing evidence-based teen pregnancy prevention programs through the Personal Responsibility Education Program. Washington: Office of Planning, Research and Evaluation, Administration for Children and Families; 2014.
    1. Lesesne CA, Lewis KM, Moore C, Fisher D, Green D, Wandersman A. Promoting science-based approaches to Teen Pregnancy Prevention using Getting To Outcomes. Atlanta: Centers for Disease Control and Prevention; 2007.
    1. Philliber S, Nolte K. Implementation science: promoting science-based approaches to prevent teen pregnancy. Prev Sci. 2008;9(3):166–77. doi: 10.1007/s11121-008-0094-9.
    1. Hawkins JD, Oesterle S, Brown EC, Arthur MW, Abbott RD, Fagan AA, et al. Results of a type 2 translational research trial to prevent adolescent drug use and delinquency: a test of communities that care. Arch Pediatr Adolesc Med. 2009;163(9):789–98. doi: 10.1001/archpediatrics.2009.141.
    1. Spoth R, Redmond C, Shin C, Greenberg M, Clair S, Feinberg M. Substance-use outcomes at 18 months past baseline: the PROSPER community-university partnership trial. Am J Prev Med. 2007;32(5):395–402. doi: 10.1016/j.amepre.2007.01.014.
    1. Rohrbach LA, Gunning M, Sun P, Sussman S. The Project Towards No Drug Abuse (TND) Dissemination Trial: implementation fidelity and immediate outcomes. Prev Sci. 2010;11:77–88. doi: 10.1007/s11121-009-0151-z.
    1. Chinman M, Hunter SB, Ebener P, Paddock S, Stillman L, Imm P, et al. The Getting To Outcomes demonstration and evaluation: an illustration of the prevention support system. Am J Community Psychol. 2008;41:206–24. doi: 10.1007/s10464-008-9163-2.
    1. Chinman M, Tremain B, Imm P, Wandersman A. Strengthening prevention performance using technology: a formative evaluation of interactive Getting To Outcomes. Am J Orthopsychiatry. 2009;79:469–81. doi: 10.1037/a0016705.
    1. Chinman M, Acosta J, Ebener P, Burkhart Q, Clifford M, Corsello M, et al. Intervening with practitioners to improve the quality of prevention: one year findings from a randomized trial of assets—Getting To Outcomes. J Prim Prev. 2013;34:173–91. doi: 10.1007/s10935-013-0302-7.
    1. Chinman M, Ebener P, Burkhart Q, Osilla KC, Imm P, Paddock SM, et al. Evaluating the impact of getting to outcomes-underage drinking on prevention capacity and alcohol merchant attitudes and selling behaviors. Prev Sci. 2014;15(4):485–96. doi: 10.1007/s11121-013-0389-3.
    1. Jemmott JB, Jemmott LS, Fong GT. Abstinence and safer sex HIV risk-reduction interventions for African American adolescents: a randomized controlled trial. JAMA. 1998;279:1529–36. doi: 10.1001/jama.279.19.1529.
    1. Borawski EA, Trapl ES, Adams-Tufts K, Hayman LL, Goodwin MA, Lovegreen LD. Taking be proud! Be responsible! To the suburbs: a replication study. Perspect Sex Reprod Health. 2009;41(1):12–22. doi: 10.1363/4101209.
    1. Aarons GA. Mental health provider attitudes toward adoption of evidence-based practice: The Evidence-Based Practice Attitude Scale (EBPAS) Ment Health Serv Res. 2004;6(2):61–74. doi: 10.1023/B:MHSR.0000024351.12294.65.
    1. Aarons GA, Sommerfeld DH, Walrath-Greene CM. Evidence-based practice implementation: the impact of public versus private sector organization type on organizational support, provider attitudes, and adoption of evidence-based practice. Implement Sci. 2009;4:1–13. doi: 10.1186/1748-5908-4-83.
    1. Bandura A. A social cognitive approach to the exercise of control over AIDS infection. In: DiClemente R, editor. Adolescents and AIDS: A generation in jeopardy. Newbury Park: Sage Publications; 1992. pp. 89–116.
    1. Ajzen I. Theory of planned behavior. Organ Behav Hum Decis Process. 1991;50:179–211. doi: 10.1016/0749-5978(91)90020-T.
    1. Jemmott JB, Jemmott L, Spears H, Hewitt N, Cruz-Collins M. Self-efficacy, hedonistic expectancies, and condom-use intentions among inner-city Black adolescent women: a social cognitive approach to AIDS risk behavior. J Adolesc Health. 1992;13:512–9. doi: 10.1016/1054-139X(92)90016-5.
    1. Zief S, Shapiro R, Strong D. The Personal Responsibility Education Program (PREP): launching a nationwide adolescent pregnancy prevention effort. Washington: Office of Planning, Research and Evaluation, Administration for Children and Families; 2013.
    1. Chinman M, Acosta J, Ebener P, Driver J, Keith J, Peebles D. Enhancing Quality Interventions Promoting Healthy Sexuality (EQUIPS): a novel application of translational research methods. Clin Transl Sci. 2013;6:232–7. doi: 10.1111/cts.12031.
    1. Dusenbury L, Brannigan R, Hansen W, Walsh J, Falco M. Quality of implementation: developing measures crucial to understanding the diffusion of preventive interventions. Health Educ Behav. 2005;20:308–13.
    1. Rolleri LA, Lezin N, Taylor J, Moore C, Wilson MM, Fuller TR, et al. Becoming a Responsible Teen adaptation kit. Santa Cruz: ETR Associates; 2011.
    1. Kellermann A, Romano J, Rodríguez de Gil P, Pham T, Rasmussen P, Chen Y, et al. GEN_OMEGA2: A SAS® macro for computing the generalized omega-squared effect size associated with analysis of variance models. SouthEast SAS Users Group 2013 Proceedings. Cary: SAS Institute Inc; 2013.
    1. Grissom RJ, Kim JJ. Effect sizes for research: univariate and multivariate applications. 2. New York: Routledge; 2012.
    1. Benjamini Y, Hochberg B. Controlling the false discovery rate: a practical and powerful approach to multiple testing. J R Stat Soc. 1995;57(1):289–300.
    1. Fagan AA, Hanson K, Hawkins JD, Arthur MW. Bridging science to practice: achieving prevention program implementation fidelity in the Community Youth Development Study. Am J Community Psychol. 2008;41:235–49. doi: 10.1007/s10464-008-9176-x.
    1. Spoth R, Guyll M, Lillehoj CJ, Redmond C. PROSPER study of evidence-based intervention implementation quality by community–university partnerships. J Community Psychol. 2007;35(8):981–99. doi: 10.1002/jcop.20207.
    1. Lesesne CA, Lewis KM, White CP, White DC, Green DC, Duffy JL, et al. Promoting science-based approaches to teen pregnancy prevention: proactively engaging the three systems of the interactive systems framework. Am J Community Psychol. 2008;41:379–92. doi: 10.1007/s10464-008-9175-y.

Source: PubMed

Sponsors en medewerkers

Medische omstandigheden

Geneesmiddelinterventies

3
Abonneren