Comparing, Contrasting, and Integrating Dissemination and Implementation Outcomes Included in the RE-AIM and Implementation Outcomes Frameworks

Kathryn Louise Reilly, Sarah Kennedy, Gwenndolyn Porter, Paul Estabrooks, Kathryn Louise Reilly, Sarah Kennedy, Gwenndolyn Porter, Paul Estabrooks

Abstract

As the field of dissemination and implementation science matures, there are a myriad of outcomes, identified in numerous frameworks, that can be considered across individual, organizational, and population levels. This can lead to difficulty in summarizing literature, comparing across studies, and advancing translational science. This manuscript sought to (1) compare, contrast, and integrate the outcomes included in the RE-AIM and Implementation Outcomes Frameworks (IOF) and (2) expand RE-AIM indicators to include relevant IOF dissemination and implementation outcomes. Cross tabular comparisons were made between the constitutive definitions of each construct, across frameworks, to reconcile apparent discrepancies between approaches and to distinguish between implementation outcomes and implementation antecedents. A great deal of consistency was identified across approaches, including adoption (the intention, initial decision, or action to employ an evidence-based intervention), fidelity/implementation (the degree to which an intervention was delivered as intended), organizational maintenance/sustainability (extent to which a newly implemented treatment is maintained or institutionalized), and cost. The IOF construct of penetration was defined as a higher-order construct that may encompass the reach, adoption, and organizational maintenance outcomes within RE-AIM. Within the IOF approach acceptability, appropriateness, and feasibility did not match constitutive definitions of dissemination or implementation but rather reflected theoretical antecedents of implementation outcomes. Integration of the IOF approach across RE-AIM indicators was successfully achieved by expanding the operational definitions of RE-AIM to include antecedents to reach, adoption, implementation, and organizational maintenance. Additional combined metrics were also introduced including penetration, individual level utility, service provider utility, organizational utility, and systemic utility. The expanded RE-AIM indicators move beyond the current approaches described within both the RE-AIM framework and IOF and provides additional planning and evaluation targets that can contribute to the scientific field and increase the translation of evidence into practice.

Keywords: Implementation Outcomes Framework; RE-AIM (Reach, Effectiveness, Adoption, Implementation and Maintenance); implementation outcomes; scale-up; translational reseach.

Copyright © 2020 Reilly, Kennedy, Porter and Estabrooks.

References

    1. Kessler RS, Purcell EP, Glasgow RE, Klesges LM, Benkeser RM, Peek CJ. What does it mean to “employ” the RE-AIM model? Eval Health Prof. (2013) 36:44–66. 10.1177/0163278712446066
    1. Glasgow RE, Riley WT. Pragmatic measures: what they are and why we need them. Am J Prev Med. (2013) 45:237–43. 10.1016/j.amepre.2013.03.010
    1. Moullin JC, Dickson KS, Stadnick NA, Rabin B, Aarons GA. Systematic review of the exploration, preparation, implementation, sustainment (EPIS) framework. Implement Sci. (2019) 14:1. 10.1186/s13012-018-0842-6
    1. Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, et al. . Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health. (2011) 38:65–76. 10.1007/s10488-010-0319-7
    1. Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health. (1999) 89:1322–7. 10.2105/AJPH.89.9.1322
    1. Glasgow RE, Harden SM, Gaglio B, Rabin B, Lee Smith M, Porter GC, et al. . RE-AIM planning and evaluation framework: adapting to new science and practice with a 20-year review. Front Public Health. (2019) 7:64. 10.3389/fpubh.2019.00064
    1. Lewis CC, Proctor EK, Brownson RC. Measurement issues in dissemination and implementation research. In: Brownson RC, Colditz GA, Proctor EK. editors. Dissemination and Implementation Research in Health: Translating Science to Practice. 2nd ed New York, NY: Oxford University Press; (2018). p. 15.
    1. Glasgow RE, Emmons KM. How can we increase translation of research into practice? Types of evidence needed. Annu Rev Public Health. (2007) 28:413–33. 10.1146/annurev.publhealth.28.021406.144145
    1. Estabrooks PA, Allen KC. Updating, employing, and adapting: a commentary on what does it mean to “employ” the RE-AIM model. Eval Health Prof. (2013) 36:67–72. 10.1177/0163278712460546
    1. Almeida FA, Michaud TL, Wilson KE, Schwab RJ, Goessl C, Porter GC, et al. . Preventing diabetes with digital health and coaching for translation and scalability (PREDICTS): a type 1 hybrid effectiveness-implementation trial protocol. Contemp Clin Trials. (2020) 88:105877. 10.1016/j.cct.2019.105877
    1. Zoellner JM, Hedrick VE, You W, Chen Y, Davy BM, Porter KJ, et al. . Effects of a behavioral and health literacy intervention to reduce sugar-sweetened beverages: a randomized-controlled trial. Int J Behav Nutr Phys Act. (2016) 13:38. 10.1186/s12966-016-0362-1
    1. Glasgow RE, Klesges LM, Dzewaltowski DA, Estabrooks PA, Vogt TM. Evaluating the impact of health promotion programs: using the RE-AIM framework to form summary measures for decision making involving complex issues. Health Educ Res. (2006) 21:688–94. 10.1093/her/cyl081
    1. Aarons GA, Hurlburt M, Horwitz SM. Advancing a conceptual model of evidence-based practice implementation in public service sectors. Adm Policy Ment Health. (2011) 38:4–23. 10.1007/s10488-010-0327-7
    1. Lewis CC, Fischer S, Weiner BJ, Stanick C, Kim M, Martinez RG. Outcomes for implementation science: an enhanced systematic review of instruments using evidence-based rating criteria. Implement Sci. (2015) 10:155. 10.1186/s13012-015-0342-x
    1. Rabin BA, Brownson RC. Terminology in dissemination and implementation science. In: Brownson RC, Colditz GA, Proctor EK. editors. Dissemination and Implementation Research in Health: Translating Science to Practice. 2nd ed New York, NY: Oxford University Press; (2018). p. 19–45.
    1. Eisman AB, Kilbourne AM, Dopp AR, Saldana L, Eisenberg D. Economic evaluation in implementation science: making the business case for implementation strategies. Psychiatry Res. (2020) 283:112433. 10.1016/j.psychres.2019.06.008
    1. Rabin BA, McCreight M, Battaglia C, Ayele R, Burke RE, Hess PL, et al. . Systematic, multimethod assessment of adaptations across four diverse health systems interventions. Front Public Health. (2018) 6:102. 10.3389/fpubh.2018.00102
    1. Glasgow RE, Estabrooks PE. Pragmatic applications of RE-AIM for health care initiatives in community and clinical settings. Prev Chronic Dis. (2018) 15:E02. 10.5888/pcd15.170271
    1. Ward CE, Hall SV, Barnett PG, Jordan N, Duffy SA. Cost-effectiveness of a nurse-delivered, inpatient smoking cessation intervention. Transl Behav Med. (2019). 10.1093/tbm/ibz101. [Epub ahead of print].
    1. Wiltsey Stirman S, Baumann AA, Miller CJ. The FRAME: an expanded framework for reporting adaptations and modifications to evidence-based interventions. Implement Sci. (2019) 14:58. 10.1186/s13012-019-0898-y
    1. Estabrooks PA, Gyurcsik NC. Evaluating the impact of behavioral interventions that target physical activity: issues of generalizability and public health. Psychol Sport Exerc. (2003) 4:41–55. 10.1016/S1469-0292(02)00016-X
    1. Leeman J, Baquero B, Bender M, Choy-Brown M, Ko LK, Nilsen P, et al. . Advancing the use of organization theory in implementation science. Prev Med. (2019) 129S:105832. 10.1016/j.ypmed.2019.105832
    1. Lyon AR, Munson SA, Renn BN, Atkins DC, Pullmann MD, Friedman E, et al. . Use of human-centered design to improve implementation of evidence-based psychotherapies in low-resource communities: protocol for studies applying a framework to assess usability. JMIR Res Protoc. (2019) 8:e14990. 10.2196/14990
    1. Weiner BJ, Lewis CC, Stanick C, Powell BJ, Dorsey CN, Clary AS, et al. . Psychometric assessment of three newly developed implementation outcome measures. Implement Sci. (2017) 12:1–12. 10.1186/s13012-017-0635-3
    1. Indig D, Lee K, Grunseit A, Milat A, Bauman A. Pathways for scaling up public health interventions. BMC Public Health. (2017) 18:68. 10.1186/s12889-017-4572-5
    1. Dzewaltowski DA, Estabrooks PA, Glasgow RE. The future of physical activity behavior change research: what is needed to improve translation of research into health promotion practice? Exerc Sport Sci Rev. (2004) 32:57–63. 10.1097/00003677-200404000-00004
    1. Curran GM, Bauer M, Mittman B, Pyne JM, Stetler C. Effectiveness-implementation hybrid designs: combining elements of clinical effectiveness and implementation research to enhance public health impact. Med Care. (2012) 50:217–26. 10.1097/MLR.0b013e3182408812
    1. Dopp AR, Mundey P, Beasley LO, Silovsky JF, Eisenberg D. Mixed-method approaches to strengthen economic evaluations in implementation research. Implement Sci. (2019) 14:2. 10.1186/s13012-018-0850-6
    1. McHugh RK, Murray HW, Barlow DH. Balancing fidelity and adaptation in the dissemination of empirically-supported treatments: the promise of transdiagnostic interventions. Behav Res Ther. (2009) 47:946–53. 10.1016/j.brat.2009.07.005
    1. Chambers DA, Norton WE. The adaptome: advancing the science of intervention adaptation. Am J Prev Med. (2016) 51(4 Suppl. 2):S124–31. 10.1016/j.amepre.2016.05.011
    1. Shelton RC, Chambers DA, Glasgow RE. An extension of RE-AIM to enhance sustainability: addressing dynamic context and promoting health equity over time. Front Public Health. (2020) 8:134. 10.3389/fpubh.2020.00134
    1. Glasgow RE, Brownson RC, Kessler RS. Thinking about health-related outcomes: what do we need evidence about? Clin Transl Sci. (2013) 6:286–91. 10.1111/cts.12080
    1. Jones Rhodes WC, Ritzwoller DP, Glasgow RE. Stakeholder perspectives on costs and resource expenditures: tools for addressing economic issues most relevant to patients, providers, and clinics. Transl Behav Med. (2018) 8:675–82. 10.1093/tbm/ibx003
    1. Nilsen P. Making sense of implementation theories, models and frameworks. Implement Sci. (2015) 10:53. 10.1186/s13012-015-0242-0
    1. Feldstein AC, Glasgow RE. A practical, robust implementation and sustainability model (PRISM) for integrating research findings into practice. Jt Comm J Qual Patient Saf. (2008) 34:228–43. 10.1016/S1553-7250(08)34030-6
    1. McCreight MS, Rabin BA, Glasgow RE, Ayele RA, Leonard CA, Gilmartin HM, et al. . Using the practical, robust implementation and sustainability model (PRISM) to qualitatively assess multilevel contextual factors to help plan, implement, evaluate, and disseminate health services programs. Transl Behav Med. (2019) 9:1002–11. 10.1093/tbm/ibz085
    1. Powell BJ, Fernandez ME, Williams NJ, Aarons GA, Beidas RS, Lewis CC, et al. . Enhancing the impact of implementation strategies in healthcare: a research agenda. Front Public Health. (2019) 7:3. 10.3389/fpubh.2019.00003
    1. Kwan BM, McGinnes HL, Ory MG, Estabrooks PA, Waxmonsky JA, Glasgow RE. RE-AIM in the real world: use of the RE-AIM framework for program planning and evaluation in clinical and community settings. Front Public Health. (2019) 7:345. 10.3389/fpubh.2019.00345

Source: PubMed

Sponsors and Collaborators

Medical Conditions

Drug Interventions

3
Subscribe