RE-AIM Planning and Evaluation Framework: Adapting to New Science and Practice With a 20-Year Review

Russell E Glasgow, Samantha M Harden, Bridget Gaglio, Borsika Rabin, Matthew Lee Smith, Gwenndolyn C Porter, Marcia G Ory, Paul A Estabrooks, Russell E Glasgow, Samantha M Harden, Bridget Gaglio, Borsika Rabin, Matthew Lee Smith, Gwenndolyn C Porter, Marcia G Ory, Paul A Estabrooks

Abstract

The RE-AIM planning and evaluation framework was conceptualized two decades ago. As one of the most frequently applied implementation frameworks, RE-AIM has now been cited in over 2,800 publications. This paper describes the application and evolution of RE-AIM as well as lessons learned from its use. RE-AIM has been applied most often in public health and health behavior change research, but increasingly in more diverse content areas and within clinical, community, and corporate settings. We discuss challenges of using RE-AIM while encouraging a more pragmatic use of key dimensions rather than comprehensive applications of all elements. Current foci of RE-AIM include increasing the emphasis on cost and adaptations to programs and expanding the use of qualitative methods to understand "how" and "why" results came about. The framework will continue to evolve to focus on contextual and explanatory factors related to RE-AIM outcomes, package RE-AIM for use by non-researchers, and integrate RE-AIM with other pragmatic and reporting frameworks.

Keywords: RE-AIM; dissemination; evaluation; external validity; implementation.

Figures

Figure 1
Figure 1
Revised, enhanced RE-AIM/PRISM 2019 model.

References

    1. Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health. (1999) 89:1322–7. 10.2105/AJPH.89.9.1322
    1. Glasgow RE, Askew S, Purcell P, Levine E, Warner ET, Stange KC, et al. . Use of RE-AIM to address health inequities: application in a low-income community health center-based weight loss and hypertension self-management program. Transl Behav Med. (2013) 3:200–10. 10.1007/s13142-013-0201-8
    1. Gaglio B, Phillips SM, Heurtin-Roberts S, Sanchez MA, Glasgow RE. How pragmatic is it? lessons learned using PRECIS and RE-AIM for determining pragmatic characteristics of research. Implement Sci. (2014) 9:96. 10.1186/s13012-014-0096-x
    1. Glasgow RE, Nelson CC, Strycker LA, King DK. Using RE-AIM metrics to evaluate diabetes self-management support interventions. Am J Prev Med. (2006) 30:67–73. 10.1016/j.amepre.2005.08.037
    1. Estabrooks PA, Allen KC. Updating, employing, and adapting: a commentary on what does it mean to “employ” the RE-AIM model. Eval Health Prof. (2013) 36:67–72. 10.1177/0163278712460546
    1. Estabrooks PA, Glasgow RE. Translating effective clinic-based physical activity interventions into practice. Am J Prev Med. (2006) 31 (4 Suppl.):S45–56. 10.1016/j.amepre.2006.06.019
    1. Tabak RG, Khoong EC, Chambers DA, Brownson RC. Bridging research and practice: models for dissemination and implementation research. Am J Prev Med. (2012) 43:337–50. 10.1016/j.amepre.2012.05.024
    1. Vinson C, Stamatakis K, Kerner J. Dissemination and implementation research in community and public health settings. In: Brownson RC, Colditz GA, Proctor EK. editors. Dissemination and Implementation Research in Health; Translating Research to Practice. Oxford, NY: Oxford University Press; (2018). p. 355–70.
    1. Shoup JA, Gaglio B, Varda D, Glasgow RE. Network analysis of RE-AIM framework: chronology of the field and the connectivity of its contributors. Transl Behav Med. (2015) 5:216–32. 10.1007/s13142-014-0300-1
    1. Kessler RS, Purcell EP, Glasgow RE, Klesges LM, Benkeser RM, Peek CJ. What does it mean to “employ” the RE-AIM model? Eval Heal Prof. (2013) 36:44–66. 10.1177/0163278712446066
    1. Galaviz KI, Harden SM, Smith E, Blackman KC, Berrey LM, Mama SK, et al. . Physical activity promotion in Latin American populations: a systematic review on issues of internal and external validity. Int J Behav Nutr Phys Act. (2014) 11:77. 10.1186/1479-5868-11-77
    1. Kim AE, Towers A, Renaud J, Zhu J, Shea JA, Galvin R, et al. . Application of the RE-AIM framework to evaluate the impact of a worksite-based financial incentive intervention for smoking cessation. J Occup Environ Med. (2012) 54:610–4. 10.1097/JOM.0b013e31824b2171
    1. Chen I, Money D, Yong P, Williams C, Allaire C. An evaluation model for a multidisciplinary chronic pelvic pain clinic: application of the RE-AIM framework. J Obstet Gynaecol Can. (2015) 37:804–9. 10.1016/S1701-2163(15)30151-1
    1. Ory MG, Altpeter M, Belza B, Helduser J, Zhang C, Smith ML. Perceived utility of the RE-AIM framework for health promotion/disease prevention initiatives for older adults: a case study from the U.S. evidence-based disease prevention initiative. Front Public Health. (2015) 2:143. 10.3389/fpubh.2014.00143
    1. Ory MG, Altpeter M, Belza B, Helduser J, Zhang C, Smith ML. Perceptions about community applications of RE-AIM in the promotion of evidence-based programs for older adults. Eval Heal Prof. (2015) 38:15–20. 10.1177/0163278714542335
    1. Aerts I, Cumps E, Verhagen E, Mathieu N, Van Schuerbeeck S, Meeusen R. A 3-month jump-landing training program: a feasibility study using the RE-AIM framework. J Athl Train. (2013) 48:296–305. 10.4085/1062-6050-48.3.18
    1. Altpeter M, Gwyther LP, Kennedy SR, Patterson TR, Derence K. From evidence to practice: using the RE-AIM framework to adapt the REACHII caregiver intervention to the community. Dementia. (2015) 14:104–13. 10.1177/1471301213494499
    1. Schwingel A, Gálvez P, Linares D, Sebastião E. Using a mixed-methods RE-AIM framework to evaluate community health programs for older latinas. J Aging Health. (2017) 29:551–93. 10.1177/0898264316641075
    1. Adams EJ, Chalkley AE, Esliger DW, Sherar LB. Evaluation of the implementation of a whole-workplace walking programme using the RE-AIM framework. BMC Public Health. (2017) 17:466. 10.1186/s12889-017-4376-7
    1. Brace AM, Padilla HM, DeJoy DM, Wilson MG, Vandenberg RJ, Davis M. Applying RE-AIM to the evaluation of FUEL your life : a worksite translation of DPP. Health Promot Pract. (2015) 16:28–35. 10.1177/1524839914539329
    1. Ward S, Chow AF, Humbert ML, Bélanger M, Muhajarine N, Vatanparast H, et al. . Promoting physical activity, healthy eating and gross motor skills development among preschoolers attending childcare centers: process evaluation of the Healthy Start-Départ Santé intervention using the RE-AIM framework. Eval Program Plann. (2018) 68:90–8. 10.1016/j.evalprogplan.2018.02.005
    1. Glasgow RE, Gaglio B, Estabrooks PA, Marcus AC, Ritzwoller DP, Smith TL, et al. . Long-term results of a smoking reduction program. Med Care. (2009) 47:115–20. 10.1097/MLR.0b013e31817e18d1
    1. Dzewaltowski DA, Estabrooks PA, Glasgow RE. The future of physical activity behavior change research: what is needed to improve translation of research into health promotion practice? Exerc Sport Sci Rev. (2004) 32:57–63. 10.1097/00003677-200404000-00004
    1. Estabrooks P, Dzewaltowski DA, Glasgow RE, Klesges LM. Reporting of validity from school health promotion studies published in 12 leading journals, 1996-2000. J Sch Health. (2003) 73:21–8. 10.1111/j.1746-1561.2003.tb06554.x
    1. Lee RE, Galavíz KI, Soltero EG, Rosales Chavez J, Jauregui E, Lévesque L, et al. Applying the RE-AIM conceptual framework for the promotion of physical activity in low-and middle-income countries. Rev Lat Am Enfermagem. (2017) 25:e2923 10.1590/1518-8345.1894.2923
    1. Caperchione CM, Duncan M, Kolt GS, Vandelanotte C, Rosenkranz RR, Maeder A, et al. . Examining an Australian physical activity and nutrition intervention using RE-AIM. Health Promot Int. (2015) 31:450–8. 10.1093/heapro/dav005
    1. O'Brien J, Finch CF. The implementation of musculoskeletal injury-prevention exercise programmes in team ball sports: a systematic review employing the RE-AIM framework. Sports Med. (2014) 44:1305–18. 10.1007/s40279-014-0208-4
    1. Koorts H, Gillison F. Mixed method evaluation of a community-based physical activity program using the RE-AIM framework: practical application in a real-world setting. BMC Public Health. (2015) 15:1102. 10.1186/s12889-015-2466-y
    1. Jenkinson KA, Naughton G, Benson AC. The GLAMA (Girls! Lead! Achieve! Mentor! Activate!) physical activity and peer leadership intervention pilot project: a process evaluation using the RE-AIM framework. BMC Public Health. (2012) 12:55. 10.1186/1471-2458-12-55
    1. Oldroyd JC, White S, Stephens M, Neil AA, Nanayakkara V. Program evaluation of the inner south community health oral health program for priority populations. J Health Care Poor Underserv. (2017) 28:1222–39. 10.1353/hpu.2017.0107
    1. Leenaars KE, Smit E, Wagemakers A, Molleman GR, Koelen MA. Exploring the impact of the care sport connector in the Netherlands. BMC Public Health. (2017) 17:813. 10.1186/s12889-017-4830-6
    1. Schaap FD, Dijkstra GJ, Finnema EJ, Reijneveld SA. The first use of dementia care mapping in the care for older people with intellectual disability: a process analysis according to the RE-AIM framework. Aging Ment Health. (2018) 22:912–9. 10.1080/13607863.2017.1401582
    1. Spook JE, Paulussen T, Paulissen R, Visschedijk G, Kok G, van Empelen P. Design rationale behind the serious self-regulation game intervention “balance it”: overweight prevention among secondary vocational education students in the Netherlands. Games Health J. (2015) 4:387–400. 10.1089/g4h.2014.0142
    1. Boersma P, Van Weert JC, van Meijel B, van de Ven PM, Dröes RM. Study protocol Implementation of the Veder contact method (VCM) in daily nursing home care for people with dementia: an evaluation based on the RE-AIM framework. Aging Ment Health. (2017) 21:730–41. 10.1080/13607863.2016.1154015
    1. Baba CT, Oliveira IM, Silva AE, Vieira LM, Cerri NC, Florindo AA, et al. Evaluating the impact of a walking program in a disadvantaged area: using the RE-AIM framework by mixed methods. BMC Public Health. (2017) 17:709 10.1186/s12889-017-4698-5
    1. Almeida FA, Almeida Brito F. Planning and evaluating health programs: contributions of the RE-AIM framework to Nursing. Rev Lat Am Enfermagem. (2014) 22:527–8. 10.1590/0104-1169.0000.2447
    1. Quinn AK, Bruce N, Puzzolo E, Dickinson K, Sturke R, Jack DW, et al. . An analysis of efforts to scale up clean household energy for cooking around the world. Energy Sustain Dev. (2018) 46:1–0. 10.1016/j.esd.2018.06.011
    1. Estabrooks PA, Gyurcsik NC. Evaluating the impact of behavioral interventions that target physical activity: issues of generalizability and public health. Psychol Sport Exerc. (2003) 4:41–55. 10.1016/S1469-0292(02)00016-X
    1. Harden SM, Fanning J, Motl RW, McAuley E, Estabrooks PA. Determining the reach of a home-based physical activity program for older adults within the context of a randomized controlled trial. Health Educ Res. (2014) 29:861–9. 10.1093/her/cyu049
    1. You W, Almeida FA, Zoellner JM, Hill JL, Pinard CA, Allen KC, et al. . Who participates in internet-based worksite weight loss programs? BMC Public Health. (2011) 11:709. 10.1186/1471-2458-11-709
    1. Lee RE, Reese-Smith JY, Mama SK, Medina AV, Wolfe KL, Estabrooks PA. Reach and representativeness of ethnic minority women in the health is power study: a longitudinal analysis. Transl Behav Med. (2017) 7:106–14. 10.1007/s13142-016-0385-9
    1. Harden SM, Gaglio B, Shoup JA, Kinney KA, Johnson SB, Brito F, et al. . Fidelity to and comparative results across behavioral interventions evaluated through the RE-AIM framework: a systematic review. Syst Rev. (2015) 4:155. 10.1186/s13643-015-0141-0
    1. Perry RA, Golley RK, Hartley J, Magarey AM. The adaptation and translation of the PEACHTM RCT intervention: the process and outcomes of the PEACHTM in the community trial. Public Health. (2017) 153:154–62. 10.1016/j.puhe.2017.08.009
    1. Jaipakdee J, Jiamjarasrangsi W, Lohsoonthorn V, Lertmaharit S. Effectiveness of a self-management support program for Thais with type 2 diabetes: evaluation according to the RE-AIM framework. Nurs Health Sci. (2015) 17:362–9. 10.1111/nhs.12198
    1. Smith SN, Almirall D, Prenovost K, Goodrich DE, Abraham KM, Liebrecht C, et al. . Organizational culture and climate as moderators of enhanced outreach for persons with serious mental illness: results from a cluster-randomized trial of adaptive implementation strategies. Implement Sci. (2018) 13:93. 10.1186/s13012-018-0787-9
    1. Dunton GF, Liao Y, Grana R, Lagloire R, Riggs N, Chou CP, et al. . State-wide dissemination of a school-based nutrition education programme: a RE-AIM (Reach, Efficacy, Adoption, Implementation, Maintenance) analysis. Public Health Nutr. (2014) 17:422–30. 10.1017/S1368980012005186
    1. Maxwell AE, Jo AM, Chin SY, Lee KS, Bastani R. Impact of a print intervention to increase annual mammography screening among Korean American women enrolled in the National Breast and Cervical Cancer Early Detection Program. Cancer Detect Prev. (2008) 32:229–35. 10.1016/j.cdp.2008.04.003
    1. Breitenstein SM, Fogg L, Garvey C, Hill C, Resnick B, Gross D. Measuring implementation fidelity in a community-based parenting intervention. Nurs Res. (2010) 59:158–65. 10.1097/NNR.0b013e3181dbb2e2
    1. Duffy SA, Ronis DL, Ewing LA, Waltje AH, Hall S V, Thomas PL, et al. . Implementation of the Tobacco Tactics intervention versus usual care in Trinity Health community hospitals. Implement Sci. (2016) 11:147. 10.1186/s13012-016-0511-6
    1. Santos SLZ, Tagai EK, Scheirer MA, Bowie J, Haider M, Slade J, et al. . Adoption, reach, and implementation of a cancer education intervention in African American churches. Implement Sci. (2017) 12:36. 10.1186/s13012-017-0566-z
    1. Estabrooks PA, Wilson KE, McGuire TJ, Harden SM, Ramalingam N, Schoepke L, et al. . A quasi-experiment to assess the impact of a scalable, community-based weight loss program: combining reach, effectiveness, and cost. J Gen Intern Med. (2017) 32:24–31. 10.1007/s11606-016-3971-0
    1. Gaglio B, Shoup JA, Glasgow RE. The RE-AIM framework: a systematic review of use over time. Am J Public Health. (2013) 103:e38–46. 10.2105/AJPH.2013.301299
    1. Glasgow RE, Vinson C, Chambers D, Khoury MJ, Kaplan RM, Hunter C. National Institutes of health approaches to dissemination and implementation science: current and future directions. Am J Public Health. (2012) 102:1274–81. 10.2105/AJPH.2012.300755
    1. Sweet SN, Ginis KAM, Estabrooks PA, Latimer-Cheung AE. Operationalizing the RE-AIM framework to evaluate the impact of multi-sector partnerships. Implement Sci. (2014) 9:74. 10.1186/1748-5908-9-74
    1. Harden SM, Smith ML, Ory MG, Smith-Ray RL, Estabrooks P, Glasgow R. RE-AIM in clinical, community, and corporate settings: perspectives, strategies, and recommendations to enhance public health impact. Front Public Health. (2018) 6:71. 10.3389/fpubh.2018.00071
    1. Fortney J, Enderle M, McDougall S, Clothier J, Otero J, Altman L, et al. . Implementation outcomes of evidence-based quality improvement for depression in VA community based outpatient clinics. Implement Sci. (2012) 7:30. 10.1186/1748-5908-7-30
    1. Carlfjord S, Lindberg M, Andersson A. Sustained use of a tool for lifestyle intervention implemented in primary health care: a 2-year follow-up. J Eval Clin Pract. (2013) 19:327–34. 10.1111/j.1365-2753.2012.01827.x
    1. Rabin BA, McCreight M, Battaglia C, Ayele R, Burke RE, Hess PL, et al. . Systematic, multimethod assessment of adaptations across four diverse health systems interventions. Front Public Health. (2018) 6:102. 10.3389/fpubh.2018.00102
    1. Jones Rhodes WC, Ritzwoller DP, Glasgow RE. Stakeholder perspectives on costs and resource expenditures: tools for addressing economic issues most relevant to patients, providers, and clinics. Transl Behav Med. (2018) 8:675–82. 10.1093/tbm/ibx003
    1. Akers JD, Estabrooks PA, Davy BM. Translational research: bridging the gap between long-term weight loss maintenance research and practice. J Am Diet Assoc. (2010) 110:1511–22, 1522-3. 10.1016/j.jada.2010.07.005
    1. Allen K, Zoellner J, Motley M, Estabrooks PA. Understanding the internal and external validity of health literacy interventions: a systematic literature review using the RE-AIM framework. J Health Commun. (2011) 16 (Suppl. 3):55–72. 10.1080/10810730.2011.604381
    1. Bull SS, Gillette C, Glasgow RE, Estabrooks P. Work site health promotion research: to what extent can we generalize the results and what is needed to translate research to practice? Heal Educ Behav. (2003) 30:537–49. 10.1177/1090198103254340
    1. Klesges LM, Estabrooks PA, Dzewaltowski DA, Bull SS, Glasgow RE. Beginning with the application in mind: designing and planning health behavior change interventions to enhance dissemination. Ann Behav Med. (2005) 29:66–75. 10.1207/s15324796abm2902s_10
    1. Dzewaltowski DA, Glasgow RE, Klesges LM, Estabrooks PA, Brock E. RE-AIM: evidence-based standards and a Web resource to improve translation of research into practice. Ann Behav Med. (2004) 28:75–80. 10.1207/s15324796abm2802_1
    1. Ory MG, Jordan PJ, Bazzarre T. The behavior change consortium: setting the stage for a new century of health behavior-change research. Health Educ Res. (2002) 17:500–11. 10.1093/her/17.5.500
    1. Glasgow RE, Bull SS, Gillette C, Klesges LM, Dzewaltowski DA. Behavior change intervention research in healthcare settings: a review of recent reports with emphasis on external validity. Am J Prev Med. (2002) 23:62–9. 10.1016/S0749-3797(02)00437-3
    1. Glasgow RE, Estabrooks PE. Pragmatic applications of RE-AIM for health care initiatives in community and clinical settings. Prev Chronic Dis. (2018) 15:170271. 10.5888/pcd15.170271
    1. Chambers DA, Norton WE. The adaptome: advancing the science of intervention adaptation. Am J Prev Med. (2016) 51:S124–31. 10.1016/j.amepre.2016.05.011
    1. Mittman B. Evaluating Complex Interventions: Confronting and Guiding (Versus Ignoring and Suppressing) Heterogeneity and Adaptation. PSMG Northwestern Grand Rounds webinar series. (2018) Available online at: (Accessed December 12, 2018).
    1. Patient-Centered Outcomes Research Institute (PCORI) . Methodology Standards. (2019) Available online at:
    1. Stirman SW, Miller CJ, Toder K, Calloway A. Development of a framework and coding system for modifications and adaptations of evidence-based interventions. Implement Sci. (2013) 8:65. 10.1186/1748-5908-8-65
    1. Holtrop JS, Rabin BA, Glasgow RE. Qualitative approaches to use of the RE-AIM framework: rationale and methods. BMC Health Serv Res. (2018) 18:177. 10.1186/s12913-018-2938-8
    1. Stoutenberg M, Galaviz KI, Lobelo F, Joy E, Heath GW, Hutber A, et al. . A pragmatic application of the RE-AIM framework for evaluating the implementation of physical activity as a standard of care in health systems. Prev Chronic Dis. (2018) 15:E54. 10.5888/pcd15.170344
    1. Balis LE, Strayer III TE, Ramalingam N, Harden SM. Beginning with the end in mind: contextual considerations for scaling-out a community-based intervention. Front Public Health. (2018) 6:357. 10.3389/fpubh.2018.00357
    1. Harden SM, Ramalingam NS, Breig SA, Estabrooks PA. Walk this way: our perspective on challenges and opportunities for extension statewide walking promotion programs. J Nutr Educ Behav. (2019). 10.1016/j.jneb.2018.12.010 [Epub ahead of print].
    1. Ritzwoller DP, Sukhanova A, Gaglio B, Glasgow RE. Costing behavioral interventions: a practical guide to enhance translation. Ann Behav Med. (2009) 37:218–27. 10.1007/s12160-009-9088-5
    1. Glasgow RE, Huebschmann AG, Brownson RC. Expanding the CONSORT figure: increasing transparency in reporting on external validity. Am J Prev Med. (2018) 55:422–30. 10.1016/j.amepre.2018.04.044
    1. Moher D. CONSORT: an evolving tool to help improve the quality of reports of randomized controlled trials. JAMA. (1998) 279:1489–91. 10.1001/jama.279.18.1489
    1. Glasgow R, Huebschmann A, Brownson R. Expanded CONSORT Figure for Planning and Reporting D & I Research. (2018) Available online at: (Accessed December 13, 2018).
    1. Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, et al. . Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health. (2011) 38:65–76. 10.1007/s10488-010-0319-7
    1. Proctor E, Powell B, McMillen J. Implementation strategies: recommendations for specifying and reporting. Implement Sci. (2013) 8:139. 10.1186/1748-5908-8-139
    1. Lewis CC, Klasnja P, Powell BJ, Lyon AR, Tuzzio L, Jones S, et al. . From classification to causality: advancing understanding of mechanisms of change in implementation science. Front Public Health. (2018) 6:136. 10.3389/fpubh.2018.00136
    1. Feldstein A, Glasgow R. A practical, robust implementation and sustainability model (PRISM) for integrating research findings into practice. Jt Comm J Qual Patient Saf. (2008) 34:228–43. 10.1016/S1553-7250(08)34030-6
    1. Liles EG, Schneider JL, Feldstein AC, Mosen DM, Perrin N, Rosales AG, et al. . Implementation challenges and successes of a population-based colorectal cancer screening program: a qualitative study of stakeholder perspectives. Implement Sci. (2015) 10:41. 10.1186/s13012-015-0227-z
    1. Rogers EM. Diffusion of Innovations. New York, NY: Free Press of Glencoe; (1962)
    1. Beck A, Bergman DA, Rahm AK, Dearing JW, Glasgow RE. Using implementation and dissemination concepts to spread 21st-century well-child care at a health maintenance organization. Perm J. (2009) 13:10. 10.7812/TPP/08-088
    1. Thorpe KE, Zwarenstein M, Oxman AD, Treweek S, Furberg CD, Altman DG, et al. . A pragmatic-explanatory continuum indicator summary (PRECIS): a tool to help trial designers. J Clin Epidemiol. (2009) 62:464–75. 10.1016/j.jclinepi.2008.12.011
    1. Loudon K, Treweek S, Sullivan F, Donnan P, Thorpe KE, Zwarenstein M. The PRECIS-2 tool: designing trials that are fit for purpose. BMJ. (2015) 350:h2147. 10.1136/bmj.h2147
    1. Luoma KA, Leavitt IM, Marrs JC, Nederveld AL, Regensteiner JG, Dunn AL, et al. . How can clinical practices pragmatically increase physical activity for patients with type 2 diabetes? A systematic review. Transl Behav Med. (2017) 7:751–72. 10.1007/s13142-017-0502-4
    1. Pinnock H, Barwick M, Carpenter CR, Eldridge S, Grandes G, Griffiths CJ, et al. . Standards for Reporting Implementation Studies (StaRI) Statement. BMJ. 356:i6795. 10.1136/bmj.i679

Source: PubMed

Sponsor e collaboratori

Condizioni mediche

Interventi farmacologici

3
Sottoscrivi