Development of a framework and coding system for modifications and adaptations of evidence-based interventions

Shannon Wiltsey Stirman, Christopher J Miller, Katherine Toder, Amber Calloway, Shannon Wiltsey Stirman, Christopher J Miller, Katherine Toder, Amber Calloway

Abstract

Background: Evidence-based interventions are frequently modified or adapted during the implementation process. Changes may be made to protocols to meet the needs of the target population or address differences between the context in which the intervention was originally designed and the one into which it is implemented [Addict Behav 2011, 36(6):630-635]. However, whether modification compromises or enhances the desired benefits of the intervention is not well understood. A challenge to understanding the impact of specific types of modifications is a lack of attention to characterizing the different types of changes that may occur. A system for classifying the types of modifications that are made when interventions and programs are implemented can facilitate efforts to understand the nature of modifications that are made in particular contexts as well as the impact of these modifications on outcomes of interest.

Methods: We developed a system for classifying modifications made to interventions and programs across a variety of fields and settings. We then coded 258 modifications identified in 32 published articles that described interventions implemented in routine care or community settings.

Results: We identified modifications made to the content of interventions, as well as to the context in which interventions are delivered. We identified 12 different types of content modifications, and our coding scheme also included ratings for the level at which these modifications were made (ranging from the individual patient level up to a hospital network or community). We identified five types of contextual modifications (changes to the format, setting, or patient population that do not in and of themselves alter the actual content of the intervention). We also developed codes to indicate who made the modifications and identified a smaller subset of modifications made to the ways that training or evaluations occur when evidence-based interventions are implemented. Rater agreement analyses indicated that the coding scheme can be used to reliably classify modifications described in research articles without overly burdensome training.

Conclusions: This coding system can complement research on fidelity and may advance research with the goal of understanding the impact of modifications made when evidence-based interventions are implemented. Such findings can further inform efforts to implement such interventions while preserving desired levels of program or intervention effectiveness.

Figures

Figure 1
Figure 1
Article selection procedure and results.
Figure 2
Figure 2
System of classifying modifications to evidence-based programs or interventions.

References

    1. Lundgren L. et al.Modifications of evidence-based practices in community-based addiction treatment organizations: a qualitative research study. Addict Behav. 2011;36(6):630–635. doi: 10.1016/j.addbeh.2011.01.003.
    1. Schoenwald SK. et al.Toward the effective and efficient measurement of implementation fidelity. Adm Policy Ment Health. 2011;38(1):32–43. doi: 10.1007/s10488-010-0321-0.
    1. Waltz J. et al.Testing the Integrity of a Psychotherapy Protocol: Assessment of Adherence and Competence. J Consult Clin Psychol. 1993;61(4):620–630.
    1. Stirman SW. et al.The sustainability of new programs and interventions: A review of the empirical literature and recommendations for future research. Implementation Sci. 2012;7:17. doi: 10.1186/1748-5908-7-17.
    1. Zvoch K. Treatment fidelity in multisite evaluation. Am J Evaluation. 2009;30(1):44–61. doi: 10.1177/1098214008329523.
    1. Dusenbury L. et al.A review of research on fidelity of implementation: implications for drug abuse prevention in school settings. Health Educ Res. 2003;18(2):237–256. doi: 10.1093/her/18.2.237.
    1. Schoenwald SK GA, Chapman JE, Frazier SL, Sheidow AJ, Southam-Gerow MA. Toward the effective and efficient measurement of implementation fidelity. Administration Policy Mental Health and Mental Health Serv Res. 2011;38(1):32–43. doi: 10.1007/s10488-010-0321-0.
    1. Bauman LJ, Stein REK, Ireys HT. Reinventing fidelity: the transfer of social technology among settings. Am J Community Psychol. 1991;19(4):619–639. doi: 10.1007/BF00937995.
    1. Kelly JA. et al.Transfer of research-based HIV prevention interventions to community service providers: fidelity and adaptation. AIDS Educ Prev. 2000;12:87–98.
    1. Elliot DSMS. Issues in disseminating and replicating effective prevention programs. Prev Sci. 2004;5(1):47–53.
    1. Levitt JT. et al.The flexible application of a manualized treatment for PTSD symptoms and functional impairment related to the 9/11 World Trade Center attack. Behav Res Ther. 2007;45(7):1419. doi: 10.1016/j.brat.2007.01.004.
    1. Galovski TE. et al.Manualized therapy for PTSD: Flexing the structure of cognitive processing therapy. J Consult Clin Psychol. 2012;80(6):968–981.
    1. Kalichman SC. et al.Culturally tailored HIV-AIDS risk-reduction messages targeted to African-American urban women:impact on risk sensitization and risk reduction. J Consult Clin Psychol. 1993;61(2):291–295.
    1. Kennedy MG. et al.The effect of tailoring a model HIV prevention program for local adolescent target audiences. AIDS Educ Prev. 2000;12(3):225–238.
    1. Stanton B. et al.The complex business of adapting effective interventions to new populations: an urban to rural transfer. J Adolesc Health. 2005;37(163):17–26.
    1. Kumpfer KLAR, Smith P, Bellamy N. Cultural sensitivity and adaptation in family-based prevention interventions. Prev Sci. 2002;3(3):241–246. doi: 10.1023/A:1019902902119.
    1. Kaysen D. et al.Adaptation of cognitive processing therapy for treatment of torture victims: Experience in Kurdistan, Iraq. Psychological Trauma. 2013;52(2):184–192.
    1. Salerno A. et al.Wellness self-management: an adaptation of the illness management and recovery program in New York State. Psychiatr Serv. 2011;62(5):456–458. doi: 10.1176/appi.ps.62.5.456.
    1. Leerlooijer JN. et al.The world starts with me: using intervention mapping for the systematic adaptation and transfer of school based sexuality education from Uganda to Indonesia. TBM. 2011;1(2):331–340.
    1. Hill LG, Maucione K, Hood BK. A focused approach to assessing program fidelity. Prev Sci. 2007;8:25–34. doi: 10.1007/s11121-006-0051-4.
    1. Nastasi BK. et al.The participatory intervention model: a framework for conceptualizing and promoting intervention acceptability. School Psychology Quarterly. 2000;15(2):207–232.
    1. Tortolero SR. et al.Using intervention mapping to adapt an effective HIV, sexually transmitted disease, and pregnancy prevention program for high-risk minority youth. Health Promot Pract. 2005;6(3):286–298. doi: 10.1177/1524839904266472.
    1. Devieux JG. et al.Cultural adaptation in translational research: field experiences. J Urban Health. 2005;82(2 Suppl 3):iii82–iii91.
    1. Goldstein NE. et al.Guidelines for adapting manualized interventions for new target populations: a step‒wise approach using anger management as a model. Clin Psychology: Science and Practice. 2012;19(4):385–401. doi: 10.1111/cpsp.12011.
    1. Lau AS. Making the case for selective and directed cultural adaptations of evidence-based treatments: examples from parent training. Clin Psychology: Science & Practice. 2006;13(4):295–310. doi: 10.1111/j.1468-2850.2006.00042.x.
    1. Aarons G. et al.Dynamic adaptation process to implement an evidence-based child maltreatment intervention. Implementation Science. 2012;7(32)
    1. Castro FG, Barrera M Jr, Martinez CR Jr. The cultural adaptation of prevention interventions: Resolving tensions between fidelity and fit. Prev Sci. 2004;5(1):41–45.
    1. Scheirer MA. Linking Sustainability Research to Intervention Types. Am J Public Health. 2013;103(4):e73–e80. doi: 10.2105/AJPH.2012.300976.
    1. Glaser BG, Strauss AL. The Discovery of Grounded Theory: Strategies for Qualitative Research. Chicago, IL: Aldine Publishing Company; 1967.
    1. Strauss A, Corbin J. Basics of qualitative research. Techniques and procedures for developing grounded theory. 1998.
    1. Cohen J. Weighted kappa: Nominal scale agreement provision for scaled disagreement or partial credit. Psychol Bull. 1968;70(4):213–220.
    1. Hinton DEPT, Tran M, Safren SA, Otto MW, Pollack MH. CBT for Vietnamese refugees with treatment-resistant PTSD and panic attacks: a pilot study. J Trauma Stress. 2004;17(5):429–433. doi: 10.1023/B:JOTS.0000048956.03529.fa.
    1. McCabe KMYM, Garland AF, Lau AS, Chavez G. The GANA program: a tailoring approach to adapting parent child interaction therapy for Mexican Americans. Education & Treatment of Children. 2005;28(2):111–129.
    1. Webster-Stratton C, Reid M. Adapting the Incredible Years Child Dinosaur Social, Emotionalm and problem Solving Intervention to Address Comorbid Diagnoses. J Child Serv. 2008;3(3):17–30.
    1. Aarons G, Adaptation happens: a qualitative case study of implementation of the incredible years evidence-based parent training program in a residential substance abuse treatment program. J Child Serv. in press.
    1. McIntyre LL. Adapting Webster‒Stratton's incredible years parent training for children with developmental delay: findings from a treatment group only study. J Intellect Disabil Res. 2008;52(12):1176–1192. doi: 10.1111/j.1365-2788.2008.01108.x.
    1. Owczarzak J, Dickson-Gomez J. Provider Perspectives on Evidence Based HIV Prevention Interventions: Barriers and Facilitators to Implementation. AIDS Patient Care STDS. 2011;25(3):171–179.
    1. Lyon AR. et al.Easier Said Than Done: Intervention Sustainability in an Urban After School Program. AdministrationPolicy Mental Health and Mental Health Serv Res. 2011;38(6):504–517. doi: 10.1007/s10488-011-0339-y.
    1. Malow RM. et al.Increasing access to preventative health care through cultural adaptation of effective HIV prevention interventions: a brief report from the HIV prevention in haitain youths study. ABNF J. 2004;132:127–132. November/December.
    1. Devieux JG, Malow, Robert M, Rosenberg, Rhonda, Dyer, Janyce G. Context and common ground: cultural adaptation of an intervention for minority HIV Infected individuals. J Cult Divers. 2004;11(2):49–59.
    1. Dushay RA. et al.Lowering HIV risk among ethnic minority drug users: comparing culturally targeted intervention to a standard intervention. Am J Drug Alcohol Abuse. 2001;27(3):501–524. doi: 10.1081/ADA-100104515.
    1. Miller RL. Adapting an evidence-based intervention: tales of the hustler project. AIDS Educ Prev. 2003;15(1):127–138.
    1. Nastasi BK. et al.Participatory model of mental health programming: Lessons learned from work in a developing country. School Psychology Review. 1998;27(2):260–276.
    1. Hasson H, Blomberg S, Duner A. Fidelity and moderating factors in complex interventions: a case study of a continuum of care program for frail elderly people in health and social care. Implementation Science. 2012;7(23)
    1. Williams RB, Williams VP. Adaptation and implementation of an evidence based behavioral medicine program in diverse global settings: the Williams life skills experience. TBM. 2011;1(2):303–312.
    1. Blasinsky M, Goldman HH, Unutzer J. Project IMPACT: A report on barriers and facilitators to sustainability. Administration Policy Mental Health and Mental Health Serv Res. 2006;33(6):718–729. doi: 10.1007/s10488-006-0086-7.
    1. Melde C, Esbensen F-A, Tusinski K. Addressing Program Fidelity Using Onsite Observations and Program Provider Descriptions of Program Delivery. Eval Rev. 2006;30(6):714–740. doi: 10.1177/0193841X06293412.
    1. Remien RH. et al.Moving From Theory to Research to Practice: Implementing an Effective Dyadic Intervention to Improve Antiretroviral Adherence for Clinic Patients. Acquiring Immune Definciency Syndrome. 2006;43(1):S69–S78. doi: 10.1097/01.qai.0000225014.61192.98.
    1. Noonan R. et al.Adoption, Adaptation, and Fidelity of Implementation of Sexual Violence Prevention Programs. Health Promot Pract. 2009;10(1):59S–70S.
    1. Webster-Stratton C, Reid MJ. Adapting The Incredible Years, an evidence-based parenting programme, for families involved in the child welfare system. J Child Serv. 2010;5(1):25–42. doi: 10.5042/jcs.2010.0115.
    1. Holliday J. et al.High Fidelity? How should we consider variations in the delivery of school based health promotion interventions? Health Educ J. 2009;68(44)
    1. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33(1):159–174. doi: 10.2307/2529310.
    1. Stirman SW, Modifications made to cognitive therapy by community mental health providers: Implications for effectiveness and sustainability. Psychiatr Serv. 2013. (in press)
    1. Shediac-Rizkallah MC, Bone LR. Planning for the sustainability of community-based health programs: conceptual frameworks and future directions for research, practice and policy. Health Educ Res. 1998;13(1):87–108. doi: 10.1093/her/13.1.87.
    1. Scheirer MA, Dearing JW. An agenda for research on the sustainability of public health programs. Am J Public Health. 2011;101(11):2059–2067. doi: 10.2105/AJPH.2011.300193.
    1. Bishop DC, Measuring Fidelity and Adaptation: Reliability of a Instrument for School-Based Prevention Programs. Evaluation & The Health Professions; 2013.

Source: PubMed

スポンサーと協力者

医学的状態

薬物療法

3
購読する