Measuring implementation fidelity in a cluster-randomized pragmatic trial: development and use of a quantitative multi-component approach

Miranda B Olson, Ellen M McCreedy, Rosa R Baier, Renée R Shield, Esme E Zediker, Rebecca Uth, Kali S Thomas, Vincent Mor, Roee Gutman, James L Rudolph, Miranda B Olson, Ellen M McCreedy, Rosa R Baier, Renée R Shield, Esme E Zediker, Rebecca Uth, Kali S Thomas, Vincent Mor, Roee Gutman, James L Rudolph

Abstract

Background: In pragmatic trials, on-site partners, rather than researchers, lead intervention delivery, which may result in implementation variation. There is a need to quantitatively measure this variation. Applying the Framework for Implementation Fidelity (FIF), we develop an approach for measuring variability in site-level implementation fidelity. This approach is then applied to measure site-level fidelity in a cluster-randomized pragmatic trial of Music & MemorySM (M&M), a personalized music intervention targeting agitated behaviors in residents living with dementia, in US nursing homes (NHs).

Methods: Intervention NHs (N = 27) implemented M&M using a standardized manual, utilizing provided staff trainings and iPods for participating residents. Quantitative implementation data, including iPod metadata (i.e., song title, duration, number of plays), were collected during baseline, 4-month, and 8-month site visits. Three researchers developed four FIF adherence dimension scores. For Details of Content, we independently reviewed the implementation manual and reached consensus on six core M&M components. Coverage was the total number of residents exposed to the music at each NH. Frequency was the percent of participating residents in each NH exposed to M&M at least weekly. Duration was the median minutes of music received per resident day exposed. Data elements were scaled and summed to generate dimension-level NH scores, which were then summed to create a Composite adherence score. NHs were grouped by tercile (low-, medium-, high-fidelity).

Results: The 27 NHs differed in size, resident composition, and publicly reported quality rating. The Composite score demonstrated significant variation across NHs, ranging from 4.0 to 12.0 [8.0, standard deviation (SD) 2.1]. Scaled dimension scores were significantly correlated with the Composite score. However, dimension scores were not highly correlated with each other; for example, the correlation of the Details of Content score with Coverage was τb = 0.11 (p = 0.59) and with Duration was τb = - 0.05 (p = 0.78). The Composite score correlated with CMS quality star rating and presence of an Alzheimer's unit, suggesting face validity.

Conclusions: Guided by the FIF, we developed and used an approach to quantitatively measure overall site-level fidelity in a multi-site pragmatic trial. Future pragmatic trials, particularly in the long-term care environment, may benefit from this approach.

Trial registration: Clinicaltrials.gov NCT03821844. Registered on 30 January 2019, https://ichgcp.net/clinical-trials-registry/NCT03821844 .

Keywords: Adherence; Dementia; Fidelity; Implementation; Nursing home; Pragmatic trial.

Conflict of interest statement

The authors declare that they have no competing interests.

© 2022. The Author(s).

Figures

Fig. 1
Fig. 1
An overview of the conceptual Framework for Implementation Fidelity as applied to the METRIcAL study. For each adherence dimension presented in the original model, the study-specific definition and identified data elements are summarized

References

    1. Weinfurt KP, Hernandez AF, Coronado GD, DeBar LL, Dember LM, Green BB, et al. Pragmatic clinical trials embedded in healthcare systems: generalizable lessons from the NIH Collaboratory. BMC Med Res Methodol. 2017;17(1):144. doi: 10.1186/s12874-017-0420-7.
    1. Gitlin LN, Marx K, Stanley IH. Hodgson N. Translating evidence-based dementia caregiving interventions into practice: state-of-the-science and next steps. Gerontologist. 2015;55(2):210–226. doi: 10.1093/geront/gnu123.
    1. Chalkidou K, Tunis S, Whicher D, Fowler R, Zwarenstein M. The role for pragmatic randomized controlled trials (pRCTs) in comparative effectiveness research. Clin Trials J Soc Clin Trials. 2012;2, 9(4):–446. 10.1177/1740774512450097.
    1. Curran GM, Bauer M, Mittman B, Pyne JM, Stetler C. Effectiveness-implementation hybrid designs: combining elements of clinical effectiveness and implementation research to enhance public health impact. Med Care. 2012;50(3):217–226. doi: 10.1097/MLR.0b013e3182408812.
    1. Glasgow RE, Magid DJ, Beck A, Ritzwoller D, Estabrooks PA. Practical clinical trials for translating research to practice: design and measurement recommendations. Med Care, 7. 2005;43(6):551 Available from: .
    1. Dusenbury L, Brannigan R, Falco M, Hansen WB. A review of research on fidelity of implementation: implications for drug abuse prevention in school settings. Health Educ Res. 2003;18(2):–256. 10.1093/her/18.2.237.
    1. Hill HC, Erickson A. Using implementation fidelity to aid in interpreting program impacts: a brief review. Educ Res. 2019;48(9):590–598. doi: 10.3102/0013189X19891436.
    1. Elliott DS, Mihalic S. Issues in disseminating and replicating effective prevention programs. Prev Sci. 2004;5(1):47–53. doi: 10.1023/B:PREV.0000013981.28071.52.
    1. Inouye SK, Bogardus ST, Jr, Williams CS, Leo-Summers L, Agostini JV. The role of adherence on the effectiveness of nonpharmacologic interventions: evidence from the delirium prevention trial. Arch Intern Med. 2003;163(8):958–964. doi: 10.1001/archinte.163.8.958.
    1. Gesell SB, Bushnell CD, Jones SB, Coleman SW, Levy SM, Xenakis JG, et al. Implementation of a billable transitional care model for stroke patients: the COMPASS study. BMC Health Serv Res. 2019;19(1):978. doi: 10.1186/s12913-019-4771-0.
    1. Mitchell SL, Volandes AE, Gutman R, Gozalo PL, Ogarek JA, Loomer L, et al. Advance care planning video intervention among long-stay nursing home residents: a pragmatic cluster randomized clinical trial. JAMA Intern Med. 2020;180(8):1070–1078. doi: 10.1001/jamainternmed.2020.2366.
    1. Lin S-Y, Schneider CE, Bristol AA, Clancy M, Sprague SA, Aldridge M, et al. Findings of sequential pilot trials of Aliviado Dementia Care to inform an embedded pragmatic clinical trial. Gerontologist. 2020:gnaa220 Available from. 10.1093/geront/gnaa220.
    1. Axford N, Bjornstad G, Clarkson S, Ukoumunne OC, Wrigley Z, Matthews J, et al. The effectiveness of the KiVa bullying prevention program in Wales, UK: results from a pragmatic cluster randomized controlled trial. Prev Sci. 2020;21(5):615–626. doi: 10.1007/s11121-020-01103-9.
    1. Toomey E, Matthews J, Guerin S, Hurley DA. Development of a feasible implementation fidelity protocol within a complex physical therapy–led self-management intervention. Phys Ther. 2016;96(8):1287–1298. doi: 10.2522/ptj.20150446.
    1. Palmer JA, Parker VA, Barre LR, Mor V, Volandes AE, Belanger E, et al. Understanding implementation fidelity in a pragmatic randomized clinical trial in the nursing home setting:a mixed-methods examination. Trials. 2019;20(1):656. doi: 10.1186/s13063-019-3725-5.
    1. Harris-Kojetin L, Sengupta M, Lendon J, Rome V, Valverde R, Caffrey C. Long-term care providers and services users in the United States, 2015-2016. Vital Heal Stat. 2019;3(43):23.
    1. Matthews KA, Xu W, Gaglioti AH, Holt JB, Croft JB, Mack D, et al. Racial and ethnic estimates of Alzheimer’s disease and related dementias in the United States (2015-2060) in adults aged ≥65 years. Alzheimers Dement. 2019;15(1):17–24. doi: 10.1016/j.jalz.2018.06.3063.
    1. Zuidema S, Koopmans R, Verhey F. Prevalence and predictors of neuropsychiatric symptoms in cognitively impaired nursing home patients. J Geriatr Psychiatry Neurol. 2007;29(1):–49. 10.1177/0891988706292762.
    1. Chiu Y, Bero L, Hessol NA, Lexchin J, Harrington C, et al. Health Policy (New York) 2015;119(6):802–813. doi: 10.1016/j.healthpol.2015.02.014.
    1. Why Get Certified: Music and Memory. . Accessed 1 January 2021.
    1. Jacobsen J-H, Stelzer J, Fritz TH, Chételat G, La Joie R, Turner R. Why musical memory can be preserved in advanced Alzheimer’s disease. Brain. 2015;138(8).
    1. Kwak J, Anderson K, O’Connell VK. Findings from a prospective randomized controlled trial of an individualized music listening program for persons with dementia. J Appl Gerontol. 2020;39(6):–575. 10.1177/0733464818778991.
    1. Thomas KS, Baier R, Kosar C, Ogarek J, Trepman A, Mor V, et al. Am J Geriatr Psychiatry. 2017;25(9):931–938. doi: 10.1016/j.jagp.2017.04.008.
    1. Bakerjian D, Bettega K, Cachu AM, Azzis L, Taylor S, et al. J Am Med Dir Assoc. 2020;21(8):1045–1050.e2. doi: 10.1016/j.jamda.2020.01.103.
    1. McCreedy EM, Yang X, Baier RR, Rudolph JL, Thomas KS, Mor V. Measuring effects of nondrug interventions on behaviors: Music & Memory pilot study. J Am Geriatr Soc. 2019;67(10):–2138. 10.1111/jgs.16069.
    1. McCreedy EM, Gutman R, Baier R, Rudolph JL, Thomas KS, Dvorchak F, et al. Measuring the effects of a personalized music intervention on agitated behaviors among nursing home residents with dementia: design features for cluster-randomized adaptive trial. Trials. 2021;22(1):681. doi: 10.1186/s13063-021-05620-y.
    1. Landes SJ, McBain SA, Curran GM. An introduction to effectiveness-implementation hybrid designs. Psychiatry Res. 2019;280:112513. doi: 10.1016/j.psychres.2019.112513.
    1. Carroll C, Patterson M, Wood S, Booth A, Rick J, Balain S. A conceptual framework for implementation fidelity. Implement Sci. 2007;2(1):40. doi: 10.1186/1748-5908-2-40.
    1. MUSIC & MEMORY® Certification Training. . Accessed 1 March 2021.
    1. Long-term Care: Facts on Care in the US. Brown University Center for Gerontology and Healthcare Research. 2019. . Accessed 1 January 2020.
    1. System F-SQR. Centers for Medicare and Medicaid Services. 2019.
    1. Morris JN, Fries BE, Morris SA. Scaling ADLs Within the MDS. J Gerontol Ser A. 1999;54(11):M546–M553. Available from: 10.1093/gerona/54.11.M546
    1. Medicare and Medicaid Basics - MLN Booklet. Centers for Medicare and Medicaid Services. 2018. . Accessed 11 December 2021.
    1. Hasson H. Systematic evaluation of implementation fidelity of complex interventions in health and social care. Implement Sci. 2010;5(1):67. doi: 10.1186/1748-5908-5-67.
    1. Muntinga ME, Van Leeuwen KM, Schellevis FG, Nijpels G, Jansen APD. From concept to content: assessing the implementation fidelity of a chronic care model for frail, older people who live at home. BMC Health Serv Res. 2015;15(1):18. doi: 10.1186/s12913-014-0662-6.
    1. Willeboordse F, Schellevis FG, Meulendijk MC, Hugtenburg JG, PJM E. Implementation fidelity of a clinical medication review intervention: process evaluation. Int J Clin Pharm. 2018;40(3):550–565. doi: 10.1007/s11096-018-0615-y.
    1. Berry CA, Nguyen AM, Cuthel AM, Cleland CM, Siman N, Pham-Singer H, et al. Measuring implementation strategy fidelity in HealthyHearts NYC: a complex intervention using practice facilitation in primary care. Am J Med Qual. 2020;36(4):270–276. doi: 10.1177/1062860620959450.
    1. Smith J, Green J, Siddiqi N, Inouye SK, Collinson M, Farrin A, et al. Investigation of ward fidelity to a multicomponent delirium prevention intervention during a multicentre, pragmatic, cluster randomised, controlled feasibility trial. Age Ageing. 2020;49(4):648–655. doi: 10.1093/ageing/afaa042.
    1. Hasson H, Blomberg S, Dunér A. Fidelity and moderating factors in complex interventions: a case study of a continuum of care program for frail elderly people in health and social care. Implement Sci. 2012;7:23. doi: 10.1186/1748-5908-7-23.
    1. Bragstad LK, Bronken BA, Sveen U, Hjelle EG, Kitzmüller G, Martinsen R, et al. Implementation fidelity in a complex intervention promoting psychosocial well-being following stroke: an explanatory sequential mixed methods study. BMC Med Res Methodol. 2019;19(1):59. doi: 10.1186/s12874-019-0694-z.

Source: PubMed

Patrocinadores y Colaboradores

Condiciones médicas

Intervenciones de drogas

3
Suscribir