Assessing long-term sustainment of clinic participation in NIATx200: Results and a new methodological approach

Abstract

Background: Sustainability frameworks differentiate between sustainability capacity and sustainment of organizational change. Multiple studies have examined sustainability capacity. Methodological approaches to assess long-term sustainment have not been explored. This study addresses this gap by describing the development of a long-term sustainment methodology and evaluating its application utilizing data from substance abuse clinics participating in a quality improvement collaborative.

Methods: The study involved clinics (n = 121) in three states (MI, NY and WA) participating in the 2007-2009 NIATx200 quality improvement (QI) intervention. It extended the primary analysis to focus on clinics' long-term sustainment of wait time, retention and admission improvements. Long-term sustainment was defined as two years post end of the active implementation period (Calendar Years 2010 and 2011). The analysis defined case exclusion criteria and spline "knot" time intervals; allowed for Cp statistic use to address clinic data volatility; established the structure of sustainment plots and explored differences between NIATx implementation strategies.

Results: Example spline and sustain plots highlight the application of the long-term sustainment methodology for NIATx200 clinics. In clinics with available longitudinal outcome data, 40.8% (n = 31 of 76 clinics) sustained improvements in wait time, 26.7% (n = 20 of 75 clinics) in retention, and 28.1% (n = 32 of 114 clinics) for admissions. Clinic assignment to a NIATx200 implementation strategy did not significantly influence a clinics' long-term sustainment except for lower wait time changes in the interest circle interventions. Thirty clinics (24.8%) sustained improvements for two outcomes and six clinics (5.0%) did so for all three outcomes. The clinics that sustained multiple outcome improvements were assigned to the interest circle (n = 12), learning session (n = 10), combination (n = 8), and coaching (n = 6) implementation strategies. Guidance for applying the long-term sustainment methodology in other quality improvement settings is described.

Conclusions: Research about sustainability capacity and sustainment of change has become increasingly important in dissemination and implementation research. Assessment of long-term sustainment in a multi-organizational quality improvement collaborative (QIC) is needed to identify when program drift and intervention decay occurs. If "cut-points" indicate when effects diminish, specific sustainability modules could be developed and introduced within the structure of a QIC to improve organizational long-term sustainment. Coaches and change teams could be trained to focus on organizational change sustainment and strengthen the likelihood of institutionalization. ClinicalTrials.gov Identifier: NCT00934141 Registered July 6, 2009. Retrospectively registered.

Keywords: Admissions; Long-term sustainment; NIATx200; Quality improvement collaborative; Retention; Sustain plots; Wait time.

Copyright © 2018 Elsevier Inc. All rights reserved.

Figures

Fig. 1.

Linear spline time intervals for sustainment analysis. 1The pre-intervention baseline period is equivalent to the original NIATx200 baseline period for each outcome. The NIATx200 active implementation period starts one month after the end of the baseline period, continuing to March 2009 (end of active NIATx200 intervention period). 2The short-term sustainment period corresponds to the original study nine-month short-term sustainment period (April 2009 to December 2009). 3The partial fourth year sustainment period differs by outcome due to right censoring of wait time and retention but not admission.

Fig. 2.

MI-32 wait time spline knot chart with explanation of associated table.

Fig. 3.

Clinics sustaining two or more outcomes by implementation strategy. Notes: Clinics who did not initially improve or sustain a specific outcome (e.g., wait time, retention, or admissions) during the active NIATx intervention (Period AB) but showed sustained improvement during Period CD (Jan 2010 to Dec 2011) were classified as clinics who lagged behind in their overall efforts to improve and sustain a given outcome. These clinics are represented in the “Clinics with Lagged Improvement & Sustainment” group in the Figure. Clinics that improved during the active NIATx intervention and then sustained those improvements are represented in the “Clinics that Improved and Sustained” group in the Figure.