Robust analysis of stepped wedge trials using cluster-level summaries within periods

Abstract

In stepped-wedge trials (SWTs), the intervention is rolled out in a random order over more than 1 time-period. SWTs are often analysed using mixed-effects models that require strong assumptions and may be inappropriate when the number of clusters is small. We propose a non-parametric within-period method to analyse SWTs. This method estimates the intervention effect by comparing intervention and control conditions in a given period using cluster-level data corresponding to exposure. The within-period intervention effects are combined with an inverse-variance-weighted average, and permutation tests are used. We present an example and, using simulated data, compared the method to (1) a parametric cluster-level within-period method, (2) the most commonly used mixed-effects model, and (3) a more flexible mixed-effects model. We simulated scenarios where period effects were common to all clusters, and when they varied according to a distribution informed by routinely collected health data. The non-parametric within-period method provided unbiased intervention effect estimates with correct confidence-interval coverage for all scenarios. The parametric within-period method produced confidence intervals with low coverage for most scenarios. The mixed-effects models' confidence intervals had low coverage when period effects varied between clusters but had greater power than the non-parametric within-period method when period effects were common to all clusters. The non-parametric within-period method is a robust method for analysing SWT. The method could be used by trial statisticians who want to emphasise that the SWT is a randomised trial, in the common position of being uncertain about whether data will meet the assumptions necessary for mixed-effect models.

Keywords: cluster randomised trial; confidence interval coverage; permutation test; simulation study; stepped wedge trial.

© 2018 The Authors. Statistics in Medicine published by John Wiley & Sons Ltd.

Figures

Figure 1

Simulated log‐odds for 33 clusters over 2 years with no intervention in the 4 scenarios; common period effects with a high ICC (0.08), common period effects with a low ICC (0.02), varying period effects with a high ICC (0.08), and varying period effects with a low ICC (0.02)

Figure 2

Diagrams of the 3 and 11 sequence designs. For each, we simulated trials with 3 or 11 clusters per sequence

Figure 3

Graph of mean and ½ standard deviation either side of the estimated log‐odds ratios for each method, scenario, and trial design. The vertical grey line denotes the true log‐odds ratio. The mean estimates are shown with the circles. The horizontal lines depict ½ standard deviation either side of the mean. The effects are organised first by whether the period effects vary between cluster, then by the ICC, and finally by the trial design (number of sequences x the number of clusters allocated to each sequence) [Colour figure can be viewed at http://wileyonlinelibrary.com]

Figure 4

Coverage of 95% confidence intervals for each analysis, scenario, and trial design. Coverage between 96.4% and 94.6% is nominal. The coverage estimates are organised first by whether the time‐trends vary between cluster, then by the ICC, and finally by the trial design (number of sequences x number of cluster allocated to each sequence) [Colour figure can be viewed at http://wileyonlinelibrary.com]