Analysis of cluster randomised stepped wedge trials with repeated cross-sectional samples

Karla Hemming, Monica Taljaard, Andrew Forbes, Karla Hemming, Monica Taljaard, Andrew Forbes

Abstract

Background: The stepped wedge cluster randomised trial (SW-CRT) is increasingly being used to evaluate policy or service delivery interventions. However, there is a dearth of trials literature addressing analytical approaches to the SW-CRT. Perhaps as a result, a significant number of published trials have major methodological shortcomings, including failure to adjust for secular trends at the analysis stage. Furthermore, the commonly used analytical framework proposed by Hussey and Hughes makes several assumptions.

Methods: We highlight the assumptions implicit in the basic SW-CRT analytical model proposed by Hussey and Hughes. We consider how simple modifications of the basic model, using both random and fixed effects, can be used to accommodate deviations from the underlying assumptions. We consider the implications of these modifications for the intracluster correlation coefficients. In a case study, the importance of adjusting for the secular trend is illustrated.

Results: The basic SW-CRT model includes a fixed effect for time, implying a common underlying secular trend across steps and clusters. It also includes a single term for treatment, implying a constant shift in this trend under the treatment. When these assumptions are not realistic, simple modifications can be implemented to allow the secular trend to vary across clusters and the treatment effect to vary across clusters or time. In our case study, the naïve treatment effect estimate (adjusted for clustering but unadjusted for time) suggests a beneficial effect. However, after adjusting for the underlying secular trend, we demonstrate a reversal of the treatment effect.

Conclusion: Due to the inherent confounding of the treatment effect with time, analysis of a SW-CRT should always account for secular trends or risk-biased estimates of the treatment effect. Furthermore, the basic model proposed by Hussey and Hughes makes a number of important assumptions. Consideration needs to be given to the appropriate model choice at the analysis stage. We provide a Stata code to implement the proposed analyses in the illustrative case study.

Keywords: Analysis; Cluster randomised trial; Secular trends; Stepped wedge.

Figures

Fig. 1
Fig. 1
Schematic illustration of the stepped wedge cluster randomised trial
Fig. 2
Fig. 2
Schematic representation of study design for case study
Fig. 3
Fig. 3
Model-based estimate of underlying temporal trend in primary outcome over duration of trial in unexposed clusters (black line) and model-based estimated of outcome in intervention periods (red line) – basic model for the case study. Point estimates and 95% CI for each step with smoothed (LOWESS) line overlaid (black control; red intervention)
Fig. 4
Fig. 4
Model-based estimate of treatment effect (ln odds ratio, OR) over duration of trial. Point estimates and 95% CI for each time period in which observations were both exposed and unexposed to intervention

References

    1. Brown CA, Lilford RJ. The stepped wedge trial design: a systematic review. BMC Med Res Methodol. 2006;6:54. doi: 10.1186/1471-2288-6-54.
    1. Mdege ND, Man MS, Taylor (nee Brown) CA, Torgerson DJ. Systematic review of stepped wedge cluster randomized trials shows that design is particularly used to evaluate interventions during routine implementation. J Clin Epidemiol. 2011;64(9):936–48. doi: 10.1016/j.jclinepi.2010.12.003.
    1. Hemming K, Haines TP, Chilton PJ, Girling AJ, Lilford RJ. The stepped wedge cluster randomised trial: rationale, design, analysis and reporting. BMJ. 2015;350:h391. doi: 10.1136/bmj.h391.
    1. The Gambia Hepatitis Study Group The Gambia Hepatitis Intervention Study. Cancer Res. 1987;47(21):5782–7.
    1. King G, Gakidou E, Ravishankar N, Moore RT, Lakin J, Vargas M, et al. A ‘politically robust’ experimental design for public policy evaluation, with application to the Mexican universal health insurance program. J Policy Anal Manage. 2007;26:479–506. doi: 10.1002/pam.20279.
    1. Leontjevas R, Gerritsen DL, Smalbrugge M, Teerenstra S, Vernooij-Dassen MJ, Koopmans RT. A structural multidisciplinary approach to depression management in nursing-home residents: a multicentre, stepped-wedge cluster-randomised trial. Lancet. 2013;381:2255–64. doi: 10.1016/S0140-6736(13)60590-5.
    1. Pearse R, Pedan C, Bion J, Faiz O, Holt P, Girling A, et al. HS&DR—12/5005/10. Enhanced Peri-Operative Care for High-risk patients (EPOCH) trial: a stepped wedge cluster randomised trial of a quality improvement intervention for patients undergoing emergency laparotomy [protocol]. 2013. .
    1. Davey C, Hargreaves J, Thompson JA, Copas AJ, Beard E, Lewis JJ, Fielding KL. Analysis and reporting of stepped wedge randomised controlled trials: synthesis and critical appraisal of published studies, 2010 to 2014. Trials. 2015;16:358. doi: 10.1186/s13063-015-0838-3.
    1. Martin J, Taljaard M, Girling A, Hemming K. A systematic review finds major deficiencies in sample size methodology and reporting for stepped wedge cluster randomised trials. BMJ Open. 2016;6(2):e010166. doi: 10.1136/bmjopen-2015-010166.
    1. Barker D, et al. Stepped wedge cluster randomised trials: a review of the statistical methodology used and available. BMC Med Res Methodol. 2016;16:69. doi: 10.1186/s12874-016-0176-5.
    1. Bland JM. Cluster randomised trials in the medical literature: two bibliometric surveys. BMC Med Res Methodol. 2004;4:21. doi: 10.1186/1471-2288-4-21.
    1. Ivers NM, Taljaard M, Dixon S, Bennett C, McRae A, Taleban J, Skea Z, Brehaut JC, Boruch RF, Eccles MP, Grimshaw JM, Weijer C, Zwarenstein M, Donner A. Impact of CONSORT extension for cluster randomised trials on quality of reporting and study methodology: review of random sample of 300 trials, 2000-8. BMJ. 2011;343:d5886. doi: 10.1136/bmj.d5886.
    1. Rutterford C, Taljaard M, Dixon S, Copas A, Eldridge S. Reporting and methodological quality of sample size calculations in cluster randomised trials could be improved: a review. J Clin Epidemiol. In Press.
    1. Dreischulte T, Donnan P, Grant A, Hapca A, McCowan C, Guthrie B. Safer prescribing—A trial of education, informatics, and financial incentives. N Engl J Med. 2016;374(11):1053–64. doi: 10.1056/NEJMsa1508955.
    1. Heo M, Kim N, Rinke ML, Wylie-Rosett J. Sample size determinations for stepped-wedge clinical trials from a three-level data hierarchy perspective. Stat Methods Med Res. 2016. [Epub ahead of print].
    1. Hussey MA, Hughes JP. Design and analysis of stepped wedge cluster randomized trials. Contemp Clin Trials. 2007;28(2):182–191.
    1. Parienti JJ, Kuss O. Cluster-crossover design: a method for limiting clusters level effect in community-intervention studies. Contemp Clin Trials. 2007;28(3):316–23. doi: 10.1016/j.cct.2006.10.004.
    1. Baio G, Copas A, Ambler G, Hargreaves J, Beard E, Omar RZ. Sample size calculation for a stepped wedge trial. Trials. 2015;16:354. doi: 10.1186/s13063-015-0840-9.
    1. S Kenyon, S Dann, L Ingram, P Clarke, A Hogan, D Jenkinson, K Hemming. Evaluation of bespoke training to midwifery teams to increase membrane sweeping to reduce induction of labour: a stepped wedge cluster randomised trial. Submitted for publication.
    1. Ukoumunne OC, Thompson SG. Analysis of cluster randomized trials with repeated cross-sectional binary measurements. Stat Med. 2001;20(3):417–33.7. doi: 10.1002/1097-0258(20010215)20:3<417::AID-SIM802>;2-G.
    1. Turner RM, White IR, Croudace T, PIP Study Group Analysis of cluster randomized cross-over trial data: a comparison of methods. Stat Med. 2007;26(2):274–89. doi: 10.1002/sim.2537.
    1. Hooper R, Bourke L. Cluster randomised trials with repeated cross sections: alternatives to parallel group designs. BMJ. 2015;350:h2925. doi: 10.1136/bmj.h2925.
    1. Girling AJ, Hemming K. Statistical efficiency and optimal design for stepped cluster studies under linear mixed effects models. Stat Med. 2016. doi:10.1002/sim.6850 [Epub ahead of print].
    1. Hooper R, Teerenstra S, de Hoop E, Eldridge S. Sample size calculation for stepped wedge and other longitudinal cluster randomised trials. Stat Med. 2016;35(26):4718–28. doi: 10.1002/sim.7028.
    1. Taljaard M, Teerenstra S, Ivers NM, Fergusson DA. Substantial risks associated with few clusters in cluster randomized and stepped wedge designs. Clin Trials. 2016. [Epub ahead of print].
    1. Brown H, Prescott R. Generalised linear mixed models, Applied mixed models. In: Medicine. 3rd ed. Chichester: Wiley; 2014. doi:10.1002/9781118778210.ch3.
    1. Hughes JP, Granston TS, Heagerty PJ. Current issues in the design and analysis of stepped wedge trials. Contemp Clin Trials. 2015;45(Pt A):55–60. doi: 10.1016/j.cct.2015.07.006.
    1. Eldridge SM, Ukoumunne OC, Carlin JB. The intra-cluster correlation coefficient in cluster randomized trials: a review of definitions. Int Stat Rev. 2009;77(3):378–94. doi: 10.1111/j.1751-5823.2009.00092.x.
    1. van den Broek IV, van Bergen JE, Brouwers EE, Fennema JS, Götz HM, Hoebe CJ, Koekenbier RH, Kretzschmar M, Over EA, Schmid BV, Pars LL, van Ravesteijn SM, van der Sande MA, de Wit GA, Low N, Op de Coul EL. Effectiveness of yearly, register based screening for chlamydia in the Netherlands: controlled trial with randomised stepped wedge implementation. BMJ. 2012;345:e4316. doi: 10.1136/bmj.e4316.
    1. Dixon-Woods M, Leslie M, Tarrant C, Bion J. Explaining Matching Michigan: an ethnographic study of a patient safety program. Implement Sci. 2013;8:70. doi: 10.1186/1748-5908-8-70.
    1. Rothman KJ, Greenland S, Lash TL. Modern epidemiology. 3. Philadelphia: Lippincott, Williams & Wilkins; 2008.
    1. Long J, Ryoo J. Using fractional polynomials to model non-linear trends in longitudinal data. Br J Math Stat Psychol. 2010;63:177–203. doi: 10.1348/000711009X431509.
    1. Fitzmaurice GM, Laird N, Ware JH. Applied longitudinal analysis. London: Wiley; 2004.
    1. Twisk JWR, Hoogendijk EO, Zwijsen SA, de Boer MR. Different methods to analyze stepped wedge trial designs revealed different aspects of intervention effects. J Clin Epidemiol. 2016;72:75–83. doi: 10.1016/j.jclinepi.2015.11.004.
    1. Abad AA, Litière S, Molenberghs G. Testing for misspecification in generalized linear mixed models. Biostatistics. 2010;11(4):771–86. doi: 10.1093/biostatistics/kxq019.
    1. Morgan KE, Forbes AB, Keogh RH, Jairath V, Kahan BC. Choosing appropriate analysis methods for cluster randomised cross-over trials with a binary outcome. Stat Med. 2017;36(2):318–33. doi: 10.1002/sim.7137.
    1. Hubbard AE, Ahern J, Fleischer NL, Van der Laan M, Lippman SA, Jewell N, Bruckner T, Satariano WA. To GEE or not to GEE: comparing population average and mixed models for estimating the associations between neighborhood risk factors and health. Epidemiology. 2010;21(4):467–74. doi: 10.1097/EDE.0b013e3181caeb90.
    1. Scott JM, deCamp A, Juraska M, Fay MP, Gilbert PB. Finite-sample corrected generalized estimating equation of population average treatment effects in stepped wedge cluster randomized trials. Stat Methods Med Res. 2014. [Epub ahead of print] PubMed PMID: 25267551; PubMed Central PMCID: PMC4411204.
    1. Donner A, Klar N. Design and analysis of cluster randomization trials in health research. London: Arnold; 2000.
    1. Hayes RJ, Moulton LH. Cluster randomised trials. London: Chapman & Hall/CRC Press. 2009.

Source: PubMed

Sponzoři a spolupracovníci

Zdravotní podmínky

Drogové intervence

3
Předplatit