Performance of model-based vs. permutation tests in the HEALing (Helping to End Addiction Long-termSM) Communities Study, a covariate-constrained cluster randomized trial

Xiaoyu Tang, Timothy Heeren, Philip M Westgate, Daniel J Feaster, Soledad A Fernandez, Nathan Vandergrift, Debbie M Cheng, Xiaoyu Tang, Timothy Heeren, Philip M Westgate, Daniel J Feaster, Soledad A Fernandez, Nathan Vandergrift, Debbie M Cheng

Abstract

Background: The HEALing (Helping to End Addiction Long-termSM) Communities Study (HCS) is a multi-site parallel group cluster randomized wait-list comparison trial designed to evaluate the effect of the Communities That Heal (CTH) intervention compared to usual care on opioid overdose deaths. Covariate-constrained randomization (CCR) was applied to balance the community-level baseline covariates in the HCS. The purpose of this paper is to evaluate the performance of model-based tests and permutation tests in the HCS setting. We conducted a simulation study to evaluate type I error rates and power for model-based and permutation tests for the multi-site HCS as well as for a subgroup analysis of a single state (Massachusetts). We also investigated whether the maximum degree of imbalance in the CCR design has an impact on the performance of the tests.

Methods: The primary outcome, the number of opioid overdose deaths, is count data assessed at the community level that will be analyzed using a negative binomial regression model. We conducted a simulation study to evaluate the type I error rates and power for 3 tests: (1) Wald-type t-test with small-sample corrected empirical standard error estimates, (2) Wald-type z-test with model-based standard error estimates, and (3) permutation test with test statistics calculated by the difference in average residuals for the two groups.

Results: Our simulation results demonstrated that Wald-type t-tests with small-sample corrected empirical standard error estimates from the negative binomial regression model maintained proper type I error. Wald-type z-tests with model-based standard error estimates were anti-conservative. Permutation tests preserved type I error rates if the constrained space was not too small. For all tests, the power was high to detect the hypothesized 40% reduction in opioid overdose deaths for the intervention vs. comparison group both for the overall HCS and the subgroup analysis of Massachusetts (MA).

Conclusions: Based on the results of our simulation study, the Wald-type t-test with small-sample corrected empirical standard error estimates from a negative binomial regression model is a valid and appropriate approach for analyzing cluster-level count data from the HEALing Communities Study.

Trial registration: ClinicalTrials.gov http://www.

Clinicaltrials: gov ; Identifier: NCT04111939.

Keywords: Cluster randomized trials; Covariate-constrained randomization; Model-based tests; Negative binomial regression; Permutation tests.

Conflict of interest statement

Debbie Cheng serves on Data Safety Monitoring Boards for Janssen Research & Development. No other conflicts to disclose.

© 2022. The Author(s).

Figures

Fig. 1
Fig. 1
Type I error rate and power with varying maximum degrees of covariate imbalance for different tests for overall HCS
Fig. 2
Fig. 2
Type I error rate and power with varying maximum degrees of covariate imbalance for different tests for MA only

References

    1. Murray DM, Varnell SP, Blitstein JL. Design and analysis of group-randomized trials: a review of recent methodological developments. Am J Public Health. 2004;94(3):423–432. doi: 10.2105/AJPH.94.3.423.
    1. Rosenberger W. Randomization in clinical trials. New York: Wiley; 2002.
    1. Senn S. Testing for baseline balance in clinical trials. Stat Med. 1994;13(17):1715–1726. doi: 10.1002/sim.4780131703.
    1. Martin DC, et al. The effect of matching on the power of randomized community intervention studies. Stat Med. 1993;12(3-4):329–338. doi: 10.1002/sim.4780120315.
    1. Kernan WN, et al. Stratified randomization for clinical trials. J Clin Epidemiol. 1999;52(1):19–26. doi: 10.1016/S0895-4356(98)00138-3.
    1. Taves DR. Minimization: a new method of assigning patients to treatment and control groups. Clin Pharmacol Ther. 1974;15(5):443–453. doi: 10.1002/cpt1974155443.
    1. Moulton LH. Covariate-based constrained randomization of group-randomized trials. Clin Trials. 2004;1(3):297–305. doi: 10.1191/1740774504cn024oa.
    1. Raab GM, Butcher I. Randomization inference for balanced cluster-randomized trials. Clin Trials. 2005;2(2):130–140. doi: 10.1191/1740774505cn075oa.
    1. Ludbrook J. Advantages of permutation (randomization) tests in clinical and experimental pharmacology and physiology. Clin Exp Pharmacol Physiol. 1994;21(9):673–686. doi: 10.1111/j.1440-1681.1994.tb02570.x.
    1. Fu D, Murray DM, Wong S. Comparison study of general linear mixed model and permutation tests in group-randomizwd trials under non-normal error distributions, in The Joint Statistical Meetings. Washington D.C; 2009.
    1. Murray DM, et al. A comparison of permutation and mixed-model regression methods for the analysis of simulated data in the context of a group-randomized trial. Stat Med. 2006;25(3):375–388. doi: 10.1002/sim.2233.
    1. Li F, et al. An evaluation of constrained randomization for the design and analysis of group-randomized trials. Stat Med. 2016;35(10):1565–1579. doi: 10.1002/sim.6813.
    1. Li F, et al. An evaluation of constrained randomization for the design and analysis of group-randomized trials with binary outcomes. Stat Med. 2017;36(24):3791–3806. doi: 10.1002/sim.7410.
    1. El-Bassel N, et al. Introduction to the special issue on the HEALing Communities Study. Drug Alcohol Depend. 2020;217:108327. doi: 10.1016/j.drugalcdep.2020.108327.
    1. Chandler RK, et al. Addressing opioid overdose deaths: the vision for the HEALing communities study. Drug Alcohol Depend. 2020;217:108329. doi: 10.1016/j.drugalcdep.2020.108329.
    1. Walsh SL, El-Bassel N, Jackson RD. The HEALing (Helping to End Addiction Long-termsm) Communities Study: protocol for a cluster randomized trial at the community level to reduce opioid overdose deaths through implementation of an integrated set of evidence-based practices. Drug Alcohol Dependence. 2020;217:108335. doi: 10.1016/j.drugalcdep.2020.108335.
    1. Westgate PM, et al. Marginal modeling in community randomized trials with rare events: utilization of the negative binomial regression model. Clin Trials. 2022;19(2):162–171. doi: 10.1177/17407745211063479.
    1. Simon R. Restricted randomization designs in clinical trials. Biometrics. 1979;35(2):503–512. doi: 10.2307/2530354.
    1. Mancl LA, DeRouen TA. A covariance estimator for GEE with improved small-sample properties. Biometrics. 2001;57(1):126–134. doi: 10.1111/j.0006-341X.2001.00126.x.
    1. Kauermann G, Carrol R. A note on the efficiency of sandwich covariance matrix estimation. J Am Stat Assoc. 2001;96:1387–1396. doi: 10.1198/016214501753382309.
    1. Ford WP, Westgate PM. Improved standard error estimator for maintaining the validity of inference in cluster randomized trials with a small number of clusters. Biom J. 2017;59(3):478–495. doi: 10.1002/bimj.201600182.
    1. Good PI. Parametric and bootstrap tests of hypotheses. Springer; 2005.
    1. Gail MH, et al. On design considerations and randomization-based inference for community intervention trials. Stat Med. 1996;15(11):1069–1092. doi: 10.1002/(SICI)1097-0258(19960615)15:11<1069::AID-SIM220>;2-Q.
    1. Fay MP, Graubard BI. Small-sample adjustments for Wald-type tests using sandwich estimators. Biometrics. 2001;57(4):1198–1206. doi: 10.1111/j.0006-341X.2001.01198.x.

Source: PubMed

Sponsor e collaboratori

Condizioni mediche

Interventi farmacologici

3
Sottoscrivi