The risks and rewards of covariate adjustment in randomized trials: an assessment of 12 outcomes from 8 studies

Brennan C Kahan, Vipul Jairath, Caroline J Doré, Tim P Morris, Brennan C Kahan, Vipul Jairath, Caroline J Doré, Tim P Morris

Abstract

Background: Adjustment for prognostic covariates can lead to increased power in the analysis of randomized trials. However, adjusted analyses are not often performed in practice.

Methods: We used simulation to examine the impact of covariate adjustment on 12 outcomes from 8 studies across a range of therapeutic areas. We assessed (1) how large an increase in power can be expected in practice; and (2) the impact of adjustment for covariates that are not prognostic.

Results: Adjustment for known prognostic covariates led to large increases in power for most outcomes. When power was set to 80% based on an unadjusted analysis, covariate adjustment led to a median increase in power to 92.6% across the 12 outcomes (range 80.6 to 99.4%). Power was increased to over 85% for 8 of 12 outcomes, and to over 95% for 5 of 12 outcomes. Conversely, the largest decrease in power from adjustment for covariates that were not prognostic was from 80% to 78.5%.

Conclusions: Adjustment for known prognostic covariates can lead to substantial increases in power, and should be routinely incorporated into the analysis of randomized trials. The potential benefits of adjusting for a small number of possibly prognostic covariates in trials with moderate or large sample sizes far outweigh the risks of doing so, and so should also be considered.

Figures

Figure 1
Figure 1
Simulation results for ‘underpowered’ trials. Change in power through covariate adjustment as compared with unadjusted analysis for ‘underpowered’ trials (where an unadjusted analysis gives 50% power).
Figure 2
Figure 2
Simulation results for ‘adequately powered’ trials. Change in power through covariate adjustment as compared with unadjusted analysis for ‘adequately powered’ trials (where an unadjusted analysis gives 80% power).

References

    1. Hernandez AV, Eijkemans MJ, Steyerberg EW. Randomized controlled trials with time-to-event outcomes: how much does prespecified covariate adjustment increase power? Ann Epidemiol. 2006;16(1):41–48. doi: 10.1016/j.annepidem.2005.09.007.
    1. Hernandez AV, Steyerberg EW, Butcher I, Mushkudiani N, Taylor GS, Murray GD, Marmarou A, Choi SC, Lu J, Habbema JD, Maas AI. Adjustment for strong predictors of outcome in traumatic brain injury trials: 25% reduction in sample size requirements in the IMPACT study. J Neurotrauma. 2006;23(9):1295–1303. doi: 10.1089/neu.2006.23.1295.
    1. Hernandez AV, Steyerberg EW, Habbema JD. Covariate adjustment in randomized controlled trials with dichotomous outcomes increases statistical power and reduces sample size requirements. J Clin Epidemiol. 2004;57(5):454–460. doi: 10.1016/j.jclinepi.2003.09.014.
    1. Pocock SJ, Assmann SE, Enos LE, Kasten LE. Subgroup analysis, covariate adjustment and baseline comparisons in clinical trial reporting: current practice and problems. Stat Med. 2002;21(19):2917–2930. doi: 10.1002/sim.1296.
    1. McHugh GS, Butcher I, Steyerberg EW, Marmarou A, Lu J, Lingsma HF, Weir J, Maas AI, Murray GD. A simulation study evaluating approaches to the analysis of ordinal outcome data in randomized controlled trials in traumatic brain injury: results from the IMPACT project. Clin Trials. 2010;7(1):44–57. doi: 10.1177/1740774509356580.
    1. Negassa A, Hanley JA. The effect of omitted covariates on confidence interval and study power in binary outcome analysis: a simulation study. Contemp Clin Trials. 2007;28(3):242–248. doi: 10.1016/j.cct.2006.08.007.
    1. Senn S. Statistical Issues in Drug Development. Chichester: Wiley; 2007.
    1. Turner EL, Perel P, Clayton T, Edwards P, Hernandez AV, Roberts I, Shakur H, Steyerberg EW. CRASH trial collaborators: Covariate adjustment increased power in randomized controlled trials: an example in traumatic brain injury. J Clin Epidemiol. 2012;65(5):474–481. doi: 10.1016/j.jclinepi.2011.08.012.
    1. Hauck WW, Anderson S, Marcus SM. Should we adjust for covariates in nonlinear regression analyses of randomized trials? Control Clin Trials. 1998;19(3):249–256. doi: 10.1016/S0197-2456(97)00147-5.
    1. Robinson LD, Jewell NP. Some surprising results about covariate adjustment in logistic regression models. Int Stat Rev. 1991;58:227–240.
    1. Kahan BC, Morris TP. Improper analysis of trials randomised using stratified blocks or minimisation. Stat Med. 2012;31(4):328–340. doi: 10.1002/sim.4431.
    1. Kahan BC, Morris TP. Reporting and analysis of trials using stratified randomisation in leading medical journals: review and reanalysis. BMJ. 2012;345:e5840. doi: 10.1136/bmj.e5840.
    1. Kahan BC, Morris TP. Assessing potential sources of clustering in individually randomised trials. BMC Med Res Methodol. 2013;13(1):58. doi: 10.1186/1471-2288-13-58.
    1. Parzen M, Lipsitz SR, Dear KBG. Does clustering affect the usual test statistics of no treatment effect in a randomized clinical trial? Biom J. 1998;40:385–402. doi: 10.1002/(SICI)1521-4036(199808)40:4<385::AID-BIMJ385>;2-#.
    1. Austin PC, Manca A, Zwarenstein M, Juurlink DN, Stanbrook MB. A substantial and confusing variation exists in handling of baseline covariates in randomized controlled trials: a review of trials published in leading medical journals. J Clin Epidemiol. 2010;63(2):142–153. doi: 10.1016/j.jclinepi.2009.06.002.
    1. Hernandez AV, Steyerberg EW, Taylor GS, Marmarou A, Habbema JD, Maas AI. Subgroup analysis and covariate adjustment in randomized clinical trials of traumatic brain injury: a systematic review. Neurosurgery. 2005;57(6):1244–1253. doi: 10.1227/01.NEU.0000186039.57548.96. Discussion, 1253.
    1. Assmann SF, Pocock SJ, Enos LE, Kasten LE. Subgroup analysis and other (mis)uses of baseline data in clinical trials. Lancet. 2000;355(9209):1064–1069. doi: 10.1016/S0140-6736(00)02039-0.
    1. Yu LM, Chan AW, Hopewell S, Deeks JJ, Altman DG. Reporting on covariate adjustment in randomised controlled trials before and after revision of the 2001 CONSORT statement: a literature review. Trials. 2010;11:59. doi: 10.1186/1745-6215-11-59.
    1. Saquib N, Saquib J, Ioannidis JP. Practices and impact of primary outcome adjustment in randomized controlled trials: meta-epidemiologic study. BMJ. 2013;347:f4313. doi: 10.1136/bmj.f4313.
    1. Schott JM, Bartlett JW, Barnes J, Leung KK, Ourselin S, Fox NC. Reduced sample sizes for atrophy outcomes in Alzheimer’s disease trials: baseline adjustment. Neurobiol Aging. 2010;31(8):1452–1462. doi: 10.1016/j.neurobiolaging.2010.04.011. e2.
    1. Rahman NM, Maskell NA, West A, Teoh R, Arnold A, Mackinlay C, Peckham D, Davies CW, Ali N, Kinnear W, Bentley A, Kahan BC, Wrightson JM, Davies HE, Hooper CE, Lee YC, Hedley EL, Crosthwaite N, Choo L, Helm EJ, Gleeson FV, Nunn AJ, Davies RJ. Intrapleural use of tissue plasminogen activator and DNase in pleural infection. N Engl J Med. 2011;365(6):518–526. doi: 10.1056/NEJMoa1012740.
    1. Pocock SJ, Simon R. Sequential treatment assignment with balancing for prognostic factors in the controlled clinical trial. Biometrics. 1975;31(1):103–115. doi: 10.2307/2529712.
    1. Rosenberger WF, Lachin JM. Randomization in Clinical Trials: Theory and Practice. New York: Wiley; 2002.
    1. Scott NW, McPherson GC, Ramsay CR, Campbell MK. The method of minimization for allocation to clinical trials: a review. Control Clin Trials. 2002;23(6):662–674. doi: 10.1016/S0197-2456(02)00242-8.
    1. Taves DR. Minimization: a new method of assigning patients to treatment and control groups. Clin Pharmacol Ther. 1974;15(5):443–453.
    1. Van Breukelen GJ. ANCOVA versus change from baseline: more power in randomized studies, more bias in nonrandomized studies [corrected] J Clin Epidemiol. 2006;59(9):920–925. doi: 10.1016/j.jclinepi.2006.02.007.
    1. Egbewale BE, Lewis M, Sim J. Bias, precision and statistical power of analysis of covariance in the analysis of randomized trials with baseline imbalance: a simulation study. BMC Med Res Methodol. 2014;14:49. doi: 10.1186/1471-2288-14-49. doi:10.1186/1471-2288-14-49.
    1. Kahan BC, Morris TP. Analysis of multicentre trials with continuous outcomes: when and how should we account for centre effects? Stat Med. 2013;32(7):1136–1149. doi: 10.1002/sim.5667.
    1. Kahan BC, Morris TP. Adjusting for multiple prognostic factors in the analysis of randomised trials. BMC Med Res Methodol. 2013;13:99. doi: 10.1186/1471-2288-13-99.
    1. White IR, Thompson SG. Adjusting for partially missing baseline measurements in randomized trials. Stat Med. 2005;24(7):993–1007. doi: 10.1002/sim.1981.
    1. Raab GM, Day S, Sales J. How to select covariates to include in the analysis of a clinical trial. Control Clin Trials. 2000;21(4):330–342. doi: 10.1016/S0197-2456(00)00061-1.
    1. Jairath V, Kahan BC, Logan RF, Hearnshaw SA, Dore CJ, Travis SP, Murphy MF, Palmer KR. National audit of the use of surgery and radiological embolization after failed endoscopic haemostasis for non-variceal upper gastrointestinal bleeding. Br J Surg. 2012;99(12):1672–1680. doi: 10.1002/bjs.8932.
    1. Jairath V, Kahan BC, Logan RF, Hearnshaw SA, Dore CJ, Travis SP, Murphy MF, Palmer KR. Outcomes following acute nonvariceal upper gastrointestinal bleeding in relation to time to endoscopy: results from a nationwide study. Endoscopy. 2012;44(8):723–730.
    1. Jairath V, Kahan BC, Logan RF, Hearnshaw SA, Travis SP, Murphy MF, Palmer KR. Mortality from acute upper gastrointestinal bleeding in the United Kingdom: does it display a ‘weekend effect’? Am J Gastroenterol. 2011;106(9):1621–1628. doi: 10.1038/ajg.2011.172.
    1. Jairath V, Kahan BC, Stanworth SJ, Logan RF, Hearnshaw SA, Travis SP, Palmer KR, Murphy MF. Prevalence, management, and outcomes of patients with coagulopathy after acute nonvariceal upper gastrointestinal bleeding in the United Kingdom. Transfusion. 2013;53(5):1069–1076. doi: 10.1111/j.1537-2995.2012.03849.x.
    1. McGregor AH, Dore CJ, Morris TP, Morris S, Jamrozik K. ISSLS prize winner: Function After Spinal Treatment, Exercise, and Rehabilitation (FASTER): a factorial randomized trial to determine whether the functional outcome of spinal surgery can be improved. Spine (Phila Pa 1976) 2011;36(21):1711–1720. doi: 10.1097/BRS.0b013e318214e3e6.
    1. Craig SE, Kohler M, Nicoll D, Bratton DJ, Nunn A, Davies R, Stradling J. Continuous positive airway pressure improves sleepiness but not calculated vascular risk in patients with minimally symptomatic obstructive sleep apnoea: the MOSAIC randomised controlled trial. Thorax. 2012;67(12):1090–1096. doi: 10.1136/thoraxjnl-2012-202178.
    1. Christensen E, Neuberger J, Crowe J, Altman DG, Popper H, Portmann B, Doniach D, Ranek L, Tygstrup N, Williams R. Beneficial effect of azathioprine and prediction of prognosis in primary biliary cirrhosis. Final results of an international trial. Gastroenterology. 1985;89(5):1084–1091.
    1. Carr R, Brocklehurst P, Dore CJ, Modi N. Granulocyte-macrophage colony stimulating factor administered as prophylaxis for reduction of sepsis in extremely preterm, small for gestational age neonates (the PROGRAMS trial): a single-blind, multicentre, randomised controlled trial. Lancet. 2009;373(9659):226–233. doi: 10.1016/S0140-6736(09)60071-4.
    1. Medical Research Council Renal Cancer Collaborators. Interferon-α and survival in metastatic renal carcinoma: early results of a randomised controlled trial. Lancet. 1999;353(9146):14–17.
    1. Davies HE, Mishra EK, Kahan BC, Wrightson JM, Stanton AE, Guhan A, Davies CW, Grayez J, Harrison R, Prasad A, Crosthwaite N, Lee YC, Davies RJ, Miller RF, Rahman NM. Effect of an indwelling pleural catheter vs chest tube and talc pleurodesis for relieving dyspnea in patients with malignant pleural effusion: the TIME2 randomized controlled trial. JAMA. 2012;307(22):2383–2389. doi: 10.1001/jama.2012.5535.
    1. Chan AW, Tetzlaff JM, Gotzsche PC, Altman DG, Mann H, Berlin JA, Dickersin K, Hróbjartsson A, Schulz KF, Parulekar WR, Krleza-Jeric K, Laupacis A, Moher D. SPIRIT 2013 explanation and elaboration: guidance for protocols of clinical trials. BMJ. 2013;346:e7586. doi: 10.1136/bmj.e7586.
    1. ICH. Harmonised tripartite guideline ICH. Statistical principles for clinical trials. International Conference on Harmonisation E9 Expert Working Group. Stat Med. 1999;18(15):1905–1942.
    1. Chu R, Thabane L, Ma J, Holbrook A, Pullenayegum E, Devereaux PJ. Comparing methods to estimate treatment effects on a continuous outcome in multicentre randomized controlled trials: a simulation study. BMC Med Res Methodol. 2011;11:21. doi: 10.1186/1471-2288-11-21.
    1. Agresti A, Hartzel J. Strategies for comparing treatments on a binary response with multi-centre data. Stat Med. 2000;19(8):1115–1139. doi: 10.1002/(SICI)1097-0258(20000430)19:8<1115::AID-SIM408>;2-X.

Source: PubMed

Sponsorer og samarbejdspartnere

Medicinske tilstande

Narkotikainterventioner

3
Abonner