New Approaches for Estimation of Effect Sizes and their Confidence Intervals for Treatment Effects from Randomized Controlled Trials

Abstract

Although Cohen's d and the growth modeling analysis (GMA) d from linear models are common standardized effect sizes used to convey treatment effects, popular statistical software packages do not include them in their standard outputs. This article demonstrated the use of statistical software with user-prescribed parameter functions (e.g., Mplus) to produce d for treatment effects from both classical analysis and GMA--along with their associated standard errors (SEs) and confidence intervals (CIs). A Monte Carlo study was conducted to examine bias in the SE and CI for GMA d obtained with Mplus and found that both estimates were more accurate when calculated by the software with the standard bootstrap than with the delta method, but the delta method estimates were less biased than respective estimates from extant post hoc equations. Thus, users of many statistical software packages (including SAS, R, and LISREL) should obtain d or GMA d and associated CIs directly. Researchers employing less versatile software--and meta-analysts including ds and GMA ds in their syntheses of treatment effects--should continue to use the conventional post hoc equations. Biases in SEs and CIs for effect sizes obtained with them are ignorable and point estimates of d and GMA d are the same whether obtained directly from the software or with post hoc equations.

Keywords: confidence intervals; effect sizes; latent growth models; multilevel analysis.