Alternatives to P value: confidence interval and effect size

Abstract

The previous articles of the Statistical Round in the Korean Journal of Anesthesiology posed a strong enquiry on the issue of null hypothesis significance testing (NHST). P values lie at the core of NHST and are used to classify all treatments into two groups: "has a significant effect" or "does not have a significant effect." NHST is frequently criticized for its misinterpretation of relationships and limitations in assessing practical importance. It has now provoked criticism for its limited use in merely separating treatments that "have a significant effect" from others that do not. Effect sizes and CIs expand the approach to statistical thinking. These attractive estimates facilitate authors and readers to discriminate between a multitude of treatment effects. Through this article, I have illustrated the concept and estimating principles of effect sizes and CIs.

Keywords: Confidence intervals; Effect sizes; P value.

Figures

Fig. 1. The changes in CI of the mean and alpha error values in accordance with sample size. All three data samples are randomly extracted using R system, under conditions of normal distribution with mean = 100, SD = 10. Each datum includes 10, 100, or 1000 samples. With the increase in sample size, the range of the 95% CI is considerably decreased from 8.8 for n = 10, 3.9 for n = 100 to 1.3 for n = 1000. The limits of 95% probability (5% alpha error limits) remains relatively unchanged as all three data samples were originated with the same mean and SD. This phenomenon implies that increase in sample size results in a more precise statistical inference (narrower CI) as well as increased statistical power. Critical values for alpha error probability are not much affected by increased sample size. These imaginary data are presented under the assumption of normal distribution with similar dispersions. SD: standard deviation, SEM: standard error of the mean.