Post hoc power analysis: is it an informative and meaningful analysis?

Abstract

Power analysis is a key component for planning prospective studies such as clinical trials. However, some journals in biomedical and psychosocial sciences ask for power analysis for data already collected and analysed before accepting manuscripts for publication. In this report, post hoc power analysis for retrospective studies is examined and the informativeness of understanding the power for detecting significant effects of the results analysed, using the same data on which the power analysis is based, is scrutinised. Monte Carlo simulation is used to investigate the performance of posthoc power analysis.

Keywords: continuous outcome; monte carlo; post-hoc power; retrospective study; simulation.

Conflict of interest statement

Competing interests: None declared.

Figures

Figure 1

Histograms of post hoc power, along with true power, based on 1000 Monte Carlo sample sizes with the mean difference: (A) δ=0.5; (B) δ=1 and (C) δ=2 and a sample size n=50.

Figure 2

Histograms of post hoc power, along with true power, based on 1000 Monte Carlo sample sizes with the mean difference: (A) δ=0.5, (B) δ=1 and (C) δ=2 and a sample size n=100.