Which is more generalizable, powerful and interpretable in meta-analyses, mean difference or standardized mean difference?

Nozomi Takeshima, Takashi Sozu, Aran Tajika, Yusuke Ogawa, Yu Hayasaka, Toshiaki A Furukawa, Nozomi Takeshima, Takashi Sozu, Aran Tajika, Yusuke Ogawa, Yu Hayasaka, Toshiaki A Furukawa

Abstract

Background: To examine empirically whether the mean difference (MD) or the standardised mean difference (SMD) is more generalizable and statistically powerful in meta-analyses of continuous outcomes when the same unit is used.

Methods: From all the Cochrane Database (March 2013), we identified systematic reviews that combined 3 or more randomised controlled trials (RCT) using the same continuous outcome. Generalizability was assessed using the I-squared (I2) and the percentage agreement. The percentage agreement was calculated by comparing the MD or SMD of each RCT with the corresponding MD or SMD from the meta-analysis of all the other RCTs. The statistical power was estimated using Z-scores. Meta-analyses were conducted using both random-effects and fixed-effect models.

Results: 1068 meta-analyses were included. The I2 index was significantly smaller for the SMD than for the MD (P < 0.0001, sign test). For continuous outcomes, the current Cochrane reviews pooled some extremely heterogeneous results. When all these or less heterogeneous subsets of the reviews were examined, the SMD always showed a greater percentage agreement than the MD. When the I2 index was less than 30%, the percentage agreement was 55.3% for MD and 59.8% for SMD in the random-effects model and 53.0% and 59.8%, respectively, in the fixed effect model (both P < 0.0001, sign test). Although the Z-scores were larger for MD than for SMD, there were no differences in the percentage of statistical significance between MD and SMD in either model.

Conclusions: The SMD was more generalizable than the MD. The MD had a greater statistical power than the SMD but did not result in material differences.

Figures

Figure 1
Figure 1
Flowchart for selection of meta-analysis.
Figure 2
Figure 2
Histogram of I2 index for MD and SMD.

References

    1. Higgins JPT, Green S, editor. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration, 2011. Available from .
    1. Cohen J. Statistical Power Analysis for the Behavioral Sciences. Hillsdale, New Jersey: Lawrence Erlbaum Associates: Routledge; 1988.
    1. Glass GV. Primary, secondary, and meta-analysis of research. Educ Res. 1976;5(10):3–8.
    1. Hedges LV, Olkin I, Statistiker M, Olkin I, Olkin I. Statistical Methods for Meta-Analysis. New York: Academic Press; 1985.
    1. Wilkinson L. Statistical methods in psychology journals: guidelines and explanations. Am Psychol. 1999;54(8):594.
    1. Egger M, Smith GD, Altman D. Systematic Reviews in Health Care: Meta-Analysis in Context. 2008. Wiley. com.
    1. Association AP. Publication Manual of the American Psychological Association. 6. Washington, DC: American Psychological Association; 2009.
    1. Borenstein M, Hedges LV, Higgins JP, Rothstein HR. Introduction to Meta-Analysis. 2011. Wiley. com.
    1. Tian L. Inferences on standardized mean difference: the generalized variable approach. Stat Med. 2007;26(5):945–953. doi: 10.1002/sim.2589.
    1. Furukawa TA, Guyatt GH, Griffith LE. Can we individualize the ‘number needed to treat’? An empirical study of summary effect measures in meta-analyses. Int J Epidemiol. 2002;31(1):72–76. doi: 10.1093/ije/31.1.72.
    1. Jørgensen AW, Hilden J, Gøtzsche PC. Cochrane reviews compared with industry supported meta-analyses and other meta-analyses of the same drugs: systematic review. BMJ. 2006;333(7572):782. doi: 10.1136/bmj.38973.444699.0B.
    1. Jadad AR, Cook DJ, Jones A, Klassen TP, Tugwell P, Moher M, Moher D. Methodology and reports of systematic reviews and meta-analyses. JAMA. 1998;280(3):278–280. doi: 10.1001/jama.280.3.278.
    1. Higgins JP, Thompson SG, Deeks JJ, Altman DG. Measuring inconsistency in meta-analyses. BMJ. 2003;327(7414):557. doi: 10.1136/bmj.327.7414.557.
    1. DerSimonian R, Laird N. Meta-analysis in clinical trials. Control Clin Trials. 1986;7(3):177–188. doi: 10.1016/0197-2456(86)90046-2.
    1. DeMets DL. Methods for combining randomized clinical trials: strengths and limitations. Stat Med. 1987;6(3):341–348. doi: 10.1002/sim.4780060325.
    1. Cooper H, Hedges LV, Valentine JC. Handbook of Research Synthesis and Meta-Analysis. New York: Russell Sage Foundation; 2009.
    1. Friedrich JO, Adhikari NK, Beyene J. Ratio of means for analyzing continuous outcomes in meta-analysis performed as well as mean difference methods. J Clin Epidemiol. 2011;64(5):556–564. doi: 10.1016/j.jclinepi.2010.09.016.
    1. Friedrich JO, Adhikari NK, Beyene J. Ratio of geometric means to analyze continuous outcomes in meta-analysis: comparison to mean differences and ratio of arithmetic means using empiric data and simulation. Stat Med. 2012;31(17):1857–1886. doi: 10.1002/sim.4501.
    1. Hamilton M. A rating scale for depression. J Neurol Neurosurg Psychiatry. 1960;23:56–62. doi: 10.1136/jnnp.23.1.56.
    1. Ware JE, Snow KK, Kosinski M. SF-36 Health Survey: Manual and Interpretation Guide. London, RI: Quality Metric Incorporated; 1993.
    1. Kunz B, Oranje A, Labreze L, Stalder J-F, Ring J, Taieb A. Clinical validation and guidelines for the SCORAD index: consensus report of the European Task Force on Atopic Dermatitis. Dermatology. 1997;195(1):10–19.
    1. Furukawa TA. From effect size into number needed to treat. Lancet. 1999;353(9165):1680.
    1. da Costa BR, Rutjes AW, Johnston BC, Reichenbach S, Nüesch E, Tonia T, Gemperli A, Guyatt GH, Jüni P. Methods to convert continuous outcomes into odds ratios of treatment response and numbers needed to treat: meta-epidemiological study. Int J Epidemiol. 2012;41(5):1445–1459. doi: 10.1093/ije/dys124.
    1. Guyatt GH, Thorlund K, Oxman AD, Walter SD, Patrick D, Furukawa TA, Johnston BC, Karanicolas P, Akl EA, Vist G. et al.GRADE guidelines: 13 Preparing summary of findings tables and evidence profiles-continuous outcomes. J Clin Epidemiol. 2013;66(2):173–183. doi: 10.1016/j.jclinepi.2012.08.001.
    1. Thorlund K, Walter SD, Johnston BC, Furukawa TA, Guyatt GH. Pooling health‒related quality of life outcomes in meta‒analysis—a tutorial and review of methods for enhancing interpretability. Res Synth Methods. 2011;2(3):188–203. doi: 10.1002/jrsm.46.

Source: PubMed

Спонсоры и соавторы

Заболевания

Медикаментозные вмешательства

Подписаться