Planning future studies based on the conditional power of a meta-analysis

Verena Roloff, Julian P T Higgins, Alex J Sutton, Verena Roloff, Julian P T Higgins, Alex J Sutton

Abstract

Systematic reviews often provide recommendations for further research. When meta-analyses are inconclusive, such recommendations typically argue for further studies to be conducted. However, the nature and amount of future research should depend on the nature and amount of the existing research. We propose a method based on conditional power to make these recommendations more specific. Assuming a random-effects meta-analysis model, we evaluate the influence of the number of additional studies, of their information sizes and of the heterogeneity anticipated among them on the ability of an updated meta-analysis to detect a prespecified effect size. The conditional powers of possible design alternatives can be summarized in a simple graph which can also be the basis for decision making. We use three examples from the Cochrane Database of Systematic Reviews to demonstrate our strategy. We demonstrate that if heterogeneity is anticipated, it might not be possible for a single study to reach the desirable power no matter how large it is.

Copyright © 2012 John Wiley & Sons, Ltd.

Figures

Figure 1
Figure 1
Forest plots of meta-analyses for the examples used in the paper: (a) antibiotic prophylaxis in ear surgery (fixed-effect meta-analysis) , (b) preoperative chemotherapy for oesophageal cancer (random-effects meta-analysis) and (c) sublingual immunotherapy for allergic rhinitis (random-effects meta-analysis) . OR, odds ratio; HR, hazard ratio; SMD, standardized mean difference.
Figure 2
Figure 2
Conditional power of an updated meta-analysis to detect an odds ratio of 0.61, having observed an odds ratio of 0.73 (95% CI 0.44 to 1.2), for a meta-analysis of antibiotics for postoperative infection within three weeks after ear surgery , assuming that the additional study does not introduce heterogeneity.
Figure 3
Figure 3
Conditional power of an updated meta-analysis to detect a hazard ratio of 0.82, having observed a hazard ratio of 0.88 (95% CI 0.75 to 1.04), for a meta-analysis of overall survival after preoperative chemotherapy for oesophageal cancer , assuming that is equal to that in the previous studies. Different curves represent different numbers of future studies (m).
Figure 4
Figure 4
Contours for a conditional power of 90% for an updated meta-analysis to detect a hazard ratio of 0.82, having observed a hazard ratio of 0.88 (95% CI 0.75 to 1.04], based on a meta-analysis of overall survival after preoperative chemotherapy versus surgery for oesophageal cancer . The graph illustrates the relationship between additional information size (expressed as number of additional events) and number of planned studies for four values of future heterogeneity .
Figure 5
Figure 5
Conditional power of an updated meta-analysis to detect a standardized mean difference of − 0.5, having observed a standardized mean difference of − 0.58 (95% CI − 1.43 to 0.27), for a meta-analysis of allergic rhinitis symptom scores after sublingual immunotherapy for allergic rhinitis caused by house dust mites , assuming that is equal to that in the previous studies. Different curves represent different numbers of future studies (m).
Figure 6
Figure 6
Contours for conditional power of 90% for an updated meta-analysis to detect a standardized mean difference of − 0.5, having observed a standardized mean difference of − 0.58 (95% CI − 1.43 to 0.27), based on a meta-analysis of allergic rhinitis symptom scores after sublingual immunotherapy for allergic rhinitis caused by house dust mites . The graph illustrates the relationship between additional information size (expressed as sample size) and number of planned studies for three values of future heterogeneity, .
Figure 7
Figure 7
Precision of an updated meta-analysis (expressed as relative reduction in confidence interval width) for a meta-analysis of allergic rhinitis symptom scores after sublingual immunotherapy for allergic rhinitis caused by house dust mites , assuming that is equal to that in the previous studies. Different curves represent different numbers of future studies (m).

References

    1. Sutton A, Cooper N, Jones D, Lambert P, Thompson J, Abrams K. Evidence-based sample size calculations based upon updated meta-analysis. Statistics in Medicine. 2007;26:2479–2500. DOI: .
    1. Higgins JPT, Whitehead A, Simmonds M. Sequential methods for random-effects meta-analysis. Statistics in Medicine. 2011;30(9):903–921.
    1. Chow SC, Chang M. Adaptive Design Methods in Clinical Trials. Boca Raton, FL: Chapman & Hall/CRC; 2006.
    1. Clarke L, Clarke M, Clarke T. How useful are Cochrane reviews in identifying research needs? Journal of Health Services Research Policy. 2007;12:101–103. DOI: .
    1. Sutton AJ, Donegan S, Takwoingi Y, Garner P, Gamble C, Donald A. An encouraging assessment of methods to inform priorities for updating systematic reviews. Journal of Clinical Epidemiology. 2009;62:241–251. DOI: .
    1. Clarke M, Hopewell S, Chalmers I. Reports of clinical trials should begin and end with up-to-date systematic reviews of other relevant evidence: a status report. Journal of the Royal Society of Medicine. 2007;100:187–190. DOI: .
    1. MRC Additional Requirements for Trial Grants . Available from: [Accessed on 10-5-2012]
    1. Young C, Horton R. Putting clinical trials into context. The Lancet. 2005;366:107–108. DOI: .
    1. Wetterslev J, Thorlund K, Brok J, Gluud C. Estimating required information size by quantifying diversity in random-effects model meta-analyses. BMC Medical Research Methodology. 2009;9:86. DOI: .
    1. Whitehead A. Meta-Analysis of Controlled Clinical Trials. Chichester, U.K: John Wiley & Sons, Ltd; 2002.
    1. Hedges LV, Pigott T. The power of statistical tests in meta-analysis. Psychological Methods. 2001;6:203–217. DOI: .
    1. Julious SA. Sample sizes for clinical trials with Normal data. Statistics in Medicine. 2004;23:1921–1986. DOI: .
    1. Borenstein M, Hedges LV, Higgins JPT, Rothstein HR. Introduction to Meta Analysis. Chichester, U.K: Wiley; 2009.
    1. Spiegelhalter DJ, Abrams KR, Myles JP. Bayesian Approaches to Clinical Trials and Health-care Evaluation. Chichester, U.K: Wiley; 2003.
    1. Denne JS. Sample size recalculation using conditional power. Statistics in Medicine. 2001;20:2645–2660. DOI: .
    1. Jennison C, Turnbull BW. Group Sequential Methods with Applications to Clinical Trials. Boca Raton, FL: Chapman and Hall/CRC; 1999.
    1. Verschuur HP, de WW, van-Benthem PP. Antibiotic prophylaxis in clean and clean-contaminated ear surgery. Cochrane Database of Systematic Reviews. 2004 DOI: .
    1. Vogt K, Fenlon D, Rhodes S, Malthaner R. Preoperative chemotherapy for resectable thoracic esophageal cancer. Cochrane Database of Systematic Reviews. 2006 DOI: .
    1. Wilson DR, Torres LI, Durham SR. Sublingual immunotherapy for allergic rhinitis. Cochrane Database of Systematic Reviews. 2003 DOI: .
    1. Parmar MK, Torri V, Stewart L. Extracting summary statistics to perform meta-analyses of the published literature for survival endpoints. Statistics in Medicine. 1998;17:2815–2834. DOI: .
    1. O'Hagan A, Stevens JW, Campbell MJ. Assurance in clinical trial design. Pharmaceutical Statistics. 2005;4:187–201. DOI: .
    1. Devereaux PJ, Beattie WS, Choi PT-L, Badner NH, Guyatt GH, Villar JC, Cinà CS, Leslie K, Jacka MJ, Montori VM, Bhandari M, Avezum A, Cavalcanti AB, Giles JW, Schricker T, Yang H, Jakobsen C-J, Yusuf S. How strong is the evidence for the use of perioperative beta blockers in non-cardiac surgery? Systematic review and meta-analysis of randomised controlled trials. BMJ. 2005;331:313–321. DOI: .
    1. Wetterslev J, Thorlund K, Brok J, Gluud C. Trial sequential analysis may establish when firm evidence is reached in cumulative meta-analysis. Journal of Clinical Epidemiology. 2008;61:64–75. DOI: .
    1. Hu M, Cappelleri JC, Lan KKG. Applying the law of iterated logarithm to control type I error in cumulative meta-analysis of binary outcomes. Clinical Trials. 2007;4:329–340. DOI: .
    1. Pogue JM, Yusuf S. Cumulating evidence from randomized trials: utilizing sequential monitoring boundaries for cumulative meta-analysis. Controlled Clinical Trials. 1997;18:580–593. DOI: .
    1. Bauer P, Kohne K. Evaluation of experiments with adaptive interim analyses. Biometrics. 1994;50:1029–1041.
    1. Proschan MA, Hunsberger SA. Designed extension of studies based on conditional power. Biometrics. 1995;51:1315–1324.
    1. Müller HH, Schäfer H. A general statistical principle for changing a design any time during the course of a trial. Statistics in Medicine. 2004;23:2497–2508. DOI: .
    1. Julious S, Wang S. How biased are indirect comparisons, particularly when comparisons are made over time in controlled trials? Drug Information Journal. 2011;42:625–633.
    1. Julious S, Tan SB, Machin D. An Introduction to Statistics in Early Phase Trials. Chichester, U.K: John Wiley & Sons; 2010.
    1. Claxton K, Ginnelly L, Sculpher M, Philips Z, Palmer S. A pilot study on the use of decision theory and value of information analysis as part of the NHS Health Technology Assessment programme. Health Technology Assessment. 2004;8:1–103.
    1. Claxton K, Sculpher M, Drummond M. A rational framework for decision making by the National Institute For Clinical Excellence (NICE) The Lancet. 2002;360:711–715. DOI: .

Source: PubMed

赞助者和合作者

医疗条件

药物干预

3
订阅