Planning future studies based on the conditional power of a meta-analysis
Verena Roloff, Julian P T Higgins, Alex J Sutton, Verena Roloff, Julian P T Higgins, Alex J Sutton
Abstract
Systematic reviews often provide recommendations for further research. When meta-analyses are inconclusive, such recommendations typically argue for further studies to be conducted. However, the nature and amount of future research should depend on the nature and amount of the existing research. We propose a method based on conditional power to make these recommendations more specific. Assuming a random-effects meta-analysis model, we evaluate the influence of the number of additional studies, of their information sizes and of the heterogeneity anticipated among them on the ability of an updated meta-analysis to detect a prespecified effect size. The conditional powers of possible design alternatives can be summarized in a simple graph which can also be the basis for decision making. We use three examples from the Cochrane Database of Systematic Reviews to demonstrate our strategy. We demonstrate that if heterogeneity is anticipated, it might not be possible for a single study to reach the desirable power no matter how large it is.
Copyright © 2012 John Wiley & Sons, Ltd.
Figures
References
- Sutton A, Cooper N, Jones D, Lambert P, Thompson J, Abrams K. Evidence-based sample size calculations based upon updated meta-analysis. Statistics in Medicine. 2007;26:2479–2500. DOI: .
- Higgins JPT, Whitehead A, Simmonds M. Sequential methods for random-effects meta-analysis. Statistics in Medicine. 2011;30(9):903–921.
- Chow SC, Chang M. Adaptive Design Methods in Clinical Trials. Boca Raton, FL: Chapman & Hall/CRC; 2006.
- Clarke L, Clarke M, Clarke T. How useful are Cochrane reviews in identifying research needs? Journal of Health Services Research Policy. 2007;12:101–103. DOI: .
- Sutton AJ, Donegan S, Takwoingi Y, Garner P, Gamble C, Donald A. An encouraging assessment of methods to inform priorities for updating systematic reviews. Journal of Clinical Epidemiology. 2009;62:241–251. DOI: .
- Clarke M, Hopewell S, Chalmers I. Reports of clinical trials should begin and end with up-to-date systematic reviews of other relevant evidence: a status report. Journal of the Royal Society of Medicine. 2007;100:187–190. DOI: .
- MRC Additional Requirements for Trial Grants . Available from: [Accessed on 10-5-2012]
- Young C, Horton R. Putting clinical trials into context. The Lancet. 2005;366:107–108. DOI: .
- Wetterslev J, Thorlund K, Brok J, Gluud C. Estimating required information size by quantifying diversity in random-effects model meta-analyses. BMC Medical Research Methodology. 2009;9:86. DOI: .
- Whitehead A. Meta-Analysis of Controlled Clinical Trials. Chichester, U.K: John Wiley & Sons, Ltd; 2002.
- Hedges LV, Pigott T. The power of statistical tests in meta-analysis. Psychological Methods. 2001;6:203–217. DOI: .
- Julious SA. Sample sizes for clinical trials with Normal data. Statistics in Medicine. 2004;23:1921–1986. DOI: .
- Borenstein M, Hedges LV, Higgins JPT, Rothstein HR. Introduction to Meta Analysis. Chichester, U.K: Wiley; 2009.
- Spiegelhalter DJ, Abrams KR, Myles JP. Bayesian Approaches to Clinical Trials and Health-care Evaluation. Chichester, U.K: Wiley; 2003.
- Denne JS. Sample size recalculation using conditional power. Statistics in Medicine. 2001;20:2645–2660. DOI: .
- Jennison C, Turnbull BW. Group Sequential Methods with Applications to Clinical Trials. Boca Raton, FL: Chapman and Hall/CRC; 1999.
- Verschuur HP, de WW, van-Benthem PP. Antibiotic prophylaxis in clean and clean-contaminated ear surgery. Cochrane Database of Systematic Reviews. 2004 DOI: .
- Vogt K, Fenlon D, Rhodes S, Malthaner R. Preoperative chemotherapy for resectable thoracic esophageal cancer. Cochrane Database of Systematic Reviews. 2006 DOI: .
- Wilson DR, Torres LI, Durham SR. Sublingual immunotherapy for allergic rhinitis. Cochrane Database of Systematic Reviews. 2003 DOI: .
- Parmar MK, Torri V, Stewart L. Extracting summary statistics to perform meta-analyses of the published literature for survival endpoints. Statistics in Medicine. 1998;17:2815–2834. DOI: .
- O'Hagan A, Stevens JW, Campbell MJ. Assurance in clinical trial design. Pharmaceutical Statistics. 2005;4:187–201. DOI: .
- Devereaux PJ, Beattie WS, Choi PT-L, Badner NH, Guyatt GH, Villar JC, Cinà CS, Leslie K, Jacka MJ, Montori VM, Bhandari M, Avezum A, Cavalcanti AB, Giles JW, Schricker T, Yang H, Jakobsen C-J, Yusuf S. How strong is the evidence for the use of perioperative beta blockers in non-cardiac surgery? Systematic review and meta-analysis of randomised controlled trials. BMJ. 2005;331:313–321. DOI: .
- Wetterslev J, Thorlund K, Brok J, Gluud C. Trial sequential analysis may establish when firm evidence is reached in cumulative meta-analysis. Journal of Clinical Epidemiology. 2008;61:64–75. DOI: .
- Hu M, Cappelleri JC, Lan KKG. Applying the law of iterated logarithm to control type I error in cumulative meta-analysis of binary outcomes. Clinical Trials. 2007;4:329–340. DOI: .
- Pogue JM, Yusuf S. Cumulating evidence from randomized trials: utilizing sequential monitoring boundaries for cumulative meta-analysis. Controlled Clinical Trials. 1997;18:580–593. DOI: .
- Bauer P, Kohne K. Evaluation of experiments with adaptive interim analyses. Biometrics. 1994;50:1029–1041.
- Proschan MA, Hunsberger SA. Designed extension of studies based on conditional power. Biometrics. 1995;51:1315–1324.
- Müller HH, Schäfer H. A general statistical principle for changing a design any time during the course of a trial. Statistics in Medicine. 2004;23:2497–2508. DOI: .
- Julious S, Wang S. How biased are indirect comparisons, particularly when comparisons are made over time in controlled trials? Drug Information Journal. 2011;42:625–633.
- Julious S, Tan SB, Machin D. An Introduction to Statistics in Early Phase Trials. Chichester, U.K: John Wiley & Sons; 2010.
- Claxton K, Ginnelly L, Sculpher M, Philips Z, Palmer S. A pilot study on the use of decision theory and value of information analysis as part of the NHS Health Technology Assessment programme. Health Technology Assessment. 2004;8:1–103.
- Claxton K, Sculpher M, Drummond M. A rational framework for decision making by the National Institute For Clinical Excellence (NICE) The Lancet. 2002;360:711–715. DOI: .
Source: PubMed