Agreement of treatment effects for mortality from routinely collected data and subsequent randomized trials: meta-epidemiological survey

Lars G Hemkens, Despina G Contopoulos-Ioannidis, John P A Ioannidis, Lars G Hemkens, Despina G Contopoulos-Ioannidis, John P A Ioannidis

Abstract

Objective: To assess differences in estimated treatment effects for mortality between observational studies with routinely collected health data (RCD; that are published before trials are available) and subsequent evidence from randomized controlled trials on the same clinical question.

Design: Meta-epidemiological survey.

Data sources: PubMed searched up to November 2014.

Methods: Eligible RCD studies were published up to 2010 that used propensity scores to address confounding bias and reported comparative effects of interventions for mortality. The analysis included only RCD studies conducted before any trial was published on the same topic. The direction of treatment effects, confidence intervals, and effect sizes (odds ratios) were compared between RCD studies and randomized controlled trials. The relative odds ratio (that is, the summary odds ratio of trial(s) divided by the RCD study estimate) and the summary relative odds ratio were calculated across all pairs of RCD studies and trials. A summary relative odds ratio greater than one indicates that RCD studies gave more favorable mortality results.

Results: The evaluation included 16 eligible RCD studies, and 36 subsequent published randomized controlled trials investigating the same clinical questions (with 17,275 patients and 835 deaths). Trials were published a median of three years after the corresponding RCD study. For five (31%) of the 16 clinical questions, the direction of treatment effects differed between RCD studies and trials. Confidence intervals in nine (56%) RCD studies did not include the RCT effect estimate. Overall, RCD studies showed significantly more favorable mortality estimates by 31% than subsequent trials (summary relative odds ratio 1.31 (95% confidence interval 1.03 to 1.65; I(2)=0%)).

Conclusions: Studies of routinely collected health data could give different answers from subsequent randomized controlled trials on the same clinical questions, and may substantially overestimate treatment effects. Caution is needed to prevent misguided clinical decision making.

Conflict of interest statement

Competing interests: All authors have completed the ICMJE uniform disclosure form at www.icmje.org/coi_disclosure.pdf and declare: DCI and JPAI had no financial support for this project; LGH had support from the Commonwealth Fund for the submitted work; all authors declare no financial relationships with any organization that might have an interest in the submitted work in the previous three years and no other relationships or activities that could appear to have influenced the submitted work.

Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://group.bmj.com/group/rights-licensing/permissions.

Figures

https://www.ncbi.nlm.nih.gov/pmc/articles/instance/4772787/bin/heml029712.f1_default.jpg
Fig 1 Study flow diagram. RCT=randomized controlled trial
https://www.ncbi.nlm.nih.gov/pmc/articles/instance/4772787/bin/heml029712.f2_default.jpg
Fig 2 Meta-analyses of comparative effects of medical interventions on mortality reported in randomized controlled trials published after the same clinical question was investigated in RCD studies (part one). For each clinical question investigated in a RCD study, the trials published subsequently are shown. Diamonds=result of meta-analyses combining these subsequent trials as summary odds ratios (using random effects models)
https://www.ncbi.nlm.nih.gov/pmc/articles/instance/4772787/bin/heml029712.f3_default.jpg
Fig 3 Meta-analyses of comparative effects of medical interventions on mortality reported in randomized controlled trials published after the same clinical question was investigated in RCD studies (part two). For each clinical question investigated in a RCD study, the trials published subsequently are shown. Diamonds=result of meta-analyses combining these subsequent trials as summary odds ratios (using random effects models)
https://www.ncbi.nlm.nih.gov/pmc/articles/instance/4772787/bin/heml029712.f4_default.jpg
Fig 4 Treatment effects on mortality in RCD studies and randomized controlled trials. Left panel shows comparative effects of medical interventions on mortality reported in RCD studies and results of subsequently published trials on the same treatment comparisons. White circles=effect estimates reported in RCD studies; blue circles=pooled summary effects from subsequent trials (corresponding meta-analyses are shown in fig 2 and fig 3); lines=95% confidence intervals. Right panel shows for each clinical question the ratio of mortality effects reported in trial evidence versus RCD study effects (as relative odds ratios). Blue squares (lines)=relative odds ratio (95% confidence intervals); blue diamond=pooled summary relative odds ratio (meta-analysis of relative odds ratio) across all clinical questions. A relative odds ratio greater than 1 indicates more favorable mortality outcomes in RCD studies than in subsequent trials

References

    1. Hemkens LG, Contopoulos-Ioannidis DG, Ioannidis JPA. Routinely collected data and comparative effectiveness evidence: promises and limitations. CMAJ [forthcoming].
    1. Rosenbaum PR, Rubin DB. Reducing bias in observational studies using subclassification on the propensity score. J Am Stat Assoc 1984;79:516-24. 10.1080/01621459.1984.10478078. .
    1. Sox HC, Goodman SN. The methods of comparative effectiveness research. Annu Rev Public Health 2012;33:425-45. 10.1146/annurev-publhealth-031811-124610. ..
    1. Hlatky MA, Winkelmayer WC, Setoguchi S. Epidemiologic and statistical methods for comparative effectiveness research. Heart Fail Clin 2013;9:29-36. 10.1016/j.hfc.2012.09.007. ..
    1. Johnson ML, Crown W, Martin BC, Dormuth CR, Siebert U. Good research practices for comparative effectiveness research: analytic methods to improve causal inference from nonrandomized studies of treatment effects using secondary data sources: the ISPOR good research practices for retrospective database analysis task force report—Part III. Value Health 2009;12:1062-73. 10.1111/j.1524-4733.2009.00602.x. ..
    1. Hemkens LG, Contopoulos-Ioannidis DG, Ioannidis JPA. Do routinely collected health data complement randomized evidence? A survey. CMAJ Open [forthcoming].
    1. Safran C. Using routinely collected data for clinical research. Stat Med 1991;10:559-64. 10.1002/sim.4780100407. .
    1. Spasoff RA. Epidemiologic methods for health policy.Oxford University Press, Inc, 1999.
    1. Gliklich R, Dreyer N, Leavy M, eds. Registries for evaluating patient outcomes: a user’s guide. 3rd ed. (Prepared by the Outcome DEcIDE Center [Outcome Sciences, a Quintiles company] under contract no 290 2005 00351 TO7.) AHRQ publication no 13(14)-EHC111. Agency for Healthcare Research and Quality. April 2014. .
    1. National Library of Medicine. MEDLINE / PubMed Search Strategy & Electronic Health Record Information Resources. Secondary MEDLINE / PubMed Search Strategy & Electronic Health Record Information Resources 2015. .
    1. Richardson WS, Wilson MC, Nishikawa J, Hayward RS. The well-built clinical question: a key to evidence-based decisions. ACP J Club 1995;123:A12-3..
    1. The Cochrane Collaboration. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. Secondary Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011] 2011. .
    1. Sterne JAC, Higgins JPT, Reeves BC, et al. A Cochrane risk of bias assessment tool: for non-randomized studies of interventions (ACROBAT-NRSI), version 1.0.0, 24 September 2014, 2014. .
    1. Lau J, Ioannidis JP, Schmid CH. Quantitative synthesis in systematic reviews. Ann Intern Med 1997;127:820-6. 10.7326/0003-4819-127-9-199711010-00008. .
    1. Bradburn MJ, Deeks JJ, Berlin JA, Russell Localio A. Much ado about nothing: a comparison of the performance of meta-analytical methods with rare events. Stat Med 2007;26:53-77. 10.1002/sim.2528. ..
    1. Kim DH, Daskalakis C, Silvestry SC, et al. Aspirin and clopidogrel use in the early postoperative period following on-pump and off-pump coronary artery bypass grafting. J Thorac Cardiovasc Surg 2009;138:1377-84. 10.1016/j.jtcvs.2009.07.027. .
    1. Hahn J-Y, Song YB, Choi J-H, et al. DATE Registry Investigators. Three-month dual antiplatelet therapy after implantation of zotarolimus-eluting stents: the DATE (Duration of Dual Antiplatelet Therapy AfterImplantation of Endeavor Stent) registry. Circ J 2010;74:2314-21. 10.1253/circj.CJ-10-0347. .
    1. Butler MJ, Eccleston D, Clark DJ, et al. Melbourne Interventional Group. The effect of intended duration of clopidogrel use on early and late mortality and major adverse cardiac events in patients with drug-eluting stents. Am Heart J 2009;157:899-907. 10.1016/j.ahj.2009.02.018. .
    1. Holman WL, Li Q, Kiefe CI, et al. Prophylactic value of preincision intra-aortic balloon pump: analysis of a statewide experience. J Thorac Cardiovasc Surg 2000;120:1112-9. 10.1067/mtc.2000.110459. .
    1. Shavelle DM, Parsons L, Sada MJ, French WJ, Every NR. National Registry of Myocardial Infarction 2. Is there a benefit to early angiography in patients with ST-segment depression myocardial infarction? An observational study. Am Heart J 2002;143:488-96. 10.1067/mhj.2002.120970. .
    1. Winkelmayer WC, Glynn RJ, Mittleman MA, Levin R, Pliskin JS, Avorn J. Journal of the American Society of Nephrology. Comparing mortality of elderly patients on hemodialysis versus peritoneal dialysis: a propensity score approach. J Am Soc Nephrol 2002;13:2353-62. 10.1097/01.ASN.0000025785.41314.76. .
    1. Karthik S, Musleh G, Grayson AD, et al. Effect of avoiding cardiopulmonary bypass in non-elective coronary artery bypass surgery: a propensity score analysis. Eur J Cardiothorac Surg 2003;24:66-71. 10.1016/S1010-7940(03)00255-0. .
    1. Guru V, Fremes SE, Tu JV. How many arterial grafts are enough? A population-based study of midterm outcomes. J Thorac Cardiovasc Surg 2006;131:1021-8. 10.1016/j.jtcvs.2005.09.036. .
    1. Wu C, Hannan EL, Walford G, Faxon DP. Utilization and outcomes of unprotected left main coronary artery stenting and coronary artery bypass graft surgery. Ann Thorac Surg 2008;86:1153-9. 10.1016/j.athoracsur.2008.05.059. .
    1. Ascione R, Narayan P, Rogers CA, Lim KH, Capoun R, Angelini GD. Early and midterm clinical outcome in patients with severe left ventricular dysfunction undergoing coronary artery surgery. Ann Thorac Surg 2003;76:793-9. 10.1016/S0003-4975(03)00664-7. .
    1. Polkinghorne KR, McDonald SP, Atkins RC, Kerr PG. Vascular access and all-cause mortality: a propensity score analysis. J Am Soc Nephrol 2004;15:477-86. 10.1097/01.ASN.0000109668.05157.05. .
    1. Gnerlich J, Jeffe DB, Deshpande AD, Beers C, Zander C, Margenthaler JA. Surgical removal of the primary tumor increases overall survival in patients with metastatic breast cancer: analysis of the 1988-2003 SEER data. Ann Surg Oncol 2007;14:2187-94. 10.1245/s10434-007-9438-0. .
    1. Lindenauer PK, Pekow P, Wang K, Gutierrez B, Benjamin EM. Lipid-lowering therapy and in-hospital mortality following major noncardiac surgery. JAMA 2004;291:2092-9. 10.1001/jama.291.17.2092. ..
    1. Cabell CH, Abrutyn E, Fowler VG Jr, et al. International Collaboration on Endocarditis Merged Database (ICE-MD) Study Group Investigators. Use of surgery in patients with native valve infective endocarditis: results from the International Collaboration on Endocarditis Merged Database. Am Heart J 2005;150:1092-8. 10.1016/j.ahj.2005.03.057. .
    1. Moss RR, Humphries KH, Gao M, et al. Outcome of mitral valve repair or replacement: a comparison by propensity score analysis. Circulation 2003;108(Suppl 1):II90-7. 10.1161/. ..
    1. Fonarow GC, Abraham WT, Albert NM, et al. OPTIMIZE-HF Investigators and Coordinators. Influence of beta-blocker continuation or withdrawal on outcomes in patients hospitalized with heart failure: findings from the OPTIMIZE-HF program. J Am Coll Cardiol 2008;52:190-9. 10.1016/j.jacc.2008.03.048. .
    1. Muneretto C, Bisleri G, Negri A, et al. Off-pump coronary artery bypass surgery technique for total arterial myocardial revascularization: a prospective randomized study. Ann Thorac Surg 2003;76:778-82, discussion 783. 10.1016/S0003-4975(03)00564-2. .
    1. Masoumi M, Saidi MR, Rostami F, Sepahi H, Roushani D. Off-pump coronary artery bypass grafting in left ventricular dysfunction. Asian Cardiovasc Thorac Ann 2008;16:16-20. 10.1177/021849230801600105. .
    1. Arogundade FA, Ishola DA Jr, , Sanusi AA, Akinsola A. An analysis of the effectiveness and benefits of peritoneal dialysis and haemodialysis using Nigerian made PD fluids. Afr J Med Med Sci 2005;34:227-33..
    1. Korevaar JC, Feith GW, Dekker FW, et al. NECOSAD Study Group. Effect of starting with hemodialysis compared with peritoneal dialysis in patients new on dialysis treatment: a randomized controlled trial. Kidney Int 2003;64:2222-8. 10.1046/j.1523-1755.2003.00321.x. ..
    1. Thiele H, Rach J, Klein N, et al. LIPSIA-NSTEMI Trial Group. Optimal timing of invasive angiography in stable non-ST-elevation myocardial infarction: the Leipzig Immediate versus early and late PercutaneouS coronary Intervention triAl in NSTEMI (LIPSIA-NSTEMI Trial). Eur Heart J 2012;33:2035-43. 10.1093/eurheartj/ehr418. ..
    1. Montalescot G, Cayla G, Collet JP, et al. ABOARD Investigators. Immediate vs delayed intervention for acute coronary syndromes: a randomized clinical trial. JAMA 2009;302:947-54. 10.1001/jama.2009.1267. ..
    1. Neumann FJ, Kastrati A, Pogatsa-Murray G, et al. Evaluation of prolonged antithrombotic pretreatment (“cooling-off” strategy) before intervention in patients with unstable coronary syndromes: a randomized controlled trial. JAMA 2003;290:1593-9. 10.1001/jama.290.12.1593. ..
    1. Badings EA, The SH, Dambrink JH, et al. Early or late intervention in high-risk non-ST-elevation acute coronary syndromes: results of the ELISA-3 trial. EuroIntervention 2013;9:54-61. 10.4244/EIJV9I1A9. ..
    1. Medved I, Anić D, Ostrić M, Zrnić B, Ivancić A, Tomulić V. Is mitral valve repair safe procedure in elderly patients?Coll Antropol 2010;34(Suppl 2):213-5..
    1. Acker MA, Parides MK, Perrault LP, et al. CTSN. Mitral-valve repair versus replacement for severe ischemic mitral regurgitation. N Engl J Med 2014;370:23-32. 10.1056/NEJMoa1312808. ..
    1. Fattouch K, Guccione F, Dioguardi P, et al. Off-pump versus on-pump myocardial revascularization in patients with ST-segment elevation myocardial infarction: a randomized trial. J Thorac Cardiovasc Surg 2009;137:650-6, discussion 656-7. 10.1016/j.jtcvs.2008.11.033. ..
    1. Dunkelgrun M, Boersma E, Schouten O, et al. Dutch Echocardiographic Cardiac Risk Evaluation Applying Stress Echocardiography Study Group. Bisoprolol and fluvastatin for the reduction of perioperative cardiac mortality and myocardial infarction in intermediate-risk patients undergoing noncardiovascular surgery: a randomized controlled trial (DECREASE-IV). Ann Surg 2009;249:921-6. 10.1097/SLA.0b013e3181a77d00. ..
    1. Durazzo AE, Machado FS, Ikeoka DT, et al. Reduction in cardiovascular events after vascular surgery with atorvastatin: a randomized trial. J Vasc Surg 2004;39:967-75, discussion 975-6. 10.1016/j.jvs.2004.01.004. ..
    1. Schouten O, Boersma E, Hoeks SE, et al. Dutch Echocardiographic Cardiac Risk Evaluation Applying Stress Echocardiography Study Group. Fluvastatin and perioperative events in patients undergoing vascular surgery. N Engl J Med 2009;361:980-9. 10.1056/NEJMoa0808207. ..
    1. Rooijens PP, Burgmans JP, Yo TI, et al. Autogenous radial-cephalic or prosthetic brachial-antecubital forearm loop AVF in patients with compromised vessels? A randomized, multicenter study of the patency of primary hemodialysis access. J Vasc Surg 2005;42:481-6, 487. 10.1016/j.jvs.2005.05.025. ..
    1. Keuter XH, De Smet AA, Kessels AG, van der Sande FM, Welten RJ, Torr JH. A randomized multicenter study of the outcome of brachial-basilic arteriovenous fistula and prosthetic brachial-antecubital forearm loop as vascular access for hemodialysis. J Vasc Surg 2008;47:395-401. 10.1016/j.jvs.2007.09.063. ..
    1. Kang DH, Kim YJ, Kim SH, et al. Early surgery versus conventional treatment for infective endocarditis. N Engl J Med 2012;366:2466-73. 10.1056/NEJMoa1112843. ..
    1. Soran A, Ozmen V, Ozbas S, et alAbstract S2-03: Early follow up of a randomized trial evaluating resection of the primary breast tumor in women presenting with de novo stage IV breast cancer; Turkish study (protocol MF07-01). Cancer Res 2013;73(24 suppl):S2-03-S2-03.
    1. Badwe R, Parmar V, Hawaldar R, et alAbstract S2-02: Surgical removal of primary tumor and axillary lymph nodes in women with metastatic breast cancer at first presentation: A randomized controlled trial. Cancer Res 2013;73(24 suppl):S2-02-S2-02.
    1. Damgaard S, Wetterslev J, Lund JT, et al. One-year results of total arterial revascularization vs. conventional coronary surgery: CARRPO trial. Eur Heart J 2009;30:1005-11. 10.1093/eurheartj/ehp048. ..
    1. Morice MC, Serruys PW, Kappetein AP, et al. Outcomes in patients with de novo left main disease treated with either percutaneous coronary intervention using paclitaxel-eluting stents or coronary artery bypass graft treatment in the Synergy Between Percutaneous Coronary Intervention with TAXUS and Cardiac Surgery (SYNTAX) trial. Circulation 2010;121:2645-53. 10.1161/CIRCULATIONAHA.109.899211. ..
    1. Boudriot E, Thiele H, Walther T, et al. Randomized comparison of percutaneous coronary intervention with sirolimus-eluting stents versus coronary artery bypass grafting in unprotected left main stem stenosis. J Am Coll Cardiol 2011;57:538-45. 10.1016/j.jacc.2010.09.038. ..
    1. Park SJ, Kim YH, Park DW, et al. Randomized trial of stents versus bypass surgery for left main coronary artery disease. N Engl J Med 2011;364:1718-27. 10.1056/NEJMoa1100452. ..
    1. Ranucci M, Castelvecchio S, Biondi A, et al. Surgical and Clinical Outcome Research (SCORE) Group. A randomized controlled trial of preoperative intra-aortic balloon pump in coronary patients with poor left ventricular function undergoing coronary artery bypass surgery. Crit Care Med 2013;41:2476-83. 10.1097/CCM.0b013e3182978dfc. ..
    1. Lomivorotov VV, Boboshko VA, Efremov SM, et al. Levosimendan versus an intra-aortic balloon pump in high-risk cardiac patients. J Cardiothorac Vasc Anesth 2012;26:596-603. 10.1053/j.jvca.2011.09.006. ..
    1. Sun JC, Teoh KH, Lamy A, et al. Randomized trial of aspirin and clopidogrel versus aspirin alone for the prevention of coronary artery bypass graft occlusion: the Preoperative Aspirin and Postoperative Antiplatelets in Coronary Artery Bypass Grafting study. Am Heart J 2010;160:1178-84. 10.1016/j.ahj.2010.07.035. ..
    1. Mannacio VA, Di Tommaso L, Antignan A, De Amicis V, Vosa C. Aspirin plus clopidogrel for optimal platelet inhibition following off-pump coronary artery bypass surgery: results from the CRYSSA (prevention of Coronary arteRY bypaSS occlusion After off-pump procedures) randomised study. Heart 2012;98:1710-5. 10.1136/heartjnl-2012-302449. ..
    1. Gao G, Zheng Z, Pi Y, Lu B, Lu J, Hu S. Aspirin plus clopidogrel therapy increases early venous graft patency after coronary artery bypass surgery a single-center, randomized, controlled trial. J Am Coll Cardiol 2010;56:1639-43. 10.1016/j.jacc.2010.03.104. ..
    1. Kulik A, Le May MR, Voisine P, et al. Aspirin plus clopidogrel versus aspirin alone after coronary artery bypass grafting: the clopidogrel after surgery for coronary artery disease (CASCADE) Trial. Circulation 2010;122:2680-7. 10.1161/CIRCULATIONAHA.110.978007. ..
    1. Gasparovic H, Petricevic M, Kopjar T, Djuric Z, Svetina L, Biocina B. Impact of dual antiplatelet therapy on outcomes among aspirin-resistant patients following coronary artery bypass grafting. Am J Cardiol 2014;113:1660-7. 10.1016/j.amjcard.2014.02.024. ..
    1. Valgimigli M, Borghesi M, Tebaldi M, Vranckx P, Parrinello G, Ferrari R. PROlonging Dual antiplatelet treatment after Grading stent-induced Intimal hyperplasia studY Investigators. Should duration of dual antiplatelet therapy depend on the type and/or potency of implanted stent? A pre-specified analysis from the PROlonging Dual antiplatelet treatment after Grading stent-induced Intimal hyperplasia studY (PRODIGY). Eur Heart J 2013;34:909-19. 10.1093/eurheartj/ehs460. ..
    1. Gwon HC, Hahn JY, Park KW, et al. Six-month versus 12-month dual antiplatelet therapy after implantation of drug-eluting stents: the Efficacy of Xience/Promus Versus Cypher to Reduce Late Loss After Stenting (EXCELLENT) randomized, multicenter study. Circulation 2012;125:505-13. 10.1161/CIRCULATIONAHA.111.059022. ..
    1. Kim BK, Hong MK, Shin DH, et al. RESET Investigators. A new strategy for discontinuation of dual antiplatelet therapy: the RESET Trial (REal Safety and Efficacy of 3-month dual antiplatelet Therapy following Endeavor zotarolimus-eluting stent implantation). J Am Coll Cardiol 2012;60:1340-8. 10.1016/j.jacc.2012.06.043. ..
    1. Feres F, Costa RA, Abizaid A, et al. OPTIMIZE Trial Investigators. Three vs twelve months of dual antiplatelet therapy after zotarolimus-eluting stents: the OPTIMIZE randomized trial. JAMA 2013;310:2510-22. 10.1001/jama.2013.282183. ..
    1. Colombo A, Chieffo A, Frasheri A, et al. Second-generation drug-eluting stent implantation followed by 6- versus 12-month dual antiplatelet therapy: the SECURITY randomized clinical trial. J Am Coll Cardiol 2014;64:2086-97. 10.1016/j.jacc.2014.09.008. ..
    1. Jondeau G, Neuder Y, Eicher JC, et al. B-CONVINCED Investigators. B-CONVINCED: Beta-blocker CONtinuation Vs. INterruption in patients with Congestive heart failure hospitalizED for a decompensation episode. Eur Heart J 2009;30:2186-92. 10.1093/eurheartj/ehp323. ..
    1. Anglemyer A, Horvath HT, Bero L. Healthcare outcomes assessed with observational study designs compared with those assessed in randomized trials. Cochrane Database Syst Rev 2014;4MR000034. 10.1002/14651858.MR000034.pub2. ..
    1. Lonjon G, Boutron I, Trinquart L, et al. Comparison of treatment effect estimates from prospective nonrandomized studies with propensity score analysis and randomized controlled trials of surgical procedures. Ann Surg 2014;259:18-25. 10.1097/SLA.0000000000000256. ..
    1. Kuss O, Legler T, Börgermann J. Treatments effects from randomized trials and propensity score analyses were similar in similar populations in an example from cardiac surgery. J Clin Epidemiol 2011;64:1076-84. 10.1016/j.jclinepi.2011.01.005. ..
    1. Dahabreh IJ, Sheldrick RC, Paulus JK, et al. Do observational studies using propensity score methods agree with randomized trials? A systematic comparison of studies on acute coronary syndromes. Eur Heart J 2012;33:1893-901. 10.1093/eurheartj/ehs114. ..
    1. Pereira TV, Horwitz RI, Ioannidis JP. Empirical evaluation of very large treatment effects of medical interventions. JAMA 2012;308:1676-84. 10.1001/jama.2012.13444. ..
    1. Hemkens LG, Contopoulos-Ioannidis DG, Ioannidis JP. Concordance of effects of medical interventions on hospital admission and readmission rates with effects on mortality. CMAJ 2013;85:E827-37. 10.1503/cmaj.130430. .
    1. Pereira TV, Ioannidis JP. Statistically significant meta-analyses of clinical trials have modest credibility and inflated effects. J Clin Epidemiol 2011;64:1060-9. 10.1016/j.jclinepi.2010.12.012. ..
    1. Ioannidis JP, Haidich AB, Lau J. Any casualties in the clash of randomised and observational evidence?BMJ 2001;322:879-80. 10.1136/bmj.322.7291.879. .
    1. Ioannidis JP, Haidich AB, Pappa M, et al. Comparison of evidence of treatment effects in randomized and nonrandomized studies. JAMA 2001;286:821-30. 10.1001/jama.286.7.821. .
    1. Yusuf S, Collins R, Peto R. Why do we need some large, simple randomized trials?Stat Med 1984;3:409-22. 10.1002/sim.4780030421. .
    1. Ioannidis JP. Mega-trials for blockbusters. JAMA 2013;309:239-40. 10.1001/jama.2012.168095. ..

Source: PubMed

Patrocinadores e Colaboradores

Condições médicas

Intervenções de drogas

3
Se inscrever