Use of primary care electronic medical record database in drug efficacy research on cardiovascular outcomes: comparison of database and randomised controlled trial findings

Richard L Tannen, Mark G Weiner, Dawei Xie, Richard L Tannen, Mark G Weiner, Dawei Xie

Abstract

Objectives: To determine whether observational studies that use an electronic medical record database can provide valid results of therapeutic effectiveness and to develop new methods to enhance validity.

Design: Data from the UK general practice research database (GPRD) were used to replicate previously performed randomised controlled trials, to the extent that was feasible aside from randomisation. Studies Six published randomised controlled trials.

Main outcome measure: Cardiovascular outcomes analysed by hazard ratios calculated with standard biostatistical methods and a new analytical technique, prior event rate ratio (PERR) adjustment.

Results: In nine of 17 outcome comparisons, there were no significant differences between results of randomised controlled trials and database studies analysed using standard biostatistical methods or PERR analysis. In eight comparisons, Cox adjusted hazard ratios in the database differed significantly from the results of the randomised controlled trials, suggesting unmeasured confounding. In seven of these eight, PERR adjusted hazard ratios differed significantly from Cox adjusted hazard ratios, whereas in five they didn't differ significantly, and in three were more similar to the hazard ratio from the randomised controlled trial, yielding PERR results more similar to the randomised controlled trial than Cox (P<0.05).

Conclusions: Although observational studies using databases are subject to unmeasured confounding, our new analytical technique (PERR), applied here to cardiovascular outcomes, worked well to identify and reduce the effects of such confounding. These results suggest that electronic medical record databases can be useful to investigate therapeutic effectiveness.

Conflict of interest statement

Competing interests: None declared.

Figures

https://www.ncbi.nlm.nih.gov/pmc/articles/instance/4795668/bin/tanr576298.f1_default.jpg
Fig 1 Selection process for participants in database studies
https://www.ncbi.nlm.nih.gov/pmc/articles/instance/4795668/bin/tanr576298.f2_default.jpg
Fig 2 Comparisons between hazard ratios from randomised controlled trials (RCT) and adjusted hazard ratios for respective database studies. Data plotted as natural logarithms, so 0 on x axis indicates no difference between exposed and unexposed cohort. Database adjusted hazard ratios shown with both Cox and prior event rate ratio (PERR) adjustment analysis. Results are shown for myocardial infarction, stroke, and coronary revascularisation (CABG/PTCA). GPRD=general practice research database

References

    1. Britton A, McKee M, Black N, McPherson K, Sanderson C, Bain C. Choosing between randomised and non-randomised studies: a systematic review. Health Technol Assess 1998;2:1-124.
    1. MacLehose RR, Reeves BC, Harvey IM, Sheldon TA, Russell IT, Black AMS. A systematic review of comparisons of effect-sizes derived from randomized and non-randomised studies. Health Technol Assess 2000;4:1-154.
    1. Benson K, Hartz AJ. A comparison of observational studies and randomized controlled trials. N Engl J Med 2000;342:1878-86.
    1. Concato J, Horwitz RI. Beyond randomized versus observational studies. Lancet 2004;363:1660-1.
    1. Sacks HS. Observational studies and randomized trials. N Engl J Med 2000;343:1195.
    1. Ioannidis JPA, Haidich AB, Lau J. Any casualties in the clash of randomised and observational evidence? BMJ 2001;322:879-80.
    1. Avorn J. In defense of pharmacoepidemiology—embracing the yin and yang of drug research. N Engl J Med 2007;357:2219-21.
    1. Ioannidis JPA, Haidich AB, Pappa M, Pantazis N, Kokori SI, Tektonidou MG, et al. Comparison of evidence of treatment effects in randomized and nonrandomized studies. JAMA 2001;286:821-30.
    1. Black N. Why we need observational studies to evaluate the effectiveness of health care. BMJ 1996;312:1215-8.
    1. Kunz R, Oxman AD. The unpredictability paradox: review of empirical comparisons of randomised and non-randomised clinical trials. BMJ 1998;317:1185-90.
    1. Popcock SJ, Elbourne DR. Randomized trials or observational tribulations? N Engl J Med 2000;342:1907-9.
    1. McMahon AD. Observation and experiment with the efficacy of drugs: a warning example from a cohort of nonsteroidal anti-inflammatory and ulcer-healing drug users. Am J Epidemiol 2001;154:557-62.
    1. MacMahon S, Collins R. Reliable assessment of the effects of treatment on mortality and major morbidity, II: observational studies. Lancet 2001;357:455-6.
    1. Califf RM, Pryor DB, Greenfield J Jr. Beyond randomized clinical trials: applying clinical experience in the treatment of patients with coronary artery disease. Circulation 1986;74:1191-4.
    1. Deeks JJ, Dinnes J, D’Amico RA, Sowden AJ, Sakarovitch C, Song F, Petticrew M, Altman DG. Evaluating non-randomised intervention studies. Health Technol Assess 2003;7:1-173.
    1. McKee M, Britton A, Black N, McPherson K, Sanderson C, Bain C. Interpreting the evidence choosing between randomised and non-randomised studies. BMJ 1999;319:312-5.
    1. Concato J, Shah N, Horwitz RI. Randomized, controlled trials, observational studies, and the hierarchy of research designs. N Engl J Med 2000;342:1887-92.
    1. Grodstein F, Stampfer MJ, Manson JE, Colditz GA, Willett WC, Rosner B, et al. Postmenopausal estrogen and progestin use and the risk of cardiovascular disease. J Engl J Med 1996;335:453-61.
    1. Grodstein F, Manson JE, Colditz GA, Willett WC, Speizer FE, Stampfel MJ. A prospective, observational study of postmenopausal hormone therapy and primary prevention of cardiovascular disease. Ann Intern Med 2000;133:933-41.
    1. Writing Group for the Women’s Health Initiative Investigators. Risks and benefits of estrogen plus progestin in health postmenopausal women. Principal results from the Women’s Health Initiative randomized controlled trial. JAMA 2002;288:321-33.
    1. Vandenbroucke JP. When are observational studies as credible as randomized trials? Lancet 2004;363:1728-31.
    1. Coronary artery surgery study (CASS): a randomized trial of coronary bypass surgery. Comparability of entry characteristics and survival in randomized patients and nonrandomized patients meeting randomization criteria. J Am Coll Cardiol 1984;3:114-28.
    1. Horwitz RI, Viscoli CM, Clemens JD, Sadock RT. Developing improved observational methods for evaluating therapeutic effectiveness. Am J Med 1990;89:630-8
    1. Gelfand JM, Margolis DJ, Dattani H. The UK general practice research database. In: Strom BL, ed. Pharmacoepidemiology. 4th ed. New York: John Wiley, 2005:337-46.
    1. Tannen RL, Weiner MG, Marcus SM. Simulation of the Syst-Eur randomized control trial using a primary care electronic medical record was feasible. J Clin Epidemiol 2006;59:254-64.
    1. Tannen RL, Weiner MG, Xie D, Barnhart K. A simulation using data from a primary care practice database closely replicated the women’s health initiative trial. J Clin Epidemiol 2007;60:686-95.
    1. Tannen RL, Weiner MG, Xie D, Barnhart K. Estrogen affects post-menopausal women differently than estrogen plus progestin therapy. Hum Reprod 2007;22:1769-77.
    1. Weiner MG, Xie D, Tannen RL. Replication of the Scandinavian Simvastatin Survival Study using a primary care medical record database prompted exploration of a new method to address unmeasured confounding. Pharmacoepidemiol Drug Saf 2008;17:661-70.
    1. Tannen RL, Weiner MG, Xie D. Replicated studies of two randomized trials of angiotensin converting enzyme inhibitors: further empiric validation of the “prior event rate ratio” to adjust for unmeasured confounding by indication. Pharmacoepidemiol Drug Saf 2008;17:671-85.
    1. Staessen JA, Fagard R, Thijs L, Celis H, Arabidze GG, Birkenhager WH, et al. Randomised double-blind comparison of placebo and active treatment for older patients with isolated systolic hypertension. Lancet 1997;350:757-64.
    1. Women’s Health Initiative Steering Committee. Effects of conjugated equine estrogen in postmenopausal women with hysterectomy. The Women’s Health Initiative randomized controlled trial. JAMA 2004;291:1701-12.
    1. Scandinavian Simvastatin Survival Study Group. Randomised trial of cholesterol lowering in 4444 patients with coronary heart disease: the Scandinavian Simvastatin survival study (4S). Lancet 1994;344:1383-9.
    1. Heart Outcomes Prevention Evaluation Study Investigators. Effects of an angiotensin-converting-enzyme inhibitor, ramipril, on cardiovascular events in high risk patients. N Engl J Med 2000;342:145-53.
    1. European trial on reduction of cardiac events with Perindopril in stable coronary artery disease investigators. Efficacy of perindopril in reduction of cardiovascular events among patients with stable coronary artery disease: randomized double-blind, placebo-controlled, multicentre trial (the EUROPA study). Lancet 2003;362:782-8.
    1. Raghunathan TE, Solenberger PW, Van Hoewyk J. IVEware: imputation and variance estimation software. Ann Arbor, MI: Institute for Social Research, University of Michigan, 1998.
    1. Rubin D. Multiple imputation for non-response in surveys. New York: John Wiley, 1987.
    1. Raghunathan TE, Lepkowski JM, Van Hoewyk J, Solenberger PW. A multivariate technique for multiply imputing missing values using a sequence of regression models. Surv Methodol 2001;27:85-95.
    1. Rosenbaum P, Rubin D. The central role of the propensity score in observational studies for causal effects. Biometrika 1983;70:41-55.
    1. Rosenbaum P, Rubin D. Reducing bias in observational studies using subclassification on the propensity score. J Am Stat Assoc 1984;79:516-24.
    1. D’Agostino RB, Rubin DB, Estimating and using propensity scores with partially missing data. J Am Stat Assoc 2000;95:749-59.
    1. Rossouw JE, Prentice RL, Manson JE, Wu L, Barad D, Barnabei VM, et al. Postmenopausal hormone therapy and risk of cardiovascular disease by age and years since menopause. JAMA 2007;297:1465-77.
    1. McMahon AD. Approaches to combat confounding by indication in observational studies of intended drug effects. Pharmacoepidemiol Drug Saf 2003;12:551-8.
    1. Rosenbaum PR. Discussing hidden bias in observational studies. Ann Intern Med 1991;115:901-5.
    1. Rubin DB. Estimating causal effects from large data sets using propensity scores. Ann Intern Med 1997;127:757-63.
    1. Braitman LE, Rosenbaum PR. Rare outcomes, common treatments: analytic strategies using propensity scores. Ann Intern Med 2002;137:693-5.
    1. Sturmer T, Schneeweiss, Rothman KJ, Avorn J, Glynn RJ. Performance of propensity score calibration. A simulation study. Am J Epidemiol 2007;165:1110-8.
    1. Oakes JM, Church TR. Invited commentary: advancing propensity study methods in epidemiology. Am J Epidemiol 2007;165:1119-21.
    1. McClellan M, McNeil BJ, Newhouse JP. Does more intensive treatment of acute myocardial infarction in the elderly reduce mortality? Analysis using instrumental variables. JAMA 1994;272:859-66.
    1. Newhouse JP, McClellan M. Econometrics in outcomes research: the use of instrumental variables. Annu Rev Public Health 1998;19:17-34.
    1. Sukel TA, Fisher ES, Wennberg DE, Alter DA, Gottleib DJ, Vermeulen MJ. Analysis of observational studies in the presence of treatment selection bias. Effect of invasive cardiac management on AMI survival using propensity score and instrumental variable methods. JAMA 2007;297:278-85.
    1. Martens EP, Pestman WR, de Boer A, Belitser SV, Klungel OH. Instrumental variables. Applications and limitations. Epidemiology 2006;17:260-7.
    1. Brookhart MA, Wang PS, Solomon DH, Schneeweiss S. Evaluating short-term drug effects using a physician-specific prescribing preference as an instrumental variable. Epidemiology 2006;17:268-75.
    1. Gruber J. The incidence of mandated maternity benefits. American Economic Review 1994;84:622-41.
    1. Meyer BD. Natural and quasi-experiments in economics. J Business Econom Stat 1995;13:151-61.
    1. Angrist JD, Krueger AB. Empirical strategies in labor economics. In: Ashenfelter O, Card D, eds. Handbook of labor economics. Amsterdam: Elsevier Science, 1999:1277-366.
    1. Campbell D, Stanley J. Experimental and quasi-experimental designs for research. Boston, MA: Houghton-Mifflin, 1963.
    1. Shadish WR, Cook TD, Campbell DT. Experimental and quasi-experimental designs for generalized causal inference. Boston, MA: Houghton Mifflin, 2002.
    1. Weiner MG, Barnhart K, Xie D, Tannen RL. Hormone replacement therapy and coronary heart disease in young women. Menopause 2008;15:86-93.
    1. Tannen RL, Weiner MG, Xie D, Barnhart K, Perspectives on hormone replacement therapy: the Women’s Health Initiative and new observational studies sampling the overall population. Fertil Steril 2008;90:258-64.

Source: PubMed

Sponsorer och medarbetare

Medicinska tillstånd

Läkemedelsinterventioner

3
Prenumerera