Clinical Value of Predicting Individual Treatment Effects for Intensive Blood Pressure Therapy

Abstract

Background: The absolute risk reduction (ARR) in cardiovascular events from therapy is generally assumed to be proportional to baseline risk-such that high-risk patients benefit most. Yet newer analyses have proposed using randomized trial data to develop models that estimate individual treatment effects. We tested 2 hypotheses: first, that models of individual treatment effects would reveal that benefit from intensive blood pressure therapy is proportional to baseline risk; and second, that a machine learning approach designed to predict heterogeneous treatment effects-the X-learner meta-algorithm-is equivalent to a conventional logistic regression approach.

Methods and results: We compared conventional logistic regression to the X-learner approach for prediction of 3-year cardiovascular disease event risk reduction from intensive (target systolic blood pressure <120 mm Hg) versus standard (target <140 mm Hg) blood pressure treatment, using individual participant data from the SPRINT (Systolic Blood Pressure Intervention Trial; N=9361) and ACCORD BP (Action to Control Cardiovascular Risk in Diabetes Blood Pressure; N=4733) trials. Each model incorporated 17 covariates, an indicator for treatment arm, and interaction terms between covariates and treatment. Logistic regression had lower C statistic for benefit than the X-learner (0.51 [95% CI, 0.49-0.53] versus 0.60 [95% CI, 0.58-0.63], respectively). Following the logistic regression's recommendation for individualized therapy produced restricted mean time until cardiovascular disease event of 1065.47 days (95% CI, 1061.04-1069.35), while following the X-learner's recommendation improved mean time until cardiovascular disease event to 1068.71 days (95% CI, 1065.42-1072.08). Calibration was worse for logistic regression; it over-estimated ARR attributable to intensive treatment (slope between predicted and observed ARR of 0.73 [95% CI, 0.30-1.14] versus 1.06 [95% CI, 0.74-1.32] for the X-learner, compared with the ideal of 1). Predicted ARRs using logistic regression were generally proportional to baseline pretreatment cardiovascular risk, whereas the X-learner observed-correctly-that individual treatment effects were often not proportional to baseline risk.

Conclusions: Predictions for individual treatment effects from trial data reveal that patients may experience ARRs not simply proportional to baseline cardiovascular disease risk. Machine learning methods may improve discrimination and calibration of individualized treatment effect estimates from clinical trial data.

Clinical trial registration: URL: https://www.clinicaltrials.gov . Unique identifiers: NCT01206062; NCT00000620.

Keywords: blood pressure; calibration; cardiovascular disease; machine learning; risk factors.

Figures

Figure 1:

Conceptualization of the X-learner approach to detect heterogeneous treatment effects from trial data. We first estimate expectations of outcomes given predictors under control and treatment separately, then estimate imputed treatment effects (defined as the difference between expected outcomes and actual outcomes), and finally predict individual treatment effects as weighted averages of the estimated imputed treatment effects. For each step in the process we use random forests with 1000–2000 trees as base learners, and predict out-of-bag samples to prevent over-fitting.

Figure 2:

Participants with uncensored outcomes at 3 years are grouped into predicted buckets of benefit (ARR > 0) and no benefit (ARR ≤ 0) via machine learning and conventional methods. Using these buckets, we bootstrap 95% confidence intervals for the observed absolute risk reduction (ARR) in each bucket, and calculate corresponding P values via the Wald test. The X-learner machine learning approach yielded more discriminative buckets than the conventional logistic regression approach, with the benefit bucket exhibiting higher observed ARR and the no benefit bucket exhibiting lower observed ARR.

Figure 3:

Calibration plots for predicted absolute risk reduction versus observed absolute risk reduction (difference between intensive and standard treatment arms in cardiovascular event rates) using the X-learner machine learning approach and the conventional logistic regression approach, evaluated at quintiles of predicted absolute risk reduction and using Kaplan-Meier estimates to adjust for censoring.

Figure 4:

Predicted risk reduction across deciles of predicted baseline CVD risk. We employed predictions of baseline risk calculated by ACC/AHA ASCVD risk score estimates to group trial participants into baseline risk deciles and compare the median and quartiles of predicted absolute risk reduction (ARR) from intensive blood pressure treatment across each decile. The logistic regression approach generally predicted absolute risk reduction to be proportional to baseline risk, whereas the X-learner machine learning approach predicted wider ranges of risk reduction per decile, not necessarily proportional to baseline risk.