Improving 1-year mortality prediction in ACS patients using machine learning

Abstract

Background: The Global Registry of Acute Coronary Events (GRACE) score is an established clinical risk stratification tool for patients with acute coronary syndromes (ACS). We developed and internally validated a model for 1-year all-cause mortality prediction in ACS patients.

Methods: Between 2009 and 2012, 2'168 ACS patients were enrolled into the Swiss SPUM-ACS Cohort. Biomarkers were determined in 1'892 patients and follow-up was achieved in 95.8% of patients. 1-year all-cause mortality was 4.3% (n = 80). In our analysis we consider all linear models using combinations of 8 out of 56 variables to predict 1-year all-cause mortality and to derive a variable ranking.

Results: 1.3% of 1'420'494'075 models outperformed the GRACE 2.0 Score. The SPUM-ACS Score includes age, plasma glucose, NT-proBNP, left ventricular ejection fraction (LVEF), Killip class, history of peripheral artery disease (PAD), malignancy, and cardio-pulmonary resuscitation. For predicting 1-year mortality after ACS, the SPUM-ACS Score outperformed the GRACE 2.0 Score which achieves a 5-fold cross-validated AUC of 0.81 (95% CI 0.78-0.84). Ranking individual features according to their importance across all multivariate models revealed age, trimethylamine N-oxide, creatinine, history of PAD or malignancy, LVEF, and haemoglobin as the most relevant variables for predicting 1-year mortality.

Conclusions: The variable ranking and the selection for the SPUM-ACS Score highlight the relevance of age, markers of heart failure, and comorbidities for prediction of all-cause death. Before application, this score needs to be externally validated and refined in larger cohorts.

Clinical trial registration: NCT01000701.

Keywords: Acute Coronary Syndromes; GRACE 2.0 Score; Machine Learning; NT-proBNP; age.

Published on behalf of the European Society of Cardiology. All rights reserved. © The Author(s) 2021. For permissions, please email: journals.permissions@oup.com.

Figures

Summarizing scheme of GRACE 2.0 and ML-based risk score for 1-year mortality and putative impact. Analyses of baseline variables in ACS patients for risk of mortality at 1 year. Easily available variables from clinical assessment build the basis for the GRACE score. The ML-derived multivariate SPUM-ACS risk score improves risk prediction compared to the GRACE 2.0 score.

Figure 1

AUCs of models predicting all-cause death at 1 year – numerous candidate risk scores improve risk prediction above GRACE 2.0. This figure shows model performance in terms of the five-fold cross-validated AUC score. A total of 1’420’494’075 models were obtained by combining the 56 baseline variables in all possible 8-variable models. The GRACE 2.0 Score performance is marked in black (AUC 0.815), the overall best and the overall best practical models’ performances are highlighted in red (AUC 0.866) and green (AUC 0.865) respectively. 17’857’817 (1.3%) models perform better than the GRACE 2.0 Score (blue curve segment on the left) (214’256’356 (15.1%) better than the GRACE 1.0 Score).

Figure 2

SPUM-ACS Score calculator available as online supplemental file “SPUM-ACS Score.html”.

Figure 3

Ranking of features according to their importance across all multivariate models shows the role of age, atherosclerosis burden, heart damage, inflammation, and novel biomarkers for 1-year all-cause mortality risk stratification. The variables on the left are ranked and each entry indicates whether the variable on the left has a certain minimum rank (column) with a certain probability (colour coding), e.g. age appears on rank 4 or better with probability close to 1, while OAC has only a minimum rank of 43 with probability 50%. The blue/orange/blue lines indicate the minimum rank that a variable achieves with probability 25%/50%/75%.

Figure 4

Variable contribution to the performance of the SPUM-ACS Score. The SPUM-ACS Score model achieves AUC 0.86 while GRACE 2.0 achieves 0.81 (GRACE 1.0 achieves 0.78); the bars indicate how much the performance drops if the respective variable is being replaced by the next best practical variable, e.g. the model where we replace the LVEF variable is 0.006 AUC worse than the original model, while replacing Killip class in the SPUM-ACS model incurs a lesser drop in model performance, hence LVEF plays a more important role for this model.

Figure 5

Pairwise correlations of variable occurrence in models outperforming GRACE 2.0. This scheme illustrates the degree to which variables complement each other in good models: Each node in the network corresponds to one variable. Its colour indicates a cluster membership. Clusters mark variables that are better combinable than expected based on both their ranks. Gray links mark good combinations of variables in different clusters, while coloured links highlight good inter-cluster combinations. The thicker a line, the higher is the correlation to appear together in good models. We only visualize the biggest 5% of positive correlations. Links that are realized in our score are highlighted with black borders. The two strongest correlations between CPR—NT-proBNP and h/o malignancy—NT-proBNP are among them.