Development and Validation of a Personalized, Sex-Specific Prediction Algorithm of Severe Atheromatosis in Middle-Aged Asymptomatic Individuals: The ILERVAS Study

Abstract

Background: Although European guidelines recommend vascular ultrasound for the assessment of cardiovascular risk in low-to-moderate risk individuals, no algorithm properly identifies patients who could benefit from it. The aim of this study is to develop a sex-specific algorithm to identify those patients, especially women who are usually underdiagnosed.

Methods: Clinical, anthropometrical, and biochemical data were combined with a 12-territory vascular ultrasound to predict severe atheromatosis (SA: ≥ 3 territories with plaque). A Personalized Algorithm for Severe Atheromatosis Prediction (PASAP-ILERVAS) was obtained by machine learning. Models were trained in the ILERVAS cohort (n = 8,330; 51% women) and validated in the control subpopulation of the NEFRONA cohort (n = 559; 47% women). Performance was compared to the Systematic COronary Risk Evaluation (SCORE) model.

Results: The PASAP-ILERVAS is a sex-specific, easy-to-interpret predictive model that stratifies individuals according to their risk of SA in low, intermediate, or high risk. New clinical predictors beyond traditional factors were uncovered. In low- and high-risk (L&H-risk) men, the net reclassification index (NRI) was 0.044 (95% CI: 0.020-0.068), and the integrated discrimination index (IDI) was 0.038 (95% CI: 0.029-0.048) compared to the SCORE. In L&H-risk women, PASAP-ILERVAS showed a significant increase in the area under the curve (AUC, 0.074 (95% CI: 0.062-0.087), p-value: < 0.001), an NRI of 0.193 (95% CI: 0.162-0.224), and an IDI of 0.119 (95% CI: 0.109-0.129).

Conclusion: The PASAP-ILERVAS improves SA prediction, especially in women. Thus, it could reduce the number of unnecessary complementary explorations selecting patients for a further imaging study within the intermediate risk group, increasing cost-effectiveness and optimizing health resources.

Clinical trial registration: [www.ClinicalTrials.gov], identifier [NCT03228459].

Keywords: atherosclerosis; cardiovascular disease; cardiovascular risk assessment; machine learning; recursive partitioning classification trees; vascular ultrasound.

Conflict of interest statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2022 Bermúdez-López, Martí-Antonio, Castro-Boqué, Bretones, Farràs, Torres, Pamplona, Lecube, Mauricio, Valdivielso and Fernández on behalf of the ILERVAS Project Collaborators.

Figures

FIGURE 1

Analysis pipeline. ASCVD, atherosclerotic cardiovascular disease; OR, odds ratio; SD, standard deviation.

FIGURE 2

Personalized Algorithm for Severe Atheromatosis Prediction in men. The structure of the classification tree in men is represented. The probability of severe atheromatosis, which corresponds to the proportion of affected patients, is shown inside circles in the final nodes. The barplot offers a clearer visualization of this prevalence with its 95% confidence interval. The colors green, yellow, and red indicate the level of risk (low, intermediate, or high) identified after calibration. The units were as follows: age, year; GFR, ml/min/m2; SBP, mmHg; total cholesterol, mg/dl. GFR, glomerular filtration rate, SBP, systolic blood pressure.

FIGURE 3

Personalized Algorithm for Severe Atheromatosis Prediction in women. The probability of severe atheromatosis, which corresponds to the proportion of affected patients, was shown inside circles in the final nodes. The barplot offers a clearer visualization of this prevalence with its 95% confidence interval. The colors green, yellow, and red indicate the level of risk (low, intermediate, or high, respectively) identified after calibration. Hypertension and dyslipidemia refer to patients who had prior clinical diagnostic of hypertension or dyslipidemia in their clinical records. The units were as follows: Age: year; BMI: kg/m2; SBP, mmHg; total cholesterol: mg/dL; uric acid: mg/dl; waist perimeter: cm. BMI, body mass index; SBP, systolic blood pressure.

FIGURE 4

Clinical predictor importance in the PASAP-ILERVAS in both sexes. A variable importance score was calculated using the improvement measure attributable to each predictor in its role as splitter, plus the goodness for all splits in which it was a surrogate. The values of importance were scaled up to sum 100%. SBP, systolic blood pressure; BMI, body mass index; GFR, glomerular filtration rate.

FIGURE 5

Comparison of the PASAP-ILERVAS with the SCORE risk score. Barplots contain the main performance metrics of PASAP-ILERVAS and SCORE risk model in L&H-risk individuals: (A) specificity; (B) sensitivity; (C) accuracy; (D) positive predictive model; (E) ROC curve in men; (F) ROC curve in women. The ROC curves reflect the area under the curve between sensitivity and complementary of specificity with both models. SCORE, Systematic COronary risk Evaluation; ROC, Receiver Operating Characteristic.