External Validation of the FedHist Dynamic Early Warning Model for Mortality Risk in Critically Ill Patients: A Prospective Multicenter Study

Critically ill patients can deteriorate rapidly, creating a need for timely and repeated assessment of mortality risk. Conventional severity assessments, including the Acute Physiology and Chronic Health Evaluation II (APACHE II) and Sequential Organ Failure Assessment (SOFA) scores, provide clinically useful information. However, risk assessments based on data from defined assessment periods may not fully capture evolving clinical trajectories. Models that incorporate longitudinal clinical data may support more frequent assessment of short-term mortality risk.

Artificial intelligence offers opportunities to integrate clinical information collected over time for dynamic risk prediction. However, models developed using data from a single center or database may perform differently in other settings. Differences in patient characteristics, disease severity, measurement frequency, and clinical workflows can affect predictive performance. Privacy requirements, data governance policies, and institutional control over data also limit the pooling of patient records for conventional centralized model development.

FedHist was developed using a federated learning framework incorporating electronic health records from more than 250,000 critically ill patients across five countries and regions. This framework enabled collaborative model development while raw patient data remained at the contributing institutions. FedHist generates an updated estimate of the risk of ICU death within the next 24 hours at 6-hour intervals.

Previous evaluations showed a macro-averaged area under the receiver operating characteristic curve (AUROC) of 0.893 in internal validation. AUROCs were 0.881 in the NWICU external validation cohort and 0.961 in the prospective CDIC cohort. FedHist outperformed the corresponding locally trained models. When only 10% of the development data at each site were used, its performance approached or exceeded that of local models trained on the full development datasets. These findings support further prospective evaluation following integration into routine clinical information systems across multiple hospitals.

This prospective multicenter study will externally validate the previously developed FedHist model in participating ICUs. The model will be integrated into local clinical information systems to generate risk estimates using prospectively collected clinical data. Predictions will be compared with observed ICU outcomes to assess performance in predicting death within the subsequent 24 hours. The study will evaluate predictive performance across participating hospitals and assess the generalizability of FedHist under routine clinical conditions.