Machine learning predicts mortality based on analysis of ventilation parameters of critically ill patients: multi-centre validation

Abstract

Background: Mechanical Ventilation (MV) is a complex and central treatment process in the care of critically ill patients. It influences acid-base balance and can also cause prognostically relevant biotrauma by generating forces and liberating reactive oxygen species, negatively affecting outcomes. In this work we evaluate the use of a Recurrent Neural Network (RNN) modelling to predict outcomes of mechanically ventilated patients, using standard mechanical ventilation parameters.

Methods: We performed our analysis on VENTILA dataset, an observational, prospective, international, multi-centre study, performed to investigate the effect of baseline characteristics and management changes over time on the all-cause mortality rate in mechanically ventilated patients in ICU. Our cohort includes 12,596 adult patients older than 18, associated with 12,755 distinct admissions in ICUs across 37 countries and receiving invasive and non-invasive mechanical ventilation. We carry out four different analysis. Initially we select typical mechanical ventilation parameters and evaluate the machine learning model on both, the overall cohort and a subgroup of patients admitted with respiratory disorders. Furthermore, we carry out sensitivity analysis to evaluate whether inclusion of variables related to the function of other organs, improve the predictive performance of the model for both the overall cohort as well as the subgroup of patients with respiratory disorders.

Results: Predictive performance of RNN-based model was higher with Area Under the Receiver Operating Characteristic (ROC) Curve (AUC) of 0.72 (± 0.01) and Average Precision (AP) of 0.57 (± 0.01) in comparison to RF and LR for the overall patient dataset. Higher predictive performance was recorded in the subgroup of patients admitted with respiratory disorders with AUC of 0.75 (± 0.02) and AP of 0.65 (± 0.03). Inclusion of function of other organs further improved the performance to AUC of 0.79 (± 0.01) and AP 0.68 (± 0.02) for the overall patient dataset and AUC of 0.79 (± 0.01) and AP 0.72 (± 0.02) for the subgroup with respiratory disorders.

Conclusion: The RNN-based model demonstrated better performance than RF and LR in patients in mechanical ventilation and its subgroup admitted with respiratory disorders. Clinical studies are needed to evaluate whether it impacts decision-making and patient outcomes.

Trial registration: NCT02731898 ( https://ichgcp.net/clinical-trials-registry/NCT02731898 ), prospectively registered on April 8, 2016.

Keywords: Critical care medicine; ICU; Machine learning; Mechanical ventilation; Risk stratification.

Conflict of interest statement

The authors declare that they have no competing interests.

Figures

Fig. 1

Linear correlation of variables and the outcome (indicated by Discharge ICU). Note the correlation scale is in the interval − 0.2 to 0.2

Fig. 2

Panel a. Predictive performance (AUC and AUPRC) of our LSTM-based model versus Random Forrest (RF) and Logistic Regression (LR) for the overall patient dataset using six standard mechanical ventilation parameters. Panel b. Predictive performance of our LSTM-based model versus Random Forrest (RF) and Logistic Regression (LR) for the subgroup of patients admitted with respiratory disorders using six standard mechanical ventilation parameters. Confidence intervals are shown in grey for both panels

Fig. 3

Panel a. Predictive performance (AUC and AUPRC) of our LSTM-based model versus Random Forrest (RF) and Logistic Regression (LR) for the overall patient dataset, including also variables related to kidney and liver function. Panel b. Predictive performance of our LSTM-based model versus Random Forrest (RF) and Logistic Regression (LR) for the subgroup of patients admitted with respiratory disorders, including also variables related to kidney and liver function. Confidence intervals are shown in grey for both panels

Fig. 4

Variable importance ranking for each LSTM model: a) All patients, and b) patients admitted with respiratory disorders

Fig. 5

Calibration plots for each LSTM model: All patients (left) and patients admitted with respiratory disorders (right)