Predicting Extubation Failure Using Machine Learning

This is a prospective, multicenter, observational cohort study. Extubation and reintubation decisions are made by the attending anesthesiologists as part of routine care and are not informed by the models. The models will be developed only after data collection is complete.

Data collection:

Candidate predictors are chosen from previous studies and clinical relevance. They fall into four groups: patient characteristics, clinical condition, mechanical ventilation parameters, and laboratory results. Trained ICU staff and research assistants record them from bedside monitors, ventilator displays, paper and electronic medical records, and family interviews. Patient characteristics and severity scores (APACHE II and SAPS II) are recorded at ICU admission. Before extubation, ventilator parameters are taken within 4 hours, clinical condition within 24 hours, and laboratory results within 48 hours, using the measurement closest to extubation. Patients are followed for 7 days after planned extubation through bedside monitoring and review of medical records after transfer to the ward.

Participants:

Participants are enrolled when they receive invasive mechanical ventilation in the ICU, before it is known whether they will have a planned extubation. After enrollment, participants are excluded from the analysis if they die or have a tracheostomy without the breathing tube being removed; stay on a breathing machine for more than 2 weeks without the tube being removed; have an unplanned extubation (the tube was pulled out by the participant or by accident); are transferred to another hospital before extubation or within 7 days after; or if they or their family decline further intensive care. The number of participants enrolled is therefore larger than the number analyzed. The number analyzed will be reported in the results article.

Data management:

Data are entered into REDCap, a secure web-based data management application, using study codes without names, medical record numbers, or dates of birth. Data are stored at the Big Data Center, Indonesia Medical Education and Research Institute (IMERI), Universitas Indonesia. Research assistants periodically check the data for completeness and implausible values. Values that are not measured are left blank during data collection and will be handled in the analysis.

Sample size:

The development cohort target was calculated to estimate sensitivity (80%, precision of 10 percentage points, 95% confidence), assuming a 10% extubation failure rate and 10% dropout. The external validation target was calculated to estimate the extubation failure rate with a 5 percentage point margin of error. The development sample is smaller than formal prediction model sample size criteria would require. The study is therefore exploratory, and the external validation is preliminary.

Planned analysis:

Five supervised machine learning algorithms will be developed: LightGBM, CatBoost, XGBoost, random forest, and support vector machine. The development cohort will be randomly split, stratified by outcome, into training, validation, and holdout test sets. Missing values will be imputed, and class imbalance will be handled during model training. The classification threshold will be chosen on the validation set. Performance will be assessed on the holdout test set and on the external validation cohort, using the final models without modification. Predictor importance will be assessed with SHAP values for tree-based models and permutation importance for the support vector machine.

Reporting:

This study follows the TRIPOD+AI reporting guideline (Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis, updated for artificial intelligence).