External Validation of AI-Aided Weaning Software Using Multicenter Retrospective Data

June 17, 2026 updated by: Chieh-Liang Wu, Taichung Veterans General Hospital

Using Multicenter Retrospective Data to Validate the Performance of AI-Aided Weaning Software

This multicenter retrospective study aims to externally validate an artificial intelligence-aided weaning software developed using intensive care unit data from Taichung Veterans General Hospital between 2015 and 2019. The model predicts the optimal timing for extubation using routinely collected clinical variables including ventilator parameters, physiologic measurements, and fluid and nutrition information. De-identified data from four hospitals collected between 2020 and 2024 will be used to evaluate model performance. Performance metrics include sensitivity, specificity, accuracy, area under the receiver operating characteristic curve (AUROC), and F1 score.

Study Overview

Status

Completed

Conditions

Detailed Description

Critical care generates a large amount of digitized clinical data that may benefit from artificial intelligence-assisted decision support. The AI-Aided Weaning Software was previously developed using ICU data from Taichung Veterans General Hospital collected between 2015 and 2019.

This retrospective multicenter validation study will evaluate the external performance of the established model using independent datasets from four hospitals in Taiwan, including Taichung Veterans General Hospital, Mackay Memorial Hospital, Kaohsiung Medical University Chung-Ho Memorial Hospital, and Tungs' Taichung MetroHarbor Hospital.

The study population includes adult ICU patients with respiratory failure who received mechanical ventilation for at least 72 hours between January 2020 and December 2024. De-identified routine clinical records will be collected according to a predefined case report form and analyzed centrally.

The primary objective is to assess the external validity of the AI-Aided Weaning Software across different hospitals. Model performance will be evaluated using sensitivity, specificity, accuracy, AUROC, and F1 score.

Study Type

Observational

Enrollment (Actual)

1500

Contacts and Locations

This section provides the contact details for those conducting the study, and information on where this study is being conducted.

Study Locations

Taiwan
- - Taichung, Taiwan
    - Taichung Veterans General Hospital

Participation Criteria

Researchers look for people who fit a certain description, called eligibility criteria. Some examples of these criteria are a person's general health condition or prior treatments.

Eligibility Criteria

Ages Eligible for Study

Adult
Older Adult

Accepts Healthy Volunteers

Sampling Method

Non-Probability Sample

Study Population

Adult patients with acute respiratory failure who were admitted to participating hospitals between January 2022 and December 2024 and required invasive mechanical ventilation for at least 24 hours. This is a retrospective study using existing clinical and imaging data for model validation.

Description

Inclusion Criteria:

Adult patients aged 20 years or older.
Admitted to the intensive care unit (ICU) at one of the participating hospitals between January 1, 2020 and December 31, 2024.
Received invasive mechanical ventilation for at least 72 hours.
Availability of de-identified clinical data required for model validation.

Exclusion Criteria:

Patients who did not receive invasive mechanical ventilation.
Duration of mechanical ventilation less than 72 hours.
Missing key clinical variables required for model validation.

Study Plan

This section provides details of the study plan, including how the study is designed and what the study is measuring.

How is the study designed?

Design Details

Number of groups / cohorts

Cohorts and Interventions

Group / Cohort
Mechanically Ventilated ICU Patients Adult intensive care unit patients aged 20 years or older who received invasive mechanical ventilation for at least 72 hours between January 2020 and December 2024 at four participating hospitals. Retrospective de-identified clinical data were used to validate the performance of AI-Aided Weaning Software.

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Model Performance (AUROC) Time Frame: Using data collected during ICU admission	Area under the receiver operating characteristic curve (AUROC) for predicting successful extubation. AUROC ranges from 0.5 to 1.0, with higher values indicating better discriminative performance of the prediction model.	Using data collected during ICU admission

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Sensitivity Time Frame: ICU admission	SensitivitySensitivity of the prediction model for successful extubation. Sensitivity ranges from 0 to 1 (or 0% to 100%), with higher values indicating better identification of patients who achieve successful extubation.	ICU admission
Specificity Time Frame: ICU admission	Specificity of the prediction model for successful extubation. Specificity ranges from 0 to 1 (or 0% to 100%), with higher values indicating better identification of patients who do not achieve successful extubation.	ICU admission
Accuracy Time Frame: ICU admission	Accuracy of the prediction model for successful extubation. Accuracy ranges from 0 to 1 (or 0% to 100%), with higher values indicating better overall prediction performance.	ICU admission
F1 Score Time Frame: ICU admission	F1 score of the prediction model for successful extubation. F1 score ranges from 0 to 1, with higher values indicating better balance between precision and recall.	ICU admission

Collaborators and Investigators

This is where you will find people and organizations involved with this study.

Sponsor

Taichung Veterans General Hospital

Collaborators

Mackay Memorial Hospital

Kaohsiung Medical University Chung-Ho Memorial Hospital

Tungs' Taichung Metroharbor Hospital

Publications and helpful links

The person responsible for entering information about the study voluntarily provides these publications. These may be about anything related to the study.

General Publications

Liu CF, Hung CM, Ko SC, Cheng KC, Chao CM, Sung MI, Hsing SC, Wang JJ, Chen CJ, Lai CC, Chen CM, Chiu CC. An artificial intelligence system to predict the optimal timing for mechanical ventilation weaning for intensive care unit patients: A two-stage prediction approach. Front Med (Lausanne). 2022 Nov 18;9:935366. doi: 10.3389/fmed.2022.935366. eCollection 2022.

Study record dates

These dates track the progress of study record and summary results submissions to ClinicalTrials.gov. Study records and reported results are reviewed by the National Library of Medicine (NLM) to make sure they meet specific quality control standards before being posted on the public website.

Study Major Dates

Study Start (Actual)

January 1, 2020

Primary Completion (Actual)

December 31, 2024

Study Completion (Actual)

December 31, 2024

Study Registration Dates

First Submitted

June 14, 2026

First Submitted That Met QC Criteria