Assessing Intensive Care Unit (ICU) Indications: Human vs. ChatGPT-4o Predictions (ICU)

Evaluation of the Accuracy of Intensive Care Unit (ICU) Admission Indications in Emergency Department Patients: A Comparison Between Clinical Decisions and ChatGPT-4o Prediction

This retrospective study evaluates the accuracy of ICU admission indications by comparing clinical decisions with predictions from ChatGPT-4. Patient data, including demographics, vital signs, laboratory results, imaging findings, and clinical decisions, will be retrospectively collected and documented systematically using Case Report Forms. The model will be trained using ICU admission guidelines and tasked to predict ICU needs based on collected patient data. This study aims to systematically assess the alignment between AI-based predictions and clinical decisions for ICU admissions.

Study Overview

• Status: Completed
• Conditions: Clinical Decision-making; Emergency Department Patient; Artificial Intelligence (AI); Intensive Care Unit (ICU) Admission

Detailed Description

This study has a retrospective design. The medical data of patients admitted to the emergency department and consulted to the anesthesiology and reanimation clinic for ICU indications will be collected retrospectively. Demographic information, vital signs, laboratory results, imaging findings, and clinical decisions of the patients will be recorded. These data will be systematically collected for each patient using an individual Case Report Form.

Inclusion Criteria for the Study:

Patients aged 18 years and older who were consulted to the anesthesiology and reanimation clinic for ICU indications from the emergency department will be included in the study.

Exclusion Criteria for the Study:

Patients consulted to the anesthesiology and reanimation clinic for ICU indications from inpatient services.

Patients consulted to the anesthesiology and reanimation clinic from the emergency department for reasons other than ICU indications.

Patients consulted to the anesthesiology and reanimation clinic for ICU indications from the emergency department but with insufficient recorded data (patients with data loss).

Model Training and Prediction Analysis:

ChatGPT-4 will be trained according to the guidelines in "Yoğun Bakım Hasta Kabul Kriterleri (Rehberleri)" by Çiftçi B, Erdoğan C, and Demiraran Y (5). The collected patient data will be presented to the ChatGPT-4 model to obtain predictions regarding whether the patients require ICU admission. The predictions made by ChatGPT will be compared with clinical decisions, and accuracy rate, false positive rate, and false negative rate will be analyzed.

Statistical Analysis Methods to Be Used in the Study:

Accuracy Rate: The rate at which ChatGPT correctly predicts ICU indications will be calculated.

False Positive Rate: The rate at which ChatGPT predicts ICU need for patients who do not require ICU admission will be evaluated.

False Negative Rate: The rate at which ChatGPT predicts no ICU need for patients who require ICU admission will be analyzed.

Kappa Statistics: The agreement between ChatGPT predictions and clinical decisions will be measured.

ROC Curve and AUC: The performance of ChatGPT will be evaluated using the ROC curve and AUC.

The Case Report Form used for each patient ensures detailed and systematic data collection of clinical information, aiming to meaningfully compare the alignment of ChatGPT's predictions with clinical decisions.

• Study Type: Observational
• Enrollment (Actual): 624

Contacts and Locations

Study Locations

• Turkey (Türkiye)
  • Bursa, Turkey (Türkiye)
    • Bursa Yuksek Ihtisas Training and Research Hospital

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: Adult; Older Adult
• Accepts Healthy Volunteers: N/A

• Sampling Method: Non-Probability Sample

Study Population

500

Description

Inclusion Criteria:

• Patients aged 18 years and older who are consulted to the anesthesiology and reanimation clinic for ICU indications from the emergency department will be included in the study.

Exclusion Criteria:

• Patients consulted to the anesthesiology and reanimation clinic for ICU indications from inpatient services.
• Patients consulted to the anesthesiology and reanimation clinic from the emergency department for reasons other than ICU indications.
• Patients consulted to the anesthesiology and reanimation clinic for ICU indications from the emergency department but with insufficient recorded data (patients with data loss)

Study Plan

How is the study designed?

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Time Frame
Accuracy rate of ChatGPT-4 in predicting ICU indications	3 month
False positive rate	3 month
False negative rate	3 month

Secondary Outcome Measures

Outcome Measure	Time Frame
Kappa statistic	3 month

Collaborators and Investigators

• Sponsor: Bursa Yuksek Ihtisas Training and Research Hospital
• Investigators: Principal Investigator: İlkay Ceylan, ceylanilkay@yahoo.com

Study record dates

Study Major Dates

• Study Start (Actual): September 1, 2024
• Primary Completion (Actual): November 30, 2024
• Study Completion (Actual): February 28, 2025

Study Registration Dates

• First Submitted: December 6, 2024
• First Submitted That Met QC Criteria: December 6, 2024
• First Posted (Actual): December 10, 2024

Study Record Updates

• Last Update Posted (Actual): April 14, 2026
• Last Update Submitted That Met QC Criteria: April 13, 2026
• Last Verified: April 1, 2026

More Information

Terms related to this study

• Keywords: Intensive Care Unit; AI; Clinical decision-making; Artificial Intelligenc
• Additional Relevant MeSH Terms: Pathologic Processes; Disease Attributes; Emergencies
• Other Study ID Numbers: BYIEAH-INKA

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: NO

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No