Assessment of Organ Failure Risk Predictions in ICU (AI4ICU-Obs)
Prospective Assessment of Risk Predictions of Organ Failure in the Intensive Care Unit - Comparing Accuracy of Human and AI Risk Predictions
Study Overview
Status
Status
Conditions
Conditions
Detailed Description
Study Type
Study Type
Enrollment (Actual)
Enrollment
Contacts and Locations
Study Contact
Study Contact
- Name: Gunnar Professor Dr. rer. nat.
- Phone Number: +41 44 632 20 36
- Email: raetsch@ethz.ch
Study Contact Backup
- Name: Quinten Johnson
- Phone Number: +41 78 300 99 97
- Email: quinten.johnson@inf.ethz.ch
Study Locations
-
-
Canton of Bern
-
Bern, Canton of Bern, Switzerland, 3010
- University Hospital Inselspital, Berne
-
-
Participation Criteria
Eligibility Criteria
Eligibility Criteria
Ages Eligible for Study
- Adult
- Older Adult
Accepts Healthy Volunteers
Sampling Method
Study Population
Description
Inclusion Criteria:
- patient minimum age of 18 years
- emergency admission to the ICU
- arterial line in place
Exclusion Criteria:
- documented refusal (on the general consent form) to participate to clinical research
- patients with neurologic conditions that impair the patient's level of consciousness (including, but not limited to stroke, traumatic brain injury, intracranial hemorrhage, CNS infections; except polytrauma)
- patients on mechanical circulatory support systems (IABP, VA-ECMO, Impella, VAD) or extracorporeal membrane oxygenation (VV-ECMO) at any time during their ICU stay;
- patients receiving end-of-life care or are admitted for the sole purpose of evaluating organ donation
Study Plan
How is the study designed?
Design Details
Number of groups / cohorts
Cohorts and Interventions
Group / CohortGroup / Cohort |
|---|
|
Adult ICU patients
|
What is the study measuring?
Primary Outcome Measures
Primary Outcome Measures
Outcome Measure |
Measure Description |
Time Frame |
|---|---|---|
|
Clinician prediction of circulatory failure within 8 hours compared to published ML models
Time Frame: Assessments are collected within the first 72 hours following admission.
|
This outcome compares the area under the receiver operating characteristic curve (auROC) for two methods of predicting circulatory failure within 8 hours of each assessment time point: (1) ICU clinicians' risk estimates, and (2) previously published machine learning (ML) models applied retrospectively.
For each assessment, we compute the auROC separately for clinicians and for the ML model for the same time points and patients.
The difference in auROC (clinician minus ML) is the main measure of interest, evaluated under a non-inferiority framework with a margin of 0.025.
|
Assessments are collected within the first 72 hours following admission.
|
Secondary Outcome Measures
Secondary Outcome Measures
Outcome Measure |
Measure Description |
Time Frame |
|---|---|---|
|
Clinician prediction of respiratory failure within 24 hours compared to published ML models
Time Frame: Assessments are collected within the first 72 hours following admission.
|
This outcome compares the area under the receiver operating characteristic curve (auROC) for two methods of predicting respiratory failure within 24 hours of each assessment time point: (1) ICU clinicians' risk estimates, and (2) previously published machine learning (ML) models applied retrospectively.
For each assessment, we compute the auROC separately for clinicians and for the ML model.
The difference in auROCs (clinician minus ML) is the main measure of interest, evaluated using the same methodological framework as the primary outcome.
|
Assessments are collected within the first 72 hours following admission.
|
Other Outcome Measures
Other Outcome Measures
Outcome Measure |
Measure Description |
Time Frame |
|---|---|---|
|
Exploratory analysis of clinician prediction of renal failure within 48 hours compared to published ML models
Time Frame: Assessments are collected within the first 72 hours following admission.
|
This exploratory outcome compares the area under the receiver operating characteristic curve (auROC) for two methods of predicting renal failure within 48 hours of each assessment time point: (1) ICU clinicians' risk estimates, and (2) previously published machine learning (ML) models applied retrospectively.
For each assessment, we compute the auROC separately for clinicians and for the ML model.
The difference in auROCs is evaluated using the same methodological framework as the primary outcome.
|
Assessments are collected within the first 72 hours following admission.
|
|
Exploratory analysis of clinician mortality prediction compared to published ML models
Time Frame: Assessments are collected within the first 72 hours following admission.
|
All-cause mortality prediction by McNemar's test.
Evaluated and tested using the performance of respective assessments by clinicians (binary response) and corresponding (paired) predictions by a machine learning model (probability prediction reduced to a binary response) trained on historical data for the three individual binary outcomes of all-cause mortality <28-day, <6-months, <12-months.
For each horizon separately the machine-based probabilities are thresholded to match the sensitivity of the clinicians, a McNemar's test is performed to test for a significant difference in predictive capabilities and p-values will be adjusted to account for multiple testing if necessary.
|
Assessments are collected within the first 72 hours following admission.
|
|
Exploratory analysis of predictive accuracy of treating physicians versus treating nurses
Time Frame: Assessments are collected within the first 72 hours following admission.
|
Evaluated and tested for the same as the primary, secondary and first exploratory outcomes (i.e., circulatory, respiratory, and renal failure risks) using the respective assessments by treating physicians and treating nurses with paired time-point assessments.
|
Assessments are collected within the first 72 hours following admission.
|
|
Comparison of predictive accuracy of treating versus non-treating physicians
Time Frame: Assessments are collected within the first 72 hours following admission.
|
Evaluated and tested for the same as the primary, secondary and first exploratory outcomes (i.e., circulatory, respiratory, and renal failure risks) using the respective assessments by clinicians (treating) and solely relying on EHR data (non-treating) with paired time-point assessments.
|
Assessments are collected within the first 72 hours following admission.
|
|
Calibration analysis of clinician prediction scores
Time Frame: Assessments are collected within the first 72 hours following admission.
|
Calibration analysis of clinician prediction scores.
The study assesses the prediction capabilities of a collective of clinicians.
However, humans might be ill-calibrated amongst each other with respect to providing probability estimates.
Assessing the collective's performance without calibration of the individuals amongst each other might underestimate the actual predictive capabilities of the clinicians if they were well-calibrated.
We propose to re-assess the primary and secondary outcomes but additionally perform a risk score calibration amongst physicians.
|
Assessments are collected within the first 72 hours following admission.
|
|
Exploratory analysis of patterns (e.g., systematic errors/biases) of predictive performance in human and machine assessors
Time Frame: Assessments are collected within the first 72 hours following admission.
|
|
Assessments are collected within the first 72 hours following admission.
|
Collaborators and Investigators
Sponsor
Sponsor
Collaborators
Collaborators
Investigators
Investigators
- Principal Investigator: Martin Faltys, Dr. med., Insel Gruppe AG, University Hospital Bern
Study record dates
Study Major Dates
Study Start (Actual)
Study Start
Primary Completion (Actual)
Primary Completion
Study Completion (Estimated)
Study Completion
Study Registration Dates
First Submitted
First Submitted
First Submitted That Met QC Criteria
First Submitted That Met QC Criteria
First Posted (Actual)
First Posted
Study Record Updates
Last Update Posted (Estimated)
Last Update Posted
Last Update Submitted That Met QC Criteria
Last Update Submitted That Met QC Criteria
Last Verified
Last Verified
More Information
Terms related to this study
Additional Relevant MeSH Terms
- Urogenital Diseases
- Pathologic Processes
- Male Urogenital Diseases
- Kidney Diseases
- Urologic Diseases
- Female Urogenital Diseases
- Female Urogenital Diseases and Pregnancy Complications
- Respiratory Tract Diseases
- Respiration Disorders
- Pathological Conditions, Signs and Symptoms
- Respiratory Insufficiency
- Renal Insufficiency
- Shock
Other Study ID Numbers
Other Study ID Numbers
- BASEC-01046
Plan for Individual participant data (IPD)
Plan to Share Individual Participant Data (IPD)?
Drug and device information, study documents
Studies a U.S. FDA-regulated drug product
Studies a U.S. FDA-regulated device product
product manufactured in and exported from the U.S.
This information was retrieved directly from the website clinicaltrials.gov without any changes. If you have any requests to change, remove or update your study details, please contact register@clinicaltrials.gov. As soon as a change is implemented on clinicaltrials.gov, this will be updated automatically on our website as well.