AI Treatment Recommendations With Predicted Consequences for Sepsis Fluid and Vasopressor Decisions
Effect of Displaying Predicted Consequences of AI Treatment Recommendations on Clinicians' Fluid and Vasopressor Decisions in Sepsis: A Randomized Crossover Study
When artificial intelligence (AI) recommends a treatment, clinicians must decide whether to follow it. This study tests whether showing the predicted consequences of a recommendation, in addition to the recommendation itself, helps clinicians accept appropriate advice and reject inappropriate advice.
We developed a clinician-anchored counterfactual model that compares the on-duty clinician's intravenous fluid and vasopressor plan with alternative plans for patients with sepsis. It predicts the patient's course under each plan and recommends a change only when the predicted benefit holds across several plausible assumptions about unrecorded factors; otherwise it states that the evidence is insufficient to recommend a change.
About 72 physicians from two hospitals, stratified by seniority, will review 30 retrospective, de-identified ICU decision points from patients with sepsis. In stage 1, each physician independently specifies fluid and vasopressor doses without any AI information. In stage 2, the same cases are reviewed under three conditions assigned by counterbalanced sequences: the case with the on-duty clinician's plan and no AI (B0); B0 plus the AI recommendation (B1); and B1 plus the predicted consequences of the on-duty and recommended plans (B2). The AI recommendation is identical in B1 and B2. Decision quality is assessed against plans made independently by expert intensivists. No patients are enrolled, and participants' decisions do not affect patient care.
Studienübersicht
Status
Status
Bedingungen
Bedingungen
Intervention / Behandlung
Intervention / Behandlung
Detaillierte Beschreibung
Cases. Thirty decision points were selected from retrospective, de-identified ICU records before the experiment. Sampling was stratified so that the set includes AI recommendations to change treatment and to keep the current plan, both appropriate and inappropriate according to the expert reference. Each patient contributes one decision point, and none of the cases were used to develop the AI system.
Expert reference. A panel of intensivists who did not take part in AI development or in the experiment independently specify a fluid and vasopressor plan for every case, without AI output and without knowledge of study conditions. A participant's plan is scored by the number of dose levels by which it lies outside the range of expert plans (0 if within the range), summed over the two drugs. AI recommendations are classified as appropriate or inappropriate by the same rule before participant data are analyzed.
Analysis. Outcomes are analyzed with mixed-effects models with physician and case as crossed random effects. Secondary outcomes are tested in a prespecified order after the primary outcome to control the overall type I error.
Related non-interventional components. A retrospective comparison of clinicians' and the AI's plans and a prospective shadow deployment, in which the system ran on live ICU data without influencing treatment, are reported alongside this experiment but are not part of the randomized comparison.
Studientyp
Studientyp
Einschreibung (Geschätzt)
Einschreibung
Phase
Phase
- Unzutreffend
Kontakte und Standorte
Studienorte
-
-
Chongqing Municipality
-
Chongqing, Chongqing Municipality, China, 400016
- Rekrutierung
- The First Affiliated Hospital of Chongqing Medical University
-
Kontakt:
- Muchan Tao
- Telefonnummer: +86 18980918483
- E-Mail: orchilipairs@gmail.com
-
-
Sichuan
-
Luzhou, Sichuan, China, 646000
- Rekrutierung
- The Affiliated Hospital of Southwest Medical University
-
Kontakt:
- Muchan Tao
- Telefonnummer: +86 18980918483
- E-Mail: mooochann@163.com
-
-
Teilnahmekriterien
Zulassungskriterien
Zulassungskriterien
Studienberechtigtes Alter
Akzeptiert gesunde Freiwillige
Studienberechtigte Geschlechter
Beschreibung
Inclusion Criteria:
- Licensed physicians in critical care, emergency medicine, anaesthesiology, or respiratory medicine.
- Currently treating ICU patients.
- Able to complete the online questionnaire.
Exclusion Criteria:
- Involvement in developing or evaluating the AI system, or in case selection
- Membership of the expert reference panel
- Participation in the pilot survey
Studienplan
Wie ist die Studie aufgebaut?
Designdetails
- Hauptzweck: Versorgungsforschung
- Zuteilung: Zufällig
- Interventionsmodell: Crossover-Aufgabe
- Maskierung: Single
Anzahl der Arme
Waffen und Interventionen
Teilnehmergruppe / ArmTeilnehmergruppe / Arm |
Intervention / BehandlungIntervention / Behandlung |
|---|---|
|
Experimental: Sequence A
Stage 1: all 30 cases answered without AI information. Stage 2 (questionnaire version A): cases 1-10 under B0, cases 11-20 under B1, cases 21-30 under B2. |
Participants review the case together with the on-duty clinician's actual intravenous fluid rate and vasopressor dose and submit a final plan. No AI information is shown. This condition controls for the effect of a second review. As B0, plus the AI output: either a recommended dose level for intravenous fluid and for vasopressor, shown with the corresponding dose range, or a statement that the evidence is insufficient to recommend a change. As B1, with identical AI output, plus the model's predicted mean arterial pressure, lactate and urine output over the next 4 hours under the on-duty plan and under the recommended plan, shown as point estimates. |
|
Experimental: Sequence B
Stage 1: all 30 cases answered without AI information. Stage 2 (questionnaire version B): cases 1-10 under B1, cases 11-20 under B2, cases 21-30 under B0. |
Participants review the case together with the on-duty clinician's actual intravenous fluid rate and vasopressor dose and submit a final plan. No AI information is shown. This condition controls for the effect of a second review. As B0, plus the AI output: either a recommended dose level for intravenous fluid and for vasopressor, shown with the corresponding dose range, or a statement that the evidence is insufficient to recommend a change. As B1, with identical AI output, plus the model's predicted mean arterial pressure, lactate and urine output over the next 4 hours under the on-duty plan and under the recommended plan, shown as point estimates. |
|
Experimental: Sequence C
Stage 1: all 30 cases answered without AI information. Stage 2 (questionnaire version C): cases 1-10 under B2, cases 11-20 under B0, cases 21-30 under B1. |
Participants review the case together with the on-duty clinician's actual intravenous fluid rate and vasopressor dose and submit a final plan. No AI information is shown. This condition controls for the effect of a second review. As B0, plus the AI output: either a recommended dose level for intravenous fluid and for vasopressor, shown with the corresponding dose range, or a statement that the evidence is insufficient to recommend a change. As B1, with identical AI output, plus the model's predicted mean arterial pressure, lactate and urine output over the next 4 hours under the on-duty plan and under the recommended plan, shown as point estimates. |
Was misst die Studie?
Primäre Ergebnismessungen
Primäre Ergebnismessungen
Ergebnis Maßnahme |
Maßnahmenbeschreibung |
Zeitfenster |
|---|---|---|
|
Distance of final treatment decisions from the range of expert plans, B2 versus B1
Zeitfenster: At completion of the participant's stage-2 review (single online session, approximately 2 hours)
|
For each case, the number of dose levels by which the participant's final fluid and vasopressor plan lies outside the range of plans made independently by the expert panel (0 = within range), summed over the two drugs. Compared between B2 and B1 with a mixed-effects model with physician and case as crossed random effects, adjusted for stage-1 score, block position and sequence. |
At completion of the participant's stage-2 review (single online session, approximately 2 hours)
|
Sekundäre Ergebnismessungen
Sekundäre Ergebnismessungen
Ergebnis Maßnahme |
Maßnahmenbeschreibung |
Zeitfenster |
|---|---|---|
|
Discrimination between appropriate and inappropriate AI recommendations, B2 versus B1
Zeitfenster: At completion of the participant's stage-2 review (single online session, approximately 2 hours)
|
Among cases in which the AI recommends changing treatment, acceptance is defined as a final plan equal to the AI-recommended dose level for the changed drug. The outcome is the interaction between condition (B2 vs B1) and recommendation appropriateness (classified by the expert-range rule before analysis) on acceptance, estimated with a mixed-effects probit model with physician and case as crossed random effects. |
At completion of the participant's stage-2 review (single online session, approximately 2 hours)
|
Mitarbeiter und Ermittler
Sponsor
Sponsor
Mitarbeiter
Mitarbeiter
Studienaufzeichnungsdaten
Haupttermine studieren
Studienbeginn (Tatsächlich)
Studienbeginn
Primärer Abschluss (Geschätzt)
Primärer Abschluss
Studienabschluss (Geschätzt)
Studienabschluss
Studienanmeldedaten
Zuerst eingereicht
Zuerst eingereicht
Zuerst eingereicht, das die QC-Kriterien erfüllt hat
Zuerst eingereicht, das die QC-Kriterien erfüllt hat
Zuerst gepostet (Tatsächlich)
Zuerst gepostet
Studienaufzeichnungsaktualisierungen
Letztes Update gepostet (Tatsächlich)
Letztes Update gepostet
Letztes eingereichtes Update, das die QC-Kriterien erfüllt
Letztes eingereichtes Update, das die QC-Kriterien erfüllt
Zuletzt verifiziert
Zuletzt verifiziert
Mehr Informationen
Begriffe im Zusammenhang mit dieser Studie
Schlüsselwörter
Zusätzliche relevante MeSH-Bedingungen
Andere Studien-ID-Nummern
Andere Studien-ID-Nummern
- KY2026132
Plan für individuelle Teilnehmerdaten (IPD)
Planen Sie, individuelle Teilnehmerdaten (IPD) zu teilen?
Beschreibung des IPD-Plans
De-identified participant-level responses (treatment decisions, confidence, response times, condition assignments and participant characteristics without identifiers) will be shared. Patient case material derived from source databases will be shared only as permitted by their data use agreements.
Arzneimittel- und Geräteinformationen, Studienunterlagen
Studiert ein von der US-amerikanischen FDA reguliertes Arzneimittelprodukt
Studiert ein von der US-amerikanischen FDA reguliertes Geräteprodukt
Diese Informationen wurden ohne Änderungen direkt von der Website clinicaltrials.gov abgerufen. Wenn Sie Ihre Studiendaten ändern, entfernen oder aktualisieren möchten, wenden Sie sich bitte an register@clinicaltrials.gov. Sobald eine Änderung auf clinicaltrials.gov implementiert wird, wird diese automatisch auch auf unserer Website aktualisiert .