Point-of-Care AI Assistance and Critical Care Outcomes: A Randomized Trial (POC-AI-ICU)

Prospective Evaluation of a Point-of-Care Artificial Intelligence Model in Critical Care Outcomes

This is a prospective, unmasked, randomized, multicenter clinical trial evaluating the impact of point-of-care large language model (LLM)-based decision support on diagnostic accuracy and clinical outcomes in adult medical intensive care unit (MICU) patients.

Consecutive adult ICU admissions at participating community hospitals (initially MetroWest Medical Center and St. Vincent Hospital) will be screened for eligibility. Eligible patients will be randomized 1:1 to standard care or an AI-assisted group. In both arms, initial evaluation and management will follow usual practice. For patients randomized to AI assistance, de-identified admission data (history and physical, labs, imaging reports, and other relevant documentation) will be formatted and submitted to a state-of-the-art LLM (ChatGPT-5) at the time of admission. The AI-generated differential diagnosis and therapeutic recommendations will be provided to the admitting team for consideration. For the standard care arm, LLM output will be generated but not shared with clinicians.

After discharge, a masked chart review will determine the "ground truth" primary diagnosis and extract outcomes including: Primary Outcome - a composite of medical errors (from time of ICU admission through day 7 of ICU stay, or ICU discharge, whichever comes first); Secondary Outcomes - 90-day mortality, ICU and hospital length of stay, and ventilator-free days.

Study Overview

• Status: Not yet recruiting
• Conditions: Sepsis; Shock; Critical Illness; Acute Kidney Injury; Delirium Confusional State; Multi-organ Failure; Acute Respiratory Failure (ARF)
• Intervention / Treatment: Other: Point-of-care large language model decision support (ChatGPT-5)

Detailed Description

The rapid development of large language models (LLMs) such as ChatGPT has created new opportunities and risks for their use in medicine. Although early studies suggest high diagnostic accuracy in complex clinical scenarios and ICU admissions, the impact of LLMs on real-world clinical outcomes and the optimal mode of physician-AI interaction remain uncertain. Published work from our group showed that ChatGPT-4 achieved diagnostic accuracy comparable to board-certified intensivists for ICU admissions in a retrospective study. However, prospective, randomized data on clinical outcomes are lacking.

This trial will evaluate a pragmatic paradigm for integrating LLMs at the time of ICU admission (point-of-care AI). All eligible adult MICU admissions at participating sites will be prospectively randomized to: (1) standard care, or (2) AI-assisted care in which an LLM receives standardized, de-identified admission data and returns a proposed primary diagnosis, ranked differential diagnosis (up to five conditions), suggested additional information, and prioritized therapeutic interventions. Admitting clinicians in the AI-assisted arm will be asked to review and optionally incorporate the AI recommendations and will complete a brief questionnaire regarding perceived utility and any changes in diagnosis or management.

A masked clinical adjudication panel will perform longitudinal chart review to define the "ground truth" primary diagnosis and assess error rates and outcomes. The primary endpoint is a composite of medical errors. The specific time frame will be from the time of ICU admission through day 7 of ICU stay, or ICU discharge, whichever comes first. Secondary endpoints will include 90-day mortality, ICU and hospital length of stay, and ventilator-free days. Other exploratory secondary endpoints will be considered. The trial is designed to enroll approximately 1000 patients across multiple MICUs, with interim analysis at 12 months to assess feasibility, integrity, and futility. The study is minimal risk, uses de-identified data for AI queries, and does not alter standard diagnostic testing or therapeutic options.

• Study Type: Interventional
• Enrollment (Estimated): 1000
• Phase: Phase 2; Phase 1

Contacts and Locations

• Study Contact: Name: Eric Silverman, M.D. principal Investigator, M.D.; Phone Number: 508-344-5680; Email: esilverman@pamw.org

Study Locations

• United States
  • Massachusetts
    • Framingham, Massachusetts, United States, 01702
      • Framingham Union Hospital/MetroWest Medical Center
      • Contact: Eric Silverman, M.D.; Phone Number: 508-344-5680; Email: esilverman@pamw.org
      • Contact: Chih-Hsien Wu, M.D.; Email: lisa19950421@gmail.com

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: Adult; Older Adult
• Accepts Healthy Volunteers: No

Description

Inclusion Criteria:

1. Adult patients (≥ 18 years) admitted to the medical intensive care unit (MICU) at participating hospitals.
2. Direct admissions from the emergency department or transfers from medical wards to the MICU.
3. Critically ill patients meeting local ICU admission criteria.

Exclusion Criteria:

1. Transfers to the MICU from outside hospitals, operating room, or post-anesthesia care unit.
2. Age < 18 years.
3. Incomplete or missing essential clinical information at admission (e.g., key labs or documentation not yet available).
4. Primary surgical or cardiac (e.g., STEMI) patients.
5. Pregnant or postpartum women.
6. Prisoners.

Study Plan

How is the study designed?

Design Details

• Primary Purpose: Treatment
• Allocation: Randomized
• Interventional Model: Parallel Assignment
• Masking: Double

Number of Arms

2

Arms and Interventions

Participant Group / Arm	Intervention / Treatment
No Intervention: Standard Care; Patients receive usual ICU care per local practice. De-identified admission data may be processed and submitted to the LLM for research purposes, but AI output is not shared with treating clinicians and does not influence real-time management.
Other: AI-Assisted Care; Patients receive standard ICU care plus point-of-care LLM-based decision support at admission. De-identified admission data are formatted and submitted to an LLM (ChatGPT-5). The model returns a primary diagnosis, ranked differential diagnosis list, suggested additional information, and prioritized therapeutic recommendations. This output is provided to the admitting team for consideration in ongoing management.	Other: Point-of-care large language model decision support (ChatGPT-5); Use of a large language model (ChatGPT-5) to analyze de-identified ICU admission data (history, physical examination, laboratory results, imaging reports, and other documentation) at the time of admission. The model generates diagnostic and therapeutic recommendations that are shared with clinicians in the AI-assisted arm only.

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Composite of Medical Errors	Proportion of patients with at least one clinically important diagnostic or therapeutic error identified by masked chart review (e.g., missed or delayed critical diagnosis, major guideline-discordant therapy with potential for harm).	From the time of ICU admission through day 7 of ICU stay or ICU discharge, whichever comes first.

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
90-day All-Cause Mortality	All-cause mortality within 90 days of index ICU admission, as determined by chart review and available follow-up records.	90 days from ICU admission.
ICU Length of Stay	Total number of days spent in the ICU during the index hospitalization.	From ICU admission to ICU discharge (up to 90 days).
Ventilator-Free Days	Number of days alive and free from invasive mechanical ventilation during the first 28 days after ICU admission.	Up to 28 days after ICU admission.
Hospital Length of Stay	Total number of days from hospital admission to hospital discharge during the index hospitalization.	From hospital admission to hospital discharge (up to 90 days).

Collaborators and Investigators

• Sponsor: MetroWest Artificial Intelligence Research Workgroup
• Investigators: Principal Investigator: Eric Silverman, M.D., MetroWest Medical Center and St. Vincent Hospital

Publications and helpful links

General Publications

• Singh J, Bohra R, Mukhtiar V, Fernandes W, Bhanushali C, Chinnamuthu R, Kanamgode SS, Ellis J, Silverman E. Diagnostic Accuracy of a Large Language Model (ChatGPT-4) for Patients Admitted to a Community Hospital Medical Intensive Care Unit: A Retrospective Case Study. J Intensive Care Med. 2025 Aug 17:8850666251368270. doi: 10.1177/08850666251368270. Online ahead of print.

Study record dates

Study Major Dates

• Study Start (Estimated): January 1, 2026
• Primary Completion (Estimated): December 31, 2028
• Study Completion (Estimated): June 30, 2029

Study Registration Dates

• First Submitted: November 17, 2025
• First Submitted That Met QC Criteria: December 15, 2025
• First Posted (Actual): December 18, 2025

Study Record Updates

• Last Update Posted (Actual): December 18, 2025
• Last Update Submitted That Met QC Criteria: December 15, 2025
• Last Verified: December 1, 2025

More Information

Terms related to this study

• Keywords: Clinical Decision Support; Shock; Intensive Care Unit; Artificial Intelligence; Sepsis; Diagnostic Accuracy; Critical Care; Acute Respiratory Failure; Critical Care Outcomes; Multiorgan Failure; Large Language Model
• Additional Relevant MeSH Terms: Urogenital Diseases; Neurologic Manifestations; Nervous System Diseases; Pathologic Processes; Male Urogenital Diseases; Kidney Diseases; Urologic Diseases; Female Urogenital Diseases; Female Urogenital Diseases and Pregnancy Complications; Disease Attributes; Infections; Neurobehavioral Manifestations; Systemic Inflammatory Response Syndrome; Inflammation; Renal Insufficiency; Pathological Conditions, Signs and Symptoms; Signs and Symptoms; Critical Illness; Acute Kidney Injury; Sepsis; Shock; Multiple Organ Failure; Confusion
• Other Study ID Numbers: POC-AI-ICU-001; IRB#2025-067 (Other Identifier: MetroWest Medical Center and St. Vincent Hospital)

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: YES
• IPD Plan Description: De-identified individual participant data will be retained for reanalysis by the study team. There is no current plan for routine public sharing of individual participant-level data, but de-identified datasets may be shared with qualified investigators upon reasonable request and appropriate data use agreements.
• IPD Sharing Supporting Information Type: STUDY_PROTOCOL; SAP; CSR

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No