Artificial Intelligence as a Decision Making Tool in Emergency Department

July 29, 2026 updated by: Shahar Shelly MD, Rambam Health Care Campus

Artificial Intelligence as a Decision Making Tool in Emergency Medicine

This study will evaluate the performance of a large language model (LLM)-based clinical decision support system in the emergency department at Rambam Health Care Campus. The system analyzes structured patient data from the electronic health record and generates diagnostic and treatment recommendations for physicians.

The study will assess the system's ability to support diagnostic reasoning, its impact on diagnostic accuracy when used by physicians, and its perceived clinical usefulness. In addition, a retrospective analysis of de-identified patient records will be conducted to compare LLM-generated recommendations with actual clinical outcomes, including diagnosis, disposition decisions, and length of stay.

The study will also examine the performance of the system in a multilingual clinical environment where both Hebrew and English are used in medical documentation and communication.

Study Overview

Status

Status

Indicates the current recruitment status or the expanded access status.
Active, not recruiting

Conditions

Conditions

The disease, disorder, syndrome, illness, or injury that is being studied. On ClinicalTrials.gov, conditions may also include other health-related issues, such as lifespan, quality of life, and health risks.

Intervention / Treatment

Intervention / Treatment

A process or action that is the focus of a clinical study. Interventions include drugs, medical devices, procedures, vaccines, and other products that are either investigational or already available. Interventions can also include noninvasive approaches, such as education or modifying diet and exercise.

Detailed Description

This is a mixed-methods study combining a prospective controlled component and a retrospective chart review.

Prospective Component

  • Setting: Emergency Department, Rambam Health Care Campus
  • The LLM will receive structured patient input (chief complaint, vitals, relevant history, laboratory and imaging results) via a secure interface.
  • LLM-generated recommendations will be logged and made available to the treating physician; final clinical decisions remain entirely with the physician.
  • The system operates in decision-support mode only it does not autonomously initiate any clinical action.

Retrospective Component

• De-identified historical ED records will be used to evaluate LLM performance against documented clinical outcomes.

Primary metrics: diagnostic concordance, appropriateness of suggested workup, and disposition accuracy.

Study Type

Study Type

Describes the nature of a clinical study. Study types include interventional studies (also called clinical trials), observational studies (including patient registries), and expanded access.
Observational

Enrollment (Estimated)

Enrollment

The number of participants in a clinical study. The "anticipated" enrollment is the target number of participants that the researchers need for the study.
100000

Contacts and Locations

This section provides the contact details for those conducting the study, and information on where this study is being conducted.

Study Locations

  • Israel
      • Haifa, Israel, 3109601
        • Rambam healthcare campus

Participation Criteria

Researchers look for people who fit a certain description, called eligibility criteria. Some examples of these criteria are a person's general health condition or prior treatments.

Eligibility Criteria

Eligibility Criteria

The key requirements that people who want to participate in a clinical study must meet or the characteristics they must have. Eligibility criteria consist of both inclusion criteria (which are required for a person to participate in the study) and exclusion criteria (which prevent a person from participating). Types of eligibility criteria include whether a study accepts healthy volunteers, has age or age group requirements, or is limited by sex.

Ages Eligible for Study

  • Child
  • Adult
  • Older Adult

Accepts Healthy Volunteers

No

Sampling Method

Non-Probability Sample

Study Population

All patients (0-120 years) receiving care in emergency departments (main and pediatric)

Description

Inclusion Criteria:

Patients aged 0 to 120 years presented to the emergency department

Exclusion Criteria:

None

Study Plan

This section provides details of the study plan, including how the study is designed and what the study is measuring.

How is the study designed?

Design Details

Cohorts and Interventions

Group / Cohort

Group / Cohort

A group or subgroup of participants in an observational study that is assessed for biomedical or health outcomes.
Evaluation With AI
A scenario in which the physician receives real-time recommendations only from the model before making the final decision (the final decision will be called on the basis of senior attending, and the treating physician)
Evaluation Without AI
A scenario in which the physician is not exposed to the model's recommendations.

What is the study measuring?

Primary Outcome Measures

Primary Outcome Measures

In a clinical study's protocol, the planned outcome measure that is the most important for evaluating the effect of an intervention/treatment. Most clinical studies have one primary outcome measure, but some have more than one.
Outcome Measure
Measure Description
Time Frame
Length of Stay in Emergency Department
Time Frame: From ED registration until discharge from the emergency department or admission to a hospital ward, assessed up to 24 hours
Time from ED registration to discharge from emergency department or admission to a hospital ward, focusing in addition on consultation cycle time.
From ED registration until discharge from the emergency department or admission to a hospital ward, assessed up to 24 hours

Secondary Outcome Measures

Secondary Outcome Measures

In a clinical study's protocol, a planned outcome measure that is not as important as the primary outcome measure for evaluating the effect of an intervention but is still of interest. Most clinical studies have more than one secondary outcome measure.
Outcome Measure
Measure Description
Time Frame
Cost of Running the LLM
Time Frame: Up to 3 years

Average daily cost (in USD) to operate the large language model.

Description:

This outcome will measure the direct computational and licensing costs incurred by running the LLM in the clinical setting.

Unit of Measure: US Dollars (per day). How It Is Assessed: The total daily spend for the LLM (e.g., cloud-compute fees, licensing fees) will be recorded and divided by the total number of patient encounters that day to derive an average cost.
Up to 3 years
Staff Compliance With AI
Time Frame: Up to 3 years

This outcome will assess how often (compliance) and how willingly (acceptance) physicians, residents, and nurses use the LLM in eligible clinical scenarios.

Description: Determines how often physicians, residents, and nurses use the LLM for eligible patient encounters.

Unit of Measure: Proportion (0.0-1.0) or Percent (0-100%) How Assessed: Tracks the number of encounters in which the LLM is actually used, divided by the total number of eligible encounters. Higher percentages indicate greater compliance.
Up to 3 years
Transparency/Explainability
Time Frame: 3 years

Description: Evaluates how clearly the LLM's reasoning is communicated to clinical staff.

Unit of Measure: Score on a 5-point Likert scale (1 = not at all clear, 5 = extremely clear) How Assessed: After each LLM-guided decision, the resident or attending physician rates the clarity of the model's explanation. A mean score is reported. Higher scores indicate better clarity.
3 years
Quality of LLM-Generated Clinical Reports
Time Frame: Up to 3 years

Accuracy and Completeness Score of the LLM-Generated Clinical Report This outcome evaluates how accurately and comprehensively the LLM summarizes a patient's clinical encounter and recommended plan in a structured report (e.g., discharge instructions, progress notes).

Unit(s) of Measure: Quality Score based on a modified SOAP (Subjective, Objective, Assessment, Plan) Note Rating Scale, ranging from 0 to 10, where higher scores indicate better documentation quality. The score reflects the inclusion of required elements and correctness of clinical details.

Assessment Method: Each LLM-generated report is independently evaluated by a panel of attending physicians or trained clinical staff. Using a standardized checklist, reviewers assess completeness (e.g., inclusion of symptoms, physical findings, diagnosis, and plan) and accuracy (e.g., correct medications, identifiers, and diagnoses), then assign a score based on the predefined rubric.
Up to 3 years
Staff Acceptance of AI
Time Frame: Up to 3 years

Description: Evaluates how willingly staff integrate the LLM into their clinical workflow.

Unit of Measure: 5-point Likert scale (1 = strongly disagree, 5 = strongly agree) How Assessed: After each LLM-assisted encounter, staff complete a brief survey about perceived helpfulness and willingness to reuse the LLM. Higher scores indicate greater acceptance.
Up to 3 years

Other Outcome Measures

Other Outcome Measures

For clinical trials, a planned measurement described in the protocol that is used to determine the effect of an intervention/treatment on participants. For observational studies, a measurement or observation that is used to describe patterns of diseases or traits, or associations with exposures, risk factors, or treatment. Types of outcome measures include primary outcome measure and secondary outcome measure.
Outcome Measure
Measure Description
Time Frame
The study is organized around four pre-specified aims:
Time Frame: 3 years
Aim 1:LLM Diagnostic & Treatment Recommendation Appropriateness Appropriateness of LLM recommendations rated by senior clinicians (1=inappropriate, 5=appropriate) Timeframe:ED registration to discharge or inpatient admission, up to 24h Aim 2:Diagnostic Accuracy Rate- LLM-Assisted vs. Standard Care Clinical Decision-Making Proportion of correct diagnoses in LLM-assisted vs. standard care (%), matched to discharge diagnosis Timeframe:ED registration to final diagnosis, up to 24h Aim 3:Clinician-Rated Utility & Usability of LLM Outputs- SUS and Likert Scale Utility measured via SUS (0-100) and 5-point Likert rating, collected post-encounter with qualitative feedback Timeframe:End of each clinical encounter,up to 36 months Aim 4:LLM Retrospective Benchmark-Percent Agreement & Cohen's Kappa vs. Actual Clinical Outcomes Agreement between LLM recommendations and actual outcomes (diagnosis, disposition, LOS) in de-identified records Timeframe:Records up to 36 months prior to study initiation
3 years
Additional Pre-Specified Outcome Measures for Both Emergency Departments Due to Expansion to Pediatrics:
Time Frame: 3 years

Aim 1: Change in Specialist Consultations per ED Visit Internal specialist consults per daily case volume (trauma excluded) vs. 5-year mean; decrease = better decision-making and resource use.

Timeframe: Per ED visit (≤24h); monthly, 36 months Aim 2: Change in Ancillary Tests (Imaging & Laboratory) per ED Visit Imaging (X-ray, US, CT) and lab tests per daily case volume vs. 5-year mean; decrease = better decisions, less radiation.

Timeframe: Per ED visit (≤24h); monthly, 36 months Aim 3: Discharge Letter Quality Score (LLM-as-a-Judge) Automated quality score per discharge letter: Safety, Compliance, Patient Communication (1-10; higher=better); 20% manually validated vs. 200-letter baseline. Target: ≥15% improvement.

Timeframe: At closure of each discharge letter, up to 36 months Aim 4: Return Visits to ED Unplanned return within 7 days vs. prior periods; goal: rate maintenance (non-inferiority), confirming safety.

Timeframe: 7 days from discharge; monthly, 36 months
3 years

Collaborators and Investigators

This is where you will find people and organizations involved with this study.

Sponsor

Sponsor

The organization or person who initiates the study and who has authority and control over the study.
Rambam Health Care Campus

Collaborators

Collaborators

An organization other than the sponsor that provides support for a clinical study. This support may include activities related to funding, design, implementation, data analysis, or reporting.
Technion, Israel Institute of Technology

Investigators

Investigators

A researcher involved in a clinical study. Related terms include site principal investigator, site sub-investigator, study chair, study director, and study principal investigator.
  • Principal Investigator: Shahar Shelly, MD, Rambam Health Care Campus

Publications and helpful links

The person responsible for entering information about the study voluntarily provides these publications. These may be about anything related to the study.

General Publications

Study record dates

These dates track the progress of study record and summary results submissions to ClinicalTrials.gov. Study records and reported results are reviewed by the National Library of Medicine (NLM) to make sure they meet specific quality control standards before being posted on the public website.

Study Major Dates

Study Start (Actual)

Study Start

The actual date on which the first participant was enrolled in a clinical study. The "anticipated" study start date is the date that the researchers think will be the study start date.
January 1, 2000

Primary Completion (Estimated)

Primary Completion

The date on which the last participant in a clinical study was examined or received an intervention to collect final data for the primary outcome measure. Whether the clinical study ended according to the protocol or was terminated does not affect this date. For clinical studies with more than one primary outcome measure with different completion dates, this term refers to the date on which data collection is completed for all the primary outcome measures. The "anticipated" primary completion date is the date that the researchers think will be the primary completion date for the study.
September 1, 2026

Study Completion (Estimated)

Study Completion

The date on which the last participant in a clinical study was examined or received an intervention/treatment to collect final data for the primary outcome measures, secondary outcome measures, and adverse events (that is, the last participant's last visit). The "anticipated" study completion date is the date that the researchers think will be the study completion date.
September 1, 2026

Study Registration Dates

First Submitted

First Submitted

The date on which the study sponsor or investigator first submitted a study record to ClinicalTrials.gov. There is typically a delay of a few days between the first submitted date and the record's availability on ClinicalTrials.gov (the first posted date).
November 13, 2024

First Submitted That Met QC Criteria

First Submitted That Met QC Criteria

The date on which the study sponsor or investigator first submits a study record that is consistent with National Library of Medicine (NLM) quality control (QC) review criteria. The sponsor or investigator may need to revise and submit a study record one or more times before NLM's QC review criteria are met. It is the responsibility of the sponsor or investigator to ensure that the study record is consistent with the NLM QC review criteria.
March 27, 2025

First Posted (Actual)

First Posted

The date on which the study record was first available on ClinicalTrials.gov after National Library of Medicine (NLM) quality control (QC) review has concluded. There is typically a delay of a few days between the date the study sponsor or investigator submitted the study record and the first posted date.
March 30, 2025

Study Record Updates

Last Update Posted (Actual)

Last Update Posted

The most recent date on which changes to a study record were made available on ClinicalTrials.gov. There may be a delay between when the changes were submitted to ClinicalTrials.gov by the study's sponsor or investigator (the last update submitted date) and the last update posted date.
July 31, 2026

Last Update Submitted That Met QC Criteria

Last Update Submitted That Met QC Criteria

The most recent date on which the study sponsor or investigator submitted changes to a study record that are consistent with National Library of Medicine (NLM) quality control (QC) review criteria. It is the responsibility of the sponsor or investigator to ensure that the study record is consistent with the NLM QC review criteria.
July 29, 2026

Last Verified

Last Verified

The most recent date on which the study sponsor or investigator confirmed the information about a clinical study on ClinicalTrials.gov as accurate and current. If a study with a recruitment status of recruiting; not yet recruiting; or active, not recruiting has not been confirmed within the past 2 years, the study's recruitment status is shown as unknown.
July 1, 2026

More Information

Terms related to this study

Keywords

Additional Relevant MeSH Terms

Other Study ID Numbers

Other Study ID Numbers

Identifiers or ID numbers other than the NCT number that are assigned to a clinical study by the study's sponsor, funders, or others. These numbers may include unique identifiers from other trial registries and National Institutes of Health grant numbers.
  • 0026-24RMB

Plan for Individual participant data (IPD)

Plan to Share Individual Participant Data (IPD)?

NO

Drug and device information, study documents

Studies a U.S. FDA-regulated drug product

No

Studies a U.S. FDA-regulated device product

No

This information was retrieved directly from the website clinicaltrials.gov without any changes. If you have any requests to change, remove or update your study details, please contact register@clinicaltrials.gov. As soon as a change is implemented on clinicaltrials.gov, this will be updated automatically on our website as well.

Cookie Settings

We use cookies to ensure that we give you the best experience on our website. For more details carefully read our Privacy and Cookies Policy. ...>>>You can change your preferences or withdraw your consent at any time by deleting the cookies from your website or computer as described in the policy. Please accept it and choose your preferences by ticking the corresponding boxes in the "Manage Settings" section below. Please carefully read our Terms of Service before you proceed with using any part of this website.
«Manage Settings