Artificial Intelligent Clinical Decision Support System Simulation Center Study for Technology Acceptance
May 20, 2026 updated by: Yale University
Artificial Intelligent Clinical Decision Support System Simulation Center Study: Trust and Usefulness of Machine Learning Risk Stratification Tool for Acute Gastrointestinal Bleeding Using the Technology Acceptance Model
The purpose of this research study is to measure the effect on of a large language model interface on the usability, attitudes, and provider trust when using a machine learning algorithm-based clinical decision support system in the setting of bleeding from the upper gastrointestinal tract (upper GIB).
Specifically, the investigators are looking to assess the optimal implementation of such machine learning algorithms in simulation scenarios to best engender trust and improve usability.
Participants will be randomized to either machine learning algorithm alone or algorithm with a large language model interface and exposed to simulation cases of upper GIB.
Study Overview
Detailed Description
The experiment will deploy a previously validated machine learning algorithm trained on existing clinical datasets within simulation scenarios in which a patient with acute gastrointestinal bleeding (at low, moderate, and high risk for poor outcome) is evaluated.
Prior to the simulation, a baseline educational module about artificial intelligence, machine learning, and clinical decision support will be provided to all participants. The investigators will establish psychological safety by detailing what is available in the room, the opportunity to call a consultant, and availability of laboratory and radiographic studies. Each clinical scenario will run for approximately 10 minutes based on real patient cases where vital signs change over time and laboratory values are made available at specific points in the assessment. The study will evaluate the effect of a large language model-based interaction with the machine learning algorithm with interpretability dashboard compared to the machine learning algorithm with interpretability dashboard alone. Each participant will receive three scenarios in randomized order of risk.
For the large language model interaction arm, participants will be provided the computer workstation a LLM chatbot interface of the algorithm and interpretability dashboard For the machine learning dashboard arm, participants will be provided the computer workstation with the algorithm and interpretability dashboard.
Study Type
Interventional
Enrollment (Actual)
108
Phase
- Not Applicable
Contacts and Locations
This section provides the contact details for those conducting the study, and information on where this study is being conducted.
Study Locations
-
-
Connecticut
-
New Haven, Connecticut, United States, 06510
- Yale New Haven Hospital
-
-
Participation Criteria
Researchers look for people who fit a certain description, called eligibility criteria. Some examples of these criteria are a person's general health condition or prior treatments.
Eligibility Criteria
Ages Eligible for Study
- Child
- Adult
- Older Adult
Accepts Healthy Volunteers
Yes
Description
Inclusion Criteria:
- Internal Medicine residency trainees at study institution
- Emergency Medicine residency trainees at study institution
Exclusion Criteria:
- N/A
Study Plan
This section provides details of the study plan, including how the study is designed and what the study is measuring.
How is the study designed?
Design Details
- Primary Purpose: Health Services Research
- Allocation: N/A
- Interventional Model: Parallel Assignment
- Masking: None (Open Label)
Arms and Interventions
Participant Group / Arm |
Intervention / Treatment |
|---|---|
|
Experimental: Large Language Model-based Interaction
LLM-powered chatbot with the machine learning dashboard to provide the risk assessment and provide rationale based on interpretability metrics provided by the dashboard in which study participants can directly interact with using natural language.
Participants will be provided the Generative Pre-trained Transformer (GPT) chatbot powered machine learning model dashboard.
|
Use of a Large Language Model (LLM) chatbot interface to Interact with the Machine Learning Algorithm and interpretability dashboard.
|
|
No Intervention: Machine Learning Dashboard
Machine learning algorithm output with an interactive dashboard that can be used to explain, or interpret the input factors that contribute most towards the generated risk score.
Participants will have access to the machine learning dashboard only.
|
What is the study measuring?
Primary Outcome Measures
Outcome Measure |
Measure Description |
Time Frame |
|---|---|---|
|
Median Change in Attitudes Towards Machine Learning Algorithms in Clinical Care Using UTAUT
Time Frame: Approximately 60 minutes
|
The study will use a common set of dependent variables to assess baseline and post-intervention attitudes towards machine learning algorithms in clinical care using an adapted Unified Theory of Acceptance and Use of Technology (UTAUT) survey assessing perceived usefulness of the system, perceived ease of use, attitudes towards using it, behavioral intentions, and trust, measured with a 5-point Likert scale.
Percent change in UTAUT survey response between Large Language Model-based Interaction and Machine Learning Dashboard at recruitment prior to administration of scenarios and immediately after completion of scenarios.
The difference in time between the two will be approximately 60 minutes.
Higher change indicates greater acceptance/intention to use the GutGPT+Dashboard.
|
Approximately 60 minutes
|
Secondary Outcome Measures
Outcome Measure |
Measure Description |
Time Frame |
|---|---|---|
|
Clinician Decision Making of Triage of GI Bleeding
Time Frame: Approximately 60 minutes
|
Mean percentage of decision accuracy per participant.
Accuracy is defined as the percentage of times participants accurately choose the correct clinical decision for each simulation scenario of acute upper GI bleeding for each treatment condition.
Immediately after completion of scenarios (60 minutes from initiation of study for each participant).
No further follow up afterwards.
|
Approximately 60 minutes
|
Collaborators and Investigators
This is where you will find people and organizations involved with this study.
Sponsor
Investigators
- Principal Investigator: Dennis Shung, MD, Yale School of Medicine Section of Digestive Diseases
Publications and helpful links
The person responsible for entering information about the study voluntarily provides these publications. These may be about anything related to the study.
General Publications
- Laine L, Jensen DM. Management of patients with ulcer bleeding. Am J Gastroenterol. 2012 Mar;107(3):345-60; quiz 361. doi: 10.1038/ajg.2011.480. Epub 2012 Feb 7.
- Laine L. Risk Assessment Tools for Gastrointestinal Bleeding. Clin Gastroenterol Hepatol. 2016 Nov;14(11):1571-1573. doi: 10.1016/j.cgh.2016.08.003. Epub 2016 Aug 10. No abstract available.
- Leonardi, P. M. 2009. Why do people reject new technologies and stymie organizational changes of which they are in favor? Exploring misalignments between social interactions and materiality. Human Communication Research, 35(3): 407-441.
- Venkatesh, V., Morris, M. G., Davis, G. B., & Davis, F. D. (2003). User Acceptance of Information Technology: Toward a Unified View. MIS Quarterly, 27(3), 425-478
- Chung S, Giuffre M, Rajashekar N, Pu Y, Shin YE, Kresevic S, Chan C, Nakamura-Sakai S, You K, Saarinen T, Hsiao A, Wong AH, Evans L, McCall T, Kizilcec RF, Sekhon J, Laine L, Shung DL. Usability and adoption in a randomized trial of GutGPT a GenAI tool for gastrointestinal bleeding. NPJ Digit Med. 2025 Aug 18;8(1):527. doi: 10.1038/s41746-025-01896-5.
Study record dates
These dates track the progress of study record and summary results submissions to ClinicalTrials.gov. Study records and reported results are reviewed by the National Library of Medicine (NLM) to make sure they meet specific quality control standards before being posted on the public website.
Study Major Dates
Study Start (Actual)
May 23, 2023
Primary Completion (Actual)
December 31, 2024
Study Completion (Actual)
December 31, 2024
Study Registration Dates
First Submitted
March 13, 2023
First Submitted That Met QC Criteria
April 14, 2023
First Posted (Actual)
April 18, 2023
Study Record Updates
Last Update Posted (Actual)
May 22, 2026
Last Update Submitted That Met QC Criteria
May 20, 2026
Last Verified
May 1, 2026
More Information
Terms related to this study
Keywords
Additional Relevant MeSH Terms
Other Study ID Numbers
- 2000034521
- 1K23DK125718-01A1 (U.S. NIH Grant/Contract)
Plan for Individual participant data (IPD)
Plan to Share Individual Participant Data (IPD)?
NO
Drug and device information, study documents
Studies a U.S. FDA-regulated drug product
No
Studies a U.S. FDA-regulated device product
No
This information was retrieved directly from the website clinicaltrials.gov without any changes. If you have any requests to change, remove or update your study details, please contact register@clinicaltrials.gov. As soon as a change is implemented on clinicaltrials.gov, this will be updated automatically on our website as well.