Large Language Models To Improve the Quality of Care of Cardiology Patients

May 13, 2025 updated by: Jack O'Sullivan, Stanford University

Towards Bridging Generalists to Subspecialists With Large Language Models

This study evaluates the impact of large language models (LLMs) versus traditional decision support tools on clinical decision-making in cardiology. General cardiologists will be randomized to manage real patient cases from a cardiovascular genetic cardiomyopathy clinic, with or without AI assistance. Each case will be assessed by two cardiologists, and their responses will be graded by blinded subspecialty experts using a standardized evaluation rubric.

Study Overview

Status

Recruiting

Conditions

Intervention / Treatment

Detailed Description

Large language models have been shown to improve physician performance in simulated settings. Large language models have demonstrated promise in various healthcare contexts, including medical note-writing, addressing patient inquiries, and facilitating medical consultation. However, it remains uncertain whether large language models improve clinical reasoning of clinicians using real world cases.

Clinicians dedicate years of training to develop expertise, with clinical knowledge a key component. Clinicians have different areas of expertise, from generalists spanning diseases of all organ systems and patients of all ages, to subspecialists dedicated to often a handful of diseases effecting a specific organ. Both skill sets are vital to a well-functioning medical system, as generalists generally care for patients and refer to specialists when dedicated, specialty knowledge is required. There is a paucity of specialists, and thus the quality of triaging and referral to specialists is of upmost importance. We hypothesis that large language models may be able help generalists management complex patients, and improve their triage to specialists and subspecialists.

The scarcity of subspecialist medical expertise, particularly in rare, complex and life-threatening diseases, poses a significant challenge for healthcare delivery. This issue is particularly acute in cardiology where timely, accurate management determines outcomes. In this study, we will recruit General Cardiologists as participants who will be randomized to answer clinical management cases with or without access to a large language model. Each case is a real patient case of a patient referred to a subspeciality cardiovascular genetic cardiomyopathy clinic. Each case will be performed by two general cardiologists (one with access to a large language model and one without access). Each case has multiple components, and the participants will be asked to answer questions related to the management. Answers will be graded by independent, blinded subspeciality Cardiologists with expertise and training in genetic cardiomyopathies. An evaluation rubric was developed by 10 expert discussants.

Study Type

Interventional

Enrollment (Estimated)

12

Phase

  • Not Applicable

Contacts and Locations

This section provides the contact details for those conducting the study, and information on where this study is being conducted.

Study Contact

Study Contact Backup

Study Locations

  • United States
    • California
      • Palo Alto, California, United States, 94303
        • Recruiting
        • Stanford
        • Contact:
        • Principal Investigator:
          • Euan A Ashley, MD, PhD
        • Sub-Investigator:
          • Jack W O'Sullivan, MD, PhD
        • Contact:

Participation Criteria

Researchers look for people who fit a certain description, called eligibility criteria. Some examples of these criteria are a person's general health condition or prior treatments.

Eligibility Criteria

Ages Eligible for Study

  • Adult
  • Older Adult

Accepts Healthy Volunteers

No

Description

Inclusion Criteria:

  • Board certified or board eligible Cardiologist.

Exclusion Criteria:

  • Not currently practicing clinically

Study Plan

This section provides details of the study plan, including how the study is designed and what the study is measuring.

How is the study designed?

Design Details

  • Primary Purpose: Supportive Care
  • Allocation: Randomized
  • Interventional Model: Parallel Assignment
  • Masking: Single

Arms and Interventions

Participant Group / Arm
Intervention / Treatment
Active Comparator: Large Language Model
This group will be given access to a Large Language Model
Other: Large Language Model
The intervention is a Large Language Model.
Other Names:
  • AMIE (Articulate Medical Intelligence Explorer)
No Intervention: Usual resources
Group will not be given access to a Large Language Model but will be encouraged to use any resources they usually use in their practice besides large language models (UpToDate, Dynamed etc).

What is the study measuring?

Primary Outcome Measures

Outcome Measure
Measure Description
Time Frame
Subspecialist Preference
Time Frame: Subspecialist evaluation will occur within 1 month of participant completing their assessment
The primary outcome is the preference of the subspecialist between answers provided by a) Cardiologist with access to Large Language Model vs. b) Cardiologist without access to Large Language Model.
Subspecialist evaluation will occur within 1 month of participant completing their assessment

Secondary Outcome Measures

Outcome Measure
Measure Description
Time Frame
Participants perspective on use of Large Language model
Time Frame: Within one-hour
Percentage of Cardiologists that felt the use of the Large Language Model helped their assessment.
Within one-hour

Collaborators and Investigators

This is where you will find people and organizations involved with this study.

Sponsor

Stanford University

Collaborators

Google LLC.

Investigators

  • Principal Investigator: Euan A Ashley, BSc, MB ChB, DPhil, Stanford University

Study record dates

These dates track the progress of study record and summary results submissions to ClinicalTrials.gov. Study records and reported results are reviewed by the National Library of Medicine (NLM) to make sure they meet specific quality control standards before being posted on the public website.

Study Major Dates

Study Start (Actual)

January 10, 2025

Primary Completion (Estimated)

November 1, 2025

Study Completion (Estimated)

December 1, 2025

Study Registration Dates

First Submitted

April 11, 2025

First Submitted That Met QC Criteria

April 17, 2025

First Posted (Actual)

April 20, 2025

Study Record Updates

Last Update Posted (Actual)

May 15, 2025

Last Update Submitted That Met QC Criteria

May 13, 2025

Last Verified

May 1, 2025

More Information

Terms related to this study

Keywords

Additional Relevant MeSH Terms

Other Study ID Numbers

  • 78695

Plan for Individual participant data (IPD)

Plan to Share Individual Participant Data (IPD)?

YES

IPD Plan Description

The patient cases that will be used in this study will be deidentified and made publicly available. The code to conduct the statistical analysis will also be made available.

IPD Sharing Time Frame

The deidentified patient cases and statistical analysis code will be made available within 6 months of study completion.

IPD Sharing Access Criteria

It will be made publicly available and accessible by all.

IPD Sharing Supporting Information Type

  • STUDY_PROTOCOL
  • ANALYTIC_CODE

Drug and device information, study documents

Studies a U.S. FDA-regulated drug product

No

Studies a U.S. FDA-regulated device product

No

product manufactured in and exported from the U.S.

No

This information was retrieved directly from the website clinicaltrials.gov without any changes. If you have any requests to change, remove or update your study details, please contact register@clinicaltrials.gov. As soon as a change is implemented on clinicaltrials.gov, this will be updated automatically on our website as well.

Subscribe