Using Large Language Models Such As GPT-4 to Assess Guideline Adherence in Patients With Chronic Obstructive Pulmonary Disease (IMPL-AI-MENT)

According to studies in the US and the Netherlands, 33-40% of patients with chronic conditions receive care that does not follow guideline recommendations. These findings have also been demonstrated in the management of COPD. This leads to under- or over-treatment of patients and, in the case of COPD, to exacerbations and hospitalisations. These exacerbations are a significant clinical problem, affecting patient's lung function, quality of life and mortality. They are also a burden on the healthcare system. Technological advances in artificial intelligence offer the opportunity to address these issues in COPD management. In the past year, there have been remarkable innovations in the field of natural language processing, especially through large language models such as GPT-4 from OpenAI and Bard or Gemini from Google. These models offer an opportunity to improve the implementation of evidence-based care in clinical practice.

This study is a prospective, randomised trial that will compare therapy on discharge for patients with COPD. One arm will receive no intervention, while the other arm will receive a treatment recommendation from an LLM. The study will compare the percentage of patients treated according to the guideline.

Study Overview

• Status: Completed
• Conditions: COPD
• Intervention / Treatment: Other: LLM

• Study Type: Interventional
• Enrollment (Actual): 78
• Phase: Not Applicable

Contacts and Locations

Study Locations

• Germany
  • State of Berlin
    • Berlin, State of Berlin, Germany, 10117
      • Charite University

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: Adult; Older Adult
• Accepts Healthy Volunteers: No

Description

Inclusion Criteria:

• Diagnosis of COPD
• Consent
• Discharge after hospitalization

Exclusion Criteria:

• Lack of Consent

Study Plan

How is the study designed?

Design Details

• Primary Purpose: Treatment
• Allocation: Randomized
• Interventional Model: Parallel Assignment
• Masking: Triple

Number of Arms

2

Arms and Interventions

Participant Group / Arm	Intervention / Treatment
No Intervention: No Intervention; This arm will be treated as usual. No intervention will be performed.
Active Comparator: LLM Assessement; During hospital stay and after written informed consent, an LLM is asked to indicate if the treatment the patient receives is guideline-concordant. The information is ascertained by two study physicians (human-in-the-loop) and later provided to the treating physician who can recommend a change in therapy to the patient (outside of the study).	Other: LLM; A LLM-based comparison between treatment and guideline.

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Adherence to treatment guidelines at the time of hospital discharge	The primary endpoint will assess whether the treatment at the time of discharge is consistent with the guidelines' recommendations. This is a binary outcome measure of yes or no.	From date of admission (which is enrollment) to the date of discharge, assessed up to one month
Percentage of patients treated in concordance with treatment guidelines at the time of hospital discharge	This primary endpoint will assess the percentage of guideline-concordant treatments in each study arm.	From date of admission (which is enrollment) to the date of discharge, assessed up to one month

Collaborators and Investigators

• Sponsor: Charite University, Berlin, Germany
• Investigators: Principal Investigator: Matthias Gröschel, MD PhD, Charité

Study record dates

Study Major Dates

• Study Start (Actual): May 15, 2024
• Primary Completion (Actual): April 1, 2025
• Study Completion (Actual): April 1, 2025

Study Registration Dates

• First Submitted: April 24, 2024
• First Submitted That Met QC Criteria: May 7, 2024
• First Posted (Actual): May 13, 2024

Study Record Updates

• Last Update Posted (Actual): March 20, 2026
• Last Update Submitted That Met QC Criteria: March 18, 2026
• Last Verified: March 1, 2026

More Information

Terms related to this study

• Additional Relevant MeSH Terms: Pathologic Processes; Chronic Disease; Disease Attributes; Respiratory Tract Diseases; Lung Diseases; Lung Diseases, Obstructive; Pathological Conditions, Signs and Symptoms; Pulmonary Disease, Chronic Obstructive
• Other Study ID Numbers: EA2/322/23

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: YES
• IPD Plan Description: Access to trial IPD can be requested by qualified researchers engaging in independent scientific research, and will be provided following review and approval of a research proposal and Statistical Analysis Plan (SAP) and execution of a Data Sharing Agreement (DSA).

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No
• product manufactured in and exported from the U.S.: No