Al to Improve the Diagnosis of Rare Rheumatic Diseases (AIDRARER)

May 28, 2025 updated by: Philipps University Marburg
This trial aims to assess the impact of providing medical students with access to ChatGPT, a state-of-the-art large language model, in comparison to conventional diagnostic decision support tools, on their diagnostic accuracy for rare rheumatic diseases.

Study Overview

Status

Completed

Conditions

Intervention / Treatment

Detailed Description

Advanced artificial intelligence (AI) technologies, particularly large language models such as OpenAI's ChatGPT, hold significant potential for enhancing medical decision-making. While ChatGPT was not specifically designed for medical applications, it has shown utility in various healthcare scenarios, including answering patient inquiries, drafting medical documentation, and aiding consultations. Despite these advancements, its role in supporting diagnostic reasoning-especially among less experienced medical students-and for complex rare diseases remains underexplored.

Diagnostic reasoning is a multifaceted process that combines pattern recognition, knowledge synthesis, and probabilistic thinking. Tools like ChatGPT could potentially alleviate cognitive burden, enhance diagnostic accuracy, and ultimately accelerate the diagnosis for rare diseases. However, ChatGPT is not tailored for diagnostic reasoning and lacks comprehensive validation in this domain. Additionally, it is susceptible to generating misinformation or plausible-sounding but inaccurate responses, which may hinder rather than support clinical decision-making. Therefore, understanding how medical students utilize such AI tools is essential before they are integrated into educational or clinical workflows. This study will also assess a standardized prompt to facilitate ChatGPT usage and will give students direct access to enable a realistic scenario.

This study will investigate the impact of ChatGPT on the diagnostic accuracy of medical students when tackling cases of rare rheumatic diseases. Participants will be randomized into two groups: one with access to ChatGPT and one using conventional diagnostic tools. Each participant will analyze diagnostic cases by providing up to 5 differential diagnoses and and rating the diagnostic confidence. Independent reviewers, blinded to group allocation, will evaluate the accuracy and quality of their responses. This study hence aims to provide insights into the potential benefits and limitations of integrating AI tools like ChatGPT.

Study Type

Interventional

Enrollment (Actual)

68

Phase

  • Not Applicable

Contacts and Locations

This section provides the contact details for those conducting the study, and information on where this study is being conducted.

Study Locations

  • Germany
      • Marburg, Germany, 35043
        • Institute for Digital Medicine, University Hospital of Giessen and Marburg, Philipps University Marburg

Participation Criteria

Researchers look for people who fit a certain description, called eligibility criteria. Some examples of these criteria are a person's general health condition or prior treatments.

Eligibility Criteria

Ages Eligible for Study

  • Adult
  • Older Adult

Accepts Healthy Volunteers

Yes

Description

Inclusion Criteria:

  • Medical students having started with clinical subjects (Internal medicine)

Exclusion Criteria:

  • Not being a medical student

Study Plan

This section provides details of the study plan, including how the study is designed and what the study is measuring.

How is the study designed?

Design Details

  • Primary Purpose: Diagnostic
  • Allocation: Randomized
  • Interventional Model: Parallel Assignment
  • Masking: Single

Arms and Interventions

Participant Group / Arm
Intervention / Treatment
Active Comparator: Intervention group
Group will be given access to ChatGPT and standardized initial prompt
Other: ChatGPT
OpenAI's ChatGPT language model with chat interface.
No Intervention: Control group
Group will not be given access to any LLMs including ChatGPT but will be motivated to use other resources (such as online search enginges, Pubmed)

What is the study measuring?

Primary Outcome Measures

Outcome Measure
Measure Description
Time Frame
Diagnostic accuracy of top diagnosis
Time Frame: during evaluation
Participants in each group will make at least one disease suggestion (top diagnosis) and up to a total of a maximum of 5 suggestions. Percentage of exact matches of the top suggestion with the actual diagnosis will be analyzed
during evaluation

Secondary Outcome Measures

Outcome Measure
Measure Description
Time Frame
Diagnostic accuracy of top 5 suggestions
Time Frame: during evaluation
Participants in each group will make at least one disease suggestion (top diagnosis) and up to a total of a maximum of 5 suggestions. Percentage of exact matches with the actual diagnosis included in the top 5 suggestions will be analyzed
during evaluation
Diagnostic reasoning
Time Frame: during evaluation
For each case, participants will receive 1 point for each plausible diagnosis and 2 points for a completely correct response. The total scores will be compared between the randomized groups.
during evaluation
Diagnostic confidence
Time Frame: during evaluation
For each case participants will be asked for their diagnostic confidence (VAS 0-10). The mean score will be compared between groups.
during evaluation
Time spent for diagnosis
Time Frame: during evaluation
We will compare how much time (in seconds) participants spend per case between the two study arms.
during evaluation

Collaborators and Investigators

This is where you will find people and organizations involved with this study.

Sponsor

Philipps University Marburg

Study record dates

These dates track the progress of study record and summary results submissions to ClinicalTrials.gov. Study records and reported results are reviewed by the National Library of Medicine (NLM) to make sure they meet specific quality control standards before being posted on the public website.

Study Major Dates

Study Start (Actual)

January 7, 2025

Primary Completion (Actual)

March 30, 2025

Study Completion (Actual)

March 30, 2025

Study Registration Dates

First Submitted

December 16, 2024

First Submitted That Met QC Criteria

December 20, 2024

First Posted (Actual)

December 27, 2024

Study Record Updates

Last Update Posted (Actual)

May 29, 2025

Last Update Submitted That Met QC Criteria

May 28, 2025

Last Verified

May 1, 2025

More Information

Terms related to this study

Keywords

Additional Relevant MeSH Terms

Other Study ID Numbers

  • 24-221 ANZ

Plan for Individual participant data (IPD)

Plan to Share Individual Participant Data (IPD)?

NO

Drug and device information, study documents

Studies a U.S. FDA-regulated drug product

No

Studies a U.S. FDA-regulated device product

No

This information was retrieved directly from the website clinicaltrials.gov without any changes. If you have any requests to change, remove or update your study details, please contact register@clinicaltrials.gov. As soon as a change is implemented on clinicaltrials.gov, this will be updated automatically on our website as well.

Clinical Trials on Rheumatic Diseases

Clinical Trials on ChatGPT

Search Similar Trials

Sponsors and Collaborators

Medical Conditions

Drug Interventions

CROs by country

CROs in Gambia

Conditions

Rare Diseases

Drug Interventions

Dietary Supplements

Sponsor/Collaborators

Locations

Subscribe