The Big Unknown: A Journey Into Generative AI's Transformative Effect on Medical Professions

January 21, 2026 updated by: Maastricht University

The Big Unknown: A Journey Into Generative AI's Transformative Effect on Meical Professions

A parallel group randomized controlled trial using a superiority framework. Clinical vignettes will be used to assess the impact of a large language model on the clinical reasoning of physicians. Quantitative analyses will be performed on graded vignette responses.

Study Overview

Status

Completed

Conditions

Intervention / Treatment

Detailed Description

This study is a multi-country, parallel-group randomized controlled trial designed to evaluate whether access to a large language model (LLM) improves physician clinical decision-making. The trial uses a superiority framework and compares physicians randomized to either complete standardized clinical vignettes with access to GPT-4o or without any AI assistance.

Clinical vignettes simulate common primary care conditions such as cardiovascular, respiratory, musculoskeletal, fatigue-related, and infectious diseases. Each vignette includes multiple steps in the clinical reasoning process, from initial history-taking to diagnosis, treatment, and follow-up. Physician responses are graded using rubrics developed from evidence-based, context-specific best-practice guidelines.

The study is conducted across three countries-Indonesia, Kenya, and the Netherlands-representing different income levels and health system contexts. The primary outcome is performance on clinical vignettes, defined as adherence to best-practice guidelines. Secondary objectives include examining cross-country variation in physician performance, variation in performance distributions, and the role of engagement with the LLM in shaping outcomes.

Study Type

Interventional

Enrollment (Actual)

249

Phase

  • Not Applicable

Contacts and Locations

This section provides the contact details for those conducting the study, and information on where this study is being conducted.

Study Locations

  • Indonesia
      • Jakarta, Indonesia
        • Universitas Indonesia
  • Kenya
      • Nairobi, Kenya
        • Aga Khan University Hospital
  • Netherlands
      • Maastricht, Netherlands
        • Maastricht University

Participation Criteria

Researchers look for people who fit a certain description, called eligibility criteria. Some examples of these criteria are a person's general health condition or prior treatments.

Eligibility Criteria

Ages Eligible for Study

  • Child
  • Adult
  • Older Adult

Accepts Healthy Volunteers

Yes

Description

Inclusion Criteria:

  • Registered medical physicians
  • Training in internal or family medicine

Exclusion Criteria:

  • Not currently practicing clinically

Study Plan

This section provides details of the study plan, including how the study is designed and what the study is measuring.

How is the study designed?

Design Details

  • Primary Purpose: Diagnostic
  • Allocation: Randomized
  • Interventional Model: Parallel Assignment
  • Masking: Single

Arms and Interventions

Participant Group / Arm
Intervention / Treatment
No Intervention: Own Knowlege
Group will not be given access to GPT-4 or other online resources
Active Comparator: GPT-4o
Group given GPT-4o access
Other: GPT-4o
GPT-4o provided via an iFrame in the online Qualtrics environment

What is the study measuring?

Primary Outcome Measures

Outcome Measure
Measure Description
Time Frame
Percentage Correct Score
Time Frame: During Evaluation
Following Peabody et al (2000), the primary outcome is a percentage correct score across all steps in a vignette. This is generated by dividing the weighted total sum of rubric items assessed as present by the total number of rubric items possible in a vignette. Rubric items will be weighted with regards to their relevance by our expert panel.
During Evaluation

Secondary Outcome Measures

Outcome Measure
Measure Description
Time Frame
Quality Per Answer
Time Frame: During Evaluation
This outcome is generated as the average weight of rubric items assessed as present across vignettes. As each item is provided a weight (0.33,0.5,1), the average weight is the sum of weights divided by the number of answers marked as present.
During Evaluation
Number of Answers
Time Frame: During evaluation
This outcome is generated as the count of the total number of answers assessed as present by reviewers per vignette
During evaluation
Less obvious answers
Time Frame: During evaluation
This outcome is generated as the number of answers given that are less obvious, i.e. mentioned less frequently by the control group. If the answer is mentioned by 25% or less of the control group, it is considered less obvious.
During evaluation

Collaborators and Investigators

This is where you will find people and organizations involved with this study.

Sponsor

Maastricht University

Collaborators

Aga Khan University
University of Indonesia, Jakarta, Indonesia

Investigators

  • Principal Investigator: Mark Levels, PhD, Maastricht University

Study record dates

These dates track the progress of study record and summary results submissions to ClinicalTrials.gov. Study records and reported results are reviewed by the National Library of Medicine (NLM) to make sure they meet specific quality control standards before being posted on the public website.

Study Major Dates

Study Start (Actual)

August 1, 2024

Primary Completion (Actual)

January 17, 2025

Study Completion (Actual)

January 17, 2025

Study Registration Dates

First Submitted

January 21, 2026

First Submitted That Met QC Criteria

January 21, 2026

First Posted (Actual)

January 29, 2026

Study Record Updates

Last Update Posted (Actual)

January 29, 2026

Last Update Submitted That Met QC Criteria

January 21, 2026

Last Verified

September 1, 2025

More Information

Terms related to this study

Keywords

Additional Relevant MeSH Terms

Other Study ID Numbers

  • ERCIC_572_25_04_2024

Drug and device information, study documents

Studies a U.S. FDA-regulated drug product

No

Studies a U.S. FDA-regulated device product

No

This information was retrieved directly from the website clinicaltrials.gov without any changes. If you have any requests to change, remove or update your study details, please contact register@clinicaltrials.gov. As soon as a change is implemented on clinicaltrials.gov, this will be updated automatically on our website as well.

Clinical Trials on Diagnosis

Clinical Trials on GPT-4o

Search Similar Trials

Sponsors and Collaborators

Medical Conditions

Drug Interventions

CROs by country

CROs in South Africa

Conditions

Rare Diseases

Drug Interventions

Dietary Supplements

Sponsor/Collaborators

Locations

Subscribe