Artificial Intelligence-Generated vs Academician-Developed Multiple True/False Questions in Anesthesiology Education

April 30, 2026 updated by: Serkan TELLİ, Kutahya Health Sciences University

Comparison of Artificial Intelligence-Generated and Academician-Developed Multiple True/False Questions in Anesthesiology Education: A Prospective Cohort Study

This prospective observational study aims to evaluate the effectiveness and educational value of artificial intelligence (AI)-generated multiple true/false questions compared to those developed by experienced academicians in anesthesiology training.

A total of 27 anesthesiology residents will be included in the study. Question sets consisting of 200 multiple true/false items will be created, with half generated by academicians and the other half generated using an artificial intelligence model (ChatGPT-based system). The questions will be based on standardized educational materials from the anesthesiology training curriculum.

Participants will complete the test in a single session. Each correct answer will be scored as one point, and total scores will be calculated. In addition to test performance, item difficulty, discrimination indices, and test reliability will be analyzed. Furthermore, participants' perceptions regarding question quality will be evaluated.

The study aims to determine whether AI-generated questions can provide a reliable and effective alternative to traditional question development methods in medical education and contribute to more objective and standardized assessment processes.

Study Overview

Status

Completed

Conditions

Detailed Description

This single-center, prospective observational cohort study is designed to evaluate the effectiveness, reliability, and educational value of artificial intelligence (AI)-generated multiple true/false (MTF) questions compared to those developed by experienced academicians in anesthesiology training.

The study will be conducted at the Department of Anesthesiology and Reanimation, Kütahya Health Sciences University. A total of 27 anesthesiology residents will be included.

A total of 200 MTF questions will be developed based on standardized anesthesiology educational materials. Half of the questions (n=100) will be prepared by experienced academicians, while the remaining half (n=100) will be generated using an artificial intelligence model (ChatGPT-based system). All questions will be structured according to predefined criteria, including difficulty level (easy, moderate, difficult), clinical relevance, and educational appropriateness.

Participants will complete the question sets in a single session under standardized conditions. Each correct answer will be scored as 1 point, and incorrect answers will be scored as 0. Total test scores will be calculated for each participant.

Item analysis will be performed to evaluate the psychometric properties of the questions. Item difficulty index, item discrimination index, and overall test reliability will be calculated. Additionally, perceived question quality will be assessed using participant feedback.

Statistical analysis will be conducted using SPSS software. The distribution of variables will be assessed, and appropriate parametric or non-parametric tests will be used accordingly. Comparisons between groups (junior vs senior residents) and between question sources (AI-generated vs academician-developed) will be performed. A p-value of <0.05 will be considered statistically significant.

The study does not involve any clinical intervention, drug administration, or invasive procedure. Participation is voluntary, and written informed consent will be obtained from all participants. All data will be collected anonymously and used solely for research purposes.

The results of this study are expected to provide insight into the potential role of artificial intelligence in medical education, particularly in the development of assessment tools, and may contribute to more objective, standardized, and efficient evaluation methods in anesthesiology training.

Study Type

Observational

Enrollment (Actual)

26

Contacts and Locations

This section provides the contact details for those conducting the study, and information on where this study is being conducted.

Study Locations

Participation Criteria

Researchers look for people who fit a certain description, called eligibility criteria. Some examples of these criteria are a person's general health condition or prior treatments.

Eligibility Criteria

Ages Eligible for Study

  • Adult
  • Older Adult

Accepts Healthy Volunteers

Yes

Sampling Method

Non-Probability Sample

Study Population

Anesthesiology residents undergoing training at Kütahya Health Sciences University.

Description

Inclusion Criteria:

  • Being an anesthesiology resident
  • Voluntary participation in the study
  • Providing informed consent

Exclusion Criteria:

  • Refusal to participate
  • Incomplete test responses

Study Plan

This section provides details of the study plan, including how the study is designed and what the study is measuring.

How is the study designed?

Design Details

Cohorts and Interventions

Group / Cohort
AI-Generated Questions
Multiple true/false questions generated using an artificial intelligence model
Academician-Developed Question
Multiple true/false questions prepared by experienced academicians in anesthesiology.

What is the study measuring?

Primary Outcome Measures

Outcome Measure
Measure Description
Time Frame
Item Difficulty Index of AI-generated and expert-authored questions
Time Frame: Assessed once after completion of each participant's single 60-minute examination session; final item analysis performed after all participants complete the examination, up to 1 month.
For each question, the item difficulty index will be calculated as the proportion of participants who answer the item correctly. Item difficulty indices will be compared between AI-generated and expert-authored questions.
Assessed once after completion of each participant's single 60-minute examination session; final item analysis performed after all participants complete the examination, up to 1 month.

Collaborators and Investigators

This is where you will find people and organizations involved with this study.

Sponsor

Kutahya Health Sciences University

Publications and helpful links

The person responsible for entering information about the study voluntarily provides these publications. These may be about anything related to the study.

General Publications

Study record dates

These dates track the progress of study record and summary results submissions to ClinicalTrials.gov. Study records and reported results are reviewed by the National Library of Medicine (NLM) to make sure they meet specific quality control standards before being posted on the public website.

Study Major Dates

Study Start (Actual)

April 14, 2026

Primary Completion (Actual)

April 20, 2026

Study Completion (Actual)

April 30, 2026

Study Registration Dates

First Submitted

March 28, 2026

First Submitted That Met QC Criteria

April 8, 2026

First Posted (Actual)

April 13, 2026

Study Record Updates

Last Update Posted (Actual)

May 6, 2026

Last Update Submitted That Met QC Criteria

April 30, 2026

Last Verified

April 1, 2026

More Information

Terms related to this study

Keywords

Other Study ID Numbers

  • E-41997688-050.99-212840

Plan for Individual participant data (IPD)

Plan to Share Individual Participant Data (IPD)?

NO

Drug and device information, study documents

Studies a U.S. FDA-regulated drug product

No

Studies a U.S. FDA-regulated device product

No

This information was retrieved directly from the website clinicaltrials.gov without any changes. If you have any requests to change, remove or update your study details, please contact register@clinicaltrials.gov. As soon as a change is implemented on clinicaltrials.gov, this will be updated automatically on our website as well.

Clinical Trials on Medical Education

Search Similar Trials

Sponsors and Collaborators

Medical Conditions

Drug Interventions

CROs by country

CROs in El Salvador

Conditions

Rare Diseases

Drug Interventions

Dietary Supplements

Sponsor/Collaborators

Locations

Subscribe