LLM in Urodynamic Education
Large Language Model (LLM) as a Tutor in Urodynamic Education: A Prospective Learning Curve Study Among Urology Residents
Study Overview
Status
Conditions
Intervention / Treatment
Detailed Description
Pressure-flow urodynamic interpretation requires simultaneous assessment of numeric parameters (PdetQmax, Qmax, cystometric capacity, voided volume), derived indices (BOOI, BCI), technical quality verification, and integration of ICS-defined diagnostic criteria across seven diagnostic categories (bladder outlet obstruction, detrusor overactivity, detrusor underactivity, hypocompliant bladder, hypercompliant bladder, atonic bladder, and normal urodynamic study). Mastery of this skill during residency training remains highly dependent on faculty availability, case exposure, and the quality of feedback provided, limiting standardization across training environments.
This prospective, single-center, assessor-blinded educational intervention study evaluates whether a standardized, modular LLM-assisted tutoring program can improve residents' diagnostic agreement with an expert-derived reference standard over the course of training. Eligible participants are urology residents currently enrolled in an accredited urology training program without prior formal urodynamic course certification.
A total of 83 archived pressure-flow urodynamic studies, performed using a single urodynamic system, were screened; after exclusion of studies with missing summary data, inadequate technical quality, or pediatric age, a final pool of 65 fully anonymized studies was retained. From this pool, two mutually exclusive case sets were constructed: an assessment case pool of 30 cases (10 cases each for pre-test, mid-test, and post-test, balanced for diagnostic category distribution and sex) and an educational case pool used exclusively during training modules. No case was used in more than one assessment time point or in both pools, minimizing the influence of case familiarity on assessed performance.
The reference standard for all assessment cases was established independently of the educational intervention: two experienced urologists, blinded to each other's assessments, independently classified each case across the seven diagnostic categories; disagreements were adjudicated by a third independent experienced urologist, whose decision served as the final reference standard. This reference standard was finalized prior to the start of resident assessments and remained unchanged throughout the study.
The educational intervention consists of five sequential modules delivered via standardized prompts using the Gemini LLM: (1A) theoretical instruction on ICS terminology and pressure-flow principles without case analysis; (1B) structured conceptual reinforcement addressing operational thresholds, sex-specific interpretation, and ambiguous findings; (2) observational learning, in which the LLM models expert-level step-by-step interpretation of training cases; (2B) visual pattern recognition, focused specifically on graphical identification of detrusor overactivity from filling-phase pressure curves; and (3) interactive learning, in which residents independently interpret cases before receiving structured comparative feedback against the LLM's structured assessment. All residents completed modules in an identical sequence using identical materials.
Resident performance is assessed at three time points - pre-test (before the intervention), mid-test (after modules 1A, 1B, 2, and 2B), and post-test (after module 3) - each comprising 10 independent pressure-flow cases evaluated under proctored, closed-book conditions without access to LLMs, reference materials, or prior assessment feedback. For each case, residents record numeric parameters, derived indices (BOOI, BCI), classification across the seven diagnostic categories (Present/Absent for detrusor overactivity, detrusor underactivity, hypocompliant bladder, hypercompliant bladder, atonic bladder, and normal urodynamic study; Present/Absent/Indeterminate for bladder outlet obstruction, with an additional Not Applicable option reserved for female patients), and an overall technical quality assessment.
The primary outcome is the resident-level pooled, unweighted Cohen's kappa between resident classifications and the expert reference standard, calculated by combining all diagnostic category decisions per resident per time point (10 cases × 7 categories, with female bladder outlet obstruction decisions excluded) into a single multicategory agreement table. Secondary outcomes include numeric extraction accuracy, BOOI/BCI calculation accuracy, technical quality assessment agreement, and diagnosis-specific agreement, each evaluated across the three time points. Change in pooled kappa across pre-test, mid-test, and post-test is analyzed using a within-subject repeated-measures approach.
All urodynamic data were fully anonymized prior to use, and no patient-identifiable information was retained or shared. Participation was voluntary, and written informed consent was obtained from all residents. The study was conducted in accordance with the Declaration of Helsinki and approved by the institutional non-interventional clinical research ethics committee.
Study Type
Enrollment (Actual)
Phase
- Not Applicable
Contacts and Locations
Study Locations
-
-
-
Erzurum, Turkey (Türkiye)
- University of Health Sciences, Erzurum City Hospital, Department of Urology
-
-
Participation Criteria
Eligibility Criteria
Ages Eligible for Study
- Adult
- Older Adult
Accepts Healthy Volunteers
Description
Inclusion Criteria:
- Urology residents currently enrolled in an accredited urology training program
- No prior formal certification in urodynamic training
Exclusion Criteria:
- Prior completion of a formal urodynamic training course
- Declining to provide informed consent
Study Plan
How is the study designed?
Design Details
- Primary Purpose: Other
- Allocation: N/A
- Interventional Model: Single Group Assignment
- Masking: None (Open Label)
Arms and Interventions
Participant Group / Arm |
Intervention / Treatment |
|---|---|
|
Experimental: LLM-Based Urodynamic Education
Participants receive a structured urodynamic education program supported by a large language model acting as an interactive tutor.
|
Participants receive a structured urodynamic education program supported by a large language model acting as an interactive tutor.
Other Names:
|
What is the study measuring?
Primary Outcome Measures
Outcome Measure |
Measure Description |
Time Frame |
|---|---|---|
|
Change in Pressure-Flow Urodynamic Interpretation Agreement (Pooled Cohen's Kappa)
Time Frame: From baseline (pre-test) to mid-test and post-test; individualized per resident based on completion of the corresponding training modules, with total duration from pre-test to post-test ranging from approximately 2 to 4 weeks.
|
Resident-level agreement with an expert-derived reference standard, quantified as a pooled, unweighted Cohen's kappa combining classification decisions across seven diagnostic categories (bladder outlet obstruction, detrusor overactivity, detrusor underactivity, hypocompliant bladder, hypercompliant bladder, atonic bladder, and normal urodynamic study) for 10 cases per assessment (70 categorical decisions per resident per time point, with female bladder outlet obstruction decisions excluded).
Kappa values range from -1 to 1, with higher values indicating greater agreement with the reference standard (values ≤0 indicating agreement no better than chance, and 1 indicating perfect agreement).
Values are compared across pre-test, mid-test, and post-test assessments to evaluate learning curve progression.
|
From baseline (pre-test) to mid-test and post-test; individualized per resident based on completion of the corresponding training modules, with total duration from pre-test to post-test ranging from approximately 2 to 4 weeks.
|
Collaborators and Investigators
Sponsor
Investigators
- Principal Investigator: Hüseyin Koçakgöl, MD, University of Health Sciences, Erzurum City Hospital
Publications and helpful links
Study record dates
Study Major Dates
Study Start (Actual)
Primary Completion (Actual)
Study Completion (Actual)
Study Registration Dates
First Submitted
First Submitted That Met QC Criteria
First Posted (Actual)
Study Record Updates
Last Update Posted (Actual)
Last Update Submitted That Met QC Criteria
Last Verified
More Information
Terms related to this study
Other Study ID Numbers
- URO-LLM-UDS-2026
Plan for Individual participant data (IPD)
Plan to Share Individual Participant Data (IPD)?
IPD Plan Description
Drug and device information, study documents
Studies a U.S. FDA-regulated drug product
Studies a U.S. FDA-regulated device product
This information was retrieved directly from the website clinicaltrials.gov without any changes. If you have any requests to change, remove or update your study details, please contact register@clinicaltrials.gov. As soon as a change is implemented on clinicaltrials.gov, this will be updated automatically on our website as well.