Clinical Language Evaluation With AI for Residents (CLEAR2)

Clinical Language Evaluation With AI for Residents (CLEAR2) - A Pilot Randomized Controlled Trial

The purpose of this study is to refine and test existing enterprise-grade large language model (LLM) based on generative artificial intelligence (AI), to assess the feasibility and acceptability of LLM-based feedback, to assess the ability of LLM-based feedback to improve residents' communications,to explore the ability of standardized patients to assess residents' communication and to explore the ability of residents to self-assess their communication complexity

Study Overview

• Status: Not yet recruiting
• Conditions: Patient Communication
• Intervention / Treatment: Behavioral: educational LLM-based feedback tool

• Study Type: Interventional
• Enrollment (Estimated): 64
• Phase: Not Applicable

Contacts and Locations

• Study Contact: Name: Krislynn M Mueck, MD, MS, MPH; Phone Number: (713) 500-7409; Email: Krislynn.M.Mueck@uth.tmc.edu
• Study Contact Backup: Name: William D Rieger; Phone Number: (713) 500-7300; Email: William.D.Rieger@uth.tmc.edu

Study Locations

• United States
  • Texas
    • Houston, Texas, United States, 77030
      • The University of Texas Health Science Center at Houston
      • Contact: Krislynn M Mueck, MD, MS, MPH; Phone Number: (713) 500-7409; Email: Krislynn.M.Mueck@uth.tmc.edu
      • Contact: William D Rieger; Phone Number: (713) 500-7300; Email: William.D.Rieger@uth.tmc.edu

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: Adult
• Accepts Healthy Volunteers: Yes

Description

Inclusion Criteria:

• McGovern Medical School (MMS) general surgery residents
• postgraduate year (PGY) 1-5

Study Plan

How is the study designed?

Design Details

• Primary Purpose: Other
• Allocation: Randomized
• Interventional Model: Parallel Assignment
• Masking: None (Open Label)

Number of Arms

2

Arms and Interventions

Participant Group / Arm	Intervention / Treatment
No Intervention: Control
Experimental: Educational LLM-based feedback tool	Behavioral: educational LLM-based feedback tool; Participants will have their verbal communications with standardized patients (SP) regarding 3 different scenarios recorded, transcribed, and analyzed in real-time by the large language model (LLM) and will receive feedback as suggestions and alternative scripts. These will be reviewed by residents between SP scenarios

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Readability discernment as assessed by a survey	This will be scored by the by Cohen's Kappa values from 1-5. Higher Cohen's kappa scores mean better outcome	end of intervention ( 1 hour after baseline)
Quality discernment as assessed by a survey	This will be scored by the by Cohen's Kappa values from 1-5. Higher Cohen's kappa scores mean better outcome	end of intervention ( 1 hour after baseline)
Correctness of recommendations as assessed by a survey	This will be reported on a 5 point Likert scale form 1 very incorrect to 5 very correct	end of intervention ( 1 hour after baseline)
Applicability of recommendations as assessed by a survey	This will be reported on a 5 point Likert scale form 1 very inapplicable to 5 very applicable	end of intervention ( 1 hour after baseline)
Perceived readability of resident-standardized patient (SP) interactions as assessed by a survey: schooling level	This will be categorically reported in the following categories: Elementary middle high college graduate	end of intervention ( 1 hour after baseline)
confidence in communication ability	This is scored from 1( very unconfident) to 5 (very confident)	end of intervention ( 1 hour after baseline)
usefulness of the LLM	This is scored from 1( very useless) to 5 (very useful)	end of intervention ( 1 hour after baseline)
acceptability of future use	This is scored from 1( very unlikely) to 5 (very likely)	end of intervention ( 1 hour after baseline)

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
confidence in communication ability	This is scored from 1( very unconfident) to 5 (very confident)	end of intervention ( 1 hour after baseline)
Survey feedback on the LLM interface	This is scored from 1( very unrealistic) to 5 (very realistic)	end of intervention ( 1 hour after baseline)
readability grade level of resident-SP transcripts as assessed by the Flesch-Kincaid Grade Level (FKGL) readability tool	Readability of resident-SP encounter transcripts will be assessed using the Flesch-Kincaid Grade Level formula, which estimates the U.S. school grade level required to understand the text. Higher scores indicate a higher reading grade level (i.e., lower readability). Formula used: Grade level= 0.39(total words/total sentences) + 11.8 (total syllables/total words)-15.59	end of intervention ( 1 hour after baseline)
Quality based on Ensuring Quality Information for Patients (EQIP) score of resident-SP transcripts	Percentage score based on a validated questionnaire This has 20 questions and each is scored from 1(yes), 0.5(partly), 0 (no) and question is removed if it does not apply.Scores are reported as a percentage and higher percentage score indicates better quality	end of intervention ( 1 hour after baseline)
Perceived readability of SP-resident interactions as assessed by a standardized survey	This will be categorically reported in the following categories: Elementary middle high college graduate	end of intervention ( 1 hour after baseline)

Collaborators and Investigators

• Sponsor: The University of Texas Health Science Center, Houston
• Collaborators: Health Science Education Small Grants Program
• Investigators: Principal Investigator: Krislynn M Mueck, MD, MS, MPH, The University of Texas Health Science Center, Houston

Study record dates

Study Major Dates

• Study Start (Estimated): October 23, 2025
• Primary Completion (Estimated): March 26, 2026
• Study Completion (Estimated): May 28, 2026

Study Registration Dates

• First Submitted: October 28, 2025
• First Submitted That Met QC Criteria: October 28, 2025
• First Posted (Estimated): October 30, 2025

Study Record Updates

• Last Update Posted (Estimated): October 30, 2025
• Last Update Submitted That Met QC Criteria: October 28, 2025
• Last Verified: October 1, 2025

More Information

Terms related to this study

• Other Study ID Numbers: HSC-MS-25-0920

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: NO

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No