Evaluating AI-Generated Plain Language Summaries on Patient Comprehension of Ophthalmology Notes Among English-Speaking Patients

This clinical trial is testing whether plain language summaries made by artificial intelligence help people understand their eye doctor's notes better. Adults receiving eye care at the Jules Stein Eye Institute will get either the usual medical notes or a note with the addition of an AI-generated summary that explains the information in simple, everyday words. Participants will then answer a short survey and receive a follow-up call to share how clear the information was, how well they understood their diagnosis and treatment, and whether they feel more confident about their care. The goal is to find out if these plain language summaries can make it easier for people to understand their eye care and improve communication between patients and health care providers.

Study Overview

• Status: Recruiting
• Conditions: Ophthalmic Disease; Large Language Model; Artifical Intelligence
• Intervention / Treatment: Behavioral: AI-Generated Plain Language Summaries

Detailed Description

This study employs a two-arm randomized controlled trial to evaluate whether artificial intelligence (AI)-generated plain language summaries (PLSs) can improve patient comprehension of ophthalmology notes. Eligible participants are recruited during their routine visits at the Jules Stein Eye Institute, and once screened using standardized clinical criteria, they are randomly assigned to either receive the standard ophthalmology note (SON) or the SON supplemented with an AI-generated PLS. The randomization process uses a computer-generated sequence with concealed allocation to ensure unbiased group assignment.

The AI system used in this study is deployed locally on a secured UCLA intranet. It leverages a large language model (LLM) that has been customized and validated for generating plain language explanations of complex ophthalmologic information. All processing occurs on UCLA-approved, encrypted devices, and no data are transmitted externally. Before the PLS is provided to participants, each summary is reviewed by an ophthalmologist to verify accuracy and ensure that essential clinical details are correctly and clearly communicated.

Data collection is performed using survey instruments. The survey includes a series of 5-point Likert scale items, open-ended questions, and structured response sections designed to assess comprehension of diagnosis, treatment plans, and follow-up instructions. Participants complete the survey immediately after their clinic visit, and a follow-up telephone interview is conducted approximately seven days later by trained research staff to capture additional feedback on clarity and retention of the information provided. The study does not employ audio or video recording; all responses are either directly recorded by research personnel or entered electronically into a secured database.

Statistical analyses will be conducted using standard software packages to compare outcomes between the intervention and control groups. Primary analyses include independent t-tests or Mann-Whitney U tests for continuous variables, chi-square tests for categorical variables, and multivariable regression models to adjust for confounding variables such as age, education level, and baseline health literacy. The sample size was calculated to detect clinically meaningful differences in comprehension scores, with power analyses indicating a need for between 460 and 2030 participants depending on the effect size.

Data security is maintained through rigorous measures. Electronic data are stored on encrypted, UCLA-secured laptops and in a secure Box repository. All data handling follows UCLA policies and IRB guidelines for data retention and destruction, with identifiable information destroyed using secure methods once it is no longer required.

Quality control procedures include periodic audits of data entry, regular review meetings with study personnel, and cross-checks of survey responses against clinical records where applicable. An independent monitoring process is in place to ensure compliance with the study protocol and to address any deviations promptly.

Overall, this study is designed to provide robust evidence on the feasibility and effectiveness of AI-generated PLSs in enhancing patient understanding of complex medical information. By integrating technical safeguards, rigorous statistical methods, and a streamlined data collection process, the research aims to deliver insights that may lead to improved patient communication strategies and more effective health care delivery across multiple specialties.

• Study Type: Interventional
• Enrollment (Estimated): 460
• Phase: Not Applicable

Contacts and Locations

• Study Contact: Name: Prashant Tailor, MD; Phone Number: 4043166920; Email: ptailor@mednet.ucla.edu

Study Locations

• United States
  • California
    • Los Angeles, California, United States, 90095
      • Recruiting
      • UCLA
      • Contact: Prashant D Tailor, MD; Phone Number: 4043166920; Email: ptailor@mednet.ucla.edu

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: Adult; Older Adult
• Accepts Healthy Volunteers: Yes

Description

Inclusion Criteria:

• Age ≥ 18 years English-speaking Receiving ophthalmology care at the Jules Stein Eye Institute Able to provide informed consent

Exclusion Criteria:

• Known cognitive impairments (e.g., dementia, intellectual disability) that would affect comprehension Prisoners or wards of the state Unable to provide informed consent

Study Plan

How is the study designed?

Design Details

• Primary Purpose: Health Services Research
• Allocation: Randomized
• Interventional Model: Parallel Assignment
• Masking: None (Open Label)

Number of Arms

2

Arms and Interventions

Participant Group / Arm	Intervention / Treatment
No Intervention: Standard Ophthalmology Notes (SON) Only; Participants in this arm receive the standard ophthalmology notes typically provided after their clinic visit, with no additional plain language summary. They will complete surveys that measure their comprehension and satisfaction with the visit notes.
Experimental: SON + AI-Generated Plain Language Summaries; Participants in this arm receive the standard ophthalmology notes plus an AI-generated plain language summary, reviewed for accuracy before distribution. They will complete the same surveys to assess whether the additional summary improves their understanding and satisfaction compared to the control group.	Behavioral: AI-Generated Plain Language Summaries; Participants receive standard ophthalmology notes plus an AI-generated summary that explains medical information in simpler language. Each summary is reviewed by an ophthalmologist for accuracy before being given to the participant. The goal is to help participants better understand their diagnosis, treatment plan, and follow-up instructions.

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Patient Comprehension Score (Immediate Post-Visit)	Mean score on a 5-point scale assessing participants' understanding of their ophthalmology visit notes (diagnosis, treatment plan, follow-up instructions) immediately after the clinic visit. Higher scores indicate better comprehension.	Immediately post-visit (Day 0)

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Patient Comprehension Score (1-Week Follow-Up)	Mean score on a 1-5 scale assessing retention of ophthalmology information one week after the clinic visit. Higher scores indicate better long-term comprehension.	1 week post-visit
Patient Satisfaction	Mean satisfaction score (1-5 scale) measuring clarity, detail, and usefulness of the visit notes. Higher scores indicate greater satisfaction.	Immediately post-visit (Day 0)
Comprehension Gap Reduction	Difference in comprehension scores between participants with lower vs. higher baseline health literacy. A smaller gap indicates greater reduction in literacy-related disparities.	Day 0 and 1 week post-visit
Time Efficiency for Ophthalmologists	Average additional time (in minutes) required for ophthalmologists to review and edit AI-generated Plain Language Summaries, reported in the ophthalmologist survey. Lower times indicate better efficiency.	Day 0
Inbasket Message Rates	Number of patient-initiated messages (e.g., via patient portal) within 2 weeks after the visit. Lower message rates may indicate improved clarity and fewer follow-up questions.	2 weeks post-visit
Medication Fill Compliance	Percentage of prescribed medications filled within 2 weeks after the visit. Higher percentages indicate better adherence and understanding of treatment plans.	2 weeks post-visit
Ophthalmologist Satisfaction	Mean score (1-5 scale) from the ophthalmologist survey measuring satisfaction with the AI-generated summary's clarity and accuracy. Higher scores indicate greater satisfaction.	Day 0
LLM Summarization Error Rate	Proportion of AI-generated summaries identified as having any errors by the reviewing ophthalmologist. Lower percentages indicate more accurate summaries.	Day 0
Error Rate for Ophthalmologist Overreads	Percentage of critical inaccuracies in the AI-generated summaries that could lead to misinterpretation of the patient's condition or plan. Lower rates indicate higher-quality summaries.	Day 0

Collaborators and Investigators

• Sponsor: University of California, Los Angeles
• Investigators: Principal Investigator: Prashant Tailor, MD, University of California, Los Angeles

Study record dates

Study Major Dates

• Study Start (Actual): March 1, 2025
• Primary Completion (Estimated): April 30, 2026
• Study Completion (Estimated): June 1, 2026

Study Registration Dates

• First Submitted: February 23, 2025
• First Submitted That Met QC Criteria: February 27, 2025
• First Posted (Actual): March 5, 2025

Study Record Updates

• Last Update Posted (Actual): March 5, 2026
• Last Update Submitted That Met QC Criteria: March 3, 2026
• Last Verified: March 1, 2026

More Information

Terms related to this study

• Keywords: Patient education; LLM; Plain Language Summary; Artifical Intelligence
• Other Study ID Numbers: 24-5649

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: YES
• IPD Plan Description: De-identified individual participant data (IPD) underlying the study's results will be shared. The IPD includes patient survey responses on comprehension of ophthalmology visit notes (both immediately post-visit and at 1-week follow-up), patient satisfaction ratings, and demographic information (age, gender, education level, and previous ophthalmology experience). Additionally, survey responses from ophthalmologists regarding the accuracy, clarity, and time efficiency of the AI-generated plain language summaries will be provided. All data will be fully de-identified in compliance with HIPAA and UCLA guidelines using unique study IDs to replace personal identifiers, and no code keys linking data to individual participants will be shared.
• IPD Sharing Time Frame: Three years post-study completion
• IPD Sharing Access Criteria: Access to the de-identified individual participant data (IPD) and supporting documentation will be available to qualified researchers who meet our eligibility criteria. Eligible researchers must be affiliated with academic or research institutions, healthcare organizations, or other reputable entities engaged in scientific research. They must submit a detailed research proposal outlining the study objectives, methodology, and anticipated benefits, and demonstrate that their proposed use of the data aligns with advancing scientific knowledge and patient care, particularly in health communication or patient comprehension. Upon submission, proposals will be reviewed by the Principal Investigator (and an advisory committee if necessary) to ensure compliance with ethical standards and participant confidentiality. Approved researchers will be granted access via a secure online platform where they can download the de-identified IPD and supporting information. This access will be governed by a
• IPD Sharing Supporting Information Type: STUDY_PROTOCOL

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No