Evaluate the Performance of Large Language Models in Ophthalmologic Patient Consultation

Evaluate the Performance of Large Language Models in Ophthalmologic Patient Consultation: A Randomized Clinical Trial

The intelligent image models lack an understanding of diagnostic and treatment logic, and have not considered textual information such as symptoms and signs. Large language models like ChatGPT, can learn medical knowledge, understand, and generate human natural language, offering new technologies for medical knowledge-based intelligent question answering and the creation of smart medical documents. Therefore, our team plan to verify large language models' feasibility and effectiveness in ophthalmology clinics for medical history collection and examination recommendations during consultations, comparing its performance with traditional methods.

Study Overview

• Status: Completed
• Conditions: Non-emergency Ocular Diseases
• Intervention / Treatment: Other: Consultation Model of large language model in Ophthalmology Clinics

• Study Type: Interventional
• Enrollment (Actual): 172
• Phase: Not Applicable

Contacts and Locations

Study Locations

• China
  • Guangdong
    • Guangzhou, Guangdong, China
      • Zhongshan Ophthalmic Center, Sun Yat-sen University

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: Child; Adult; Older Adult
• Accepts Healthy Volunteers: Yes

Description

Inclusion Criteria:

• No age or gender restrictions for patients.
• Non-emergency ocular diseases including corneal diseases, lens disorders, and vitreoretinal diseases.
• Voluntary participation with signed informed consent.

Exclusion Criteria:

• Top 10 Ocular Emergencies: globe perforation, ocular chemical injury, corneal ulcer perforation, Pseudomonas aeruginosa keratitis, acute angle-closure glaucoma, acute panophthalmitis, central retinal artery occlusion, acute optic neuritis, endophthalmitis, and orbital cellulitis.

Study Plan

How is the study designed?

Design Details

• Primary Purpose: Screening
• Allocation: Randomized
• Interventional Model: Parallel Assignment
• Masking: Single

Number of Arms

2

Arms and Interventions

Participant Group / Arm	Intervention / Treatment
Experimental: Large language models in consultation; After the patient signs the informed consent form, the large language model completes the medical history collection and recommends examinations.	Other: Consultation Model of large language model in Ophthalmology Clinics; Large language model completes the medical history collection and recommends examinations.
No Intervention: Traditional consultation; After the patient signs the informed consent form, doctor completes the medical history collection and recommends examinations.

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Medical History Collection Scoring	The medical history collection is performed using the standard outpatient medical record form. The scoring criteria are developed collaboratively by clinical doctors from multiple specialties and researchers. Scoring is independently conducted in a blinded manner by higher-level specialists.	through study completion, up to 1 week.

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Accuracy of Recommended Tests	The gold standard for both the experimental and control groups consists of test items independently selected by senior specialists, who are not involved in the study.	through study completion, up to 1 week.
Duration of Medical History Collection	The experimental group uses the developed system to record the consultation and medical record writing completion times, while the control group records the consultation time through audio recording and the medical record writing completion time through the developed system.	through study completion, up to 1 week.
Patient satisfaction	Collected through a questionnaire.	through study completion, up to 1 week.

Collaborators and Investigators

• Sponsor: Zhongshan Ophthalmic Center, Sun Yat-sen University

Study record dates

Study Major Dates

• Study Start (Actual): May 10, 2025
• Primary Completion (Actual): June 10, 2025
• Study Completion (Actual): June 17, 2025

Study Registration Dates

• First Submitted: February 5, 2025
• First Submitted That Met QC Criteria: February 12, 2025
• First Posted (Actual): February 13, 2025

Study Record Updates

• Last Update Posted (Actual): January 8, 2026
• Last Update Submitted That Met QC Criteria: January 5, 2026
• Last Verified: January 1, 2026

More Information

Terms related to this study

• Other Study ID Numbers: 2025KYPJ004

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: NO

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No