Multimodal Machine Learning for Auxiliary Diagnosis of Eye Diseases

Multimodal Machine Learning for Auxiliary Diagnosis of Eye Diseases Using ChatGPT-based Natural Language Processing and Image Processing Techniques

With rapid advancements in natural language processing and image processing, there is a growing potential for intelligent diagnosis utilizing chatGPT trained through high-quality ophthalmic consultation. Furthermore, by incorporating patient selfies, eye examination photos, and other image analysis techniques, the diagnostic capabilities can be further enhanced. The multi-center study aims to develop an auxiliary diagnostic program for eye diseases using multimodal machine learning techniques and evaluate its diagnostic efficacy in real-world outpatient clinics.

Study Overview

• Status: Completed
• Conditions: Eye Diseases
• Intervention / Treatment: Diagnostic test: Multimodal Machine Learning Program for Auxiliary Diagnosis of Eye Diseases

• Study Type: Observational
• Enrollment (Actual): 9825

Contacts and Locations

Study Locations

• China
  • Nanjing, China
    • The Affiliated Eye Hospital of Nanjing Medical University
  • Shanghai, China
    • Fudan Eye & ENT Hospital
  • Suqian, China
    • Suqian First People's Hospital

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: Child; Adult; Older Adult
• Accepts Healthy Volunteers: Yes

• Sampling Method: Non-Probability Sample

Study Population

The "normal participants" refers to individuals with no concerns or issues related to their eyes.

The "participants with eye-related chief complaints" refers to patients from various eye clinics across China. Each participant must undergo comprehensive medical tests and have their medical records reviewed for diagnosis.

Description

Inclusion Criteria:

• Informed consent obtained;
• Participants should be able to have Chinese as their mother tongue, and be sufficiently able to read, write and understand Chinese;
• For normal participants: individuals should have no concerns related to their eyes.
• For participants with eye-related chief complaints: individuals should have specific concerns or issues related to their eyes.

Exclusion Criteria:

• Incomplete clinical data to support final diagnosis;
• Patients who, in the opinion of the attending physician or clinical study staff, are too medically unstable to participate in the study safely.

Study Plan

How is the study designed?

Number of groups / cohorts

2

Cohorts and Interventions

Group / Cohort	Intervention / Treatment
Normal participants; Healthy individuals who have no concerns related to their eyes.
Patients with Eye-related Chief Complaints; Individuals who have specific concerns or issues related to their eyes, which they consider as the main reason for seeking medical attention or making a complaint.	Diagnostic test: Multimodal Machine Learning Program for Auxiliary Diagnosis of Eye Diseases; Patients presenting with eye-related chief complaints initially complete a mobile phone application. This application utilizes patient medical history and relevant images (such as selfies and photos from eye examinations) to provide intelligent diagnosis. The diagnosis remains undisclosed to the patients. Subsequently, patients seek medical attention and undergo clinical examination by a skilled clinician. The clinical diagnosis is subsequently reviewed by a second experienced clinician. If the diagnoses align, it is considered the gold standard. In cases of discrepancy, the consensus reached by the two clinicians becomes the gold standard.

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Diagnostic accuracy of multimodal machine learning program	For each patient, the diagnoses generated by the multimodal machine learning program and the clinical diagnosis provided by skilled clinicians were documented and compared. Consistency between the two diagnoses indicates the program's precision in clinical practice.	from July 2023 to March 2024

Collaborators and Investigators

• Sponsor: Eye & ENT Hospital of Fudan University
• Collaborators: The Affiliated Eye Hospital of Nanjing Medical University; Suqian First People's Hospital

Publications and helpful links

General Publications

• Peng Z, Ma R, Zhang Y, Yan M, Lu J, Cheng Q, Liao J, Zhang Y, Wang J, Zhao Y, Zhu J, Qin B, Jiang Q, Shi F, Qian J, Chen X, Zhao C. Development and evaluation of multimodal AI for diagnosis and triage of ophthalmic diseases using ChatGPT and anterior segment images: protocol for a two-stage cross-sectional study. Front Artif Intell. 2023 Dec 8;6:1323924. doi: 10.3389/frai.2023.1323924. eCollection 2023.

Helpful Links

• Development and evaluation of multimodal AI for diagnosis and triage of ophthalmic diseases using ChatGPT and anterior segment images: protocol for a two-stage cross-sectional study

Study record dates

Study Major Dates

• Study Start (Actual): July 21, 2023
• Primary Completion (Actual): March 10, 2024
• Study Completion (Actual): March 31, 2024

Study Registration Dates

• First Submitted: June 25, 2023
• First Submitted That Met QC Criteria: June 26, 2023
• First Posted (Actual): July 5, 2023

Study Record Updates

• Last Update Posted (Actual): November 15, 2024
• Last Update Submitted That Met QC Criteria: November 13, 2024
• Last Verified: November 1, 2024

More Information

Terms related to this study

• Additional Relevant MeSH Terms: Eye Diseases
• Other Study ID Numbers: FD-EENT-20230625

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: YES
• IPD Sharing Time Frame: The protocol has been published on 08 December 2023.
• IPD Sharing Supporting Information Type: STUDY_PROTOCOL

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No
• product manufactured in and exported from the U.S.: No