Efficacy of Using Large Language Model to Assist in Diabetic Retinopathy Detection

With the increase in population and the rising prevalence of various diseases, the workload of disease diagnosis has sharply increased. The accessibility of healthcare services and long waiting times have become common issues in the public health medical system, with many primary patients having to wait for extended periods to receive medical services. There is an urgent need for rapid, accurate, and low-cost diagnostic services.

Study Overview

• Status: Completed
• Conditions: Diabetic Retinopathy; Diagnosis
• Intervention / Treatment: Other: A self-evlaution tool based on Large Language Model

• Study Type: Interventional
• Enrollment (Actual): 535
• Phase: Not Applicable

Contacts and Locations

Study Locations

• China
  • Guangdong
    • Guangzhou, Guangdong, China, 510000
      • Zhognshan Ophthalmic Center, Sun Yat-sen University

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: 18 years and older (Adult, Older Adult)
• Accepts Healthy Volunteers: No

Description

Inclusion Criteria The study will include adults aged 18 years and above who have been diagnosed with Type 2 diabetes but have not previously been screened for DR. Participants must demonstrate good compliance with clinical examinations, and provide informed consent.

Exclusion criteria The study will exclude patients who have previously been diagnosed with DR, those who have recently undergone eye surgery, and those with other significant eye diseases that could potentially confound the results of DR screening. Individuals with ocular, auditory, or cognitive impairments that prevent the use of mobile phones or reading will also be excluded.

Study Plan

How is the study designed?

Design Details

• Primary Purpose: Other
• Allocation: N/A
• Interventional Model: Single Group Assignment
• Masking: None (Open Label)

Number of Arms

1

Arms and Interventions

Participant Group / Arm

Intervention / Treatment

Experimental: A self-evlaution tool based on Large Language Model; The self-evlaution tool, powered by a large language model, processes user queries through a comprehensive generation, decision, action, and safety framework to deliver optimal responses. The system's key features include retrieval-augmented in-context learning, which enhances the responses generated by sourcing information from reliable websites. It also incorporates a Guardrail module to mitigate potential harmful content in the responses by validating the content before delivery. Additionally, the system features a Self-checking memory module that maintains essential clinical characteristics across multi-turn dialogues, ensuring consistent and continuous interactions with users.

Other: A self-evlaution tool based on Large Language Model; Following the baseline assessment, participants will be guided to use a self-evaluation tool independently to assess their risk of diabetic retinopathy (DR). This tool is a fusion of a conversational AI system based on LLM and an existing logistic diagnostic model. The AI system is designed to collect clinical variables, including age, duration of diabetes, Body Mass Index (BMI), and insulin usage. Additionally, clinical test data such as mean arterial pressure, HbA1c, serum creatinine, and microalbuminuria will be extracted from a local dataset using the patient's name and ID. Once collected, these data will be transmitted to a server-based diagnostic model for further analysis to determine the presence of DR.

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
AUROC of the self-evaluation tool	The performance of the self-evaluation tool is evaluated with accuracy with reference to the diagnostic labels by senior ophthalmologists based on fundus photos.	Immediately after using the chatbot

Collaborators and Investigators

• Sponsor: Sun Yat-sen University
• Investigators: Principal Investigator: Yingfeng Zheng, Zhongshan Ophthalmic Center, Sun Yat-sen University

Study record dates

Study Major Dates

• Study Start (Actual): May 1, 2023
• Primary Completion (Actual): July 30, 2023
• Study Completion (Actual): July 30, 2023

Study Registration Dates

• First Submitted: January 29, 2022
• First Submitted That Met QC Criteria: January 29, 2022
• First Posted (Actual): February 9, 2022

Study Record Updates

• Last Update Posted (Estimated): January 19, 2024
• Last Update Submitted That Met QC Criteria: January 17, 2024
• Last Verified: January 1, 2024

More Information

Terms related to this study

• Additional Relevant MeSH Terms: Cardiovascular Diseases; Vascular Diseases; Eye Diseases; Endocrine System Diseases; Diabetic Angiopathies; Diabetes Complications; Diabetes Mellitus; Retinal Diseases; Diabetic Retinopathy
• Other Study ID Numbers: 2022KYPJ258

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: NO

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No