Accuracy of an Artificial Intelligence-assisted Diagnostic System for Caries Diagnosis: a Prospective Multicenter Clinical Study

This clinical trial was designed as a prospective, multicenter, multi-reader multi-case (MRMC), superiority, parallel-controlled study. Participants who met the trial criteria and signed the informed consent form were enrolled. The trial group involved diagnoses of caries on panoramic radiographs using an artificial intelligence-assisted diagnostic system, while the control group involved diagnoses made by dental practitioners specializing in operative dentistry and endodontics with five years of experience, who interpreted oral panoramic radiographs to determine the presence and severity of caries.

Study Overview

• Status: Not yet recruiting
• Conditions: Dental Caries; Machine Learning; Diagnosis; Artificial Intellegence

• Study Type: Observational
• Enrollment (Estimated): 220

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: Adult; Older Adult
• Accepts Healthy Volunteers: No

• Sampling Method: Probability Sample

Study Population

Patients visiting the all the three centers during the study period with symptoms of caries as their chief complaint.

Description

Inclusion Criteria:

-

Inclusion Criteria:

1. Patients presenting with clinical manifestations of caries as their chief complaint;
2. Age ≥18 and ≤70 years, irrespective of gender;
3. Oral panoramic radiographs showing a complete dentition, specifically with the second molars and all premolars intact in each quadrant;
4. On the oral panoramic radiographs, the number of teeth with restorations or fillings does not exceed one in any quadrant;
5. Oral panoramic radiographs that are clear, easy to interpret, and free from significant artifacts;
6. Participants who voluntarily agree to partake, can comprehend the purpose of the study, and are capable of signing an informed consent form.

Exclusion Criteria:

1. Oral panoramic radiographs that are unclear, with overlapping, blurring, or artifacts present;
2. Insufficient number of teeth available for study;
3. Severe tooth wear or erosion leading to significant alteration in tooth morphology;
4. Presence of supernumerary teeth, microdontia, or missing teeth;
5. Conditions not suitable for oral radiography, such as pregnancy or undergoing radiation therapy for tumors;
6. Limited mouth opening that precludes clinical examination;
7. Neurological disorders, psychiatric illnesses, or psychological impairments;
8. Participation in another clinical trial within the last three months;
9. Any other condition deemed by the researchers as unsuitable for inclusion in the study.

Study Plan

How is the study designed?

Number of groups / cohorts

2

Cohorts and Interventions

Group / Cohort
Control group; Dentist specializing in endodontics with over five years of experience independently reviewed the participants' oral panoramic radiographs in a random order.
Trial group; The diagnoses of caries on oral panoramic radiographs were made with the assistance of an artificial intelligence-aided oral panoramic diagnostic system.

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Sensitivity, specificity, and accuracy of caries diagnosis	Sensitivity: Sensitivity assesses the diagnostic method's ability to identify the disease, that is, how many true cases of caries can be correctly identified among all actual cases of caries. Specificity: Specificity evaluates the diagnostic tool's ability to recognize the absence of disease, meaning how many true non-caries cases can be correctly identified among all actual non-caries cases. Accuracy: Accuracy evaluates the overall correctness of the diagnostic tool in diagnosing the disease.	Immediately after the completion of all participant information collection.

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Miss rate and false positive rate of caries diagnosis	Miss rate (False Negative Rate): The miss rate refers to the proportion of true caries cases that the diagnostic method fails to correctly identify among all actual cases of caries. In this study, the effectiveness of the artificial intelligence-assisted diagnostic system is evaluated by comparing the miss rate of caries diagnosis between the artificial intelligence-assisted diagnostic system and dentists with five years of experience. False positive rate: The false positive rate refers to the proportion of cases that are incorrectly identified as having caries among all actual non-caries cases. In this study, the effectiveness of the artificial intelligence-assisted diagnostic system is evaluated by comparing the false positive rate of caries diagnosis between the artificial intelligence-assisted diagnostic system and dentists with five years of experience.	Immediately after the completion of all participant information collection.
Accuracy of Caries Severity Level Diagnosis	This study utilizes the International Caries Detection and Assessment System (ICDAS) to evaluate the grading of caries severity, categorizing caries severity into five levels: E0, E1, E2, D1, and D2. Wherein: E0 indicates no apparent signs of caries; E1 represents early caries, primarily initial enamel lesions; E2 designates enamel caries without involvement of the dentin; D1 denotes superficial dentin caries, not reaching half of the dentin thickness; D2 signifies deep dentin caries, reaching or exceeding half of the dentin thickness. This study will assess and compare the diagnostic accuracy at each grading level between the trial group and the control group.	Immediately after the completion of all participant information collection.

Collaborators and Investigators

• Sponsor: Zhejiang Provincial People's Hospital

Study record dates

Study Major Dates

• Study Start (Estimated): August 1, 2024
• Primary Completion (Estimated): December 1, 2026
• Study Completion (Estimated): June 1, 2027

Study Registration Dates

• First Submitted: May 20, 2024
• First Submitted That Met QC Criteria: May 20, 2024
• First Posted (Actual): May 24, 2024

Study Record Updates

• Last Update Posted (Actual): July 10, 2024
• Last Update Submitted That Met QC Criteria: July 8, 2024
• Last Verified: July 1, 2024

More Information

Terms related to this study

• Keywords: Dental Caries; AI Diagnosis
• Additional Relevant MeSH Terms: Pathologic Processes; Stomatognathic Diseases; Tooth Demineralization; Tooth Diseases; Disease; Dental Caries
• Other Study ID Numbers: KY2024060

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No