Large Language Models for Dental Radiology Report Generation From Structured Textual Data (DENT-LLM)

Evaluation of Large Language Models for Transforming Structured Dental Radiology Data Into Narrative Radiology Reports

The purpose of this observational methodological study is to evaluate whether large language models can transform structured dental radiology data into clear narrative radiology reports. Large language models are computer programs that can generate text from information provided to them. In this study, the input will consist of organized dental radiology findings, such as chart-style or diagram-based information about teeth and surrounding structures.

Dental radiology reports are used by dentists and other health care providers to understand imaging findings and support clinical documentation. Preparing narrative reports may be time-consuming, and the wording of reports may vary between clinicians. This study will examine whether language-model-assisted report generation can produce reports that are complete, accurate, understandable, and clinically useful.

The study will compare reports generated with support from large language models with traditionally prepared reports. Researchers will also assess how the wording of the prompt and selected model parameters influence report quality. In addition, the study will analyze errors and safety risks in generated reports and evaluate whether such a system could be practical in a dental radiology workflow. The language model will not make treatment decisions, and generated reports will be used for research evaluation only.

Study Overview

• Status: Not yet recruiting
• Conditions: Oral Health; Radiography; Dentistry; Large Language Model; Natural Language Processing (NLP)
• Intervention / Treatment: Other: Large language model-assisted radiology report generation

Detailed Description

This study is designed to evaluate the use of large language models for converting structured dental radiology data into narrative radiology reports. The project focuses on the quality, safety, and practical usability of language-model-assisted report generation in dental radiology.

Structured dental radiology data will be used as the input for the language model. These data may include organized findings recorded in a diagram, chart, or predefined structured format. The model will be asked to transform this structured information into a narrative report resembling a conventional dental radiology description. The study does not evaluate the model as an autonomous diagnostic system. The model will not independently interpret radiographic images, establish a diagnosis, or recommend treatment. Its role is limited to generating narrative text from already structured radiological information.

The study will include several related analyses. First, the investigators will assess whether a large language model can reliably transform structured dental radiology findings into a narrative report. Generated reports will be evaluated for completeness, factual consistency with the source data, clarity, terminology, and clinical readability.

Second, the study will examine how prompt construction and model parameters affect the quality of the generated reports. Different prompt formats and selected generation settings may be compared to identify configurations associated with higher report quality and fewer errors.

Third, reports generated with model assistance will be compared with traditionally prepared narrative reports. The comparison may include blinded assessment by qualified evaluators, who will judge report quality without knowing whether a report was generated traditionally or with model support.

Fourth, the study will include an error and safety analysis. Errors may include omitted findings, added findings not present in the source data, incorrect tooth numbering, inconsistent terminology, misleading wording, or statements that could affect clinical interpretation. The purpose of this analysis is to identify types of errors that may occur when large language models are used for this task and to assess their potential clinical relevance.

Finally, the study will assess the potential implementation usefulness of the report-generation workflow. This may include evaluation of usability, perceived time savings, acceptability to users, clarity of generated text, and the need for human review before clinical use.

All generated reports will require expert evaluation in the study setting. The system is intended to support documentation research and workflow assessment, not to replace professional judgment. The study will provide evidence on whether language-model-assisted transformation of structured dental radiology data into narrative reports is feasible, accurate, safe, and potentially useful for future clinical documentation workflows.

• Study Type: Observational
• Enrollment (Estimated): 100

Contacts and Locations

• Study Contact: Name: Kamila Chęcińska, dr inż.; Phone Number: +48 694 816 344; Email: kamila.checinska@pimmswia.gov.pl

Study Locations

• Poland
  • Świętokrzyskie Voivodeship
    • Kielce, Świętokrzyskie Voivodeship, Poland, 25-375
      • Department of Maxillofacial Surgery
      • Contact: Maciej Sikora, dr hab.; Phone Number: +48 41 260 55 85; Email: maciej.sikora@pimmswia.gov.pl

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: Adult; Older Adult
• Accepts Healthy Volunteers: No

• Sampling Method: Non-Probability Sample

Study Population

The study population will consist of dental radiology records from patients admitted to the radiology department in Kielce, a city in southern Poland with approximately 200,000 inhabitants. Eligible records will include dental X-ray examinations performed on the basis of a written referral from a dentist or physician, including examinations performed for screening, diagnostic, or treatment-planning purposes. The study will include records from patients with permanent dentition after completion of exfoliation, provided that structured dental radiology data are available for transformation into narrative radiology reports.

Description

Inclusion Criteria:

• Dental radiology records based on dental X-ray examination performed on the basis of a written referral from a dentist or physician
• Dental X-ray examinations performed for screening, diagnostic, or treatment-planning purposes
• Records from patients with permanent dentition after completion of exfoliation

Exclusion Criteria:

• Records from patients with mixed dentition before completion of exfoliation
• Records with incomplete, ambiguous, or internally inconsistent structured dental radiology data preventing reliable report generation
• Records with missing information required for evaluation of the generated report
• Duplicate records from the same radiographic examination
• Records in which anonymization or pseudonymization cannot be ensured

Study Plan

How is the study designed?

Number of groups / cohorts

1

Cohorts and Interventions

Group / Cohort

Intervention / Treatment

Dental radiology records; Structured dental radiology records used to evaluate large language model-assisted generation of narrative dental radiology reports.

Other: Large language model-assisted radiology report generation; Structured dental radiology data will be processed using a large language model to generate narrative dental radiology reports. The model will transform predefined structured findings into report text for research evaluation. The model will not independently interpret radiographic images, make clinical diagnoses, recommend treatment, or replace professional review. Generated reports will be assessed for completeness, factual consistency with the source data, clarity, terminology, errors, safety, and potential workflow usefulness.

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Factual consistency of large language model-generated dental radiology reports with structured source data	Factual consistency will be assessed by comparing each large language model-generated narrative dental radiology report with the corresponding structured dental radiology source data. Expert evaluators will assess whether the generated report accurately reflects the source data without adding findings, omitting findings, changing tooth numbering, or altering the clinical meaning of the structured findings. The outcome will be reported as the proportion of generated reports without clinically relevant factual inconsistency and/or as the number and type of factual inconsistencies per report.	At the time of report generation and expert evaluation, up to 12 months

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Completeness of large language model-generated dental radiology reports	Completeness will be assessed by determining whether all predefined findings present in the structured dental radiology source data are included in the generated narrative report. The outcome will be reported as the proportion of required findings correctly included in each report and/or the proportion of complete reports.	At the time of report generation and expert evaluation, up to 12 months
Error rate and error categories in large language model-generated dental radiology reports	Generated reports will be reviewed for predefined error categories, including omitted findings, added findings not present in the source data, incorrect tooth numbering, inconsistent terminology, ambiguous wording, and statements with potential clinical relevance. The outcome will be reported as the number and frequency of each error category per report and across all generated reports.	At the time of report generation and expert evaluation, up to 12 months
Overall quality score of dental radiology reports	Overall report quality will be assessed by qualified evaluators using a predefined rating scale that may include clarity, readability, terminology, organization, completeness, and clinical usefulness. The outcome will be reported as the mean or median quality score for generated reports and, where applicable, for traditionally prepared reports.	At the time of blinded or non-blinded expert evaluation, up to 12 months
Difference in expert-rated quality between traditional and large language model-assisted dental radiology reports	Traditional narrative dental radiology reports and large language model-assisted reports will be compared using expert assessment. Evaluators may be blinded to the report-generation method where feasible. The outcome will be reported as the difference in predefined quality scores between traditional and model-assisted reports.	At the time of comparative expert evaluation, up to 12 months
Effect of prompt design and model parameters on generated report quality	The quality of reports generated using different prompt formats and selected model-generation parameters will be compared. Outcomes may include factual consistency, completeness, error rate, and overall quality score. The analysis will identify prompt and parameter configurations associated with higher report quality and fewer errors.	At the time of prompt and parameter comparison, up to 12 months
Usability of the large language model-assisted dental radiology reporting workflow	Usability will be assessed among users involved in evaluating or testing the model-assisted reporting workflow. Measures may include perceived usefulness, ease of use, clarity of generated reports, perceived need for editing, and potential workflow acceptability. The outcome will be reported using predefined questionnaire items or usability ratings.	At the time of usability assessment, up to 12 months

Collaborators and Investigators

• Sponsor: Hospital of the Ministry of Interior, Kielce, Poland

Study record dates

Study Major Dates

• Study Start (Estimated): June 29, 2026
• Primary Completion (Estimated): July 26, 2026
• Study Completion (Estimated): July 26, 2026

Study Registration Dates

• First Submitted: June 24, 2026
• First Submitted That Met QC Criteria: June 24, 2026
• First Posted (Actual): June 30, 2026

Study Record Updates

• Last Update Posted (Actual): June 30, 2026
• Last Update Submitted That Met QC Criteria: June 24, 2026
• Last Verified: June 1, 2026

More Information

Terms related to this study

• Other Study ID Numbers: CT/2026/1

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: YES
• IPD Plan Description: The investigators plan to share a de-identified structured dataset containing symbolic dental pathology notation derived from panoramic dental radiographs, for example tooth-level coded entries such as "16DR", "15M", or "14C". The shared dataset will not include radiographic images, names, dates of birth, personal identifiers, or other directly identifying information. Data sharing will be performed in accordance with the approval and conditions specified by the Bioethics Committee. The dataset will be shared to support transparency, reproducibility, and independent verification of the large language model-assisted report generation task.
• IPD Sharing Time Frame: Beginning at the time of publication of the main study results and available for at least 5 years.
• IPD Sharing Access Criteria: The de-identified structured dental pathology notation dataset will be made available as supplementary material accompanying the publication of the main study results or through a scientific data repository. The shared data will not include radiographic images, direct identifiers, dates of birth, names, or other directly identifying information.
• IPD Sharing Supporting Information Type: STUDY_PROTOCOL; SAP; ANALYTIC_CODE

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No