Large Language Model-Assisted cTNM Annotation From Chinese PSMA PET/CT Reports (PSMA-LLM-cTNM)

July 15, 2026 updated by: Qi Lin, MD, First Affiliated Hospital of Wenzhou Medical University

Large Language Model-Assisted Imaging cTNM Staging Annotation and Uncertainty Recognition for Prostate Cancer Based on Chinese PSMA PET/CT Reports

This observational study will develop and validate a large language model-assisted workflow for imaging cTNM staging annotation and uncertainty recognition in prostate cancer using Chinese PSMA PET/CT report texts generated during routine clinical care. The study will use de-identified report texts and necessary baseline clinical information only. No additional imaging examination, blood test, treatment, or follow-up visit will be assigned for this study.

The main objective is to evaluate whether a locally or institutionally controlled large language model can identify report-derived imaging cT, cN, and cM categories, extract supporting evidence from the original report, and recognize uncertainty expressions. Model performance will be assessed using an internal independent validation set, external validation reports from two collaborating hospitals, and a prospective validation set of 100 consecutive routine PSMA PET/CT reports. A human-AI comparison will also be performed using physicians from urology and imaging-related specialties with different seniority levels.

Study Overview

Status

Status

Indicates the current recruitment status or the expanded access status.
Not yet recruiting

Conditions

Conditions

The disease, disorder, syndrome, illness, or injury that is being studied. On ClinicalTrials.gov, conditions may also include other health-related issues, such as lifespan, quality of life, and health risks.

Intervention / Treatment

Intervention / Treatment

A process or action that is the focus of a clinical study. Interventions include drugs, medical devices, procedures, vaccines, and other products that are either investigational or already available. Interventions can also include noninvasive approaches, such as education or modifying diet and exercise.

Detailed Description

This is a multicenter observational diagnostic accuracy validation study based on Chinese PSMA PET/CT report texts from patients with prostate cancer or suspected prostate cancer. The study is not designed to evaluate a drug, device, surgical procedure, or imaging intervention. PSMA PET/CT examinations will be performed as part of routine clinical care, and the study will only analyze de-identified report texts and necessary baseline information after the reports have been finalized.

The study consists of retrospective and prospective components. Retrospectively, approximately 4,000 PSMA PET/CT reports from the First Affiliated Hospital of Wenzhou Medical University will be systematically annotated to construct a research database. An internal independent validation set of 300 reports, not used for model development or prompt optimization, will be used to evaluate the performance of the large language model. The reference standard for this 300-report validation set will be established by two experienced urologists through joint annotation, with adjudication by a nuclear medicine expert when needed. External validation will be performed using 110 de-identified reports from the First Affiliated Hospital of Ningbo University and 102 de-identified reports from Liuzhou People's Hospital. In addition, after ethics approval, 100 consecutive routine PSMA PET/CT reports from the First Affiliated Hospital of Wenzhou Medical University will be prospectively included to evaluate the accuracy and operational stability of the frozen model and prompt versions.

The large language model workflow will be deployed locally or in an institutionally controlled environment. The model will be instructed to generate structured JSON outputs, including cT_report, cN_report, cM_report, cT_uncertain, cN_uncertain, cM_uncertain, evidence_T, evidence_N, and evidence_M. The target task is report-derived imaging cTNM staging annotation, not pathological TNM staging or overall AJCC stage grouping. The model output will be used only for research evaluation and methodological analysis and will not be used for clinical diagnosis, treatment decision-making, or patient notification.

A human-AI comparison will be conducted on the 300-report internal validation set. Eight human evaluators from urology and imaging-related specialties, including trainees, residents, attending physicians, and associate chief physicians, will independently annotate the reports before and after learning the annotation manual. Annotation time will be recorded for each round. The performance of human evaluators and the large language model will be compared against the expert consensus reference standard.

The main outcome will be the accuracy of the large language model in identifying cT, cN, and cM categories from Chinese PSMA PET/CT reports. Secondary outcomes will include precision, recall, F1-score, macro-F1, micro-F1, complete cTNM triplet matching rate, uncertainty recognition performance, evidence extraction quality, human-AI comparison results, annotation time, external validation performance, prospective validation performance, and error type distribution. Error analysis will focus on local tumor extent, regional versus non-regional lymph node boundaries, bone and visceral metastasis recognition, equivocal wording, treatment-related context, benign or inflammatory alternatives, and lesions not attributable to prostate cancer.

Study Type

Study Type

Describes the nature of a clinical study. Study types include interventional studies (also called clinical trials), observational studies (including patient registries), and expanded access.
Observational

Enrollment (Estimated)

Enrollment

The number of participants in a clinical study. The "anticipated" enrollment is the target number of participants that the researchers need for the study.
4600

Contacts and Locations

This section provides the contact details for those conducting the study, and information on where this study is being conducted.

Study Contact

Study Contact

The name and contact information for the person who can answer enrollment questions for a clinical study. Each location where the study is being conducted may also have a specific contact, who may be better able to answer those questions.

Study Contact Backup

Study Locations

  • China
    • Zhejiang
      • Wenzhou, Zhejiang, China
        • The First Affiliated Hospital of Wenzhou Medical University
        • Contact:
        • Contact:

Participation Criteria

Researchers look for people who fit a certain description, called eligibility criteria. Some examples of these criteria are a person's general health condition or prior treatments.

Eligibility Criteria

Eligibility Criteria

The key requirements that people who want to participate in a clinical study must meet or the characteristics they must have. Eligibility criteria consist of both inclusion criteria (which are required for a person to participate in the study) and exclusion criteria (which prevent a person from participating). Types of eligibility criteria include whether a study accepts healthy volunteers, has age or age group requirements, or is limited by sex.

Ages Eligible for Study

  • Adult
  • Older Adult

Accepts Healthy Volunteers

No

Sampling Method

Non-Probability Sample

Study Population

The study population consists of male patients aged 18 years or older with clinically diagnosed, pathologically diagnosed, or clinically suspected prostate cancer who underwent PSMA PET/CT as part of routine clinical care. The study will include de-identified Chinese PSMA PET/CT report texts from the First Affiliated Hospital of Wenzhou Medical University, the First Affiliated Hospital of Ningbo University, and Liuzhou People's Hospital, as well as a prospective set of 100 consecutive routine PSMA PET/CT reports from the First Affiliated Hospital of Wenzhou Medical University.

Description

Inclusion Criteria:

  1. Male patients aged 18 years or older.
  2. Patients with clinically diagnosed, pathologically diagnosed, or clinically suspected prostate cancer.
  3. Patients who underwent PSMA PET/CT for initial staging, recurrence assessment, treatment response evaluation, metastatic assessment, or other clinical purposes during routine care.
  4. Complete or basically complete Chinese PSMA PET/CT report text is available, including imaging findings and/or diagnostic impression.
  5. The report text contains information that can be used to evaluate at least one target field, such as local prostate lesion, regional lymph nodes, non-regional lymph nodes, bone metastasis, visceral metastasis, or uncertainty expressions.
  6. The research data can be de-identified and replaced by a study identification number before analysis.

Exclusion Criteria:

  1. PSMA PET/CT reports unrelated to prostate cancer, or reports clearly irrelevant to the research task.
  2. Reports with severely missing, unreadable, or unavailable main text, imaging findings, or diagnostic impression.
  3. Reports that cannot be adequately de-identified or contain residual direct personal identifiers that cannot be safely removed.
  4. Duplicate records, repeated exports of the same examination, or records for which the unique report version cannot be confirmed.
  5. Reports judged by the research team to be of insufficient quality for manual annotation, model evaluation, or statistical analysis.

Study Plan

This section provides details of the study plan, including how the study is designed and what the study is measuring.

How is the study designed?

Design Details

Cohorts and Interventions

Group / Cohort

Group / Cohort

A group or subgroup of participants in an observational study that is assessed for biomedical or health outcomes.
Intervention / Treatment

Intervention / Treatment

A process or action that is the focus of a clinical study. Interventions include drugs, medical devices, procedures, vaccines, and other products that are either investigational or already available. Interventions can also include noninvasive approaches, such as education or modifying diet and exercise.
PSMA PET/CT Report Text Validation Cohort
Patients with prostate cancer or suspected prostate cancer who underwent PSMA PET/CT as part of routine clinical care. De-identified Chinese PSMA PET/CT report texts and necessary baseline information will be used for manual annotation, large language model-assisted imaging cTNM staging annotation, uncertainty recognition, internal validation, external validation, prospective validation, and human-AI comparison. No additional examination, treatment, or follow-up will be assigned for this study.
Other: Large Language Model-Assisted Report Annotation
A locally or institutionally controlled large language model workflow will analyze de-identified Chinese PSMA PET/CT report texts and generate structured outputs for report-derived imaging cTNM staging annotation, uncertainty recognition, and supporting evidence extraction. This workflow is used only for research evaluation and methodological analysis. It will not assign any examination, treatment, medication, procedure, or follow-up to participants, and it will not guide clinical diagnosis or treatment decisions.

What is the study measuring?

Primary Outcome Measures

Primary Outcome Measures

In a clinical study's protocol, the planned outcome measure that is the most important for evaluating the effect of an intervention/treatment. Most clinical studies have one primary outcome measure, but some have more than one.
Outcome Measure
Measure Description
Time Frame
Accuracy of LLM-Assisted Imaging cTNM Staging Annotation
Time Frame: After freezing the model and prompt versions, through completion of internal, external, and prospective validation, up to 18 months
The primary outcome is the accuracy of the large language model in identifying report-derived imaging cT, cN, and cM categories from de-identified Chinese PSMA PET/CT report texts. The LLM-generated cT_report, cN_report, and cM_report will be compared with the expert consensus reference standard. Accuracy, precision, recall, F1-score, macro-F1, micro-F1, complete cTNM triplet matching rate, and confusion matrices will be calculated in the internal 300-report validation set, external validation sets, and prospective 100-report validation set.
After freezing the model and prompt versions, through completion of internal, external, and prospective validation, up to 18 months

Secondary Outcome Measures

Secondary Outcome Measures

In a clinical study's protocol, a planned outcome measure that is not as important as the primary outcome measure for evaluating the effect of an intervention but is still of interest. Most clinical studies have more than one secondary outcome measure.
Outcome Measure
Measure Description
Time Frame
Component-Level Accuracy of LLM-Based Uncertainty Recognition
Time Frame: After freezing the model and prompt versions, through completion of all validation analyses, up to 18 months.

This outcome measures the component-level accuracy of the large language model in recognizing uncertainty labels for report-derived imaging cTNM staging. The LLM-generated cT_uncertain, cN_uncertain, and cM_uncertain labels will be compared with the expert consensus reference standard. Accuracy will be calculated as the number of correctly classified uncertainty labels divided by the total number of component-level uncertainty labels across all reports. The three uncertainty components will be aggregated into one percentage value.

中文对应
After freezing the model and prompt versions, through completion of all validation analyses, up to 18 months.
Complete cTNM Triplet Matching Rate for Human Evaluators and the LLM
Time Frame: During pre-training and post-training human annotation rounds and LLM batch inference, up to 18 months.
This outcome measures the proportion of reports for which the complete report-derived imaging cTNM triplet assigned by human evaluators and by the large language model exactly matches the expert consensus reference standard. A report will be counted as correct only when all three components, cT_report, cN_report, and cM_report, are correct. The result will be reported as the percentage of reports with complete cTNM triplet agreement. Results will be summarized separately for pre-training human annotation, post-training human annotation, and LLM batch inference.
During pre-training and post-training human annotation rounds and LLM batch inference, up to 18 months.
Annotation Time per Report for Human Evaluators and the LLM
Time Frame: During pre-training and post-training human annotation rounds and LLM batch inference, up to 18 months.
This outcome measures the mean time required to complete report-derived imaging cTNM annotation per report. For human evaluators, annotation time will be recorded during the annotation rounds and divided by the number of annotated reports. For the LLM workflow, batch inference time will be divided by the number of processed reports. Results will be reported separately for pre-training human annotation, post-training human annotation, and LLM batch inference.
During pre-training and post-training human annotation rounds and LLM batch inference, up to 18 months.

Collaborators and Investigators

This is where you will find people and organizations involved with this study.

Sponsor

Sponsor

The organization or person who initiates the study and who has authority and control over the study.
First Affiliated Hospital of Wenzhou Medical University

Study record dates

These dates track the progress of study record and summary results submissions to ClinicalTrials.gov. Study records and reported results are reviewed by the National Library of Medicine (NLM) to make sure they meet specific quality control standards before being posted on the public website.

Study Major Dates

Study Start (Estimated)

Study Start

The actual date on which the first participant was enrolled in a clinical study. The "anticipated" study start date is the date that the researchers think will be the study start date.
June 17, 2026

Primary Completion (Estimated)

Primary Completion

The date on which the last participant in a clinical study was examined or received an intervention to collect final data for the primary outcome measure. Whether the clinical study ended according to the protocol or was terminated does not affect this date. For clinical studies with more than one primary outcome measure with different completion dates, this term refers to the date on which data collection is completed for all the primary outcome measures. The "anticipated" primary completion date is the date that the researchers think will be the primary completion date for the study.
June 30, 2027

Study Completion (Estimated)

Study Completion

The date on which the last participant in a clinical study was examined or received an intervention/treatment to collect final data for the primary outcome measures, secondary outcome measures, and adverse events (that is, the last participant's last visit). The "anticipated" study completion date is the date that the researchers think will be the study completion date.
December 31, 2027

Study Registration Dates

First Submitted

First Submitted

The date on which the study sponsor or investigator first submitted a study record to ClinicalTrials.gov. There is typically a delay of a few days between the first submitted date and the record's availability on ClinicalTrials.gov (the first posted date).
June 30, 2026

First Submitted That Met QC Criteria

First Submitted That Met QC Criteria

The date on which the study sponsor or investigator first submits a study record that is consistent with National Library of Medicine (NLM) quality control (QC) review criteria. The sponsor or investigator may need to revise and submit a study record one or more times before NLM's QC review criteria are met. It is the responsibility of the sponsor or investigator to ensure that the study record is consistent with the NLM QC review criteria.
July 15, 2026

First Posted (Actual)

First Posted

The date on which the study record was first available on ClinicalTrials.gov after National Library of Medicine (NLM) quality control (QC) review has concluded. There is typically a delay of a few days between the date the study sponsor or investigator submitted the study record and the first posted date.
July 16, 2026

Study Record Updates

Last Update Posted (Actual)

Last Update Posted

The most recent date on which changes to a study record were made available on ClinicalTrials.gov. There may be a delay between when the changes were submitted to ClinicalTrials.gov by the study's sponsor or investigator (the last update submitted date) and the last update posted date.
July 16, 2026

Last Update Submitted That Met QC Criteria

Last Update Submitted That Met QC Criteria

The most recent date on which the study sponsor or investigator submitted changes to a study record that are consistent with National Library of Medicine (NLM) quality control (QC) review criteria. It is the responsibility of the sponsor or investigator to ensure that the study record is consistent with the NLM QC review criteria.
July 15, 2026

Last Verified

Last Verified

The most recent date on which the study sponsor or investigator confirmed the information about a clinical study on ClinicalTrials.gov as accurate and current. If a study with a recruitment status of recruiting; not yet recruiting; or active, not recruiting has not been confirmed within the past 2 years, the study's recruitment status is shown as unknown.
July 1, 2026

More Information

Terms related to this study

Keywords

Additional Relevant MeSH Terms

Other Study ID Numbers

Other Study ID Numbers

Identifiers or ID numbers other than the NCT number that are assigned to a clinical study by the study's sponsor, funders, or others. These numbers may include unique identifiers from other trial registries and National Institutes of Health grant numbers.
  • KY2026-278

Plan for Individual participant data (IPD)

Plan to Share Individual Participant Data (IPD)?

NO

IPD Plan Description

Individual participant-level data will not be shared. The study data consist of de-identified Chinese PSMA PET/CT report texts and necessary baseline clinical information generated during routine clinical care. Although direct identifiers will be removed, the free-text report data may still carry a potential risk of re-identification. Therefore, individual-level raw data will not be made publicly available. Study findings will be reported in aggregate form. De-identified summary data or analysis methods may be made available upon reasonable request and with approval from the ethics committee and the institutional data governance authority, when applicable.

Drug and device information, study documents

Studies a U.S. FDA-regulated drug product

No

Studies a U.S. FDA-regulated device product

No

This information was retrieved directly from the website clinicaltrials.gov without any changes. If you have any requests to change, remove or update your study details, please contact register@clinicaltrials.gov. As soon as a change is implemented on clinicaltrials.gov, this will be updated automatically on our website as well.

Clinical Trials on Prostate Cancer

Clinical Trials on Large Language Model-Assisted Report Annotation

Search Similar Trials

Sponsors and Collaborators

Medical Conditions

Drug Interventions

Conditions

Rare Diseases

Drug Interventions

Dietary Supplements

Sponsor/Collaborators

Locations

Cookie Settings

We use cookies to ensure that we give you the best experience on our website. For more details carefully read our Privacy and Cookies Policy. ...>>>You can change your preferences or withdraw your consent at any time by deleting the cookies from your website or computer as described in the policy. Please accept it and choose your preferences by ticking the corresponding boxes in the "Manage Settings" section below. Please carefully read our Terms of Service before you proceed with using any part of this website.
«Manage Settings