AI-Assisted Differentiation of Pancreatic Cystic Lesions Using EUS Images and Clinical Features

Artificial Intelligence-Assisted Differentiation of Pancreatic Mucinous Cystic Neoplasms, Serous Cystic Neoplasms, and Pseudocysts Using Endoscopic Ultrasound Images and Clinical Features: An Ambispective Observational Study

Pancreatic mucinous cystic neoplasms (MCNs), serous cystic neoplasms (SCNs), and pancreatic pseudocysts (PPCs) often exhibit overlapping imaging features on endoscopic ultrasound (EUS). However, these conditions require different clinical management strategies. Accurate differentiation is therefore important for appropriate treatment planning.

This study aims to develop and validate an artificial intelligence (AI) model that integrates EUS images and clinical features to differentiate MCNs, SCNs, and PPCs. The study is designed as an ambispective observational diagnostic study consisting of retrospective model development, prospective temporal validation, and a reader study involving EUS physicians.

Approximately 400 adult patients with confirmed MCN, SCN, or PPC will be included. Retrospective clinical data and EUS images will be used for model development and internal validation. After model development is completed, newly diagnosed patients will be prospectively enrolled to independently evaluate the diagnostic performance of the model.

The primary outcome is the patient-level macro-averaged area under the receiver operating characteristic curve (macro-AUROC) for three-class classification in the prospective validation cohort. Secondary outcomes include diagnostic accuracy, sensitivity, specificity, calibration, clinical net benefit, and the performance of EUS physicians with and without AI assistance.

This study is expected to provide evidence for the potential clinical value of AI-assisted differentiation of pancreatic cystic lesions. No additional invasive procedures or study-specific treatments will be performed.

Study Overview

• Status: Not yet recruiting
• Conditions: Pancreatic Pseudocyst; Pancreatic Mucinous Cystic Neoplasm; Pancreatic Serous Cystic Neoplasm

Detailed Description

This study is an ambispective observational diagnostic study designed to develop and independently validate a multimodal artificial intelligence (AI) model for differentiating pancreatic mucinous cystic neoplasms (MCNs), serous cystic neoplasms (SCNs), and pancreatic pseudocysts (PPCs) using endoscopic ultrasound (EUS) images and clinical features.

**Study Design**

Approximately 400 adult patients with a final diagnosis of MCN, SCN, or PPC will be included. The study consists of two stages: retrospective model development and prospective temporal validation. A reader study will also be conducted to evaluate the potential benefits of AI assistance for EUS physicians.

**Stage 1: Retrospective Model Development**

Approximately 310 patients will be retrospectively identified from existing clinical databases. EUS images, clinical characteristics available before EUS examination, and final diagnostic information will be collected.

MCN and SCN diagnoses will be established using surgical or histopathological findings. PPC will be diagnosed according to a predefined composite reference standard based on clinical history, imaging findings, and other relevant diagnostic evidence.

Separate models will be developed using EUS images alone, clinical features alone, and the combination of EUS images and clinical features. Multiple EUS images from the same patient will be aggregated to generate a single patient-level prediction. Patient-level stratified cross-validation will be used for internal validation, with measures implemented to prevent data leakage.

**Stage 2: Prospective Temporal Validation**

After the AI model and analysis procedures have been finalized, approximately 80-100 consecutive eligible patients will be prospectively enrolled for independent temporal validation. These patients will not be used for model training, feature selection, or parameter optimization.

The primary outcome will be the patient-level macro-averaged area under the receiver operating characteristic curve (macro-AUROC) for three-class classification of MCN, SCN, and PPC. Each disease category will be evaluated using a one-vs-rest approach, and the three AUROC values will be averaged. A 95% confidence interval will be estimated using bootstrap resampling.

Secondary evaluations will include overall accuracy, macro-F1 score, weighted F1 score, class-specific sensitivity, specificity, positive predictive value, negative predictive value, model calibration, and clinical net benefit.

**Reader Study**

EUS physicians with different levels of experience will independently evaluate selected cases. Diagnostic performance will be compared among physicians without AI assistance, the AI model alone, and physicians with AI assistance. Diagnostic confidence, interobserver agreement, and interpretation time will also be assessed.

**Model Interpretability**

Gradient-weighted class activation mapping (Grad-CAM) or attention-based visualization will be used to identify image regions contributing to model predictions. Shapley additive explanations (SHAP) will be used to evaluate the contributions of clinical variables.

Exploratory analyses will investigate model performance across predefined patient subgroups, the potential incremental value of EUS videos, and differences among pathological subtypes of MCN.

**Clinical Procedures and Patient Safety**

All EUS examinations, pathological assessments, and clinical treatments will be performed according to routine clinical practice. No additional invasive procedures or study-specific therapeutic interventions will be required. AI predictions during prospective validation will not be used to guide clinical decisions.

• Study Type: Observational
• Enrollment (Estimated): 400

Contacts and Locations

• Study Contact: Name: Beibei Liu; Phone Number: +86-21-64370045; Email: liubeibei915@gmail.com

Study Locations

• China
  • Shanghai Municipality
    • Shanghai, Shanghai Municipality, China, 200025
      • Ruijin Hospital, Shanghai Jiao Tong University School of Medicine

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: 18 years and older (Adult, Older Adult)
• Accepts Healthy Volunteers: No
• Genders Eligible for Study: All

• Sampling Method: Non-Probability Sample

Study Population

The study population will consist of approximately 400 adult patients aged 18 years or older with a final diagnosis of pancreatic mucinous cystic neoplasm (MCN), serous cystic neoplasm (SCN), or pancreatic pseudocyst (PPC) who have undergone endoscopic ultrasound (EUS) at Ruijin Hospital, Shanghai Jiao Tong University School of Medicine. Approximately 310 retrospective patients will be identified from existing clinical records for AI model development and internal validation. An additional 80-100 eligible patients will be consecutively enrolled for prospective temporal validation after model finalization. MCN and SCN diagnoses will be confirmed by pathology, whereas PPC diagnoses will be established using a predefined composite reference standard. Each patient will contribute one independent study sample regardless of the number of available EUS images.

Description

Inclusion Criteria:

• **Inclusion Criteria:**

  1. Adults aged 18 years or older, regardless of sex.
  2. Patients who have undergone endoscopic ultrasound (EUS) examination.
  3. A final diagnosis of pancreatic mucinous cystic neoplasm (MCN), serous cystic neoplasm (SCN), or pancreatic pseudocyst (PPC).
  4. For MCN and SCN, a confirmed diagnosis based on surgical or histopathological findings.
  5. For PPC, a diagnosis established according to a predefined composite reference standard incorporating clinical history, imaging findings, and relevant diagnostic evidence.
  6. Availability of at least one EUS image clearly showing the target lesion.
  7. EUS examination performed before surgery, ablation, drainage, or other treatment of the target lesion.
  8. Availability of essential clinical information corresponding to the EUS examination.
  9. For the prospective validation cohort, enrollment after model finalization and provision of written informed consent according to the approved ethics protocol.

**Exclusion Criteria:**

1. Severely degraded EUS image quality preventing reliable identification of the target lesion.
2. Availability of only post-treatment or postoperative EUS images, without pretreatment imaging.
3. Lack of reliable pathological confirmation for MCN or SCN.
4. Inability to establish a clear correspondence between the EUS images and the pathologically confirmed lesion.
5. Uncertain or disputed final diagnosis that remains unresolved after expert review.
6. Concurrent pancreatic cystic neoplasms for which the target lesion cannot be clearly identified.
7. Other pancreatic lesions, including intraductal papillary mucinous neoplasms (IPMNs), solid pseudopapillary neoplasms (SPNs), cystic neuroendocrine tumors, or cystic degeneration of solid tumors.
8. Substantial missing clinical information preventing multimodal analysis.
9. Refusal to provide or withdrawal of informed consent during the prospective phase.

Exclusion Criteria:

-

Study Plan

How is the study designed?

Design Details

• Observational Models: COHORT
• Time Perspectives: OTHER

Number of groups / cohorts

3

Cohorts and Interventions

Group / Cohort
Mucinous Cystic Neoplasm (MCN); Adult patients with pancreatic mucinous cystic neoplasms (MCNs) confirmed by surgical or histopathological examination. Eligible patients will have undergone endoscopic ultrasound (EUS) as part of routine clinical care. EUS images and pre-examination clinical data will be collected for AI model development and validation. Both retrospective and prospective cases will be included.
Serous Cystic Neoplasm (SCN); Adult patients with pancreatic serous cystic neoplasms (SCNs) confirmed by surgical or histopathological examination. Eligible patients will have undergone endoscopic ultrasound (EUS) as part of routine clinical care. EUS images and pre-examination clinical data will be collected for AI model development and validation. Both retrospective and prospective cases will be included.
Pancreatic Pseudocyst (PPC); Adult patients diagnosed with pancreatic pseudocysts (PPCs) according to a predefined composite reference standard incorporating clinical history, imaging findings, and relevant diagnostic evidence. Eligible patients will have undergone endoscopic ultrasound (EUS) as part of routine clinical care. EUS images and pre-examination clinical data will be collected for AI model development and validation. Both retrospective and prospective cases will be included.

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Patient-Level Macro-AUROC of the Multimodal AI Model for Three-Class Classification of MCN, SCN, and PPC	The diagnostic performance of the locked multimodal artificial intelligence (AI) model will be evaluated in an independent prospective temporal validation cohort using the patient-level macro-averaged area under the receiver operating characteristic curve (macro-AUROC). One-vs-rest AUROC will be calculated separately for mucinous cystic neoplasms (MCNs), serous cystic neoplasms (SCNs), and pancreatic pseudocysts (PPCs). The three AUROC values will then be averaged with equal weighting. Each patient will contribute one final prediction, and 95% confidence intervals will be estimated using bootstrap resampling. Higher values indicate better discrimination.	At completion of diagnostic assessment in the prospective temporal validation cohort, through December 2028.

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Overall Patient-Level Classification Accuracy of the AI Model	Overall accuracy will be calculated as the proportion of patients correctly classified as MCN, SCN, or PPC among all patients evaluated in the prospective validation cohort. Accuracy will be reported as a percentage with a 95% confidence interval.	At completion of diagnostic assessment in the prospective temporal validation cohort, through December 2028.
Macro-Averaged F1 Score of the AI Model	The F1 score will be calculated separately for MCN, SCN, and PPC using the harmonic mean of precision and recall. The macro-F1 score will be calculated as the unweighted average of the three class-specific F1 scores. Values range from 0 to 1, with higher values indicating better classification performance.	At completion of diagnostic assessment in the prospective temporal validation cohort, through December 2028.
Difference in Patient-Level Macro-AUROC Between Multimodal and Unimodal AI Models	The difference in patient-level macro-AUROC between the locked multimodal AI model and each unimodal model (EUS image-only and clinical feature-only) will be calculated in the independent prospective validation cohort. Each comparison will be reported as the arithmetic difference in macro-AUROC, with positive values indicating better discrimination by the multimodal model.	At completion of diagnostic assessment in the prospective temporal validation cohort, through December 2028.
Calibration Intercept of the Multimodal AI Model	The calibration intercept of the locked multimodal AI model will be estimated in the prospective validation cohort to assess systematic overestimation or underestimation of predicted probabilities. A value closer to 0 indicates better calibration.	At completion of diagnostic assessment in the prospective temporal validation cohort, through December 2028.
Change in Physicians' Diagnostic Accuracy With AI Assistance	EUS physicians with different levels of experience will evaluate cases with and without AI assistance. The primary measure for the reader study will be the paired difference in patient-level three-class diagnostic accuracy between AI-assisted and unaided interpretation. Additional reader outcomes will include macro-AUROC, macro-F1 score, diagnostic confidence, interobserver agreement, and interpretation time.	At completion of diagnostic assessment in the prospective temporal validation cohort, through December 2028.
Difference in Patient-Level Classification Accuracy Between Multimodal and Unimodal AI Models	The difference in overall patient-level classification accuracy between the locked multimodal AI model and each unimodal model (EUS image-only and clinical feature-only) will be calculated in the independent prospective validation cohort. Differences will be expressed in percentage points, with positive values indicating higher accuracy for the multimodal model.	At completion of diagnostic assessment in the prospective temporal validation cohort, through December 2028.
Calibration Slope of the Multimodal AI Model	The calibration slope of the locked multimodal AI model will be estimated in the prospective validation cohort to assess the agreement between the spread of predicted probabilities and observed outcomes. A value closer to 1 indicates better calibration.	At completion of diagnostic assessment in the prospective temporal validation cohort, through December 2028.
Multiclass Brier Score of the Multimodal AI Model	The multiclass Brier score will be calculated as the mean, across patients, of the sum of squared differences between the predicted probabilities for MCN, SCN, and PPC and their corresponding observed class indicators. The unnormalised score ranges from 0 to 2, with lower values indicating better probabilistic predictions.	At completion of diagnostic assessment in the prospective temporal validation cohort, through December 2028.

Collaborators and Investigators

• Sponsor: Ruijin Hospital

Study record dates

Study Major Dates

• Study Start (Estimated): November 1, 2026
• Primary Completion (Estimated): June 30, 2028
• Study Completion (Estimated): December 30, 2028

Study Registration Dates

• First Submitted: October 22, 2026
• First Submitted That Met QC Criteria: October 30, 2026
• First Posted (Actual): November 5, 2026

Study Record Updates

• Last Update Posted (Actual): November 5, 2026
• Last Update Submitted That Met QC Criteria: October 30, 2026
• Last Verified: October 1, 2026

More Information

Terms related to this study

• Keywords: Artificial intelligence; Endoscopic ultrasound; Deep learning; Prospective validation; Pancreatic cystic lesions; Multimodal learning
• Additional Relevant MeSH Terms: Neoplasms; Digestive System Diseases; Pancreatic Diseases; Cysts; Pancreatic Cyst; Pancreatic Pseudocyst
• Other Study ID Numbers: MR-31-26-085874

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: NO

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No