Prospective User Study and Multicenter Validation of Multimodal Medical Imaging Large Models

Prospective User Study and Multicenter Validation of Multimodal Medical Imaging Large Models in the Diagnosis of Common Systemic Diseases

Following model development and locking, the fixed model is evaluated in prospectively collected CT cohorts from two centers. The study is observational and does not affect clinical care. A subset of cases is used in a randomized crossover reader study.

Study Overview

• Status: Completed
• Conditions: Diagnostic Imaging; Artificial Intelligence (AI); Common Systemic Diseases
• Intervention / Treatment: Other: Standalone Radiologist Interpretation; Other: AI-assisted Radiologist Interpretation

Detailed Description

After model locking, CT data are prospectively collected at two centers for observational validation without retraining or parameter adjustment. Model outputs do not influence patient management. A subset of eligible cases is selected for the randomized crossover reader study.

• Study Type: Observational
• Enrollment (Actual): 310

Contacts and Locations

Study Locations

• China
  • Guangdong
    • Guangzhou, Guangdong, China, 510630
      • The Third Affiliated Hospital of Southern Medical University

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: Adult; Older Adult
• Accepts Healthy Volunteers: Yes

• Sampling Method: Non-Probability Sample

Study Population

Patients from multiple medical centers in China who underwent systemic CT imaging for various clinical indications, representing a broad range of common systemic diseases.

Description

Inclusion Criteria:

• Patients who underwent CT examinations for common systemic diseases.
• Imaging data must have confirmed clinical reference standards, expert consensus, or pathological diagnosis.
• Availability of complete DICOM format images with standard acquisition protocols.

Exclusion Criteria:

• Poor image quality (e.g., severe motion or metal artifacts) that precludes definitive diagnosis.
• Cases with incomplete clinical or pathological reference standards.
• Corrupted image files or duplicate cases.

Study Plan

How is the study designed?

Number of groups / cohorts

1

Cohorts and Interventions

Group / Cohort

Intervention / Treatment

Validation Cohort; Prospective Observational Validation Cohorts CT data are prospectively collected at two centers after model locking and used for observational model validation. A subset is included in the randomized crossover reader study.

Other: Standalone Radiologist Interpretation; Radiologists interpret the medical images independently without any assistance from the AI model to establish a baseline performance.; Other: AI-assisted Radiologist Interpretation; Radiologists interpret the same set of medical images with the assistance of the multimodal medical imaging large model to evaluate the improvement in diagnostic performance.

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Case-level Diagnostic Accuracy and Area Under the ROC Curve (AUC)	Evaluation of case-level diagnostic accuracy (defined as the proportion of diagnostic decisions matching the clinical ground-truth label) and discrimination performance (measured by AUC) to compare unaided radiologist performance versus AI-assisted performance.	Up to 1 week per evaluation period

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Diagnostic Efficiency (Reading and Reporting Time)	Measurement of diagnostic efficiency recorded as the time (in seconds) taken by radiologists to complete the case review and generate findings, with and without AI assistance.	Up to 1 week per evaluation period
Inter-rater Agreement (Fleiss' Kappa)	Assessment of diagnostic consensus and inter-rater consistency among participating radiologists measured using Fleiss' kappa (κ).	Up to 1 week per evaluation period
Clinical Report Quality and Semantic Accuracy Score	Assessment of AI-generated draft report quality evaluated by senior experts on a 5-point Likert scale (focusing on semantic accuracy and clinical relevance) and automated metrics (GREEN and ROUGE-L).	Up to 1 week per evaluation period

Collaborators and Investigators

• Sponsor: The Third Affiliated Hospital of Southern Medical University
• Investigators: Principal Investigator: Yinghua Zhao, PhD, The Third Affiliated Hospital of Southern Medical University

Study record dates

Study Major Dates

• Study Start (Actual): January 1, 2026
• Primary Completion (Actual): June 8, 2026
• Study Completion (Actual): July 10, 2026

Study Registration Dates

• First Submitted: April 21, 2026
• First Submitted That Met QC Criteria: April 21, 2026
• First Posted (Actual): April 28, 2026

Study Record Updates

• Last Update Posted (Actual): August 7, 2026
• Last Update Submitted That Met QC Criteria: August 4, 2026
• Last Verified: August 1, 2026

More Information

Terms related to this study

• Keywords: Radiology; Deep Learning; Multicenter Study; Diagnostic Performance; Multimodal Large Model
• Other Study ID Numbers: 2025-ER-141

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: NO
• IPD Plan Description: To protect patient privacy and comply with institutional data security regulations.

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No