Validation on Clinical Adaptability of the Foundation Model Specific to Neuroimaging Diagnosis

Validation on Clinical Adaptability of the Foundation Model Specific to Neuroimaging Diagniosis

This clinic trial aims to investigate whether artificial intelligence (AI) diagnostic tools at neurological diseases diagnosis on brain CT/MRI can improve the work efficiency of specialized neuroimaging physicians, with a specific focus on its clinical value in distinguishing normal from abnormal findings, critical value identification, and neurological disease classification. Using pathological and/or discharge diagnoses of neurological diseases as the gold standard, an AI model will be trained on over 10,000 CT/MRI cases to achieve diagnostic performance comparable to that of neurological radiologists before being transformed and putted to use. Furthermore, clinical trials will be conducted in sub-studies (abnormal cases identification, critical value assessment, and neurological disease classification) to validate the clinical utility of AI and human-AI collaboration in the precise diagnosis of neurological disorders. The expected outcomes include reducing missed and misdiagnosis rates, enabling rapid screening of critical conditions, and achieving precise imaging-based diagnosis by using AI tools.

Study Overview

• Status: Enrolling by invitation
• Conditions: MRI; Central Nervous System Disease; Diagnose Disease; CT; AI (Artificial Intelligence); Classification
• Intervention / Treatment: Diagnostic test: Foundation Model Specific to Neurological Diagnosis

Detailed Description

Neurological disorders pose a severe threat to human health and create a substantial socio-economic burden. Imaging examinations, including CT and MRI, play an indispensable role in disease screening, noninvasive diagnosis, and guiding treatment decision. Artificial intelligence (AI) tools have shown promising clinical application prospects in releasing the productivity of radiologists and shortening patients' waiting time, particularly in critical care settings and medically underserved regions. Although AI tools trained on foundation model are supposed to have reliable generalization and can adapt to complex clinical scenarios, current AI systems often lack robust validation in real-world clinical practice.

In the face of growing demands for precision medicine and the deluge of medical imaging data, clinical trials are essential for validating the diagnostic efficacy of AI-assisted systems and their applicability in broader clinical settings. Based on a multidisciplinary team (integrating expertise in AI, radiology, emergency, neurology, and pathology) and prior research experience, this study has designed a comprehensive and robust research protocol to ensure the reliability of the trial, ultimately facilitating clinical translation.

This study hypothesizes that the working performance of the radiologists collaborating with the neuroimaging foundation model for brain CT and MRI is non-inferior to those who work standalone. For the secondary end-points, we investigate the performance of AI-radiologist collaboration of AI tools in real clinical environment. The clinic trial contains three sub-studies:

1. Identifying abnormal cases Brain scanning, including normal and all neurological disease coverage on both CT and MRI, were randomly given to two groups of radiologists to classify into normal or abnormal cases with or without the predicted label from the AI-assisted system. Comparing the sensitivity, specificity, and efficiency of radiologists collaborating with AI or working standalone.
2. Study for critical value identification The study involved a prospective, randomized selection of emergency CT data from a 1-to-3-day window. The performance of three paradigms-human-only, AI-only, and human+AI-was separately evaluated based on diagnostic accuracy, time efficiency, and the critical metric of lead time in identifying urgent findings when AI was integrated compared to that of human-only.
3. Disease classification experiment A prospectively selected dataset from a specific time period, comprising both CT and MRI examinations, was utilized. The experiment compares the performance of three diagnostic approaches, including human-only interpretation, AI-only analysis and human-AI collaborative interpretation. Part I, task completion time, diagnostic accuracy, and diagnostic recall rate (for cases with multiple labels) were calculated and compared across the three methods as follows: (1)Radiologists independently select and utilize provided disease templates for report generation, then extract diagnosis labels from the reports; (2) Radiologists generate reports while browsing images with the aid of AI-generated category labels and AI-assigned report templates based on the "midnights" major classification system; (3)AI automatically generates the complete report and classification labels. Part II, the original reporting radiologists were recalled to re-interpret the studies, and this re-evaluation was performed with the assistance of AI-generated labels and matched templates. Part Ⅲ, leveraging the classification capabilities of a large language model/foundation model, the system was evaluated on multiple large-sample external test sets to measure accuracy and processing time. Additionally, a randomized subset of cases from various categories was selected for assessment by neuroradiologists to calculate the rate of missed diagnoses and misdiagnoses.

• Study Type: Observational
• Enrollment (Estimated): 50000

Contacts and Locations

Study Locations

• China
  • Beijing, China
    • Beijing Tiantan Hospital

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: Child; Adult; Older Adult
• Accepts Healthy Volunteers: Yes

• Sampling Method: Non-Probability Sample

Study Population

All patient exams are suspected of harboring brain tumors or the other neurological diseases, with or without neurological symptoms, and without a history of prior brain surgery or inpatient/outpatient medical records. Patients underwent brain CT and MRI, and were primarily examined at Beijing Tiantan hospital, between 2012-2026.

Description

Inclusion Criteria:

• For MRI: patients suspected of harboring ischemic, hemorrhagic, brain tumors, degenerative brain disease, or traumatic brain injury at initiating or other institution, who subsequently underwent brain MRI;
• For CT: patients with or without neurological symptoms, suspected of harboring ischemic, hemorrhagic, space-occupying, degenerative brain disease, or traumatic brain injury, who subsequently underwent brain CT.

Exclusion Criteria:

Exclusion Criteria:

• Patients who opted-out or did not give permission to reuse clinical data.
• Patients with a history of prior brain surgery.
• Patients whose brain CT or MRI exhibit severe artifacts (e.g. heavy warping due to air, metal artifacts, heavy motion artifacts), thereby impeding the usage of the data.

Study Plan

How is the study designed?

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
The diagnostic performance of human-AI collaborative approach in identifying abnormal case is not inferior to that of human-only and AI-only	AI models and over 50 radiologists at neurological diseases on brain CT/MRI to assess the working performance of neuroimaging AI diagnostic tools for differentiating normal and abnormal examinations	6 months
The diagnostic performance of human-AI collaborative approach in critical value judgment is not inferior to that of human-only and AI-only	To improve the turnaround time and quality of urgent diagnostic reports on brain CT/MRI, the performance of three paradigms-human-only, AI-only, and human+AI-was evaluated based on diagnostic accuracy, time efficiency, and the critical metric of lead time in identifying urgent findings when AI was integrated compared to that of human-only.	6 months

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
The diagnostic performance of human-AI collaborative approach in disease classification is not inferior to that of human-only and AI-only	The interpretation for disease classification experiment including such steps: (1)Radiologists independently select and utilize provided disease templates for report generation, then extract diagnosis labels from the reports; (2) Radiologists generate reports while browsing images with the aid of AI-generated category labels (91 classes) and AI-assigned report templates based on the "midnights" major classification system; (3) AI automatically generates the complete report and classification labels.	6 months
The diagnostic performance of human-AI collaborative approach in disease classification is superior to that of human-only	The original reporting physicians are recalled to re-interpret the studies, and this re-evaluation is performed with the assistance of AI-generated labels and matched templates.	6 months
The diagnostic performance of current model in disease classification on brain CT and MRI is superior to comparable models on a large-scale external dataset	This standalone test evaluates the classification capabilities of current model and comparable models on multiple large scale external data, with main measurement indicators of accuracy and processing time. A randomized subset will be selected for assessment by neuroradiologists to calculate the rate of missed diagnoses and misdiagnoses.	6 months

Collaborators and Investigators

• Sponsor: Yaou Liu

Study record dates

Study Major Dates

• Study Start (Actual): May 1, 2025
• Primary Completion (Actual): October 1, 2025
• Study Completion (Estimated): December 1, 2030

Study Registration Dates

• First Submitted: March 5, 2026
• First Submitted That Met QC Criteria: March 11, 2026
• First Posted (Actual): March 13, 2026

Study Record Updates

• Last Update Posted (Actual): March 13, 2026
• Last Update Submitted That Met QC Criteria: March 11, 2026
• Last Verified: September 1, 2025

More Information

Terms related to this study

• Additional Relevant MeSH Terms: Nervous System Diseases; Central Nervous System Diseases
• Other Study ID Numbers: 2025-A10; 2025-A09 (Other Grant/Funding Number: Beijing Tiantan hospital)

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No