Clinical Validation of an Artificial Intelligence-Based Scoring System for the Modified Rankin Scale (mRS) in Patients With Stroke

This study aims to validate the clinical performance of an artificial intelligence (AI)-based automatic scoring system for the Modified Rankin Scale (mRS). The core comparison is the consistency and accuracy between the AI-generated scores and standardized manual mRS follow-up assessments performed by trained professionals. The goal is to provide a convenient, efficient, and objective tool for stroke prognosis assessment, reduce the subjective variability of manual scoring, and optimize the stroke follow-up workflow.

Study Overview

• Status: Not yet recruiting
• Conditions: Stroke

Detailed Description

This is a prospective, multicenter, observational study designed to validate the diagnostic performance of an AI-based automated scoring system for the Modified Rankin Scale (mRS) in patients with stroke. The primary objective is to evaluate the agreement between AI-generated mRS scores and standardized manual assessments conducted by trained clinicians. Secondary endpoints include the system's sensitivity, specificity, and area under the receiver operating characteristic curve (AUC) in classifying functional outcomes.

• Study Type: Observational
• Enrollment (Estimated): 490

Contacts and Locations

• Study Contact: Name: Qingfeng Ma, MD; Phone Number: +8613601069493; Email: m.qingfeng@163.com
• Study Contact Backup: Name: Zixin Wang, MD Candidate; Phone Number: +8615031041048; Email: wzx15031041048@163.com

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: Adult; Older Adult
• Accepts Healthy Volunteers: No

• Sampling Method: Non-Probability Sample

Study Population

Consecutive stroke patients from multiple centers who are ≥18 years old, with confirmed stroke, and able to complete the 1-week follow-up will be enrolled. Patients with severe non-stroke neurological diseases, inability to communicate, or loss to follow-up will be excluded.

Description

Inclusion Criteria:

1. Age ≥ 18 years, male or female.
2. Clinically diagnosed with stroke, and confirmed by cranial CT/MRI to have stroke.
3. Clinically stable, with basic communication ability at discharge or outpatient visit. The patient or a fixed family caregiver is able to cooperate with telephone follow-up at 1 week after discharge or outpatient visit.
4. Signed informed consent by the patient or their legally authorized representative.

Exclusion Criteria:

1. Neurological deficits caused by non-stroke etiologies (e.g., brain tumor, traumatic brain injury, encephalitis).
2. Presence of severe disturbance of consciousness, severe cognitive impairment, psychiatric disorders, or global aphasia at discharge/outpatient visit, preventing effective communication; neither the patient nor family can cooperate with follow-up or assessment.
3. Combined with severe multi-organ failure (e.g., cardiac, hepatic, renal, respiratory), with an expected survival of less than 1 month, making completion of the 1-week follow-up impossible.
4. Long-term bedridden without a fixed caregiver, with no confirmed contact for follow-up, or refusal to participate in telephone follow-up and mRS assessment.
5. Incomplete clinical data, preventing baseline data collection.

Study Plan

How is the study designed?

Number of groups / cohorts

2

Cohorts and Interventions

Group / Cohort
AI-first interview group; Participants first receive telephone assessment by AI, followed by telephone assessment by human assessors.
Human-first group; Participants first receive telephone assessment by human assessors, followed by telephone assessment by AI.

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Agreement Between Artificial Intelligence (AI)-Based and Manual Modified Rankin Scale (mRS) Assessments	The weighted kappa coefficient quantifies the level of agreement between the Artificial Intelligence (AI)-generated Modified Rankin Scale (mRS) scores and the standardized manual mRS assessments performed by trained clinicians	7 days post-discharge or post-outpatient visit, ± 2 days

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Agreement Between AI-based and Manual Assessments of Dichotomized Modified Rankin Scale (mRS)	The simple kappa coefficient quantifies the level of agreement between the Artificial Intelligence (AI)-generated dichotomized Modified Rankin Scale (mRS) scores (0-2 vs. 3-6) and the standardized manual mRS assessments performed by trained clinicians	7 days post-discharge or post-outpatient visit, ± 2 days
Bland-Altman Limits of Agreement Between AI and Manual Modified Rankin Scale (mRS) Scores	The Bland-Altman limits of agreement analysis evaluates the consistency between the Artificial Intelligence (AI)-generated and manually assessed Modified Rankin Scale (mRS) scores. The difference between manual and AI scores will be plotted on the y-axis against their mean on the x-axis, with limits of agreement (mean difference ± 1.96 × standard deviation) calculated. The analysis aims to visually assess how agreement varies across the range of mRS scores and identify any proportional bias, such as greater disagreement in patients with severe disability.	7 days post-discharge or post-outpatient visit, ± 2 days
Diagnostic Performance of AI-Based vs. Manual Modified Rankin Scale (mRS) Dichotomization	The diagnostic performance analysis evaluates the ability of the Artificial Intelligence (AI)-based Modified Rankin Scale (mRS) scoring system to classify functional outcomes, using manual assessment as the reference standard. A 2×2 contingency table will be constructed for the dichotomized mRS categories (good outcome: 0-2 vs. poor outcome: 3-6). The analysis will calculate sensitivity, specificity, positive predictive value, negative predictive value, accuracy, and Youden's index. A receiver operating characteristic (ROC) curve will be plotted, and the area under the curve (AUC) will be computed.	7 days post-discharge or post-outpatient visit, ± 2 days

Collaborators and Investigators

• Sponsor: Xuanwu Hospital, Beijing
• Collaborators: Beijing Tiantan Hospital; Capital Medical University; The First Hospital of Fangshan District,Beijing; People's Hospital of Beijing Daxing District
• Investigators: Study Chair: Qingfeng Ma, MD, Xuanwu Hospital, Beijing

Study record dates

Study Major Dates

• Study Start (Estimated): April 10, 2026
• Primary Completion (Estimated): December 31, 2028
• Study Completion (Estimated): December 31, 2028

Study Registration Dates

• First Submitted: April 8, 2026
• First Submitted That Met QC Criteria: April 15, 2026
• First Posted (Actual): April 22, 2026

Study Record Updates

• Last Update Posted (Actual): April 22, 2026
• Last Update Submitted That Met QC Criteria: April 15, 2026
• Last Verified: April 1, 2026

More Information

Terms related to this study

• Keywords: Stroke; Artificial Intelligence; Modified Rankin Scale
• Additional Relevant MeSH Terms: Cerebrovascular Disorders; Brain Diseases; Central Nervous System Diseases; Nervous System Diseases; Vascular Diseases; Cardiovascular Diseases; Stroke
• Other Study ID Numbers: qingfeng ma AI mRS

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: NO
• IPD Plan Description: Due to the protection of participant privacy and institutional review board requirements, individual participant data (IPD) will not be shared publicly.

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No