Machine Vision Based MDS-UPDRS III Machine Rating

Machine Vision Based Machine Rating of MDS-UPDRS III

The Movement Disorders Society (MDS) Unified Parkinson's Disease Rating Scale (UPDRS) Part III (MDS-UPDRS III) is the primary assessment method for motor symptoms in Parkinson's disease patients. Currently, movement disorder specialists conduct semi-quantitative scoring, which entails limitations such as subjectivity, weak sensitivity, and a limited number of professional physicians. This study, based on machine vision, establishes gold standard labels according to expert scoring. By using machine learning, we develop a machine rating model and compare the model's performance with gold standard rating and general clinical rating to investigate the accuracy of machine vision-based MDS-UPDRS III machine rating.

Study Overview

• Status: Completed
• Conditions: Parkinsonian Disorders; Machine Learning
• Intervention / Treatment: Other: video recording

• Study Type: Observational
• Enrollment (Actual): 1679

Contacts and Locations

Study Locations

• China
  • Beijing Municipality
    • Beijing, Beijing Municipality, China, 100050
      • Center for Movement Disorders, Department of Neurology, Beijing Tiantan Hospital, Capital Medical University
    • Beijing, Beijing Municipality, China, 100730
      • Beijing Hospital, Neurology Department
  • Fujian
    • Fuzhou, Fujian, China, 350001
      • Department of Neurology, Fujian Medical University Union Hospital
  • Guangdong
    • Guangzhou, Guangdong, China, 510080
      • Department of Neurology, Guangdong Neuroscience Institute, Guangdong General Hospital, Guangdong Academy of Medical Sciences
  • Hubei
    • Wuhan, Hubei, China, 430022
      • Department of Neurology, Union Hospital, Tongji Medical College, Huazhong University of Science and Technology
  • Jiangsu
    • Suzhou, Jiangsu, China, 215004
      • Department of Neurology and Clinical Research Center of Neurological Disease, The Second Affiliated Hospital of Soochow University
  • Shanghai Municipality
    • Shanghai, Shanghai Municipality, China, 200025
      • Department of Neurology and Institute of Neurology, Ruijin Hospital, Shanghai Jiao Tong University School of Medicine
  • Sichuan
    • Chengdu, Sichuan, China, 610044
      • Department of Neurology, West China Hospital, Sichuan University

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: Adult; Older Adult
• Accepts Healthy Volunteers: No

• Sampling Method: Probability Sample

Study Population

In this study, participants are mainly recruited from the Department of Neurology at Ruijin Hospital, Shanghai Jiao Tong University School of Medicine, as well as from movement disorder clinics at 7 other centers. Participants are enrolled through outpatient assessment and screening. Two specialists in movement disorders independently diagnosed participants as having "Parkinsonism". In cases where there is a diagnostic disagreement, the Movement Disorder Specialist Group at the Department of Neurology, Ruijin Hospital collectively discusses and makes the decision.

Description

Inclusion Criteria:

• Meeting the diagnostic criteria for Parkinsonism established by the International Movement Disorder Society: having bradykinesia, and meeting at least one of the two criteria for resting tremor or muscle rigidity
• 20 to 80 years old
• Good compliance, voluntarily joining the study, and able to sign an informed consent form or have it signed by a legal representative

Exclusion Criteria:

• Significant cognitive impairment (MMSE ≤ 23)
• Unable to sign written informed consent or unable to complete the trial due to other reasons
• Other situations in which the researcher deems the participant unsuitable for this study
• Participation in other clinical trials

Study Plan

How is the study designed?

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Item-level: MAE	At the item level, mean absolute error (MAE) between paired AI scores and consensus reference scores, calculated as the average absolute difference across items. Lower values indicate closer agreement.	1 day
Item-level: P(|Δ|≥2)	At the item level, the proportion of paired AI and consensus reference scores with an absolute difference of 2 or more points. Lower values indicate fewer large item-level scoring disagreements.	1 day
Score-level: ICC	Intraclass correlation coefficient (ICC) between AI-derived scores and consensus reference scores, calculated separately for each of the four subdomain scores and the total score. Higher ICC values indicate greater agreement.	1 day
Score-level: SEM	Standard error of measurement (SEM) for AI-derived scores relative to consensus reference scores, calculated separately for each of the four subdomain scores and the total score. SEM quantifies measurement error in the units of the corresponding score, with lower values indicating greater measurement precision.	1 day
Score-level: SDC	Smallest detectable change (SDC), derived from the measurement error and calculated separately for each of the four subdomain scores and the total score. SDC represents the minimum score change required to exceed expected measurement error. Lower values indicate greater measurement precision.	1 day

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Item-level: Within-one agreement (ACC1; P(|Δ| ≤ 1))	At the item level, ACC1 is defined as the proportion of paired AI and consensus reference scores with an absolute difference of no more than 1 point, i.e., P(|Δ| ≤ 1). Higher values indicate closer item-level agreement.	1 day
Item-level: Exact-error proportions [P(|Δ| = k)]	At the item level, exact-error proportions, P(|Δ| = k), represent the proportions of paired AI and consensus reference scores with each exact absolute error magnitude k. In particular, P(|Δ| = 0) corresponds to exact-match accuracy (ACC), defined as the proportion of items for which the AI score exactly matches the consensus reference score. The remaining values of k characterize the distribution of item-level scoring errors.	1 day
Item-level: Per-class exact recall	At the item level, for each consensus reference score class, the proportion of items for which the AI score exactly matches the consensus reference score among all items belonging to that reference class. Higher values indicate better class-specific exact agreement.	1 day
Item-level: Confusion matrix	At the item level, a cross-tabulation of AI scores against consensus reference scores, showing the number of paired ratings for each combination of reference and AI score categories. Rows represent consensus reference scores and columns represent AI scores.	1 day
Item-level: Row-normalised confusion matrix	At the item level, the confusion matrix normalised within each consensus reference score row so that each row sums to 1, showing the distribution of AI scores conditional on each reference score class.	1 day
Score-level: Limits of agreement	Bland-Altman limits of agreement between AI-derived scores and consensus reference scores, calculated separately for each of the four subdomain scores and the total score. The limits characterize the range within which most paired differences between AI and reference scores are expected to fall.	1 day
Score-level: MAE	Mean absolute error (MAE) between AI-derived scores and consensus reference scores, calculated separately for each of the four subdomain scores and the total score as the average absolute difference between paired scores. Lower values indicate smaller scoring errors.	1 day
Score-level: RMSE	Root mean square error (RMSE) between AI-derived scores and consensus reference scores, calculated separately for each of the four subdomain scores and the total score. RMSE gives greater weight to larger scoring errors, with lower values indicating closer agreement.	1 day
Score-level: Spearman correlation	Spearman rank correlation between AI-derived scores and consensus reference scores, calculated separately for each of the four subdomain scores and the total score. Higher values indicate a stronger monotonic association between AI-derived and reference scores.	1 day

Collaborators and Investigators

• Sponsor: Ruijin Hospital
• Collaborators: Beijing Tiantan Hospital; West China Hospital; Beijing Hospital; Guangdong Provincial People's Hospital; Fujian Medical University Union Hospital; Second Affiliated Hospital of Soochow University; Union Hospital, Tongji Medical College, Huazhong University of Science and Technology

Study record dates

Study Major Dates

• Study Start (Actual): March 1, 2023
• Primary Completion (Actual): December 31, 2025
• Study Completion (Actual): December 31, 2025

Study Registration Dates

• First Submitted: June 7, 2023
• First Submitted That Met QC Criteria: June 7, 2023
• First Posted (Actual): June 18, 2023

Study Record Updates

• Last Update Posted (Actual): August 21, 2026
• Last Update Submitted That Met QC Criteria: August 20, 2026
• Last Verified: August 1, 2026

More Information

Terms related to this study

• Additional Relevant MeSH Terms: Brain Diseases; Central Nervous System Diseases; Nervous System Diseases; Movement Disorders; Basal Ganglia Diseases; Parkinsonian Disorders; Technology, Industry, and Agriculture; Technology; Tape Recording; Audiovisual Aids; Educational Technology; Television; Videotape Recording
• Other Study ID Numbers: u3

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: NO

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No