Validation of the Accuracy of an AI-Based System for Diagnosing Depressive Disorders

Accuracy of an Artificial Intelligence-Based System for Diagnosing Depressive Disorders: A Paired Comparison With Psychiatrist Clinical Diagnoses

The trial aimed to evaluate the accuracy of an artificial intelligence-based system for diagnosing depressive disorders. Specifically, it sought to determine whether the system's assessment validity is non-inferior to that of psychiatric specialists.

Study Overview

• Status: Not yet recruiting
• Conditions: Depressive Disorder

Detailed Description

This study evaluates the performance of the AI-assisted diagnostic system for identifying depressive disorders and its applicability in clinical settings. Utilizing a paired design with psychiatrists' clinical diagnoses as the gold standard, the study compares the system's diagnostic results with those of physicians to determine sensitivity and specificity, thus validating its clinical effectiveness in real-world outpatient scenarios. Additionally, standardized scales are used to assess users' perceptions of the system's usability, trustworthiness, and satisfaction, offering evidence to support the clinical integration of AI technology in mental health screening.

• Study Type: Observational
• Enrollment (Estimated): 100

Contacts and Locations

Study Locations

• China
  • Shanghai Municipality
    • Shanghai, Shanghai Municipality, China, 200030
      • Shanghai Mental Health Center
      • Contact: Min Zhao, MD; Phone Number: 3193 0086-021-34289888; Email: drminzhao@smhc.org.cn
      • Contact: Yiqun Tu, MA; Phone Number: 0086-18017311090; Email: yiquntu@163.com

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: Adult
• Accepts Healthy Volunteers: Yes

• Sampling Method: Non-Probability Sample

Study Population

The study includes the subjects with depressive disorders and healthy control into the experiment. Depressive individuals for the study will be recruited from actual outpatient patients at the Shanghai Mental Health Center. The healthy control will be recruited through public advertisements, both online and offline. Psychiatrists will be recruited through public advertisements by the Shanghai Mental Health Center.

Description

1. Subjects with anxiety disorders （1）Inclusion Criteria:

   • In accordance with ICD-11 for Depressive Disorders;
   • Between the ages of 18-60;
   • Ability to use computers or smartphone;
   • Native Chinese speaker;
   • Signing informed consent. （2）Exclusion Criteria:
   • With severe psychiatric symptoms requiring hospitalization, or unable to complete the required assessment and treatment;
   • With a high risk of suicide or self-injury;
   • With severe physical diseases, central nervous system diseases, or substance abuse;
   • With intellectual, visual, or auditory impairments that affect their ability to interact with aided-diagnostic systems.

2. Health Control （1）Inclusion Criteria:

   • Not meet ICD-11 criteria for Mental Disorders;
   • Between the ages of 18-60;
   • Ability to use computers or smartphone;
   • Native Chinese speaker;
   • Signing informed consent. （2）Exclusion Criteria:
   • With mental illness, or unable to complete the required assessment and treatment;
   • With severe physical diseases, central nervous system diseases, or substance abuse;
   • With intellectual, visual, or auditory impairments that affect their ability to interact with aided-diagnostic systems.

3. Psychiatrist （1）Inclusion Criteria:

   • Over 18 years old;
   • A minimum of three years' experience in anxiety and other mental health fields;
   • Intermediate or higher professional title;
   • Currently employed in the selected test region;
   • Signing informed consent.

Study Plan

How is the study designed?

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Diagnostic Accuracy (Sensitivity, Specificity, and Area Under the Curve) of the AI-Based Screening System for Depressive Disorders	In this study, the evaluation results of the subjects were taken as the gold standard based on the diagnostic conclusions of psychiatrists. The screening results of the aided-diagnostic system were compared with the diagnostic conclusions of the psychiatrists to verify the screening effectiveness of this system. Sensitivity : Calculated as [True Positives / (True Positives + False Negatives)], evaluating the AI system's ability to correctly identify patients with depression. Specificity : Calculated as [True Negatives / (True Negatives + False Positives)], evaluating the AI system's ability to correctly exclude non-depressed individuals.Area Under the Receiver Operating Characteristic Curve (AUC), which comprehensively evaluates the diagnostic discriminative power of the system by plotting sensitivity versus 1-specificity across different decision thresholds and calculating the AUC value .	through study completion, an average of 1 week

Collaborators and Investigators

• Sponsor: Shanghai Mental Health Center

Study record dates

Study Major Dates

• Study Start (Estimated): February 1, 2026
• Primary Completion (Estimated): March 15, 2026
• Study Completion (Estimated): March 15, 2026

Study Registration Dates

• First Submitted: November 17, 2025
• First Submitted That Met QC Criteria: December 29, 2025
• First Posted (Actual): December 31, 2025

Study Record Updates

• Last Update Posted (Actual): December 31, 2025
• Last Update Submitted That Met QC Criteria: December 29, 2025
• Last Verified: August 1, 2025

More Information

Terms related to this study

• Additional Relevant MeSH Terms: Mental Disorders; Mood Disorders; Depressive Disorder
• Other Study ID Numbers: MZhao-023

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: NO

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No