Effects of Expert Arbitration on Clinical Outcomes When Disputes Over Diagnosis Arise Between Physicians and Their Artificial Intelligence Counterparts: a Randomized, Multicenter Trial in Pediatric Outpatients

We have recently developed an artificial intelligence (AI) framework to diagnose common pediatric diseases. This randomized controlled clinical trial aims to investigate the effects of expert arbitration on clinical outcomes in the situation where the AI-based diagnosis differs from the diagnosis made by pediatricians.

Study Overview

• Status: Unknown
• Conditions: Pediatric Outpatients Encountered in Three Specialty Clinics, i.e. Respirology, Gastroenterology, and Genito-urology
• Intervention / Treatment: Other: expert arbitration over discordant diagnoses made by AI diagnostic system and human doctors, respectively

Detailed Description

Based on the historical clinical data of more than 1 million pediatric outpatients in the Guangzhou Women and Children's Medical Center, an AI diagnostic framework has recently been developed for common pediatric diseases [Liang H et al. evaluation and accurate diagnosis of pediatric disease using artificial intelligence. Nat Med. 2019;25(3):433-8]. This AI framework utilizes predefined schema to extract informative clinical data from free text and reaches clinical diagnoses by hypothetico-deductive reasoning. In internal validation, the AI system showed accuracy rates ranging from 0.85 for gastrointestinal disease to 0.98 for neuropsychiatric disorders, suggesting that it might be a promising assisting diagnostic tool in clinical practice. However, there is a lack of evidence-based strategy on how to handle the scenarios where the AI-based diagnosis and the diagnosis made by pediatricians are discordant. It is legitimate to assume that diseases with discordant diagnoses present more similar clinical features; in this case it is necessary to introduce an extra arbitrator for differential and decisive diagnosis. Therefore, we conduct this randomized controlled trial to: 1) compare the accuracy of the two diagnostic modes in a real-world clinical setting where the AI-based diagnosis and the diagnosis made by pediatricians are discordant by introducing an expert arbitrator; and 2) look further into the change of clinical outcomes (hospital revisit and hospitalization in the next 3 months after initial visit; average total outpatient cost) due to introduction of the expert arbitrator. Please note that although the aforementioned AI framework was designed for diagnosis of a wide range of diseases, this clinical trial is limited to outpatients encountered in three specialty clinics, i.e. respirology, gastroenterology, and genito-urology. The reason for this selection is that the discordant diagnoses are assumed to be more common for these two specialties according to the internal validation result.

• Study Type: Interventional
• Enrollment (Anticipated): 10000
• Phase: Not Applicable

Contacts and Locations

• Study Contact: Name: Huiying Liang, PhD; Phone Number: +86-20-3885-7692; Email: lianghuiying@hotmail.com
• Study Contact Backup: Name: Kuanrong Li, PhD; Phone Number: +86-20-33857716; Email: lik@gwcmc.org

Study Locations

• China
  • Guangdong
    • Guangzhou, Guangdong, China, 510623
      • Guangzhou Women And Children's Medical Center
      • Contact: Huiying Liang, PhD; Phone Number: +86-20-3885 7692; Email: lianghuiying@hotmail.com
      • Contact: Kuanrong Li, PhD; Phone Number: +86-20-3885 7716; Email: lik@gwcmc.org

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: No older than 18 years (Child, Adult)
• Accepts Healthy Volunteers: No
• Genders Eligible for Study: All

Description

Inclusion Criteria:

1. Outpatients who visits the respirology clinics or the gastroenterology clinics during the recruitment period.
2. Written informed consent is provided by parents/guardians

Exclusion Criteria:

1. Patients with any conditions that require immediate diagnosis and treatment.

Study Plan

How is the study designed?

Design Details

• Primary Purpose: Treatment
• Allocation: Randomized
• Interventional Model: Parallel Assignment
• Masking: None (Open Label)

Number of Arms

2

Arms and Interventions

Participant Group / Arm	Intervention / Treatment
Active Comparator: Experimental Arm; Each participant receives two diagnoses: one from the AI diagnostic system and the other from pediatricians, and the two diagnoses are discordant. Participants in the experimental arm will be referred to an expert arbitrator for differential and decisive diagnosis and will receive treatment prescribed by the expert arbitrator.	Other: expert arbitration over discordant diagnoses made by AI diagnostic system and human doctors, respectively; Each participant receives two diagnoses: one from the AI diagnostic system and the other from pediatricians, and the two diagnoses are discordant. Participants in the experimental arm will be referred to an expert arbitrator for differential and decisive diagnosis and will receive treatment prescribed by the expert arbitrator.
No Intervention: Control Arm; Each participant receives two diagnoses: one from the AI diagnostic system and the other from pediatricians, and the two diagnoses are discordant. Participants in the control arm will receive treatment prescribed by pediatricians.

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Hospital revisit	Within the first 3 months after the initial visit, active follow-up via phone call will be performed each month to collect the information on hospital revisit.	The next 3 months after the initial visit
Hospitalization in the next 3 months after the initial visit	be performed each month to collect the information on hospitalization.	The next 3 months after the initial visit
Average total outpatient cost	be performed each month to collect the information on the amount of money spending on healthcare.	The next 3 months after the initial visit

Secondary Outcome Measures

Outcome Measure	Time Frame
Accuracy rate of AI-based diagnosis and accuracy rate of the diagnoses made by pediatricians, using the diagnoses made by the expert arbitrator as the decisive diagnoses.	The next 3 months after the initial visit
Counseling time spent with each patient	The next 3 months after the initial visit

Collaborators and Investigators

• Sponsor: Guangzhou Women and Children's Medical Center

Study record dates

Study Major Dates

• Study Start (Anticipated): November 1, 2019
• Primary Completion (Anticipated): October 31, 2020
• Study Completion (Anticipated): April 30, 2021

Study Registration Dates

• First Submitted: July 4, 2019
• First Submitted That Met QC Criteria: July 4, 2019
• First Posted (Actual): July 8, 2019

Study Record Updates

• Last Update Posted (Actual): July 8, 2019
• Last Update Submitted That Met QC Criteria: July 4, 2019
• Last Verified: June 1, 2019

More Information

Terms related to this study

• Other Study ID Numbers: aiwcmc

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: NO

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No