Clinical Validation of Machine Learning Triage of Chest Radiographs

Artificial intelligence and machine learning have the potential to transform the practice of radiology, but real-world application of machine learning algorithms in clinical settings has been limited. An area in which machine learning could be applied to radiology is through the prioritization of unread studies in a radiologist's worklist. This project proposes a framework for integration and clinical validation of a machine learning algorithm that can accurately distinguish between normal and abnormal chest radiographs. Machine learning triage will be compared with traditional methods of study triage in a prospective controlled clinical trial. The investigators hypothesize that machine learning classification and prioritization of studies will result in quicker interpretation of abnormal studies. This has the potential to reduce time to initiation of appropriate clinical management in patients with critical findings. This project aims to provide a thoughtful and reproducible framework for bringing machine learning into clinical practice, potentially benefiting other areas of radiology and medicine more broadly.

Study Overview

• Status: Withdrawn
• Conditions: Chest--Diseases
• Intervention / Treatment: Other: Traditional workflow triage; Other: Machine learning workflow triage; Other: Random workflow triage

• Study Type: Interventional
• Phase: Not Applicable

Contacts and Locations

Study Locations

• United States
  • California
    • Stanford, California, United States, 94305
      • Stanford University

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: 18 years and older (Adult, Older Adult)
• Accepts Healthy Volunteers: No
• Genders Eligible for Study: All

Description

Inclusion Criteria:

• Radiologist at Stanford Hospital and Clinics

Exclusion Criteria:

• None

Study Plan

How is the study designed?

Design Details

• Primary Purpose: Diagnostic
• Allocation: Randomized
• Interventional Model: Crossover Assignment
• Masking: Single

Number of Arms

3

Arms and Interventions

Participant Group / Arm	Intervention / Treatment
Active Comparator: Traditional workflow triage; Radiologists follow standard triage of chest radiographs.	Other: Traditional workflow triage; Workflow triage is based on order location, STAT designation, and first-in-first-out status.; Other: Machine learning workflow triage; Workflow triage is based on the machine learning model's confidence of abnormality.; Other: Random workflow triage; Workflow triage is based on random order.
Active Comparator: Machine learning workflow triage; Radiologists follow machine learning triage of chest radiographs.	Other: Traditional workflow triage; Workflow triage is based on order location, STAT designation, and first-in-first-out status.; Other: Machine learning workflow triage; Workflow triage is based on the machine learning model's confidence of abnormality.; Other: Random workflow triage; Workflow triage is based on random order.
Sham Comparator: Random workflow triage; Radiologists follow randomly ordered triage of chest radiographs.	Other: Traditional workflow triage; Workflow triage is based on order location, STAT designation, and first-in-first-out status.; Other: Machine learning workflow triage; Workflow triage is based on the machine learning model's confidence of abnormality.; Other: Random workflow triage; Workflow triage is based on random order.

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Turnaround time	Time from completion of radiograph to time that radiologist issues an assessment via preliminary or final report	up to 1 hour

Collaborators and Investigators

• Sponsor: Stanford University
• Collaborators: Society of Thoracic Radiology
• Investigators: Principal Investigator: Emily Tsai, MD, Stanford University

Study record dates

Study Major Dates

• Study Start (Anticipated): August 1, 2022
• Primary Completion (Anticipated): November 1, 2022
• Study Completion (Anticipated): November 1, 2022

Study Registration Dates

• First Submitted: January 22, 2022
• First Submitted That Met QC Criteria: February 3, 2022
• First Posted (Actual): February 4, 2022

Study Record Updates

• Last Update Posted (Actual): November 1, 2022
• Last Update Submitted That Met QC Criteria: October 29, 2022
• Last Verified: October 1, 2022

More Information

Terms related to this study

• Other Study ID Numbers: 47832

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: No

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No
• product manufactured in and exported from the U.S.: No