- ICH GCP
- US Clinical Trials Registry
- Clinical Trial NCT04221100
xrAI - Improving Quality and Efficiency in Chest Radiograph Interpretation by Radiologists
xrAI (pronounced "X-ray") serves as a clinical assistance tool for trained clinical professionals who are interpreting chest radiographs. The tool is designed as a quality control and adjunct, limited, clinical decision support tool, and does not replace the role of clinical professionals. It highlights areas on chest radiographs for review by an interpreting clinician.
The objective of this study is to utilize machine learning and artificial intelligence algorithms (xrAI) to improve the quality and efficiency in the interpretation of chest radiographs by radiologists.
The hypothesis is that the addition of xrAI's analysis will reduce inter-observer variability in the interpretation of chest radiographs and increase participants' sensitivity, recall, and accuracy in pulmonary abnormality screening.
Study Overview
Status
Conditions
Intervention / Treatment
Detailed Description
To investigate the effect of xrAI for radiologists that interpret chest radiographs as part of their daily responsibilities, the investigators have designed a randomized control trial.
The pulmonary abnormalities detected by xrAI and included in the definition of abnormal are as follows: any linear scar or fibrosis, atelectasis, consolidation, abscess or cavity, nodule, pleural effusion, severe cases of emphysema and COPD (mild cases with hyperinflation but not significant emphysema are not flagged), and pneumothorax.
To assess the causal effect of xrAI the investigators randomly assign 10 to 14 radiologists to either treatment (x-ray images processed by xrAI) or control (no xrAI processing) groups. Participants will only review images once. Each participant will perform 500 radiograph interpretations in total.
Participants in the control group will be asked to interpret the same 500 images without xrAI's analysis.
To increase the precision of the estimate and better investigate potential differences between clinical professionals, investigators block randomize the assignment of treatment or control group.
To analyse the effect of xrAI, the investigators will estimate the average treatment effect (ATE) for each group by comparing the performance of the treatment and control groups using randomization-based inference (Green and Gerber, 2012).
Study Type
Phase
- Not Applicable
Participation Criteria
Eligibility Criteria
Ages Eligible for Study
- Child
- Adult
- Older Adult
Accepts Healthy Volunteers
Genders Eligible for Study
Description
Inclusion Criteria:
- Radiologist currently practicing at a Pureform Radiology clinic in Calgary, Canada.
Exclusion Criteria:
- Radiologists not currently practicing at a Pureform Radiology clinic in Calgary, Canada.
- Physicians currently practicing at a Pureform Radiology clinic in Calgary, Canada, but that are not radiologists by training.
Study Plan
How is the study designed?
Design Details
- Primary Purpose: Diagnostic
- Allocation: Randomized
- Interventional Model: Parallel Assignment
- Masking: Single
Arms and Interventions
Participant Group / Arm |
Intervention / Treatment |
---|---|
Placebo Comparator: Control
Participants will review the 500 chest radiographs without the assistance of xrAI
|
The pulmonary abnormalities detected by xrAI and included in the definition of abnormality are as follows: any linear scar or fibrosis, atelectasis, consolidation, abscess or cavity, nodule, pleural effusion, severe cases of emphysema and COPD (mild cases with hyperinflation but not significant emphysema are not flagged), pneumothorax. Participants in the treatment group will interpret 500 images presented alongside the results of xrAI's processing in a dark room and asked to categorize each image into one of the following categories: lungs are clear, at least one pulmonary abnormality is present, not sure. Participants in the control group will be asked to interpret the same 500 images as the treatment group but without xrAI's analysis. |
Experimental: Treatment
Participants will review the 500 chest radiographs with the assistance of xrAI
|
The pulmonary abnormalities detected by xrAI and included in the definition of abnormality are as follows: any linear scar or fibrosis, atelectasis, consolidation, abscess or cavity, nodule, pleural effusion, severe cases of emphysema and COPD (mild cases with hyperinflation but not significant emphysema are not flagged), pneumothorax. Participants in the treatment group will interpret 500 images presented alongside the results of xrAI's processing in a dark room and asked to categorize each image into one of the following categories: lungs are clear, at least one pulmonary abnormality is present, not sure. Participants in the control group will be asked to interpret the same 500 images as the treatment group but without xrAI's analysis. |
What is the study measuring?
Primary Outcome Measures
Outcome Measure |
Measure Description |
Time Frame |
---|---|---|
Number of abnormalities identified divided by number of total of images analyzed (accuracy)
Time Frame: Time needed to analyze 500 images. Participants will be asked to completed the exercise within 2 weeks.
|
Accuracy is defined as the ratio of the images where the physician's prediction matched the labels of the dataset. Accuracy= (TP+FP) / (TP+FP+TN+FN) TP (true positives) = cases interpreted as abnormal that are abnormal; FP (false positives) = cases wrongly interpreted to be abnormal; TN (true negatives) = cases correctly interpreted to be normal; FN (false negatives) = abnormal cases wrongly interpreted as normal. |
Time needed to analyze 500 images. Participants will be asked to completed the exercise within 2 weeks.
|
Number of true abnormalities identified divided by the total of abnormalities identified (precision)
Time Frame: Time needed to analyze 500 images. Participants will be asked to completed the exercise within 2 weeks.
|
Precision is defined as the probability of a radiograph being abnormal if a physician makes the determination that it is abnormal. Precision= TP / (TP+FP) TP (true positives) = cases interpreted as abnormal that are abnormal; FP (false positives) = cases wrongly interpreted to be abnormal; TN (true negatives) = cases correctly interpreted to be normal; FN (false negatives) = abnormal cases wrongly interpreted as normal. |
Time needed to analyze 500 images. Participants will be asked to completed the exercise within 2 weeks.
|
Number of true abnormalities identified divided by the sum of true abnormalities identified and abnormalities missed (recall)
Time Frame: Time needed to analyze 500 images. Participants will be asked to completed the exercise within 2 weeks.
|
Recall is defined as the probability of a physician catching an abnormality in an image if one exists (based on the labels of the dataset). Recall= TP / (TP+FN) TP (true positives) = cases interpreted as abnormal that are abnormal; FP (false positives) = cases wrongly interpreted to be abnormal; TN (true negatives) = cases correctly interpreted to be normal; FN (false negatives) = abnormal cases wrongly interpreted as normal. |
Time needed to analyze 500 images. Participants will be asked to completed the exercise within 2 weeks.
|
Secondary Outcome Measures
Outcome Measure |
Measure Description |
Time Frame |
---|---|---|
Mean of radiologist accuracy (as defined in outcome 1)
Time Frame: Time needed to analyze 500 images. Participants will be asked to completed the exercise within 2 weeks.
|
Investigators will calculate the mean of the accuracy of all participants in each group.
|
Time needed to analyze 500 images. Participants will be asked to completed the exercise within 2 weeks.
|
Mean of radiologist precision (as defined in outcome 2)
Time Frame: Time needed to analyze 500 images. Participants will be asked to completed the exercise within 2 weeks.
|
Investigators will calculate the mean of the precision of all participants in each group.
|
Time needed to analyze 500 images. Participants will be asked to completed the exercise within 2 weeks.
|
Mean of radiologist recall (as defined in outcome 3)
Time Frame: Time needed to analyze 500 images. Participants will be asked to completed the exercise within 2 weeks.
|
Investigators will calculate the mean of the recall of all participants in each group.
|
Time needed to analyze 500 images. Participants will be asked to completed the exercise within 2 weeks.
|
Collaborators and Investigators
Study record dates
Study Major Dates
Study Start (Anticipated)
Primary Completion (Anticipated)
Study Completion (Anticipated)
Study Registration Dates
First Submitted
First Submitted That Met QC Criteria
First Posted (Actual)
Study Record Updates
Last Update Posted (Actual)
Last Update Submitted That Met QC Criteria
Last Verified
More Information
Terms related to this study
Additional Relevant MeSH Terms
Other Study ID Numbers
- xrAI ClinicalEvaluation Rads
Plan for Individual participant data (IPD)
Plan to Share Individual Participant Data (IPD)?
IPD Plan Description
Drug and device information, study documents
Studies a U.S. FDA-regulated drug product
Studies a U.S. FDA-regulated device product
This information was retrieved directly from the website clinicaltrials.gov without any changes. If you have any requests to change, remove or update your study details, please contact register@clinicaltrials.gov. As soon as a change is implemented on clinicaltrials.gov, this will be updated automatically on our website as well.
Clinical Trials on Pulmonary Disease
-
National Taipei University of Nursing and Health...TerminatedChronic Pulmonary Disease | Chronic Obstructive Pulmonary Disease Exacerbation | Chronic Obstructive Pulmonary Disease With ExacerbationTaiwan
-
Spire, Inc.ResMedCompletedSevere Chronic Obstructive Pulmonary Disease | Moderate Chronic Obstructive Pulmonary DiseaseUnited States
-
Kırıkkale UniversityRecruitingCOPD (Chronic Obstructive Pulmonary Disease)Turkey
-
Hopital FochAir Liquide SARecruitingChronic Obstructive Pulmonary Disease SevereFrance
-
Fundación para la Investigación del Hospital Clínico...Not yet recruitingCOPD, Chronic Obstructive Pulmonary DiseaseSpain
-
Canandaigua VA Medical CenterRecruitingChronic Obstructive Pulmonary Disease ModerateUnited States
-
Mylan Inc.Theravance BiopharmaCompletedChronic Obstructive Pulmonary Disease, COPDNew Zealand
-
Astellas Pharma Global Development, Inc.CytokineticsCompletedChronic Obstructive Pulmonary Disease (COPD)United States
-
AstraZenecaParexel International Ltd; Cognizant Technology Solution; Center for Information... and other collaboratorsCompletedChronic Obstructive Pulmonary Disease COPDUnited States, Canada, France, Russian Federation, Ukraine, Hungary, Bulgaria
-
University of ZurichNational Center of Cardiology and Internal Medicine named after academician...CompletedChronic Obstructive Pulmonary Disease (COPD)Kyrgyzstan
Clinical Trials on Radiograph interpretation for pulmonary abnormalities
-
1QB Information Technologies Inc.Saskatchewan Health Authority - Regina AreaCompleted
-
Fondation LenvalRecruiting
-
University of SouthamptonEconomic and Social Research Council, United KingdomCompletedChronic Pain | Musculoskeletal Pain DisorderUnited Kingdom
-
University of DenverNational Institute of Mental Health (NIMH); University of California, Los AngelesCompletedAnxiety DisordersUnited States
-
Radboud University Medical CenterBoehringer Ingelheim; ZonMw: The Netherlands Organisation for Health Research... and other collaboratorsUnknownPulmonary Disease, Chronic Obstructive | Cough | Asthma | DyspneaNetherlands
-
Mclean HospitalRecruitingObsessive-Compulsive DisorderUnited States
-
Harvard Medical School (HMS and HSDM)Not yet recruiting
-
University of Mississippi Medical CenterCompleted
-
Mclean HospitalNational Institute of Mental Health (NIMH)CompletedDepression | Anxiety Disorders | Bipolar Disorder | Psychiatric DisorderUnited States
-
Florida State UniversityCompleted