AI Assisted Detection of Chest X-Rays (AID-CXR)

April 8, 2024 updated by: Alex Novak, Oxford University Hospitals NHS Trust

Utility of an AI-based CXR Interpretation Tool in Assisting Diagnostic Accuracy, Speed, and Confidence of Healthcare Professionals: a Study Using 500 Retrospectively Collected Inpatient and Emergency Department CXRs From Two UK Hospital Trusts

This study has been added as a sub study to the Simulation Training for Emergency Department Imaging 2 study (ClinicalTrials.gov ID NCT05427838).

The Lunit INSIGHT CXR is a validation study that aims to assess the utility of an Artificial Intelligence-based (AI) chest X-ray (CXR) interpretation tool in assisting the diagnostic accuracy, speed, and confidence of a varied group of healthcare professionals. The study will be conducted using 500 retrospectively collected inpatient and emergency department CXRs from two United Kingdom (UK) hospital trusts. Two fellowship trained thoracic radiologists will independently review all studies to establish the ground truth reference standard. The Lunit INSIGHT CXR tool will be used to analyze each CXR, and its performance will be measured against the expert readers. The study will evaluate the utility of the algorithm in improving reader accuracy and confidence as measured by sensitivity, specificity, positive predictive value, and negative predictive value. The study will measure the performance of the algorithm against ten abnormal findings, including pulmonary nodules/mass, consolidation, pneumothorax, atelectasis, calcification, cardiomegaly, fibrosis, mediastinal widening, pleural effusion, and pneumoperitoneum. The study will involve readers from various clinical professional groups with and without the assistance of Lunit INSIGHT CXR. The study will provide evidence on the impact of AI algorithms in assisting healthcare professionals such as emergency medicine and general medicine physicians who regularly review images in their daily practice.

Study Overview

Status

Active, not recruiting

Conditions

Intervention / Treatment

Study Type

Observational

Enrollment (Estimated)

33

Contacts and Locations

This section provides the contact details for those conducting the study, and information on where this study is being conducted.

Study Locations

  • United Kingdom
    • Oxfordshire
      • Oxford, Oxfordshire, United Kingdom, OX3 9DU
        • Oxford University Hospitals NHS Foundation Trust

Participation Criteria

Researchers look for people who fit a certain description, called eligibility criteria. Some examples of these criteria are a person's general health condition or prior treatments.

Eligibility Criteria

Ages Eligible for Study

  • Child
  • Adult
  • Older Adult

Accepts Healthy Volunteers

Yes

Sampling Method

Non-Probability Sample

Study Population

General radiologists/radiographers/physicians reviewing chest X-rays as part of their routine clinical practice, currently working in the National Health Service (NHS).

Description

Inclusion Criteria:

  • General radiologists/radiographers/physicians who review CXRs as part of their routine clinical practice

Exclusion Criteria:

  • Thoracic radiologists
  • Non-radiology physicians with previous formal postgraduate CXR reporting training.
  • Non-radiology physicians with previous career in radiology, respiratory medicine or thoracic surgery to registrar or consultant level

Study Plan

This section provides details of the study plan, including how the study is designed and what the study is measuring.

How is the study designed?

Design Details

Cohorts and Interventions

Group / Cohort
Intervention / Treatment
Readers/Participants

Reader Selection: 30 readers will be selected from the following five clinical specialty groups:

  • emergency medicine (ED)
  • adult intensive care (ICU)
  • adult general medicine (AGM)
  • radiographers (Rad)
  • general radiologists

Each specialty group consists of 6 members of ranked seniority. For the physicians this consists of:

  • Two 'Juniors' (Foundation Year 1 - Specialty Training 2 years)
  • Two 'Middle Grades' (Registrar from Specialty Training 3 to 6 years)
  • Two Consultants

For the radiographers, this consists of:

  • Two 'Junior/Newly qualified radiographers' (up to 18 months experience post qualification)
  • Two 'Mid-experience radiographers' (approx. 3 years' experience)
  • Two 'Reporting radiographers' (5+ years' experience)
Other: Cases reading

The reading will be done remotely via the Report and Image Quality Control site (www.RAIQC.com), an online platform allowing medical imaging viewing and reporting. Participants can work from any location, but the work must be done from a computer with internet access. For avoidance of doubt, the work cannot be performed from a phone or tablet.

The project is divided into two phases and participants are required to complete both phases. The estimated total involvement in the project is up to 20-24 hours.

Phase 1: Time allowed: 2 weeks

- Review 500 chest X-rays and express a clinical opinion through a structured reporting template (multiple choice, no open text required).

Rest/washout period of 2 weeks.

Phase 2 - Time allowed: 2 weeks

- Review 500 chest X-rays together with an AI report for each case and express your clinical opinion through the same structured reporting template used in Phase A.
Ground truthers
Two consultant thoracic radiologists. A third senior thoracic radiologist's opinion (>20 years experience) will undertake arbitration.
Other: Ground truthing
Two consultant thoracic radiologists will independently review the images to establish the 'ground truth' findings on the CXRs, where a consensus is reached this will then be used as the reference standard. In the case of disagreement, a third senior thoracic radiologist's opinion (>20 years experience) will undertake arbitration. A difficulty score will be assigned to each abnormality by the ground truthers using a 4-point Likert scale (1 being easy/obvious to 4 being hard/poorly visualised).

What is the study measuring?

Primary Outcome Measures

Outcome Measure
Measure Description
Time Frame
Performance of AI algorithm: sensitivity
Time Frame: During 4 weeks of reading time
Evaluation of the Lunit INSIGHT CXR algorithm will be performed comparing it to the reference standard in order to determine sensitivity.
During 4 weeks of reading time
Performance of AI algorithm: specificity
Time Frame: During 4 weeks of reading time
Evaluation of the Lunit INSIGHT CXR algorithm will be performed comparing it to the reference standard in order to determine specificity.
During 4 weeks of reading time
Performance of AI algorithm: Area under the ROC Curve (AU ROC)
Time Frame: During 4 weeks of reading time
Evaluation of the Lunit INSIGHT CXR algorithm will be performed comparing it to the reference standard. Continuous probability score from the algorithm will be utilized for the ROC analyses, while binary classification results with a predefined operating cut-off will be used for evaluation of sensitivity, specificity, positive predictive value, and negative predictive value.
During 4 weeks of reading time
Performance of readers with and without AI assistance: Sensitivity
Time Frame: During 4 weeks of reading time
The study will include two sessions (with and without AI overlay), with all 30 readers reviewing all 500 CXR cases each time separated by a washout period to mitigate recall bias. The cases will be randomised between the two reads and for every reader.
During 4 weeks of reading time
Performance of readers with and without AI assistance: Specificity
Time Frame: During 4 weeks of reading time
The study will include two sessions (with and without AI overlay), with all 30 readers reviewing all 500 CXR cases each time separated by a washout period to mitigate recall bias. The cases will be randomised between the two reads and for every reader.
During 4 weeks of reading time
Performance of readers with and without AI assistance: Area under the ROC Curve (AU ROC)
Time Frame: During 4 weeks of reading time
The study will include two sessions (with and without AI overlay), with all 30 readers reviewing all 500 CXR cases each time separated by a washout period to mitigate recall bias. The cases will be randomised between the two reads and for every reader.
During 4 weeks of reading time
Reader speed with vs without AI assistance.
Time Frame: During 4 weeks of reading time
Mean time taken to review a scan, with vs without AI assistance.
During 4 weeks of reading time

Collaborators and Investigators

This is where you will find people and organizations involved with this study.

Sponsor

Oxford University Hospitals NHS Trust

Publications and helpful links

The person responsible for entering information about the study voluntarily provides these publications. These may be about anything related to the study.

General Publications

Helpful Links

Study record dates

These dates track the progress of study record and summary results submissions to ClinicalTrials.gov. Study records and reported results are reviewed by the National Library of Medicine (NLM) to make sure they meet specific quality control standards before being posted on the public website.

Study Major Dates

Study Start (Actual)

October 31, 2023

Primary Completion (Estimated)

October 1, 2024

Study Completion (Estimated)

December 1, 2024

Study Registration Dates

First Submitted

October 4, 2023

First Submitted That Met QC Criteria

October 9, 2023

First Posted (Actual)

October 10, 2023

Study Record Updates

Last Update Posted (Actual)

April 10, 2024

Last Update Submitted That Met QC Criteria

April 8, 2024

Last Verified

April 1, 2024

More Information

Terms related to this study

Keywords

Additional Relevant MeSH Terms

Other Study ID Numbers

  • 310995 - B

Drug and device information, study documents

Studies a U.S. FDA-regulated drug product

No

Studies a U.S. FDA-regulated device product

No

This information was retrieved directly from the website clinicaltrials.gov without any changes. If you have any requests to change, remove or update your study details, please contact register@clinicaltrials.gov. As soon as a change is implemented on clinicaltrials.gov, this will be updated automatically on our website as well.

Clinical Trials on Pneumoperitoneum

Clinical Trials on Cases reading

Search Similar Trials

Sponsors and Collaborators

Medical Conditions

Drug Interventions

CROs by country

CROs in Kosovo

Conditions

Rare Diseases

Drug Interventions

Dietary Supplements

Sponsor/Collaborators

Locations

3
Subscribe