Multi-Reader Retrospective Study Examining Carebot AI CXR 2.0.21-v2.01 Implementation in Everyday Radiology Clinical Practice

July 19, 2023 updated by: Carebot s.r.o.
The primary objective is to evaluate the performance parameters of the proposed DLAD (Carebot AI CXR) in comparison to individual radiologists.

Study Overview

Status

Completed

Conditions

Intervention / Treatment

Detailed Description

In the period between October 18th, 2022, and November 17th, 2022, anonymized chest X-ray images of patients were collected at the Radiodiagnostic Department of the Havířov Hospital, p.o. The collection process involved utilizing the CloudPACS imaging and archiving system provided by OR-CZ spol. s r.o.

The collected X-ray images were subjected to the proposed DLAD (Carebot AI CXR) for analysis. Subsequently, the DLAD's performance was compared with the standard clinical practice, where radiologists assessed the CXR images in the simulated hospital setting with access to standard viewing tools (e.g., pan, zoom, WW/WL) and were given an unlimited amount of time to complete the review. Each radiologist determined the presence or absence of 7 indicated radiological findings, including atelectasis (ATE), consolidation (CON), pleural effusion (EFF), pulmonary lesion (LES), subcutaneous emphysema (SCE), cardiomegaly (CMG), and pneumothorax (PNO).

Study Type

Observational

Enrollment (Actual)

956

Contacts and Locations

This section provides the contact details for those conducting the study, and information on where this study is being conducted.

Study Locations

  • Czechia
      • Havířov, Czechia, 73601
        • Nemocnice Havířov, p. o.

Participation Criteria

Researchers look for people who fit a certain description, called eligibility criteria. Some examples of these criteria are a person's general health condition or prior treatments.

Eligibility Criteria

Ages Eligible for Study

  • Adult
  • Older Adult

Accepts Healthy Volunteers

N/A

Sampling Method

Non-Probability Sample

Study Population

Patient's Sex Female: 480 (50.21 %) Male: 474 (49.58 %) Unspecified: 2 (0.21 %)

Patient's Age 18-30: 58 (6.07 %) 31-50: 163 (17.05 %) 51-70: 366 38.28 %) 70+: 369 (38.60 %)

Description

Inclusion Criteria:

  • Hospital patients > 18 years who were referred for chest radiography October 18th, 2022, and November 17th, 2022 at the Radiodiagnostic Department of the Havířov Hospital, p.o.

Exclusion Criteria:

  • Patients < 18 years
  • Chest X-ray images in lateral positions
  • Duplicated chest X-ray images

Study Plan

This section provides details of the study plan, including how the study is designed and what the study is measuring.

How is the study designed?

Design Details

Cohorts and Interventions

Group / Cohort
Intervention / Treatment
Retrospective collection for the period October 18th, 2022, and November 17th, 2022
A total of 1,073 chest X-rays were acquired within the specified period at the department. The data collection remained intact and unaffected throughout the testing phase, ensuring the integrity of the dataset. The collected sample accurately represents the prevalence of findings within the observed population. After excluding ineligible studies such as X-rays from patients under 18 years of age, lateral projection X-rays, and scans of insufficient quality, a total of 956 relevant CXRs were identified for further assessment.
Device: Carebot AI CXR
The proposed DLAD (Carebot AI CXR) is a deep learning-based medical device designed to assist radiologists in interpreting chest X-ray images acquired in anteroposterior (AP) or posteroanterior (PA) projection. By employing advanced deep learning algorithms, this solution enables automatic detection of abnormal findings by analyzing visual patterns associated with specific conditions. The targeted abnormalities include atelectasis (ATE), consolidation (CON), pleural effusion (EFF), pulmonary lesion (LES), subcutaneous emphysema (SCE), cardiomegaly (CMG), and pneumothorax (PNO). The DLAD functions as a prediction algorithm complemented by various application peripherals, such as web-based communication tools, DICOM file conversion capabilities, and storage and reporting libraries supporting both DICOM Structured Report and DICOM Presentation State formats.

What is the study measuring?

Primary Outcome Measures

Outcome Measure
Measure Description
Time Frame
Performance test
Time Frame: March 2023
The primary objective is to evaluate the performance parameters of the proposed DLAD (Carebot AI CXR) in comparison to individual radiologists. The performance test includes sensitivity and specificity, positive and negative likelihood ratio, and positive and negative predictive value. The aforementioned parameters are statistically compared using confidence intervals (CI) and p-Values. The comparison procedure consists of two steps: a global hypothesis test is conducted to determine whether there are significant differences between DLAD and radiologists. If the global hypothesis test yields a significant result, individual hypothesis tests are performed. Additionally, multiple comparison methods, such as McNemar with continuity correction for Se and Sp, Holm method for LRs, and weighted generalized score statistics for PVs, are applied to control the overall error rate. All tests are performed as two-tailed tests at the 5% significance level.
March 2023

Collaborators and Investigators

This is where you will find people and organizations involved with this study.

Sponsor

Carebot s.r.o.

Publications and helpful links

The person responsible for entering information about the study voluntarily provides these publications. These may be about anything related to the study.

General Publications

  • KVAK, Daniel, Anna CHROMCOVÁ, Petra OVESNÁ, Jakub DANDÁR, Marek BIROŠ, Robert HRUBÝ, Daniel DUFEK a Marija PAJDAKOVIĆ. Can Deep Learning Reliably Recognize Abnormality Patterns on Chest X-rays? A Multi-Reader Study Examining One Month of AI Implementation in Everyday Radiology Clinical Practice. arXiv. 2023, 2305.10116, 26 s.

Study record dates

These dates track the progress of study record and summary results submissions to ClinicalTrials.gov. Study records and reported results are reviewed by the National Library of Medicine (NLM) to make sure they meet specific quality control standards before being posted on the public website.

Study Major Dates

Study Start (Actual)

October 18, 2022

Primary Completion (Actual)

November 17, 2022

Study Completion (Actual)

March 21, 2023

Study Registration Dates

First Submitted

July 12, 2023

First Submitted That Met QC Criteria

July 19, 2023

First Posted (Actual)

July 27, 2023

Study Record Updates

Last Update Posted (Actual)

July 27, 2023

Last Update Submitted That Met QC Criteria

July 19, 2023

Last Verified

July 1, 2023

More Information

Terms related to this study

Keywords

Additional Relevant MeSH Terms

Other Study ID Numbers

  • 00002

Plan for Individual participant data (IPD)

Plan to Share Individual Participant Data (IPD)?

NO

Drug and device information, study documents

Studies a U.S. FDA-regulated drug product

No

Studies a U.S. FDA-regulated device product

No

This information was retrieved directly from the website clinicaltrials.gov without any changes. If you have any requests to change, remove or update your study details, please contact register@clinicaltrials.gov. As soon as a change is implemented on clinicaltrials.gov, this will be updated automatically on our website as well.

Clinical Trials on Pneumothorax

Clinical Trials on Carebot AI CXR

Search Similar Trials

Sponsors and Collaborators

Medical Conditions

Drug Interventions

CROs by country

CROs in Zambia

Conditions

Rare Diseases

Drug Interventions

Dietary Supplements

Sponsor/Collaborators

Locations

3
Subscribe