Retrospective Study of Carebot AI CXR Performance in Preclinical Practice

July 12, 2023 updated by: Carebot s.r.o.

Chest X-Ray Abnormality Detection Using Artificial Intelligence: Retrospective Study of Carebot AI CXR Performance in Preclinical Practice

The purpose of this study is to describe the design, methodology and evaluation of the preclinical test of Carebot AI CXR software, and to provide evidence that the investigated medical device meets user requirements in accordance with its intended use. Carebot AI CXR is defined as a recommendation system (classification "prediction") based on computer-aided detection. The software can be used in a preclinical deployment at a selected site before interpretation (prioritization, display of all results and heatmaps) or after interpretation (verification of findings) of CXR images, and in accordance with the manufacturer's recommendations. Given this, a retrospective study is performed to test the clinical effectiveness on existing CXRs.

Study Overview

Status

Completed

Conditions

Intervention / Treatment

Detailed Description

The performance of the trained and internally validated Carebot AI CXR software is tested on a set of 127 CXR images from target population. This is compared to common clinical practice, i.e., image assessment by a radiologist in a hospital. Patients may have a variety of findings; at this stage of the evaluation, an abnormal finding is considered to be an abnormality in any of the defined classes. False negative images incorrectly predicted by Carebot AI CXR software result in a clinical impact determination.

To collect the CXR data for retrospective study, investigators addressed a municipal hospital in the Czech Republic that provides healthcare services to up to 130,000 residents of a medium-sized city (approximately 70,000 inhabitants) and the surrounding area. 127 anonymized CXR images were collected between August 15 and 17, 2022, and subsequently submitted to five independent radiologists of varying experience for annotation. The selected radiologists were asked to assess whether the CXR image shows any of the 12 pre-selected abnormalities. Pediatric CXR images (under 18 years of age), scans with technical problems (poor image quality, rotation), and images in lateral projection were excluded from the dataset.

Study Type

Observational

Enrollment (Actual)

127

Contacts and Locations

This section provides the contact details for those conducting the study, and information on where this study is being conducted.

Study Locations

  • Czechia
      • Havířov, Czechia, 73601
        • Nemocnice Havířov, p. o.

Participation Criteria

Researchers look for people who fit a certain description, called eligibility criteria. Some examples of these criteria are a person's general health condition or prior treatments.

Eligibility Criteria

Ages Eligible for Study

14 years and older (Adult, Older Adult)

Accepts Healthy Volunteers

No

Sampling Method

Non-Probability Sample

Study Population

To collect the CXR data for retrospective study, we addressed a municipal hospital in the Czech Republic that provides healthcare services to up to 130,000 residents of a medium-sized city (approximately 70,000 inhabitants) and the surrounding area. 127 anonymized CXR images were collected between August 15 and 17, 2022.

Description

Inclusion Criteria:

  • Hospital patients who were referred for chest radiography between August 15 and 17, 2022.

Exclusion Criteria:

  • Pediatric CXR images (under 18 years of age)
  • Scans with technical problems (poor image quality, rotation)
  • Images in lateral projection

Study Plan

This section provides details of the study plan, including how the study is designed and what the study is measuring.

How is the study designed?

Design Details

  • Observational Models: Cohort
  • Time Perspectives: Retrospective

Cohorts and Interventions

Group / Cohort
Intervention / Treatment
Retrospective collection of DICOM patient files for the period 15-17 August
To collect the CXR data for retrospective study, we addressed a municipal hospital in the Czech Republic that provides healthcare services to up to 130,000 residents of a medium-sized city (approximately 70,000 inhabitants) and the surrounding area. 127 anonymized CXR images were collected between August 15 and 17, 2022, and subsequently submitted to five independent radiologists of varying experience for annotation. The selected radiologists were asked to assess whether the CXR image shows any of the 12 abnormalities mentioned above. Pediatric CXR images (under 18 years of age), scans with technical problems (poor image quality, rotation), and images in lateral projection were excluded from the dataset.
Device: Carebot AI CXR
Carebot AI CXR is a deep learning-based software that assists radiologists in the interpretation of chest radiographs in posterior-anterior (PA) or anterior-posterior (AP) projections. The solution with artificial intelligence automatically detects abnormality based on visual patterns for the following findings: atelectasis, consolidation, cardiomegaly, mediastinal widening, pneumoperitoneum, pneumothorax, pulmonary edema, pulmonary lesion, bone fracture, hilar enlargement, subcutaneous emphysema, and pleural effusion.

What is the study measuring?

Primary Outcome Measures

Outcome Measure
Measure Description
Time Frame
Primary objective
Time Frame: 20-10-2022
Comparison of the accuracy of radiologist and Carebot AI CXR image assessment.
20-10-2022

Secondary Outcome Measures

Outcome Measure
Measure Description
Time Frame
Secondary objective
Time Frame: 20-10-2022
Comparison of the accuracy of radiologis with different experience vs. Carebot AI CXR. Weakness assessment of Carebot AI CXR.
20-10-2022

Collaborators and Investigators

This is where you will find people and organizations involved with this study.

Sponsor

Carebot s.r.o.

Study record dates

These dates track the progress of study record and summary results submissions to ClinicalTrials.gov. Study records and reported results are reviewed by the National Library of Medicine (NLM) to make sure they meet specific quality control standards before being posted on the public website.

Study Major Dates

Study Start (Actual)

August 15, 2022

Primary Completion (Actual)

August 17, 2022

Study Completion (Actual)

October 20, 2022

Study Registration Dates

First Submitted

October 21, 2022

First Submitted That Met QC Criteria

October 21, 2022

First Posted (Actual)

October 26, 2022

Study Record Updates

Last Update Posted (Actual)

July 13, 2023

Last Update Submitted That Met QC Criteria

July 12, 2023

Last Verified

July 1, 2023

More Information

Terms related to this study

Additional Relevant MeSH Terms

Other Study ID Numbers

  • 00001

Plan for Individual participant data (IPD)

Plan to Share Individual Participant Data (IPD)?

NO

IPD Plan Description

The results of the retrospective study are used to validate the Carebot AI CXR software for the target population prior to deployment in the workplace.

Drug and device information, study documents

Studies a U.S. FDA-regulated drug product

No

Studies a U.S. FDA-regulated device product

No

This information was retrieved directly from the website clinicaltrials.gov without any changes. If you have any requests to change, remove or update your study details, please contact register@clinicaltrials.gov. As soon as a change is implemented on clinicaltrials.gov, this will be updated automatically on our website as well.

Clinical Trials on Pneumonia

Clinical Trials on Carebot AI CXR

Search Similar Trials

Sponsors and Collaborators

Medical Conditions

Drug Interventions

CROs by country

CROs in France

Conditions

Rare Diseases

Drug Interventions

Dietary Supplements

Sponsor/Collaborators

Locations

Subscribe