AI-based System for Assessing Suspected Viral Pneumonia Related Lung Changes

Artificial Intelligence Based System for Assessing Suspected Viral Pneumonia Related Lung Changes According to Visual Pulmonary Lesion Grading System (CT 0-4): Retrospective Study

The AI-based system designed to process chest computed tomography (CT) aims to 1) detect the presence of pathologic patterns associated with interstitial changes in pneumonia; 2) highlight areas on the images with the probable presence of pathologies; 3) provide the physician with the results of image processing, including quantitative indicators of suspected viral pneumonia related lung changes according to visual pulmonary lesion grading system (CT0-4).

The retrospective study aims to demonstrate the clinical validation of the AI-based system. Clinical validation measures (sensitivity, specificity, accuracy, and area under the ROC curve) will be determined to provide evidence about the clinical efficacy of the AI-based system.

The hypothesis is that the measures of clinical validation of the AI-based system differ by no more than 8% from those declared by the manufacturer.

Study Overview

• Status: Recruiting
• Conditions: Pneumonia, Viral; COVID-19 Pneumonia
• Intervention / Treatment: Diagnostic test: Medical software (AI-based system)

Detailed Description

The AI-based system designed to process chest CT aims to 1) detect the presence of pathologic patterns associated with interstitial changes in pneumonia; 2) highlight areas on the images with the probable presence of pathologies; 3) provide the physician with the results of image processing, including quantitative indicators of suspected viral pneumonia related lung changes according to visual pulmonary lesion grading system (CT0-4).

This retrospective clinical study will provide the clinical validation of the AI-based system to analyze chest CT images and identify pathological patterns associated with interstitial changes in pneumonia. Clinical validation measures (sensitivity, specificity, accuracy, and area under the ROC curve) will be determined and compared with values declared by the manufacturer to provide evidence about the clinical efficacy of the AI-based system.

The first stage of clinical validation is the collection of a verified labeled dataset. For this purpose, the dataset is collected, labeled, and verified by a research group. The verified dataset should include chest CT images without infiltrative and interstitial lung changes characteristic of viral pneumonia, including COVID-19-associated (CT-0) and chest CT images of all degrees of lung involvement CT-1 (≤25%), CT-2 (25-50%), CT-3 (50-75%), CT-4 (≥75%) [1]. Forming the verified dataset will allow reliable conclusions to be drawn upon completion of the clinical validation. The verified dataset must include a sufficient volume of chest CT images. The verified dataset must be de-identified to ensure the safety of patient personal data.

The second stage of the clinical validation is assessing AI-based system performance by experts. For that purpose, the AI software is analyzed to identify radiological signs of viral pneumonia. Then an examination is made of the correctness of the quantitative assessment of lung damage associated with interstitial changes in pneumonia. The evaluation of both the ability to correctly identify signs of lung damage and to quantify the identified changes is carried out on the same verified dataset.

The third stage of clinical validation is the calculation of clinical efficacy metrics (accuracy, sensitivity, specificity, area under the ROC-curve (AUROC) of the AI-based system by testing it on a verified data set. Testing of the hypothesis to verify the main diagnostic characteristics (sensitivity and specificity) declared by the manufacturer is planned by constructing a two-sided 95% confidence interval (CI), which should not differ by more than 8% from the declared values of 95% and 97%, respectively. Those. the lower limit of the 95% CI for sensitivity should not cross the 87% threshold, and the lower limit of the 95% CI for specificity should not cross the 89% threshold.

All stages of the clinical trial must be under the control of the Principal Investigator.

Randomization of images is not provided in this clinical study, because All CT images will be assessed by the research group and AI software. Also, this design does not involve blinding or masking of the research team. The evaluation of CT images by experts and the software is carried out independently, i.e. the results of each party's assessment are not known to the other party in advance.

• Study Type: Observational
• Enrollment (Estimated): 563

Contacts and Locations

• Study Contact: Name: Victoria Zinchenko; Phone Number: +7 (495) 276-04-36; Email: ZinchenkoVV1@zdrav.mos.ru
• Study Contact Backup: Name: Anton Vladzymyrskyy; Email: VladzimirskijAV@zdrav.mos.ru

Study Locations

• Russian Federation
  • Moscow, Russian Federation, 127051
    • Recruiting
    • Research and Practical Clinical Center for Diagnostics and Telemedicine Technologies of the Moscow Health Care Department

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: Adult; Older Adult
• Accepts Healthy Volunteers: Yes

• Sampling Method: Non-Probability Sample

Study Population

Adults (patients over 18 years of age) who were referred for a chest CT study by a physician in case of suspicion or monitoring the effectiveness of treatment of diseases manifested by changes in the pulmonary parenchyma.

Description

Inclusion Criteria:

1. General

   1. Patients over 18 years old;
   2. Patients who underwent CT without contrast enhancement;
   3. Patients who underwent a CT scan according to a standardized scanning protocol: 120 kilovolts, slice thickness max. 2 mm, rigid "lung" filter (kernel) reconstruction;
   4. Patients whose studies should be of acceptable quality, performed with breath-holding, without technical artifacts, and respiratory and motor artifacts;
   5. Patients whose studies must contain DICOM tags responsible for the orientation and position of the patient in the images during the study, as well as DICOM tags responsible for the size of the scans and image parameters;
   6. Patients in whom the localization of changes is predominantly bilateral, in the basal and subpleural parts of the lungs, may be located peribronchial;

2. For group Normal

   a. Patients who do not contain COVID-19-related CT patterns;

3. For groups Mild, Moderate, Severe, and Critical

   1. Patients who contain COVID-19-related CT pattern: ground glass opacities (mild, moderate, and higher intensity);
   2. Patients who contain COVID-19-related CT pattern: pulmonary consolidation;
   3. Patients who contain COVID-19-related CT pattern: cobblestone infiltration of the lung parenchyma;
   4. Patients who contain COVID-19-related CT pattern: hydrothorax;
   5. Patients who contain a combination of one or more patterns.

Exclusion Criteria:

• Patients whose studies contain images with unreported CT patterns;
• Patients whose examinations do not conform to DICOM format;
• Patients whose examinations do not contain imaging of the lung region
• Patients whose examinations contain technical artifacts caused by malfunctions or features of CT scanners;
• Patients whose examinations contain improper patient positioning;
• Patients whose examinations contain studies with deleted DICOM tags responsible for scan size and image parameters;
• Patients whose examinations contain metal artifacts on the patient's body and clothing;
• Patients whose examinations contain the presence of other pathologic changes of lungs in patients - neoplastic, tuberculosis process, bacterial pneumonia, etc.;
• Patients under 18 years old.

Study Plan

How is the study designed?

Number of groups / cohorts

5

Cohorts and Interventions

Group / Cohort	Intervention / Treatment
Normal; CT-0. Not consistent with pneumonia (including COVID-19). [1,2] Enrollment number: 95	Diagnostic test: Medical software (AI-based system); Retrospective analysis of chest CT images with medical software (AI-based system)
Mild; СT-1. Ground glass opacities. Pulmonary parenchymal involvement =<25% OR absence CT signs in typical clinical manifestations and relevant epidemiological history [1,2]. Enrollment number: 117	Diagnostic test: Medical software (AI-based system); Retrospective analysis of chest CT images with medical software (AI-based system)
Moderate; CT-2. Ground glass opacities. Pulmonary parenchymal involvement 25-50% [1,2]. Enrollment number: 117	Diagnostic test: Medical software (AI-based system); Retrospective analysis of chest CT images with medical software (AI-based system)
Severe; CT-3. Ground glass opacities. Pulmonary consolidation. Pulmonary parenchymal involvement 50-75%. Lung involvement increased in 24-48 hours by 50% in respiratory impairment per follow-up studies [1,2]. Enrollment number: 117	Diagnostic test: Medical software (AI-based system); Retrospective analysis of chest CT images with medical software (AI-based system)
Critical; CT-4. Diffuse ground glass opacities with consolidations and reticular changes. Hydrothorax (bilateral, more on the left). Pulmonary parenchymal involvement >=75% [1,2]. Enrollment number: 117	Diagnostic test: Medical software (AI-based system); Retrospective analysis of chest CT images with medical software (AI-based system)

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Accuracy	The ability of an AI-based system to produce the correct result relative to the total number of trials	Upon completion, up to 1 year
Sensitivity	Effectiveness of the AI-based system to correctly identifies patients with the suspected viral pneumonia related lung changes	Upon completion, up to 1 year
Specificity	Effectiveness of the AI-based system to correctly identifies across a range of available measurements patients that do not have the suspected viral pneumonia related lung changes	Upon completion, up to 1 year
AUC ROC	The area under curve (AUC) of Receiver Operating Characteristic (ROC) curves of AI-based system in prediction of suspected viral pneumonia related lung changes	Upon completion, up to 1 year

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Approximate volume of affected lung tissue	Approximate volume of affected lung tissue - quantitative characteristics of lung damage volume in percent (%): separately for left lung, right lung and total percentage of damage	Time Frame: Upon completion, up to 1 year

Collaborators and Investigators

• Sponsor: Research and Practical Clinical Center for Diagnostics and Telemedicine Technologies of the Moscow Health Care Department
• Collaborators: Sciberia Co. Ltd

Publications and helpful links

Helpful Links

• Morozov S.P., et al. Diagnostic accuracy of computed tomography for identifying hospitalizations for patients with COVID-19 // Digital Diagnostics. - 2021. - Vol. 2. - N. 1. - P. 5-
• Morozov S.P., et al. How does artificial intelligence effect on the assessment of lung damage in COVID-19 on chest CT scan? // Digital Diagnostics. - 2021. - Vol. 2. - N. 1. -

Study record dates

Study Major Dates

• Study Start (Actual): June 3, 2024
• Primary Completion (Estimated): June 3, 2025
• Study Completion (Estimated): December 3, 2025

Study Registration Dates

• First Submitted: July 9, 2024
• First Submitted That Met QC Criteria: July 9, 2024
• First Posted (Actual): July 15, 2024

Study Record Updates

• Last Update Posted (Actual): July 22, 2024
• Last Update Submitted That Met QC Criteria: July 18, 2024
• Last Verified: July 1, 2024

More Information

Terms related to this study

• Keywords: Computed tomography; Artificial Intelligence; Chest CT; Machine Learning
• Additional Relevant MeSH Terms: Virus Diseases; Infections; Respiratory Tract Infections; Respiratory Tract Diseases; Lung Diseases; Pneumonia; Pneumonia, Viral
• Other Study ID Numbers: 2024-1

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: NO

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No