Clinical Evaluation of the V5-LU01 AI Software in Thoracic CT for V5med Inc.

Evaluation of V5 Lung AI Software With A Two-Arm Comparative Reader Study

STUDY DESIGN:

This is a two-arm retrospective, multi-reader, multi-case, (MRMC) randomized reader study.

OBJECTIVE:

Primary: The primary objective of this clinical study is to prove that radiologist's performance aided with V5med Lung AI is superior to the unaided for detecting qualified lung nodules.

Secondary: The secondary objective of this clinical study is to prove that the radiologist's reading time is not significantly increased when aided with V5med Lung AI.

Addition Objectives: To prove that the agreement (e.g., in kappa correlation coefficient) between experts and radiologist's Lung-RADS score aided with V5med Lung AI is superior to the unaided.

NUMBER OF SUBJECTS:

Retrospective CT studies from approximately 350 patients will be included in the study with approximately 170 true positive cases and 180 normal cases.

PRIMARY ENDPOINTS:

Scores given by the radiologists with and without V5med Lung AI will be recorded and compared to the true status of the study-cases. The frequency of the scores for each method (Aided, Unaided) will be tabulated and LROC curves constructed along with sensitivity, specificity, PPV, NPV and clinical actions. Additionally, machine nodule detection rate and false positives per patient on normal cases will be measured.

PATIENT POPULATION:

The study will target approximately two hundred (200) patients whose CT lung nodules were shown to be cancer and one hundred and eighty (150) patients who have no lung nodule greater than 4mm. The patient population will be consistent with the national lung cancer screening protocols.

Study Overview

• Status: Completed
• Conditions: Lung Cancer
• Intervention / Treatment: Device: V5med Lung AI

Detailed Description

Overview of the Study:

A reader study with sixteen participating radiologists will be conducted. A location-specific receiver operating characteristic (LROC) curve will be used to evaluate radiologists' detection performance in detecting qualified lung nodules on chest CT scans with and without the use of the V5med Lung AI system. The reading time for each case will also be recorded.

The 16 study radiologists will be split into two groups, X and Y. The 350 chest CT scans will be split into two subsets, A and B. Group X radiologists will interpret subset A without AI and subset B with V5med Lung AI. Group Y radiologist will interpret subset B without AI and subset A with AI. After a minimum of one month, each radiologist group will interpret the same images again but in the reverse order with regard to study modes (without AI and with AI). Please see Figures 1 and 2 for detailed arrangement of the radiologist interpretation sessions.

During the baseline reading session the radiologist will mark each location of qualified nodules and assign a score. If at least one suspicious nodule is observed, the radiologist will also indicate a Lung-RADS score (i.e., 2, 3, 4A, 4B, or 4X) at the end of case interpretation.

During the second reading (i.e., aided reading) session, the radiologist will be presented with a chest CT with CADe marks displayed on the "left window" and the same chest CT without CADe marks displayed on the "right window". If the radiologist confirms a nodule, the radiologist will mark the location on the chest CT displayed on the right window. This may or may not correspond to the locations of the CADe marks as displayed on the left window. The same as in the baseline study, the radiologist will assign a score (level of suspicion) to each suspicious location and provide a Lung-RADS score for the chest CT case, if at least one location on the CT is marked.

Based on the suspicious locations and their scores (level of suspicion) assessed by study radiologists, LROC curves can be constructed for both the baseline and the aided reads, and the significance of any difference will be calculated. The LOS score will also be used to calculate sensitivity, specificity, PPV and NPV.

Number and types of Chest CT cases:

Retrospective chest CT image series from approximately 350 cases will be included in the study. Approximately two hundred (200) cases will have at least one nodule size greater than 4mm. These cases will have radiology-prove report and/or biopsy-confirmed cancers. Also included as nodule images are those where a nodule was not detected at that time, but was detected and acted on based on a subsequent CT. These are the prior images where the nodule can be identified in the same location as on the "current" image confirmed by the radiologist expert panel (using a majority of three as the decision criterion). In addition, approximately one hundred and fifty (150) cases will not have a nodule size greater than 4mm.

The selected chest CT cases for the 2-arm comparative reader study, randomly selected from a larger pool of CT cases, will be enriched in the following way:

1. Lung nodules (cancers and benign nodules) will be nodule size ranging from 4mm to 30 mm. The proportion of nodules 20 mm or less may take majority of the cases with nodule(s) since this lung lesion size range is where we expect the major impact of this software to be.
2. Non-Solid (ground glass) nodules will be added to the study dataset (based on availability) to determine the performance of the system on non-solid nodules. For this group, to have sufficient cases, we may include more benign (non-malignant) non-solid nodules.
3. In this project, VIRGINIA TECH study team will perform a machine standalone test of the V5med Lung AI algorithm followed by a reader performance evaluation study. V5med Technologies will provide a system configured with the operating point set to be used for the reader studies and a configuration for an "open" system to be used for machine testing and to generate free-response ROC (FROC) curve.

The Design of Two-Arm Comparative Study:

The First Arm Study:

Perform a baseline reader study without any special tool. The reader is asked to mark all potential nodules not less than 4mm. For each mark, the size of region of interest (ROI), reader score (level of suspicion), and time spent will be recorded for the statistical analyses.

The Second Arm Study:

Perform another reader study using the V5med Lung AI as the aided read. The reader is asked to mark all potential nodules not less than 4mm assisted by V5med Lung AI. Some or majority of ROIs' parameters will be pre-filled by V5med Lung AI. The reader will make final determination for each location. The marked location, LOS score, Lung-RADS score, and time spent will be recorded for the statistical analyses and compared to the first arm study.

The primary study hypothesis is that the detection of lung nodules with the use of V5med Lung AI and associated functions is superior to the detection of lung nodules without the use of V5med Lung AI, as measured by the area under the LROC curve.

• Study Type: Interventional
• Enrollment (Actual): 16
• Phase: Not Applicable

Contacts and Locations

Study Locations

• United States
  • Virginia
    • Arlington, Virginia, United States, 22203
      • Arlington Innovation Center: Health Research

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: Adult; Older Adult
• Accepts Healthy Volunteers: No

Description

Inclusion Criteria:

1. Screening chest CT taken of asymptomatic patients age 55 - 77 with history of smoking
2. Non-screening chest CT of an adult (age 18 and older) without comorbidity
3. Primary Lung Cancer, Biopsy-proven with radiology report
4. <= 2.5mm slice spacing, no gaps
5. Standard reconstruction kernels
6. Maximum of 7 lung nodules per CT scan
7. Nodules must be 4-30mm in size (though majority of them will be 20mm or smaller)

Exclusion Criteria:

1. Acute pneumothorax
2. Incomplete inclusion of both lungs, within the field of view, including both Apices
3. Excessive motion artifacts or beam-hardening artifacts
4. Symptomatic patients with co-morbidities

Study Plan

How is the study designed?

Design Details

• Primary Purpose: Diagnostic
• Allocation: Randomized
• Interventional Model: Parallel Assignment
• Masking: Double

Number of Arms

2

Arms and Interventions

Participant Group / Arm	Intervention / Treatment
No Intervention: Reader Study without AI (Baseline ); During the baseline reading session the radiologist will mark each location of qualified nodules and assign a score. If at least one suspicious nodule is observed, the radiologist will also indicate a Lung-RADS score (i.e., 2, 3, 4A, 4B, or 4X) at the end of case interpretation.
Active Comparator: Reader Study with AI (Aided by V5med Lung AI); Perform another reader study using the V5med Lung AI as the aided read. The reader is asked to mark all potential nodules not less than 4mm assisted by V5med Lung AI. Some or majority of ROIs' parameters will be pre-filled by V5med Lung AI. The reader will make final determination for each location. The marked location, LOS score, Lung-RADS score, and time spent will be recorded for the statistical analyses and compared to the first arm study.	Device: V5med Lung AI; During the second reading session (concurrent read), the radiologist will be presented with a standard appearing CT with CADe marks placed on the "left window" and the same original without any AI mark with be display on the "right window". Deeming a nodule, the radiologist will mark location. These may or may not correspond to the locations of the CADe markers. As in the baseline study, the radiologist will assign a level of suspicious to each mark and provide a Lung-RADS score and the size of longest dimension.

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Comparing the Detection Accuracy, Sensitivity, and Specificity of Clinically Actionable Lung Nodules	The detection rates in accuracy, sensitivity, and specificity on clinically actionable lung nodules were computed and compared for Aim 1 (baseline) and Arm 2 (study participants were aided with V5med Lung AI) studies.	6 months
Comparing LROC Curve in the Detection of Clinically Actionable Lung Nodules Among Normal	The areas under location-specific Receiver Operating Characteristic (LROC) curves were computed and compared for Aim 1 (baseline) and Aim 2 (study participants were aided with V5med Lung AI).	6 months

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Average Reading Time Per Case	The average reading times per reader per case were computed and comapred in both Aim 1 and Arm 2 (study participants were aided with ClearRead CT software) studies.	4 months

Collaborators and Investigators

• Sponsor: V5med Inc.

Study record dates

Study Major Dates

• Study Start (Actual): December 18, 2023
• Primary Completion (Actual): May 17, 2024
• Study Completion (Actual): May 17, 2024

Study Registration Dates

• First Submitted: July 5, 2024
• First Submitted That Met QC Criteria: July 21, 2024
• First Posted (Actual): July 23, 2024

Study Record Updates

• Last Update Posted (Actual): July 23, 2024
• Last Update Submitted That Met QC Criteria: July 21, 2024
• Last Verified: July 1, 2024

More Information

Terms related to this study

• Other Study ID Numbers: 23-806

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No
• product manufactured in and exported from the U.S.: No