Accuracy of Deep-learning Algorithm for Detection and Risk Stratification of Lung Nodules

Feasibility Study: Accuracy and Sensitivity of Deep-learning Artificial Intelligence (AI) Algorithm for Detection and Risk Stratification of Lung Nodules in Osteogenic Sarcoma Patients

Osteosarcoma is regarded as most common malignant bone tumor in children and adolescents. Approximately 15% to 20% of patients with osteosarcoma present with detectable metastatic disease, and the majority of whom (85%) have pulmonary lesions as the sole site of metastasis. Previous studies have shown that the overall survival rate among patients with localized osteosarcoma without metastatic disease is approximately 60% to 70% whereas survival rate reduces to 10% to 30% in patients with metastatic disease. Though lately, neoadjuvant and adjuvant chemotherapeutic regimens can decline the mortality rate, 30% to 50% of patients still die of pulmonary metastases. Number, distribution and timing of lung metastases are of prognostic value for survival and hence computed tomography (CT) thorax imaging still plays a vital role in disease surveillance. In the last decade, the technology of multidetector CT scanner has enhanced the detection of numerous smaller lung lesions, which on one hand can increase the diagnostic sensitivity for lung metastasis, however, the specificity may be reduced. In recent years, deep-learning artificial intelligence (AI) algorithm in a wide variety of imaging examinations is a hot topic. Currently, an increasing number of Computer-Aided Diagnosis (CAD) systems based on deep learning technologies aiming for faster screening and correct interpretation of pulmonary nodules have been rapidly developed and introduced into the market. So far, the researches concentrating on the improving the accuracy of benign/malignant nodule classification have made substantial progress, inspired by tremendous advancement of deep learning techniques. Consequently, the majority of the existing CAD systems can perform pulmonary nodule classification with accuracy of 90% above. In clinical practice, not only the malignancy determination for pulmonary nodule, but also the distinction between primary carcinoma and intrapulmonary metastasis is crucial for patient management. However, most existing classification of pulmonary nodule applied in CAD system remains to be binary pattern (benign Vs malignant), in the lack of more thorough nodule classification characterized with splitting of primary and metastatic nodule. To the best of our knowledge, only a few studies have focuses on the performance of deep learning-based CAD system for identifying metastatic pulmonary nodule till now. In this proposed study, the investigators sought to determine the accuracy and sensitivity of one computer-aided system based on deep-learning artificial intelligence algorithm for detection and risk stratification of lung nodules in osteogenic sarcoma patients.

Study Overview

• Status: Completed
• Conditions: Osteogenic Sarcoma
• Intervention / Treatment: Radiation: computed tomography

• Study Type: Observational
• Enrollment (Actual): 100

Contacts and Locations

Study Locations

• Hong Kong
  • Shatin
    • Hong Kong, Shatin, Hong Kong
      • The Chinese University of Hong Kong, Prince of Wale Hospital

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: No older than 18 years (Child, Adult)
• Accepts Healthy Volunteers: N/A

• Sampling Method: Probability Sample

Study Population

This is a single institutional retrospective cohort study of patients diagnosed with osteogenic sarcoma between the year 2000 and 2018. All patients' data will be retrieved via the electronic patient database of our institution. Patient demographics, imaging and histological data, disease and treatment history will be recorded, including age at onset, details of chemotherapy, time interval of pulmonary metastasis from diagnosis, surgery for the primary bony tumor, subsequent pulmonary metastatectomy if performed, the length of survival, clinical outcome and so on.

Description

Inclusion Criteria:

• Patients with histologically confirmed osteogenic sarcoma
• With an age younger than 18 years old.
• Patients who underwent thin-section thoracic CT examinations for pre-treatment staging and/or subsequent post-treatment follow-up.
• With suspicious lung nodules detected on thoracic CT images.

Exclusion Criteria:

• Patients with concurring lesions that may influence analysis of lung nodules.

Study Plan

How is the study designed?

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
accuracy	proportion of true results(both true positives and true negatives) among whole instances	3 years
sensitivity	true positive rate in percentage(%) derived by ROC analysis	3 years
specificity	true negative rate in percentage (%) derived by ROC analysis	3 years
area under curve (AUC)	area under ROC curve in percentage (%)	3 years

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
average number of false positives per scan (FPs/scan)	FPs/scan in number (N) based on free-response receiver operating characteristic (FROC) analysis	3 years
competition performance metric (CPM)	Competitive performance metric (CPM) is a criterion used for CAD system evaluation. Based on FROC paradigm, CPM score is computed as an average sensitivity at seven predefined average false positive rates. CPM score ranges from 0 to 1, with higher CPM score indicating better CAD performance.	3 years

Collaborators and Investigators

• Sponsor: Chinese University of Hong Kong
• Collaborators: IBM China/Hong Kong Limited

Study record dates

Study Major Dates

• Study Start (Actual): November 6, 2019
• Primary Completion (Actual): August 1, 2023
• Study Completion (Actual): January 31, 2024

Study Registration Dates

• First Submitted: July 15, 2019
• First Submitted That Met QC Criteria: July 15, 2019
• First Posted (Actual): July 17, 2019

Study Record Updates

• Last Update Posted (Actual): February 7, 2024
• Last Update Submitted That Met QC Criteria: February 6, 2024
• Last Verified: February 1, 2024

More Information

Terms related to this study

• Keywords: Artificial intelligence; Deep learning; Lung nodules; Computer-aided diagnosis
• Additional Relevant MeSH Terms: Neoplasms, Connective and Soft Tissue; Neoplasms by Histologic Type; Neoplasms; Neoplasms, Bone Tissue; Neoplasms, Connective Tissue; Sarcoma; Osteosarcoma
• Other Study ID Numbers: 2019.421

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: NO

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No