An Artificial Intelligence-Based Chest X-ray Model on Human Nodule Detection Accuracy From a Multicenter Study

Abstract

Importance: Most early lung cancers present as pulmonary nodules on imaging, but these can be easily missed on chest radiographs.

Objective: To assess if a novel artificial intelligence (AI) algorithm can help detect pulmonary nodules on radiographs at different levels of detection difficulty.

Design, setting, and participants: This diagnostic study included 100 posteroanterior chest radiograph images taken between 2000 and 2010 of adult patients from an ambulatory health care center in Germany and a lung image database in the US. Included images were selected to represent nodules with different levels of detection difficulties (from easy to difficult), and comprised both normal and nonnormal control.

Exposures: All images were processed with a novel AI algorithm, the AI Rad Companion Chest X-ray. Two thoracic radiologists established the ground truth and 9 test radiologists from Germany and the US independently reviewed all images in 2 sessions (unaided and AI-aided mode) with at least a 1-month washout period.

Main outcomes and measures: Each test radiologist recorded the presence of 5 findings (pulmonary nodules, atelectasis, consolidation, pneumothorax, and pleural effusion) and their level of confidence for detecting the individual finding on a scale of 1 to 10 (1 representing lowest confidence; 10, highest confidence). The analyzed metrics for nodules included sensitivity, specificity, accuracy, and receiver operating characteristics curve area under the curve (AUC).

Results: Images from 100 patients were included, with a mean (SD) age of 55 (20) years and including 64 men and 36 women. Mean detection accuracy across the 9 radiologists improved by 6.4% (95% CI, 2.3% to 10.6%) with AI-aided interpretation compared with unaided interpretation. Partial AUCs within the effective interval range of 0 to 0.2 false positive rate improved by 5.6% (95% CI, -1.4% to 12.0%) with AI-aided interpretation. Junior radiologists saw greater improvement in sensitivity for nodule detection with AI-aided interpretation as compared with their senior counterparts (12%; 95% CI, 4% to 19% vs 9%; 95% CI, 1% to 17%) while senior radiologists experienced similar improvement in specificity (4%; 95% CI, -2% to 9%) as compared with junior radiologists (4%; 95% CI, -3% to 5%).

Conclusions and relevance: In this diagnostic study, an AI algorithm was associated with improved detection of pulmonary nodules on chest radiographs compared with unaided interpretation for different levels of detection difficulty and for readers with different experience.

Conflict of interest statement

Conflict of Interest Disclosures: Drs Conjeti, Wang, Preuhs, Ghesu, Mansoor, Huemmer, Fieselmann, Joerger, and Mirshahzadeh are full-time paid employees of Siemens Healthineers. Dr Digumarthy reported receiving an honorarium from Siemens Medical Solutions; he reported receiving grant funding from Lunit Inc; he reported consultation fees for independent image analysis from Pfizer, Merck, Novartis, Roche, Bayer, Abbvie, Zai Laboratories, Biengen, Gradalis, and Bristol Mayer Squibb outside the submitted work; he reported receiving grant funding from GE outside the submitted work. Dr Rueckel reported receiving compensation for conference presentations from Siemens Healthineers GmbH outside the submitted work. Dr Sabel reported receiving grants from Siemens Healthcare GmbH Department of Radiology and University Hospital of Ludwig Maximilian University of Munich; he reported receiving compensation for conference presentations from Siemens Healthcare GmbH during the conduct of the study. Dr Sistermanns reported grants from Siemens Healthineers outside the submitted work. Mrs Mirshahzadeh reported receiving personal fees from Siemens Healthcare GmbH during the conduct of the study. Dr Kalra reported receiving grants from Riverain Technologies outside the submitted work. No other disclosures were reported.

Figures

Figure 1.. Flowchart of Chest Radiograph Selection, Inclusion and Exclusion Criteria, Ground Truthing, and Multireader Study

AI indicates artificial intelligence; DICOM, Digital Imaging and Communications in Medicine.

Figure 2.. Deidentified Radiograph Images From 4 Adult Patients With True Positive Pulmonary Nodules

Panel A, artificial intelligence (AI) helped detect right upper lobe nodule (arrowheads) for 3 junior (J) radiologists as well as improved the confidence score for 1 senior (S) radiologist (score >5 implies positive finding; score ≤5 is negative). Likewise, AI-aided interpretation led to detection of missed right upper lobe nodule (arrowheads) for the radiograph in panel B for 2 junior radiologists and improved confidence of 2 junior radiologists. For panel C, AI helped 2 junior and 2 senior radiologists detect right lower lung nodule (arrowheads) they had missed on unaided interpretation. AI also helped either detect the right mid- and left-lower lung nodules (L1 and L2; arrowheads) (1 junior and 2 senior radiologists for each nodule) or improve confidence for detecting nodules (3 junior and 1 senior radiologists).