Undetected Lung Cancer at Posteroanterior Chest Radiography: Potential Role of a Deep Learning-based Detection Algorithm

Abstract

Purpose: To evaluate the performance of a deep learning-based algorithm in detecting lung cancers not reported on posteroanterior chest radiographs during routine practice.

Materials and methods: The retrospective test dataset included 168 posteroanterior chest radiographs acquired between March 2017 and December 2018 (168 patients; mean age, 71.9 years ± 9.5 [standard deviation]; age range, 42-91 years) with 187 lung cancers (mean size, 2.3 cm ± 1.2) undetected during initial clinical evaluation, and 50 normal chest radiographs. CT served as the reference standard for ground truth. Four thoracic radiologists independently reevaluated the chest radiographs for lung nodules both without and with the aid of the algorithm. The performances of the algorithm and the radiologists were evaluated and compared on a per-chest radiograph basis and a per-lesion basis, according to the area under the receiver operating characteristic curve (AUROC) and area under the jackknife free-response ROC curve (AUFROC).

Results: The algorithm showed excellent diagnostic performances both in terms of per-chest radiograph classification (AUROC, 0.899) and per-lesion localization (AUFROC, 0.744); both of these values were significantly higher than those of the radiologists (AUROC, 0.634-0.663; AUFROC, 0.619-0.651; P < .001 for all). The algorithm also demonstrated higher sensitivity (69.6% [117 of 168] vs 47.0% [316 of 672]; P < .001) and specificity (94.0% [47 of 50] vs 78.0% [156 of 200]; P = .01). When assisted by the algorithm, the radiologists' AUROC (0.634-0.663 vs 0.685-0.724; P < 0.01 for all) and pooled AUFROC (0.636 vs 0.688; P = .03) substantially improved. The false-positive rate of the algorithm, that is, the total number of false-positive nodules divided by the total number of chest radiographs, was similar to that of pooled radiologists (21.1% [46 of 218] vs 19.0% [166 of 872]; P > .05).

Conclusion: A deep learning-based nodule detection algorithm showed excellent detection performance of lung cancers that were not reported on chest radiographs during routine practice and significantly reduced reading errors when used as a second reader.Supplemental material is available for this article.© RSNA, 2020See also commentary by White in this issue.

Conflict of interest statement

Disclosures of Conflicts of Interest: J.G.N. disclosed no relevant relationships. E.J.H. disclosed no relevant relationships. D.S.K. disclosed no relevant relationships. S.J.Y. disclosed no relevant relationships. H.C. disclosed no relevant relationships. J.M.G. Activities related to the present article: disclosed no relevant relationships. Activities not related to the present article: grants/grants pending from Dongkook Lifescience and Infinnitt Healthcare. Other relationships: disclosed no relevant relationships. C.M.P. Activities related to the present article: disclosed no relevant relationships. Activities not related to the present article: grant from Lunit. Other relationships: disclosed no relevant relationships.

2020 by the Radiological Society of North America, Inc.

Figures

Figure 1:

Flow diagram for our study design. EMR = electronic medical record, exam = examination.

Figure 2a:

(a) A 64-year-old woman with confirmed lung adenocarcinoma at the right lower lobe (arrow). (b) The lesion, located at the right retrodiaphragmatic area, was missed during routine clinical practice. (c) In the reader performance test, none of the thoracic radiologists detected the lesion. The algorithm successfully localized the lesion (light blue shaded area) with a probability score of 0.16.

Figure 2b:

(a) A 64-year-old woman with confirmed lung adenocarcinoma at the right lower lobe (arrow). (b) The lesion, located at the right retrodiaphragmatic area, was missed during routine clinical practice. (c) In the reader performance test, none of the thoracic radiologists detected the lesion. The algorithm successfully localized the lesion (light blue shaded area) with a probability score of 0.16.

Figure 2c:

(a) A 64-year-old woman with confirmed lung adenocarcinoma at the right lower lobe (arrow). (b) The lesion, located at the right retrodiaphragmatic area, was missed during routine clinical practice. (c) In the reader performance test, none of the thoracic radiologists detected the lesion. The algorithm successfully localized the lesion (light blue shaded area) with a probability score of 0.16.

Figure 3a:

(a) A 57-year-old woman with confirmed lung adenocarcinoma at the right lower lobe (arrow). (b) The lesion, located at the right hilar area, was missed during routine clinical practice. (c) In the reader performance test, all thoracic radiologists detected the lesion. The algorithm also successfully localized the lesion (green shaded area), with a probability score of 0.47.

Figure 3b:

(a) A 57-year-old woman with confirmed lung adenocarcinoma at the right lower lobe (arrow). (b) The lesion, located at the right hilar area, was missed during routine clinical practice. (c) In the reader performance test, all thoracic radiologists detected the lesion. The algorithm also successfully localized the lesion (green shaded area), with a probability score of 0.47.

Figure 3c:

(a) A 57-year-old woman with confirmed lung adenocarcinoma at the right lower lobe (arrow). (b) The lesion, located at the right hilar area, was missed during routine clinical practice. (c) In the reader performance test, all thoracic radiologists detected the lesion. The algorithm also successfully localized the lesion (green shaded area), with a probability score of 0.47.