Endotracheal Tube Position Assessment on Chest Radiographs Using Deep Learning

Abstract

Purpose: To determine the efficacy of deep learning in assessing endotracheal tube (ETT) position on radiographs.

Materials and methods: In this retrospective study, 22 960 de-identified frontal chest radiographs from 11 153 patients (average age, 60.2 years ± 19.9 [standard deviation], 55.6% men) between 2010 and 2018 containing an ETT were placed into 12 categories, including bronchial insertion and distance from the carina at 1.0-cm intervals (0.0-0.9 cm, 1.0-1.9 cm, etc), and greater than 10 cm. Images were split into training (80%, 18 368 images), validation (10%, 2296 images), and internal test (10%, 2296 images), derived from the same institution as the training data. One hundred external test radiographs were also obtained from a different hospital. The Inception V3 deep neural network was used to predict ETT-carina distance. ETT-carina distances and intraclass correlation coefficients (ICCs) for the radiologists and artificial intelligence (AI) system were calculated on a subset of 100 random internal and 100 external test images. Sensitivity and specificity were calculated for low and high ETT position thresholds.

Results: On the internal and external test images, respectively, the ICCs of AI and radiologists were 0.84 (95% CI: 0.78, 0.92) and 0.89 (95% CI: 0.77, 0.94); the ICCs of the radiologists were 0.93 (95% CI: 0.90, 0.95) and 0.84 (95% CI: 0.71, 0.90). The AI model was 93.9% sensitive (95% CI: 90.0, 96.7) and 97.7% specific (95% CI: 96.9, 98.3) for detecting ETT-carina distance less than 1 cm.

Conclusion: Deep learning predicted ETT-carina distance within 1 cm in most cases and showed excellent interrater agreement compared with radiologists. The model was sensitive and specific in detecting low ETT positions.© RSNA, 2020.

Conflict of interest statement

Disclosures of Conflicts of Interest: P.L. Activities related to the present article: disclosed no relevant relationships. Activities not related to the present article: author received honorarium from Infervision for lecture unrelated to this work. Other relationships: patent planned for AI assessment of support devices on radiography. A.F. disclosed no relevant relationships. R.G. Activities related to the present article: disclosed no relevant relationships. Activities not related to the present article: author is consultant for Bioclinica and Medtronic for clinical trial reads. Other relationships: disclosed no relevant relationships.

2020 by the Radiological Society of North America, Inc.

Figures

Figure 1:

The internal dataset was split into 80% for training (18 368 of 22 960), 10% for validation (2296 of 22 960), and 10% for test (2296 of 22 960). For interrater assessment, 100 random images were taken from the internal test dataset (blue curved arrow), and 100 images were taken from an external dataset.

Figure 2:

Bland-Altman plot of AI compared with the mean of the radiologists (reader 1 and original reader), and between the radiologists (reader 1 and original reader), on the 200 test images used for interrater assessment. The top green horizontal line represents the upper boundary (+ 1.96 SD) and the bottom red line the lower boundary (−1.96 SD) for the limits of agreement. The mean difference is −0.22 denoted by the middle orange horizontal line. AI = artificial intelligence, SD = standard deviation.

Figure 3:

Full confusion matrix of AI prediction compared with ground truth on the entire internal test set (2296 images). The

Figure 4a:

Confusion matrices on the entire internal test set (2296 images) for using (a) a less than 1-cm cutoff for low endotracheal tube (ETT) placement, (b) less than 2-cm cutoff for low ETT placement, and (c) a greater than or equal to 7-cm cutoff for high ETT placement. In c (denoted by *), 42 of 74 images were misclassified by one category, in which AI predicted 6–7 cm above carina, but ground truth was 7–8 cm. AI = artificial intelligence.

Figure 4b:

Confusion matrices on the entire internal test set (2296 images) for using (a) a less than 1-cm cutoff for low endotracheal tube (ETT) placement, (b) less than 2-cm cutoff for low ETT placement, and (c) a greater than or equal to 7-cm cutoff for high ETT placement. In c (denoted by *), 42 of 74 images were misclassified by one category, in which AI predicted 6–7 cm above carina, but ground truth was 7–8 cm. AI = artificial intelligence.

Figure 4c:

Confusion matrices on the entire internal test set (2296 images) for using (a) a less than 1-cm cutoff for low endotracheal tube (ETT) placement, (b) less than 2-cm cutoff for low ETT placement, and (c) a greater than or equal to 7-cm cutoff for high ETT placement. In c (denoted by *), 42 of 74 images were misclassified by one category, in which AI predicted 6–7 cm above carina, but ground truth was 7–8 cm. AI = artificial intelligence.

Figure 5a:

Class activation maps of the endotracheal tube (ETT) model. On the images, the class activation maps show the area of the image (colormap from red to light blue) that is most activated by the network, which is the region between the ETT and carina, indicating that the appropriate part of the image is being assessed. (a) The predicted ETT-carina distance was 2.9 cm (ground truth was 2.5 cm). (b) A low ETT position where the predicted ETT-carina distance was −0.2 cm (ground truth was 0 cm at level of carina). (c) A high ETT position, where the predicted ETT-carina distance was 6.8 cm (ground truth was 8.6 cm above carina); this illustrates an image where the AI system was off by more than one category. The black arrow denotes the true carina location; the white arrow denotes potentially what the AI system is looking at based off the heatmap, which is the junction of the azygous vein and superior vena cava that resembles the appearance of a carina. AI = artificial intelligence.

Figure 5b:

Class activation maps of the endotracheal tube (ETT) model. On the images, the class activation maps show the area of the image (colormap from red to light blue) that is most activated by the network, which is the region between the ETT and carina, indicating that the appropriate part of the image is being assessed. (a) The predicted ETT-carina distance was 2.9 cm (ground truth was 2.5 cm). (b) A low ETT position where the predicted ETT-carina distance was −0.2 cm (ground truth was 0 cm at level of carina). (c) A high ETT position, where the predicted ETT-carina distance was 6.8 cm (ground truth was 8.6 cm above carina); this illustrates an image where the AI system was off by more than one category. The black arrow denotes the true carina location; the white arrow denotes potentially what the AI system is looking at based off the heatmap, which is the junction of the azygous vein and superior vena cava that resembles the appearance of a carina. AI = artificial intelligence.

Figure 5c:

Class activation maps of the endotracheal tube (ETT) model. On the images, the class activation maps show the area of the image (colormap from red to light blue) that is most activated by the network, which is the region between the ETT and carina, indicating that the appropriate part of the image is being assessed. (a) The predicted ETT-carina distance was 2.9 cm (ground truth was 2.5 cm). (b) A low ETT position where the predicted ETT-carina distance was −0.2 cm (ground truth was 0 cm at level of carina). (c) A high ETT position, where the predicted ETT-carina distance was 6.8 cm (ground truth was 8.6 cm above carina); this illustrates an image where the AI system was off by more than one category. The black arrow denotes the true carina location; the white arrow denotes potentially what the AI system is looking at based off the heatmap, which is the junction of the azygous vein and superior vena cava that resembles the appearance of a carina. AI = artificial intelligence.