Endoscopic diagnosis and treatment planning for colorectal polyps using a deep-learning model

Abstract

We aimed to develop a computer-aided diagnostic system (CAD) for predicting colorectal polyp histology using deep-learning technology and to validate its performance. Near-focus narrow-band imaging (NBI) pictures of colorectal polyps were retrieved from the database of our institution. Of these, 12480 image patches of 624 polyps were used as a training set to develop the CAD. The CAD performance was validated with two test datasets of 545 polyps. Polyps were classified into three histological groups: serrated polyp (SP), benign adenoma (BA)/mucosal or superficial submucosal cancer (MSMC), and deep submucosal cancer (DSMC). The overall kappa value measuring the agreement between the true polyp histology and the expected histology by the CAD was 0.614-0.642, which was higher than that of trainees (n = 6, endoscopists with experience of 100 NBI colonoscopies in <6 months; 0.368-0.401) and almost comparable with that of the experts (n = 3, endoscopists with experience of 2,500 NBI colonoscopies in ≥5 years) (0.649-0.735). The areas under the receiver operating curves for CAD were 0.93-0.95, 0.86-0.89, and 0.89-0.91 for SP, BA/MSMC, and DSMC, respectively. The overall diagnostic accuracy of the CAD was 81.3-82.4%, which was significantly higher than that of the trainees (63.8-71.8%, P < 0.01) and comparable with that of experts (82.4-87.3%). The kappa value and diagnostic accuracies of the trainees improved with CAD assistance: that is, the kappa value increased from 0.368 to 0.655, and the overall diagnostic accuracy increased from 63.8-71.8% to 82.7-84.2%. CAD using a deep-learning model can accurately assess polyp histology and may facilitate the diagnosis of colorectal polyps by endoscopists.

Conflict of interest statement

The authors declare no competing interests.

Figures

Figure 1

A schematic of the training strategy of the computer-aided diagnostic system (CAD) using a 50-layered convolutional neural network and image patches. SP, serrated polyp; BA, benign conventional adenoma; MSMC, mucosal or superficial submucosal cancer; DSMC, deep submucosal cancer.

Figure 2

The receiver operating characteristic (ROC) curves evaluating the diagnostic performance of the computer-aided diagnostic system (CAD). The performance of the CAD was evaluated and compared with the performances of three expert endoscopists and three trainees using ROC curves. (A–C) The ROC curves for the CAD in the SP, BA/MSMC, and DSMC groups of test dataset I; (D–F) The ROC curves for the CAD in the SP, BA/MSMC, and DSMC groups of test dataset II. AUC, area under the ROC curve; SP, serrated polyp; BA, benign conventional adenoma; MSMC, mucosal or superficial submucosal cancer (cancer with invasion depth <1000 µm from the muscularis mucosa); DSMC, deep submucosal cancer (cancer with invasion depth ≥1000 µm from the muscularis mucosa).

Figure 3

The visualized class activation map images. The figures in small rectangles in each image show the probability of each class being predicted by the computer-aided diagnostic system (CAD). The red area represents the region that the CAD considers to be compatible with the particular histology with high probability. The blue area represents the region that CAD considers to have a low probability for the particular histology. SP, serrated polyp; BA, benign conventional adenoma; MSMC, mucosal or superficial submucosal cancer; DSMC, deep submucosal cancer.

Figure 4

Improvement of the diagnostic performance of trainees with the assistance of the computer-aided diagnostic system (CAD). All empty circles representing trainees’ performance moved to solid circles representing the performance of the CAD+trainees at the left upper side or near the yellow curved line; this suggests that the performance of the CAD+trainees was superior to that of trainees and comparable to that of the CAD (yellow curved line). (A–C) Improved diagnostic performance of the CAD+trainee in the SP, BA/MSMC, and DSMC groups of test dataset I; (D–F) Improved diagnostic performance of the CAD+trainee in the SP, BA/MSMC, and DSMC groups of test dataset II. SP, serrated polyp; BA, benign conventional adenoma; MSMC, mucosal or superficial submucosal cancer; DSMC, deep submucosal cancer.