Real-time differentiation of adenomatous and hyperplastic diminutive colorectal polyps during analysis of unaltered videos of standard colonoscopy using a deep learning model

Michael F Byrne, Nicolas Chapados, Florian Soudan, Clemens Oertel, Milagros Linares Pérez, Raymond Kelly, Nadeem Iqbal, Florent Chandelier, Douglas K Rex, Michael F Byrne, Nicolas Chapados, Florian Soudan, Clemens Oertel, Milagros Linares Pérez, Raymond Kelly, Nadeem Iqbal, Florent Chandelier, Douglas K Rex

Abstract

Background: In general, academic but not community endoscopists have demonstrated adequate endoscopic differentiation accuracy to make the 'resect and discard' paradigm for diminutive colorectal polyps workable. Computer analysis of video could potentially eliminate the obstacle of interobserver variability in endoscopic polyp interpretation and enable widespread acceptance of 'resect and discard'.

Study design and methods: We developed an artificial intelligence (AI) model for real-time assessment of endoscopic video images of colorectal polyps. A deep convolutional neural network model was used. Only narrow band imaging video frames were used, split equally between relevant multiclasses. Unaltered videos from routine exams not specifically designed or adapted for AI classification were used to train and validate the model. The model was tested on a separate series of 125 videos of consecutively encountered diminutive polyps that were proven to be adenomas or hyperplastic polyps.

Results: The AI model works with a confidence mechanism and did not generate sufficient confidence to predict the histology of 19 polyps in the test set, representing 15% of the polyps. For the remaining 106 diminutive polyps, the accuracy of the model was 94% (95% CI 86% to 97%), the sensitivity for identification of adenomas was 98% (95% CI 92% to 100%), specificity was 83% (95% CI 67% to 93%), negative predictive value 97% and positive predictive value 90%.

Conclusions: An AI model trained on endoscopic video can differentiate diminutive adenomas from hyperplastic polyps with high accuracy. Additional study of this programme in a live patient clinical trial setting to address resect and discard is planned.

Keywords: colorectal adenomas; endoscopic polypectomy; polyp.

Conflict of interest statement

Competing interests: MFB: CEO and shareholder, Satis Operations Inc, ’ai4gi’ joint venture; research support: Boston Scientific. NC: Imagia shareholder, ‘ai4gi’ joint venture. FS: Imagia shareholder, ‘ai4gi’ joint venture. CO: Imagia shareholder, ‘ai4gi' joint venture. FC: Imagia shareholder, ’ai4gi' joint venture. DKR: consultant: Olympus Corp and Boston Scientific; research support: Boston Scientific, Endochoice and EndoAid.

© Article author(s) (or their employer(s) unless otherwise stated in the text of the article) 2019. All rights reserved. No commercial use is permitted unless otherwise expressly granted.

Figures

Figure 1
Figure 1
Schematic of the deep convolutional neural network model used.
Figure 2
Figure 2
Schematic of the data preparation and training procedure of the deep convolutional neural network (DCNN) frame classifier. Raw videos are curated and tagged on a frame-by-frame basis. Then videos are split into disjoint databases: the larger serving as the training set and the smaller serving as a validation set. The purpose of the latter is to carry out ‘early stopping’ during the training procedure. Data augmentation is performed on the training frames only. After training, the resulting frame classification model can be used for prediction on new videos.
Figure 3
Figure 3
Illustration of the real-time prediction on a new video. Individual frames from the video are presented to the classification model (resulting from the training procedure), whose output is then processed by the credibility update mechanism. The result is a class probability for each frame (where the class may be one of ‘NICE Type 1’, ‘NICE Type 2’, ‘No Polyp’, ‘Unsuitable’), as well as a credibility score between 0% and 100%. NICE, narrow band imaging International Colorectal Endoscopic.
Figure 4
Figure 4
(A) Screen shot of the model during the evaluation of a NICE type 1 lesion (hyperplastic polyp). The display shows the type determined by the model (type 1) and the probability (100%). (B) Screen shot of the model in the evaluation of a NICE type 2 lesion (conventional adenoma). The display shows the type 2 determined by the model and the probability (100%) (see video). NICE, narrow band imaging International Colorectal Endoscopic.
Figure 5
Figure 5
Receiver operator characteristic curve for the model differentiation of adenomatous versus hyperplastic polyps. AUC, area under the curve; DCNN, deep convolutional neural network.

References

    1. Hewett DG, Kaltenbach T, Sano Y, et al. . Validation of a simple classification system for endoscopic diagnosis of small colorectal polyps using narrow-band imaging. Gastroenterology 2012;143:599–607. 10.1053/j.gastro.2012.05.006
    1. Hayashi N, Tanaka S, Hewett DG, et al. . Endoscopic prediction of deep submucosal invasive carcinoma: validation of the narrow-band imaging international colorectal endoscopic (NICE) classification. Gastrointest Endosc 2013;78:625–32. 10.1016/j.gie.2013.04.185
    1. Rex DK, Kahi C, O’Brien M, et al. . The American society for gastrointestinal endoscopy PIVI (Preservation and Incorporation of Valuable Endoscopic Innovations) on real-time endoscopic assessment of the histology of diminutive colorectal polyps. Gastrointest Endosc 2011;73:419–22. 10.1016/j.gie.2011.01.023
    1. Abu Dayyeh BK, Thosani N, Konda V, et al. . ASGE Technology Committee systematic review and meta-analysis assessing the ASGE PIVI thresholds for adopting real-time endoscopic assessment of the histology of diminutive colorectal polyps. Gastrointest Endosc 2015;81:502.e1–6. 10.1016/j.gie.2014.12.022
    1. Kamiński MF, Hassan C, Bisschops R, et al. . Advanced imaging for detection and differentiation of colorectal neoplasia: European Society of Gastrointestinal Endoscopy (ESGE) Guideline. Endoscopy 2014;46:435–57. 10.1055/s-0034-1365348
    1. Ponugoti PL, Cummings OW, Rex DK. Risk of cancer in small and diminutive colorectal polyps. Dig Liver Dis 2017;49:34–7. 10.1016/j.dld.2016.06.025
    1. Kessler WR, Imperiale TF, Klein RW, et al. . A quantitative assessment of the risks and cost savings of forgoing histologic examination of diminutive polyps. Endoscopy 2011;43:683–91. 10.1055/s-0030-1256381
    1. Hassan C, Pickhardt PJ, Rex DK. A resect and discard strategy would improve cost-effectiveness of colorectal cancer screening. Clin Gastroenterol Hepatol 2010;8:865–9. 10.1016/j.cgh.2010.05.018
    1. Barclay RL, Vicari JJ, Doughty AS, et al. . Colonoscopic withdrawal times and adenoma detection during screening colonoscopy. N Engl J Med 2006;355:2533–41. 10.1056/NEJMoa055498
    1. Chen SC, Rex DK. Endoscopist can be more powerful than age and male gender in predicting adenoma detection at colonoscopy. Am J Gastroenterol 2007;102:856–61. 10.1111/j.1572-0241.2006.01054.x
    1. Hetzel JT, Huang CS, Coukos JA, et al. . Variation in the detection of serrated polyps in an average risk colorectal cancer screening cohort. Am J Gastroenterol 2010;105:2656–64. 10.1038/ajg.2010.315
    1. Kahi CJ, Li X, Eckert GJ, et al. . High colonoscopic prevalence of proximal colon serrated polyps in average-risk men and women. Gastrointest Endosc 2012;75:515–20. 10.1016/j.gie.2011.08.021
    1. Ladabaum U, Fioritto A, Mitani A, et al. . Real-time optical biopsy of colon polyps with narrow band imaging in community practice does not yet meet key thresholds for clinical decisions. Gastroenterology 2013;144:81–91. 10.1053/j.gastro.2012.09.054
    1. Rees CJ, Rajasekhar PT, Wilson A, et al. . Narrow band imaging optical diagnosis of small colorectal polyps in routine clinical practice: the Detect Inspect Characterise Resect and Discard 2 (DISCARD 2) study. Gut 2017;66:887–95. 10.1136/gutjnl-2015-310584
    1. IJspeert JE, Bastiaansen BA, van Leerdam ME, et al. . Development and validation of the WASP classification system for optical diagnosis of adenomas, hyperplastic polyps and sessile serrated adenomas/polyps. Gut 2016;65:963–70. 10.1136/gutjnl-2014-308411
    1. Mori Y, Kudo SE, Wakamura K, et al. . Novel computer-aided diagnostic system for colorectal lesions by using endocytoscopy (with videos). Gastrointest Endosc 2015;81:621–9. 10.1016/j.gie.2014.09.008
    1. Kominami Y, Yoshida S, Tanaka S, et al. . Computer-aided diagnosis of colorectal polyp histology by using a real-time image recognition system and narrow-band imaging magnifying colonoscopy. Gastrointest Endosc 2016;83:643–9. 10.1016/j.gie.2015.08.004
    1. Tajbakhsh N, Gurudu SR, Liang J. A comprehensive computer-aided polyp detection system for colonoscopy videos. Inf Process Med Imaging 2015;24:327–38.
    1. East JE, Vleugels JL, Roelandt P, et al. . Advanced endoscopic imaging: European Society of Gastrointestinal Endoscopy (ESGE) Technology Review. Endoscopy 2016;48:1029–45. 10.1055/s-0042-118087
    1. LeCun Y, Bengio Y, Hinton G, et al. . Deep learning. Nature 2015;521:436–44. 10.1038/nature14539
    1. Murphy KP. Machine learning: a probabilistic perspective: MIT press, 2012.
    1. Lecun Y, Bottou L, Bengio Y, et al. . Gradient-based learning applied to document recognition. Proc IEEE Inst Electr Electron Eng 1998;86:2278–324. 10.1109/5.726791
    1. Krizhevsky A, Sutskever I, Hinton GE. ImageNet classification with deep convolutional neural networks. Adv Neural Inf Process Syst 2012:1097–105.
    1. Szegedy C Vanhoucke V, Ioffe S, et al. . Rethinking the inception architecture for computer vision. IEEE Conference on Computer Vision and Pattern Recognition 2016;2818–26.
    1. Litjens G, Kooi T, Bejnordi BE, et al. . A survey on deep learning in medical image analysis. Med Image Anal 2017;42:60–88. 10.1016/j.media.2017.07.005
    1. Goodfellow I, Bengio Y, Courville A. Deep learning: Mit press, 2016.
    1. Rex DK, Ponugoti P, Kahi C. The "valley sign" in small and diminutive adenomas: prevalence, interobserver agreement, and validation as an adenoma marker. Gastrointest Endosc 2017;85:614–21. 10.1016/j.gie.2016.10.011
    1. Rex DK, Ahnen DJ, Baron JA, et al. . Serrated lesions of the colorectum: review and recommendations from an expert panel. Am J Gastroenterol 2012;107:1315–29. 10.1038/ajg.2012.161
    1. Siegel RL, Miller KD, Jemal A, et al. . Cancer statistics, 2016. CA Cancer J Clin 2016;66:7–30. 10.3322/caac.21332
    1. Pohl H, Srivastava A, Bensen SP, et al. . Incomplete polyp resection during colonoscopy-results of the Complete Adenoma Resection (CARE) study. Gastroenterology 2013;144:74–80. 10.1053/j.gastro.2012.09.043
    1. Kaminski MF, Regula J, Kraszewska E, et al. . Quality indicators for colonoscopy and the risk of interval cancer. N Engl J Med 2010;362:1795–803. 10.1056/NEJMoa0907667
    1. Corley DA, Jensen CD, Marks AR, et al. . Adenoma detection rate and risk of colorectal cancer and death. N Engl J Med 2014;370:1298–306. 10.1056/NEJMoa1309086
    1. Rex DK, Schoenfeld PS, Cohen J, et al. . Quality indicators for colonoscopy. Am J Gastroenterol 2015;110:72–90. 10.1038/ajg.2014.385
    1. Coe SG, Crook JE, Diehl NN, et al. . An endoscopic quality improvement program improves detection of colorectal adenomas. Am J Gastroenterol 2013;108:219–26. 10.1038/ajg.2012.417

Source: PubMed

Sponsorer og samarbeidspartnere

Medisinsk tilstand

Legemiddelintervensjoner

3
Abonnere