Computer-aided detection of colorectal polyps using a newly generated deep convolutional neural network: from development to first clinical experience

Lukas Pfeifer, Clemens Neufert, Moritz Leppkes, Maximilian J Waldner, Michael Häfner, Albert Beyer, Arthur Hoffman, Peter D Siersema, Markus F Neurath, Timo Rath, Lukas Pfeifer, Clemens Neufert, Moritz Leppkes, Maximilian J Waldner, Michael Häfner, Albert Beyer, Arthur Hoffman, Peter D Siersema, Markus F Neurath, Timo Rath

Abstract

Aim: The use of artificial intelligence represents an objective approach to increase endoscopist's adenoma detection rate (ADR) and limit interoperator variability. In this study, we evaluated a newly developed deep convolutional neural network (DCNN) for automated detection of colorectal polyps ex vivo as well as in a first in-human trial.

Methods: For training of the DCNN, 116 529 colonoscopy images from 278 patients with 788 different polyps were collected. A subset of 10 467 images containing 504 different polyps were manually annotated and treated as the gold standard. An independent set of 45 videos consisting of 15 534 single frames was used for ex vivo performance testing. In vivo real-time detection of colorectal polyps during routine colonoscopy by the DCNN was tested in 42 patients in a back-to-back approach.

Results: When analyzing the test set of 15 534 single frames, the DCNN's sensitivity and specificity for polyp detection and localization within the frame was 90% and 80%, respectively, with an area under the curve of 0.92. In vivo, baseline polyp detection rate and ADR were 38% and 26% and significantly increased to 50% (P = 0.023) and 36% (P = 0.044), respectively, with the use of the DCNN. Of the 13 additionally with the DCNN detected lesions, the majority were diminutive and flat, among them three sessile serrated adenomas.

Conclusion: This newly developed DCNN enables highly sensitive automated detection of colorectal polyps both ex vivo and during first in-human clinical testing and could potentially increase the detection of colorectal polyps during colonoscopy.

Trial registration: ClinicalTrials.gov NCT04359355.

Conflict of interest statement

Timo Rath and Peter D. Siersema served as consultants and received speaking honoraries from Pentax Medical. For the remaining authors, there are no conflicts of interest.

Copyright © 2021 The Author(s). Published by Wolters Kluwer Health, Inc.

Figures

Fig. 1.
Fig. 1.
Development and diagnostic output of the system. (a) The deep convolutional neural network (DCNN) processes video data as a sequence of single video frames and generates predictions based on the visual evidence of a single video frame. The predictions from individual frames are then fused to provide a more stable detection. (b) Different examples of polyp detection with the DCNN during routine colonoscopy. The computer-aided detection (CAD) system generates the diagnostic output on a second screen on which polyps are highlighted by a bounding box. Note that the DCNN is able to detect multiple polyps in a single frame simultaneously (upper right picture).
Fig. 2.
Fig. 2.
Diagnostic performance of the system. When being analyzed at an input frame rate of 30 Hz, the deep convolutional neural network’s (DCNN’s) sensitivity and specificity for polyp detection and localization within the frame were 90% and 80%, respectively with an area under the curve (AUC) of 0.92.

References

    1. Corley DA, Jensen CD, Marks AR, Zhao WK, Lee JK, Doubeni CA, et al. . Adenoma detection rate and risk of colorectal cancer and death. N Engl J Med 2014; 370:1298–1306.
    1. Nishihara R, Wu K, Lochhead P, Morikawa T, Liao X, Qian ZR, et al. . Long-term colorectal-cancer incidence and mortality after lower endoscopy. N Engl J Med 2013; 369:1095–1105.
    1. Winawer SJ, Zauber AG, Ho MN, O’Brien MJ, Gottlieb LS, Sternberg SS, et al. . Prevention of colorectal cancer by colonoscopic polypectomy. The National Polyp Study Workgroup. N Engl J Med 1993; 329:1977–1981.
    1. Zauber AG, Winawer SJ, O’Brien MJ, Lansdorp-Vogelaar I, van Ballegooijen M, Hankey BF, et al. . Colonoscopic polypectomy and long-term prevention of colorectal-cancer deaths. N Engl J Med 2012; 366:687–696.
    1. Kaminski MF, Thomas-Gibson S, Bugajski M, Bretthauer M, Rees CJ, Dekker E, et al. . Performance measures for lower gastrointestinal endoscopy: a European Society of Gastrointestinal Endoscopy (ESGE) quality improvement initiative. Endoscopy 2017; 49:378–397.
    1. Rex DK, Boland CR, Dominitz JA, Giardiello FM, Johnson DA, Kaltenbach T, et al. . Colorectal cancer screening: recommendations for physicians and patients from the U.S. Multi-Society Task Force on Colorectal Cancer. Am J Gastroenterol 2017; 112:1016–1030.
    1. Zhao S, Wang S, Pan P, Xia T, Chang X, Yang X, et al. . Magnitude, risk factors, and factors associated with adenoma miss rate of tandem colonoscopy: a systematic review and meta-analysis. Gastroenterology 2019; 156:1661–1674.e11.
    1. Pohl H, Robertson DJ. Colorectal cancers detected after colonoscopy frequently result from missed lesions. Clin Gastroenterol Hepatol 2010; 8:858–864.
    1. Aslanian HR, Shieh FK, Chan FW, Ciarleglio MM, Deng Y, Rogart JN, et al. . Nurse observation during colonoscopy increases polyp detection: a randomized prospective study. Am J Gastroenterol 2013; 108:166–172.
    1. Buchner AM, Shahid MW, Heckman MG, Diehl NN, McNeil RB, Cleveland P, et al. . Trainee participation is associated with increased small adenoma detection. Gastrointest Endosc 2011; 73:1223–1231.
    1. Kim TS, Park DI, Lee DY, Yoon JH, Park JH, Kim HJ, et al. . Endoscopy nurse participation may increase the polyp detection rate by second-year fellows during screening colonoscopies. Gut Liver 2012; 6:344–348.
    1. Lee CK, Park DI, Lee SH, Hwangbo Y, Eun CS, Han DS, et al. . Participation by experienced endoscopy nurses increases the detection rate of colon polyps during a screening colonoscopy: a multicenter, prospective, randomized study. Gastrointest Endosc 2011; 74:1094–1102.
    1. Ahmad OF, Soares AS, Mazomenos E, Brandao P, Vega R, Seward E, et al. . Artificial intelligence and computer-aided diagnosis in colonoscopy: current evidence and future directions. Lancet Gastroenterol Hepatol 2019; 4:71–80.
    1. Kudo SE, Mori Y, Misawa M, Takeda K, Kudo T, Itoh H, et al. . Artificial intelligence and colonoscopy: Current status and future perspectives. Dig Endosc 2019; 31:363–371.
    1. Misawa M, Kudo SE, Mori Y, Cho T, Kataoka S, Yamauchi A, et al. . Artificial intelligence-assisted polyp detection for colonoscopy: initial experience. Gastroenterology 2018; 154:2027–2029.e3.
    1. Tajbakhsh N, Gurudu SR, Liang J. Automated polyp detection in colonoscopy videos using shape and context information. IEEE Trans Med Imaging 2016; 35:630–644.
    1. Wang P, Xiao X, Glissen Brown JR, Berzin TM, Tu M, Xiong F, et al. . Development and validation of a deep-learning algorithm for the detection of polyps during colonoscopy. Nat Biomed Eng 2018; 2:741–748.
    1. Zhang R, Zheng Y, Poon CCY, Shen D, Lau JYW. Polyp detection during colonoscopy using a regression-based convolutional neural network with a tracker. Pattern Recognit 2018; 83:209–219.
    1. Wang P, Berzin TM, Glissen Brown JR, Bharadwaj S, Becq A, Xiao X, et al. . Real-time automatic detection system increases colonoscopic polyp and adenoma detection rates: a prospective randomised controlled study. Gut 2019; 68:1813–1819.
    1. Liu WN, Zhang YY, Bian XQ, Wang LJ, Yang Q, Zhang XD, Huang J. Study on detection rate of polyps and adenomas in artificial-intelligence-aided colonoscopy. Saudi J Gastroenterol 2020; 26:13–19.
    1. Repici A, Badalamenti M, Maselli R, Correale L, Radaelli F, Rondonotti E, et al. . Efficacy of real-time computer-aided detection of colorectal neoplasia in a randomized trial. Gastroenterology 2020; 159:512–520.e7.
    1. Wang P, Liu X, Berzin TM, Glissen Brown JR, Liu P, Zhou C, et al. . Effect of a deep-learning computer-aided detection system on adenoma detection during colonoscopy (CADe-DB trial): a double-blind randomised study. Lancet Gastroenterol Hepatol 2020; 5:343–351.
    1. Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. International Conference on Learning Representations 2015, ICLR 2015; 7–9 May, San Diego, USA, arXiv:1409.1556.
    1. Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu C-Y, et al. . SSD: Single Shot MultiBox Detector. 8–16 October, European Conference on Computer Vision 2016, ECCV 2016; Amsterdam, Netherlands, arXiv:1512.02325.
    1. The Paris endoscopic classification of superficial neoplastic lesions: esophagus, stomach, and colon: November 30 to December 1, 2002. Gastrointest Endosc 2003;58 (6 Suppl):S3–43.
    1. Arnold M, Sierra MS, Laversanne M, Soerjomataram I, Jemal A, Bray F. Global patterns and trends in colorectal cancer incidence and mortality. Gut 2017; 66:683–691.
    1. Simons DJ, Chabris CF. Gorillas in our midst: sustained inattentional blindness for dynamic events. Perception 1999; 28:1059–1074.
    1. Simons DJ, Levin DT. Change blindness. Trends Cogn Sci 1997; 1:261–267.
    1. Nishizawa T, Suzuki H, Takahashi M, Kaneko H, Fujiyama Y, Komatsu H, et al. . Trainee participation during colonoscopy adversely affects polyp and adenoma detection rates. Digestion 2011; 84:245–246.
    1. Alagappan M, Brown JRG, Mori Y, Berzin TM. Artificial intelligence in gastrointestinal endoscopy: the future is almost here. World J Gastrointest Endosc 2018; 10:239–249.
    1. Mori Y, Kudo SE, Berzin TM, Misawa M, Takeda K. Computer-aided diagnosis for colonoscopy. Endoscopy 2017; 49:813–819.
    1. Yamada M, Saito Y, Imaoka H, Saiko M, Yamada S, Kondo H, et al. . Development of a real-time endoscopic image diagnosis support system using deep learning technology in colonoscopy. Sci Rep 2019; 9:14465.
    1. Chen PJ, Lin MC, Lai MJ, Lin JC, Lu HH, Tseng VS. Accurate classification of diminutive colorectal polyps using computer-aided analysis. Gastroenterology 2018; 154:568–575.
    1. Mori Y, Kudo SE, Misawa M, Saito Y, Ikematsu H, Hotta K, et al. . Real-time use of artificial intelligence in identification of diminutive polyps during colonoscopy: a prospective study. Ann Intern Med 2018; 169:357–366.
    1. Sánchez-Montes C, Sánchez FJ, Bernal J, Córdova H, López-Cerón M, Cuatrecasas M, et al. . Computer-aided prediction of polyp histology on white light colonoscopy using surface pattern analysis. Endoscopy 2019; 51:261–265.
    1. Butterly LF, Chase MP, Pohl H, Fiarman GS. Prevalence of clinically important histology in small adenomas. Clin Gastroenterol Hepatol 2006; 4:343–348.
    1. Gupta N, Bansal A, Rao D, Early DS, Jonnalagadda S, Wani SB, et al. . Prevalence of advanced histological features in diminutive and small colon polyps. Gastrointest Endosc 2012; 75:1022–1030.
    1. Lieberman D, Moravec M, Holub J, Michaels L, Eisen G. Polyp size and advanced histology in patients undergoing colonoscopy screening: implications for CT colonography. Gastroenterology 2008; 135:1100–1105.
    1. O’Brien MJ, Winawer SJ, Zauber AG, Gottlieb LS, Sternberg SS, Diaz B, et al. . The National Polyp Study. Patient and polyp characteristics associated with high-grade dysplasia in colorectal adenomas. Gastroenterology 1990; 98:371–379.
    1. Mori Y, Kudo SE, East JE, Rastogi A, Bretthauer M, Misawa M, et al. . Cost savings in colonoscopy with artificial intelligence-aided polyp diagnosis: an add-on analysis of a clinical trial (with video). Gastrointest Endosc 2020; 92:905–911.e1.
    1. Rex DK, Boland CR, Dominitz JA, Giardiello FM, Johnson DA, Kaltenbach T, et al. . Colorectal cancer screening: recommendations for physicians and patients from the U.S. Multi-Society Task Force on Colorectal Cancer. Gastroenterology 2017; 153:307–323.
    1. Wolf AMD, Fontham ETH, Church TR, Flowers CR, Guerra CE, LaMonte SJ, et al. . Colorectal cancer screening for average-risk adults: 2018 guideline update from the American Cancer Society. CA Cancer J Clin 2018; 68:250–281.

Source: PubMed

赞助者和合作者

医疗条件

药物干预

3
订阅