A deep learning-based system for bile duct annotation and station recognition in linear endoscopic ultrasound

Liwen Yao, Jun Zhang, Jun Liu, Liangru Zhu, Xiangwu Ding, Di Chen, Huiling Wu, Zihua Lu, Wei Zhou, Lihui Zhang, Bo Xu, Shan Hu, Biqing Zheng, Yanning Yang, Honggang Yu, Liwen Yao, Jun Zhang, Jun Liu, Liangru Zhu, Xiangwu Ding, Di Chen, Huiling Wu, Zihua Lu, Wei Zhou, Lihui Zhang, Bo Xu, Shan Hu, Biqing Zheng, Yanning Yang, Honggang Yu

Abstract

Background: Detailed evaluation of bile duct (BD) is main focus during endoscopic ultrasound (EUS). The aim of this study was to develop a system for EUS BD scanning augmentation.

Methods: The scanning was divided into 4 stations. We developed a station classification model and a BD segmentation model with 10681 images and 2529 images, respectively. 1704 images and 667 images were applied to classification and segmentation internal validation. For classification and segmentation video validation, 264 and 517 videos clips were used. For man-machine contest, an independent data set contained 120 images was applied. 799 images from other two hospitals were used for external validation. A crossover study was conducted to evaluate the system effect on reducing difficulty in ultrasound images interpretation.

Findings: For classification, the model achieved an accuracy of 93.3% in image set and 90.1% in video set. For segmentation, the model had a dice of 0.77 in image set, sensitivity of 89.48% and specificity of 82.3% in video set. For external validation, the model achieved 82.6% accuracy in classification. In man-machine contest, the models achieved 88.3% accuracy in classification and 0.72 dice in BD segmentation, which is comparable to that of expert. In the crossover study, trainees' accuracy improved from 60.8% to 76.3% (P < 0.01, 95% C.I. 20.9-27.2).

Interpretation: We developed a deep learning-based augmentation system for EUS BD scanning augmentation.

Funding: Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Hubei Province Major Science and Technology Innovation Project, National Natural Science Foundation of China.

Keywords: Biliary tract; Deep learning; Endoscopic ultrasound; Training.

Conflict of interest statement

Declaration of Competing Interest None.

Copyright © 2021 The Authors. Published by Elsevier B.V. All rights reserved.

Figures

Fig. 1
Fig. 1
BP MASTER system framework: DCNN1 was applied to filter out white light images and input the ultrasound images to DCNN2. DCNN2 was applied to classify ultrasound images into standard and non-standard categories, and activate DCNN3 with standard images. DCNN3 was used to recognize stations. DCNN4 was used to segment and annotate bile duct.
Fig. 2
Fig. 2
A schematic illustration of the stations about the visualization of bile duct in linear EUS and its representative images predicted by the DCNN3.
Fig. 3
Fig. 3
Flowchart of the study development and validation.
Fig. 4
Fig. 4
The crossover study design: a. Study design. 12 trainees were divided into 2 groups to perform reads with and without model augmentation in random order, with a 2-week washout period between. b. Unaugmented read, with original EUS videos. c. Augmented read, videos with model labeled. EUS, endoscopic ultrasound.
Fig. 5
Fig. 5
The accuracy, Dice, recall and precision in the man-machine contest. In the man-machine contest, the accuracy for DCNN 3, expert C, endoscopists D, E and F was 88.3%, 90%, 85.8%, 74.2% and 84.2%, respectively. Among the images with bile duct, the dice for DCNN 4, expert C, endoscopists D, E and F was 0.72, 0.74, 0.65, 0.67, 0.65, respectively.

References

    1. Giljaca V, Gurusamy KS, Takwoingi Y, Higgie D, Poropat G, Štimac D. Endoscopic ultrasound versus magnetic resonance cholangiopancreatography for common bile duct stones. Cochrane Database Syst Rev. 2015;(2)
    1. Polkowski M, Regula J, Tilszer A, Butruk E. Endoscopic ultrasound versus endoscopic retrograde cholangiography for patients with intermediate probability of bile duct stones: a randomized trial comparing two management strategies. Endoscopy. 2007;39(4):296–303.
    1. Meeralam Y, Al-Shammari K, Yaghoobi M. Diagnostic accuracy of EUS compared with MRCP in detecting choledocholithiasis: a meta-analysis of diagnostic test accuracy in head-to-head studies. Gastrointest Endosc. 2017;86(6):986–993.
    1. Sharma M, Pathak A, Shoukat A, Rameshbabu CS, Ajmera A, Wani ZA. Imaging of common bile duct by linear endoscopic ultrasound. World J Gastrointest Endosc. 2015;7(15):1170–1180.
    1. Irisawa A, Yamao K. Curved linear array EUS technique in the pancreas and biliary tree: focusing on the stations. Gastrointest Endosc. 2009;69(2 Suppl):S84–SS9.
    1. Committee EFS, Yamao K, Irisawa A, Inoue H, Matsuda K, Kida M. Standard imaging techniques of endoscopic ultrasound-guided fine-needle aspiration using a curved linear array echoendoscope. Digest Endosc. 2007;19:S180–S205.
    1. Domagk D, Oppong KW, Aabakken L, Czakó L, Gyökeres T, Manes G. Performance measures for ERCP and endoscopic ultrasound: a European Society of Gastrointestinal Endoscopy (ESGE) quality improvement initiative. Endoscopy. 2018;50(11):1116–1127.
    1. Wani S, Keswani R, Hall M, Han S, Ali MA, Brauer B. A prospective multicenter study evaluating learning curves and competence in endoscopic ultrasound and endoscopic retrograde cholangiopancreatography among advanced endoscopy trainees: the rapid assessment of trainee endoscopy skills study. Clin Gastroenterol Hepatol. 2017;15(11)
    1. Shahidi N, Ou G, Lam E, Enns R, Telford J. When trainees reach competency in performing endoscopic ultrasound: a systematic review. Endosc Int Open. 2017;5(4):E239–EE43.
    1. Kim GH, Bang SJ, Hwang JH. Learning models for endoscopic ultrasonography in gastrointestinal endoscopy. World J Gastroenterol. 2015;21(17):5176–5182.
    1. Tsuchiya T, Itoi T, Sofuni A, Tonozuka R, Mukai S. Endoscopic ultrasonography-guided rendezvous technique. Digest Endosc. 2016;28:96–101.
    1. Okuno N, Hara K, Mizuno N, Hijioka S, Tajika M, Tanaka T. Endoscopic ultrasound-guided rendezvous technique after failed endoscopic retrograde cholangiopancreatography: which approach route is the best? Intern Med. 2017;56(23):3135–3143.
    1. Sarkaria S, Sundararajan S, Kahaleh M. Endoscopic ultrasonographic access and drainage of the common bile duct. Gastrointest Endosc Clin. 2013;23(2):435–452.
    1. Gupta K, Perez-Miranda M, Kahaleh M, Artifon EL, Itoi T, Freeman ML. Endoscopic ultrasound-assisted bile duct access and drainage: multicenter, long-term analysis of approach, outcomes, and complications of a technique in evolution. J Clin Gastroenterol. 2014;48(1):80–87.
    1. Topol EJ. High-performance medicine: the convergence of human and artificial intelligence. Nat Med. 2019;25(1):44–56.
    1. Zhang J, Zhu L, Yao L, Ding X, Chen D, Wu H. Deep-learning–based pancreas segmentation and station recognition system in EUS: development and validation of a useful training tool (with video) Gastrointest Endosc. 2020
    1. Kaneko M, Katanuma A, Maguchi H, Takahashi K, Osanai M, Yane K. Prospective, randomized, comparative study of delineation capability of radial scanning and curved linear array endoscopic ultrasound for the pancreaticobiliary region. Endosc Int Open. 2014;2(3):E160.
    1. Abadi M, Barham P, Chen J, Chen Z, Davis A, Dean J, editors. 12th {USENIX} symposium on operating systems design and implementation ({OSDI} 16) 2016. Tensorflow: a system for large-scale machine learning.
    1. Baldi P, Sadowski P. The dropout learning algorithm. Artif Intell. 2014;210:78–122.
    1. Mikołajczyk A, Grochowski M, editors. 2018 international interdisciplinary PhD workshop (IIPhDW) IEEE; 2018. Data augmentation for improving deep learning in image classification problem.
    1. Duvenaud D, Maclaurin D, Adams R, editors. Early stopping as nonparametric variational inference. Artificial Intelligence and Statistics; 2016.
    1. Deng J, Dong W, Socher R, Li L-J, Kai L, Li F-F. 2009 IEEE conference on computer vision and pattern recognition. 2009. ImageNet: a large-scale hierarchical image database; pp. 248–255.
    1. Zhou Z, Siddiquee MMR, Tajbakhsh N, Liang J. Springer; 2018. Unet++: A nested u-net architecture for medical image segmentation. Deep learning in medical image analysis and multimodal learning for clinical decision support; pp. 3–11.
    1. He K, Zhang X, Ren S, Sun J, editors. Proceedings of the IEEE conference on computer vision and pattern recognition. 2016. Deep residual learning for image recognition.
    1. Wani S, Coté GA, Keswani R, Mullady D, Azar R, Murad F. Learning curves for EUS by using cumulative sum analysis: implications for American Society for Gastrointestinal Endoscopy recommendations for training. Gastrointest Endosc. 2013;77(4):558–565.
    1. Bhutani MS, Wong RF, Hoffman BJ. Training facilities in gastrointestinal endoscopy: an animal model as an aid to learning endoscopic ultrasound. Endoscopy. 2006;38(9):932–934.
    1. Fritscher-Ravens A, Cuming T, Dhar S, Parupudi SVJ, Patel K, Ghanbari A. Endoscopic ultrasound-guided fine needle aspiration training: evaluation of a new porcine lymphadenopathy model for in vivo hands-on teaching and training, and review of the literature. Endoscopy. 2013;45(2):114–120.
    1. Konge L, Clementsen PF, Ringsted C, Minddal V, Larsen KR, Annema JT. Simulator training for endobronchial ultrasound: a randomised controlled trial. Eur Respir J. 2015;46(4):1140–1149.
    1. Eisen GM, Dominitz JA, Faigel DO, Goldstein JA, Petersen BT, Raddawi HM. Guidelines for credentialing and granting privileges for endoscopic ultrasound. Gastrointest Endosc. 2001;54(6):811–814.
    1. Wani S, Han S, Simon V, Hall M, Early D, Aagaard E. Setting minimum standards for training in EUS and ERCP: results from a prospective multicenter study evaluating learning curves and competence among advanced endoscopy trainees. Gastrointest Endosc. 2019;89(6)
    1. Wani S, Hall M, Keswani RN, Aslanian HR, Casey B, Burbridge R. Variation in aptitude of trainees in endoscopic ultrasonography, based on cumulative sum analysis. Clin Gastroenterol Hepatol. 2015;13(7) 1318-25. e2.
    1. Brauer BC, Chen YK, Fukami N, Shah RJ. Single-operator EUS-guided cholangiopancreatography for difficult pancreaticobiliary access (with video) Gastrointest Endosc. 2009;70(3):471–479.
    1. Maranki J, Hernandez A, Arslan B, Jaffan A, Angle J, Shami V. Interventional endoscopic ultrasound-guided cholangiography: long-term experience of an emerging alternative to percutaneous transhepatic cholangiography. Endoscopy. 2009;41(06):532–538.
    1. Kim Y, Gupta K, Mallery S, Li R, Kinney T, Freeman ML. Endoscopic ultrasound rendezvous for bile duct access using a transduodenal approach: cumulative experience at a single center. A case series. Endoscopy. 2010;42(06):496–502.
    1. Shah JN, Marson F, Weilert F, Bhat YM, Nguyen-Tang T, Shaw RE. Single-operator, single-session EUS-guided anterograde cholangiopancreatography in failed ERCP or inaccessible papilla. Gastrointest Endosc. 2012;75(1):56–64.

Source: PubMed

Szponzorok és közreműködők

Egészségi állapot

Kábítószer-beavatkozások

3
Iratkozz fel