Development and validation of an endoscopic images-based deep learning model for detection with nasopharyngeal malignancies

Chaofeng Li, Bingzhong Jing, Liangru Ke, Bin Li, Weixiong Xia, Caisheng He, Chaonan Qian, Chong Zhao, Haiqiang Mai, Mingyuan Chen, Kajia Cao, Haoyuan Mo, Ling Guo, Qiuyan Chen, Linquan Tang, Wenze Qiu, Yahui Yu, Hu Liang, Xinjun Huang, Guoying Liu, Wangzhong Li, Lin Wang, Rui Sun, Xiong Zou, Shanshan Guo, Peiyu Huang, Donghua Luo, Fang Qiu, Yishan Wu, Yijun Hua, Kuiyuan Liu, Shuhui Lv, Jingjing Miao, Yanqun Xiang, Ying Sun, Xiang Guo, Xing Lv, Chaofeng Li, Bingzhong Jing, Liangru Ke, Bin Li, Weixiong Xia, Caisheng He, Chaonan Qian, Chong Zhao, Haiqiang Mai, Mingyuan Chen, Kajia Cao, Haoyuan Mo, Ling Guo, Qiuyan Chen, Linquan Tang, Wenze Qiu, Yahui Yu, Hu Liang, Xinjun Huang, Guoying Liu, Wangzhong Li, Lin Wang, Rui Sun, Xiong Zou, Shanshan Guo, Peiyu Huang, Donghua Luo, Fang Qiu, Yishan Wu, Yijun Hua, Kuiyuan Liu, Shuhui Lv, Jingjing Miao, Yanqun Xiang, Ying Sun, Xiang Guo, Xing Lv

Abstract

Background: Due to the occult anatomic location of the nasopharynx and frequent presence of adenoid hyperplasia, the positive rate for malignancy identification during biopsy is low, thus leading to delayed or missed diagnosis for nasopharyngeal malignancies upon initial attempt. Here, we aimed to develop an artificial intelligence tool to detect nasopharyngeal malignancies under endoscopic examination based on deep learning.

Methods: An endoscopic images-based nasopharyngeal malignancy detection model (eNPM-DM) consisting of a fully convolutional network based on the inception architecture was developed and fine-tuned using separate training and validation sets for both classification and segmentation. Briefly, a total of 28,966 qualified images were collected. Among these images, 27,536 biopsy-proven images from 7951 individuals obtained from January 1st, 2008, to December 31st, 2016, were split into the training, validation and test sets at a ratio of 7:1:2 using simple randomization. Additionally, 1430 images obtained from January 1st, 2017, to March 31st, 2017, were used as a prospective test set to compare the performance of the established model against oncologist evaluation. The dice similarity coefficient (DSC) was used to evaluate the efficiency of eNPM-DM in automatic segmentation of malignant area from the background of nasopharyngeal endoscopic images, by comparing automatic segmentation with manual segmentation performed by the experts.

Results: All images were histopathologically confirmed, and included 5713 (19.7%) normal control, 19,107 (66.0%) nasopharyngeal carcinoma (NPC), 335 (1.2%) NPC and 3811 (13.2%) benign diseases. The eNPM-DM attained an overall accuracy of 88.7% (95% confidence interval (CI) 87.8%-89.5%) in detecting malignancies in the test set. In the prospective comparison phase, eNPM-DM outperformed the experts: the overall accuracy was 88.0% (95% CI 86.1%-89.6%) vs. 80.5% (95% CI 77.0%-84.0%). The eNPM-DM required less time (40 s vs. 110.0 ± 5.8 min) and exhibited encouraging performance in automatic segmentation of nasopharyngeal malignant area from the background, with an average DSC of 0.78 ± 0.24 and 0.75 ± 0.26 in the test and prospective test sets, respectively.

Conclusions: The eNPM-DM outperformed oncologist evaluation in diagnostic classification of nasopharyngeal mass into benign versus malignant, and realized automatic segmentation of malignant area from the background of nasopharyngeal endoscopic images.

Keywords: Automatic segmentation; Deep learning; Differential diagnosis; Nasopharyngeal malignancy.

Figures

Fig. 1
Fig. 1
The study flowchart. *The numbers of images and cases in each subset are presented
Fig. 2
Fig. 2
Representative images of nasopharyngeal masses. a normal (adenoids hyperplasia); b Nasopharyngeal carcinoma; c fibroangioma; d malignant melanoma
Fig. 3
Fig. 3
ROC for eNPM-DM in different test sets. a ROC of eNPM-DM in nasopharyngeal malignancy detection in the test set. b Comparison of the performance between eNPM-DM and oncologists with different seniorities in nasopharyngeal malignancy detection in the prospective test set. eNPM-DM endoscopic images-based nasopharyngeal malignancy detection model, ROC receiver operating characteristic curves, AUC area under curve
Fig. 4
Fig. 4
The training curve of eNPM-DM in detecting nasopharyngeal malignancies. The orange line represents the accuracy of detecting nasopharyngeal malignancies in the validation set over the course of training, with a final accuracy of 89.1% at the final epoch. The training curve was used for model selection. In this case, the best performing model at epoch 100 was used in the test and prospective test sets for final assessment. eNPM-DM endoscopic images-based nasopharyngeal malignancies detection model, Val validation
Fig. 5
Fig. 5
Representative images of nasopharyngeal malignancies segmentation. Images from the left to the right in each row are the original endoscopic images with or without malignant area highlighted by the experts (blue), the probability map output by eNPM-DM and the merged images of the malignant area outlined by the experts (blue) and segmented by the eNPM-DM (green). eNPM-DM endoscopic images-based nasopharyngeal malignancy detection model, NPC nasopharyngeal carcinoma

References

    1. Berkiten G, Kumral TL, Yildirim G, Uyar Y, Atar Y, Salturk Z. Eight years of clinical findings and biopsy results of nasopharyngeal pathologies in 1647 adult patients: a retrospective study. B-ENT. 2014;10(4):279–284.
    1. Stelow EB, Wenig BM. Update from the 4th edition of the World Health Organization classification of head and neck tumours: nasopharynx. Head Neck Pathol. 2017;11(1):16–22. doi: 10.1007/s12105-017-0787-0.
    1. Wei KR, Zheng RS, Zhang SW, Liang ZH, Li ZM, Chen WQ. Nasopharyngeal carcinoma incidence and mortality in China, 2013. Chin J Cancer. 2017;36(1):90. doi: 10.1186/s40880-017-0257-9.
    1. Liang H, Xiang YQ, Lv X, Xie CQ, Cao SM, Wang L, et al. Survival impact of waiting time for radical radiotherapy in nasopharyngeal carcinoma: a large institution-based cohort study from an endemic area. Eur J Cancer. 2017;73:48–60. doi: 10.1016/j.ejca.2016.12.009.
    1. Yi JL, Gao L, Huang XD, Li SY, Luo JW, Cai WM, et al. Nasopharyngeal carcinoma treated by radical radiotherapy alone: ten-year experience of a single institution. Int J Radiat Oncol Biol Phys. 2006;65(1):161–168. doi: 10.1016/j.ijrobp.2005.12.003.
    1. Su SF, Han F, Zhao C, Huang Y, Chen CY, Xiao WW, et al. Treatment outcomes for different subgroups of nasopharyngeal carcinoma patients treated with intensity-modulated radiation therapy. Chin J Cancer. 2011;30(8):565–573. doi: 10.5732/cjc.010.10547.
    1. Lee N, Harris J, Garden AS, Straube W, Glisson B, Xia P, et al. Intensity-modulated radiation therapy with or without chemotherapy for nasopharyngeal carcinoma: radiation therapy oncology group phase II trial 0225. J Clin Oncol. 2009;27(22):3684–3690. doi: 10.1200/JCO.2008.19.9109.
    1. Chan AT, Felip E. Nasopharyngeal cancer: ESMO clinical recommendations for diagnosis, treatment and follow-up. Ann Oncol. 2009;20(Suppl 4):123–125.
    1. Cengiz K, Kumral TL, Yildirim G. Diagnosis of pediatric nasopharynx carcinoma after recurrent adenoidectomy. Case Rep Otolaryngol. 2013;2013:653963.
    1. Wu YP, Cai PQ, Tian L, Xu JH, Mitteer RJ, Fan Y, et al. Hypertrophic adenoids in patients with nasopharyngeal carcinoma: appearance at magnetic resonance imaging before and after treatment. Chin J Cancer. 2015;34(3):130–136. doi: 10.1186/s40880-015-0005-y.
    1. Abu-Ghanem S, Carmel NN, Horowitz G, Yehuda M, Leshno M, Abu-Ghanem Y, et al. Nasopharyngeal biopsy in adults: a large-scale study in a non endemic area. Rhinology. 2015;53(2):142–148. doi: 10.4193/Rhin14.130.
    1. Miotto R, Wang F, Wang S, Jiang X, Dudley JT. Deep learning for healthcare: review, opportunities and challenges. Brief Bioinform 2017.
    1. Mnih V, Kavukcuoglu K, Silver D, Rusu AA, Veness J, Bellemare MG, et al. Human-level control through deep reinforcement learning. Nature. 2015;518(7540):529–533. doi: 10.1038/nature14236.
    1. Silver D, Huang A, Maddison CJ, Guez A, Sifre L, van den Driessche G, et al. Mastering the game of Go with deep neural networks and tree search. Nature. 2016;529(7587):484–489. doi: 10.1038/nature16961.
    1. Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, et al. ImageNet large scale visual recognition challenge. Int J Comput Vision. 2015;115(3):211–252. doi: 10.1007/s11263-015-0816-y.
    1. Gulshan V, Peng L, Coram M, Stumpe MC, Wu D, Narayanaswamy A, et al. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA. 2016;316(22):2402–2410. doi: 10.1001/jama.2016.17216.
    1. Esteva A, Kuprel B, Novoa RA, Ko J, Swetter SM, Blau HM, et al. Dermatologist-level classification of skin cancer with deep neural networks. Nature. 2017;542(7639):115–118. doi: 10.1038/nature21056.
    1. Lakhani P, Sundaram B. Deep learning at chest radiography: automated classification of pulmonary tuberculosis by using convolutional neural networks. Radiology. 2017;284(2):574–582. doi: 10.1148/radiol.2017162326.
    1. Ertosun MG, Rubin DL. Automated grading of gliomas using deep learning in digital pathology images: a modular approach with ensemble of convolutional neural networks. AMIA Annu Symp Proc. 2015;2015:1899–1908.
    1. Shelhamer E, Long J, Darrell T. Fully convolutional networks for semantic segmentation. IEEE Trans Pattern Anal Mach Intell. 2017;39(4):640–651. doi: 10.1109/TPAMI.2016.2572683.
    1. Korbar B, Olofson AM, Miraflor AP, Nicka CM, Suriawinata MA, Torresani L, et al. Deep learning for classification of colorectal polyps on whole-slide images. J Pathol Inf. 2017;8:30. doi: 10.4103/jpi.jpi_34_17.
    1. Suzuki K. Overview of deep learning in medical imaging. Radiol Phys Technol. 2017;10(3):257–273. doi: 10.1007/s12194-017-0406-5.
    1. Sharma H, Zerbe N, Klempert I, Hellwich O, Hufnagl P. Deep convolutional neural networks for automatic classification of gastric carcinoma using whole slide images in digital histopathology. Comput Med Imaging Graph. 2017;61:2–13. doi: 10.1016/j.compmedimag.2017.06.001.
    1. Schirrmeister RT, Springenberg JT, Fiederer L, Glasstetter M, Eggensperger K, Tangermann M, et al. Deep learning with convolutional neural networks for EEG decoding and visualization. Hum Brain Mapp. 2017;38:5391–5420. doi: 10.1002/hbm.23730.
    1. Komeda Y, Handa H, Watanabe T, Nomura T, Kitahashi M, Sakurai T, et al. Computer-aided diagnosis based on convolutional neural network system for colorectal polyp classification: preliminary experience. Oncology. 2017;93(Suppl 1):30–34. doi: 10.1159/000481227.
    1. van der Burgh HK, Schmidt R, Westeneng HJ, de Reus MA, van den Berg LH, van den Heuvel MP. Deep learning predictions of survival based on MRI in amyotrophic lateral sclerosis. Neuroimage Clin. 2017;13:361–369. doi: 10.1016/j.nicl.2016.10.008.
    1. Hirasawa T, Aoyama K, Tanimoto T, Ishihara S, Shichijo S, Ozawa T, et al. Application of artificial intelligence using a convolutional neural network for detecting gastric cancer in endoscopic images. Gastric Cancer. 2018;21:653–660. doi: 10.1007/s10120-018-0793-2.
    1. Shichijo S, Nomura S, Aoyama K, Nishikawa Y, Miura M, Shinagawa T, et al. application of convolutional neural networks in the diagnosis of helicobacter pylori infection based on endoscopic images. EBioMedicine. 2017;25:106–111. doi: 10.1016/j.ebiom.2017.10.014.
    1. Byrne MF, Chapados N, Soudan F, Oertel C, Linares PM, Kelly R, et al. Real-time differentiation of adenomatous and hyperplastic diminutive colorectal polyps during analysis of unaltered videos of standard colonoscopy using a deep learning model. Gut. 2017 doi: 10.1136/gutjnl-2017-314547.
    1. Mori H, Nishiyama N, Kobara H, Fujihara S, Kobayashi N, Nagase T, et al. The use of a detachable multiple polyp catcher to facilitate accurate location and pathological diagnosis of resected polyps in the proximal colon. Gastrointest Endosc. 2016;83(1):262–263. doi: 10.1016/j.gie.2015.08.008.
    1. Ribeiro E, Uhl A, Wimmer G, Hafner M. Exploring deep learning and transfer learning for colonic polyp classification. Comput Math Methods Med. 2016;2016:6584725. doi: 10.1155/2016/6584725.
    1. Chan KC. Plasma Epstein-Barr virus DNA as a biomarker for nasopharyngeal carcinoma. Chin J Cancer. 2014;33(12):598–603.
    1. Meng R, Wei K, Xia L, Xu Y, Chen W, Zheng R, et al. Cancer incidence and mortality in Guangdong province, 2012. Chin J Cancer Res. 2016;28(3):311–320. doi: 10.21147/j.issn.1000-9604.2016.03.05.

Source: PubMed

Sponsors and Collaborators

Medical Conditions

Drug Interventions

3
Subscribe