Development of a tongue image-based machine learning tool for the diagnosis of gastric cancer: a prospective multicentre clinical cohort study
Li Yuan, Lin Yang, Shichuan Zhang, Zhiyuan Xu, Jiangjiang Qin, Yunfu Shi, Pengcheng Yu, Yi Wang, Zhehan Bao, Yuhang Xia, Jiancheng Sun, Weiyang He, Tianhui Chen, Xiaolei Chen, Can Hu, Yunlong Zhang, Changwu Dong, Ping Zhao, Yanan Wang, Nan Jiang, Bin Lv, Yingwei Xue, Baoping Jiao, Hongyu Gao, Kequn Chai, Jun Li, Hao Wang, Xibo Wang, Xiaoqing Guan, Xu Liu, Gang Zhao, Zhichao Zheng, Jie Yan, Haiyue Yu, Luchuan Chen, Zaisheng Ye, Huaqiang You, Yu Bao, Xi Cheng, Peizheng Zhao, Liang Wang, Wenting Zeng, Yanfei Tian, Ming Chen, You You, Guihong Yuan, Hua Ruan, Xiaole Gao, Jingli Xu, Handong Xu, Lingbin Du, Shengjie Zhang, Huanying Fu, Xiangdong Cheng, Li Yuan, Lin Yang, Shichuan Zhang, Zhiyuan Xu, Jiangjiang Qin, Yunfu Shi, Pengcheng Yu, Yi Wang, Zhehan Bao, Yuhang Xia, Jiancheng Sun, Weiyang He, Tianhui Chen, Xiaolei Chen, Can Hu, Yunlong Zhang, Changwu Dong, Ping Zhao, Yanan Wang, Nan Jiang, Bin Lv, Yingwei Xue, Baoping Jiao, Hongyu Gao, Kequn Chai, Jun Li, Hao Wang, Xibo Wang, Xiaoqing Guan, Xu Liu, Gang Zhao, Zhichao Zheng, Jie Yan, Haiyue Yu, Luchuan Chen, Zaisheng Ye, Huaqiang You, Yu Bao, Xi Cheng, Peizheng Zhao, Liang Wang, Wenting Zeng, Yanfei Tian, Ming Chen, You You, Guihong Yuan, Hua Ruan, Xiaole Gao, Jingli Xu, Handong Xu, Lingbin Du, Shengjie Zhang, Huanying Fu, Xiangdong Cheng
Abstract
Background: Tongue images (the colour, size and shape of the tongue and the colour, thickness and moisture content of the tongue coating), reflecting the health state of the whole body according to the theory of traditional Chinese medicine (TCM), have been widely used in China for thousands of years. Herein, we investigated the value of tongue images and the tongue coating microbiome in the diagnosis of gastric cancer (GC).
Methods: From May 2020 to January 2021, we simultaneously collected tongue images and tongue coating samples from 328 patients with GC (all newly diagnosed with GC) and 304 non-gastric cancer (NGC) participants in China, and 16 S rDNA was used to characterize the microbiome of the tongue coating samples. Then, artificial intelligence (AI) deep learning models were established to evaluate the value of tongue images and the tongue coating microbiome in the diagnosis of GC. Considering that tongue imaging is more convenient and economical as a diagnostic tool, we further conducted a prospective multicentre clinical study from May 2020 to March 2022 in China and recruited 937 patients with GC and 1911 participants with NGC from 10 centres across China to further evaluate the role of tongue images in the diagnosis of GC. Moreover, we verified this approach in another independent external validation cohort that included 294 patients with GC and 521 participants with NGC from 7 centres. This study is registered at ClinicalTrials.gov, NCT01090362.
Findings: For the first time, we found that both tongue images and the tongue coating microbiome can be used as tools for the diagnosis of GC, and the area under the curve (AUC) value of the tongue image-based diagnostic model was 0.89. The AUC values of the tongue coating microbiome-based model reached 0.94 using genus data and 0.95 using species data. The results of the prospective multicentre clinical study showed that the AUC values of the three tongue image-based models for GCs reached 0.88-0.92 in the internal verification and 0.83-0.88 in the independent external verification, which were significantly superior to the combination of eight blood biomarkers.
Interpretation: Our results suggest that tongue images can be used as a stable method for GC diagnosis and are significantly superior to conventional blood biomarkers. The three kinds of tongue image-based AI deep learning diagnostic models that we developed can be used to adequately distinguish patients with GC from participants with NGC, even early GC and precancerous lesions, such as atrophic gastritis (AG).
Funding: The National Key R&D Program of China (2021YFA0910100), Program of Zhejiang Provincial TCM Sci-tech Plan (2018ZY006), Medical Science and Technology Project of Zhejiang Province (2022KY114, WKJ-ZJ-2104), Zhejiang Provincial Research Center for Upper Gastrointestinal Tract Cancer (JBZX-202006), Natural Science Foundation of Zhejiang Province (HDMY22H160008), Science and Technology Projects of Zhejiang Province (2019C03049), National Natural Science Foundation of China (82074245, 81973634, 82204828), and Chinese Postdoctoral Science Foundation (2022M713203).
Keywords: AFP, alpha fetoprotein; AG, atrophic gastritis; AI, artificial intelligence; APINet, attentive pairwise interaction neural network; AUC, area under the curve; Artificial intelligence; BC, breast cancer; CA, carbohydrate antigen; CEA, carcinoembryonic antigen; CRC, colorectal cancer; DT, decision tree learning; EC, esophageal cancer; GC, gastric cancer; Gastric cancer; HBPC, hepatobiliary pancreatic carcinoma; HC, healthy control; KNN, K-nearest neighbours; LC, lung cancer; NGC, non-gastric cancers; PCoA, principal coordinates analysis; SG, superficial gastritis; SVM, support vector machine; TCM, traditional Chinese medicine; Tongue coating microbiome; Tongue images; Traditional Chinese medicine; TransFG, transformer architecture for fine-grained recognition.
Conflict of interest statement
All authors declare no competing interests.
© 2023 The Author(s).
Figures
References
- Sung H., Ferlay J., Siegel R.L., et al. Global cancer statistics 2020: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J Clin. 2021;71:209–249.
- Thrift A.P., El-Serag H.B. Burden of gastric cancer. Clin Gastroenterol Hepatol. 2020;18:534–542.
- Wang W.Y., Zhou H., Wang Y.F., et al. Current policies and measures on the development of traditional Chinese medicine in China. Pharmacol Res. 2021;163
- Cui J., Cui H., Yang M., et al. Tongue coating microbiome as a potential biomarker for gastritis including precancerous cascade. Protein Cell. 2019;10:496–509.
- Kanawong R., Obafemi-Ajayi T., Liu D., et al. Tongue image analysis and its mobile app development for health diagnosis. Adv Exp Med Biol. 2017;1005:99–121.
- Zhou J., Li S., Wang X., et al. Weakly supervised deep learning for tooth-marked tongue recognition. Front Physiol. 2022;13
- Han S., Yang X., Qi Q., et al. Potential screening and early diagnosis method for cancer: tongue diagnosis. Int J Oncol. 2016;48:2257–2264.
- Lu H., Ren Z., Li A., et al. Tongue coating microbiome data distinguish patients with pancreatic head cancer from healthy controls. J Oral Microbiol. 2019;11
- Lu H., Ren Z., Li A., et al. Deep sequencing reveals microbiota dysbiosis of tongue coat in patients with liver carcinoma. Sci Rep. 2016;6
- Han S., Chen Y., Hu J., et al. Tongue images and tongue coating microbiome in patients with colorectal cancer. Microb Pathog. 2014;77:1–6.
- Wu Z.F., Zou K., Xiang C.J., et al. Helicobacter pylori infection is associated with the co-occurrence of bacteria in the oral cavity and the gastric mucosa. Helicobacter. 2021;26
- Kroese J.M., Brandt B.W., Buijs M.J., et al. The oral microbiome in early rheumatoid arthritis patients and individuals at risk differs from healthy controls. Arthritis Rheumatol. 2021;73(11):1986–1993.
- Zhao Y., Mao Y.F., Tang Y.S., et al. Altered oral microbiota in chronic hepatitis B patients with different tongue coatings. World J Gastroenterol. 2018;24:3448–3461.
- Goyal H., Sherazi S.A.A., Mann R., et al. Scope of artificial intelligence in gastrointestinal oncology. Cancers (Basel) 2021;13:5494.
- Yu G., Sun K., Xu C., et al. Accurate recognition of colorectal cancer with semi-supervised deep learning on pathological images. Nat Commun. 2021;12:6311.
- Cheung C.Y., Xu D., Cheng C.Y., et al. A deep-learning system for the assessment of cardiovascular disease risk via the measurement of retinal-vessel calibre. Nat Biomed Eng. 2021;5:498–508.
- Takenaka K., Ohtsuka K., Fujii T., et al. Development and validation of a deep neural network for accurate evaluation of endoscopic images from patients with ulcerative colitis. Gastroenterology. 2020;158:2150–2157.
- Hu Y., Wen G., Luo M., et al. Fully-channel regional attention network for disease-location recognition with tongue images. Artif Intell Med. 2021;118
- Hu Y., Wen G., Liao H., et al. Automatic construction of Chinese herbal prescriptions from tongue images using CNNs and auxiliary latent therapy topics. IEEE Trans Cybern. 2021;51:708–721.
- Pang B., Zhang D., Wang K. The bi-elliptical deformable contour and its application to automated tongue segmentation in Chinese medicine. IEEE Trans Med Imaging. 2005;24:946–956.
- Wang X., Zhang B., Yang Z., et al. Statistical analysis of tongue images for feature extraction and diagnostics. IEEE Trans Image Process. 2013;22:5336–5347.
- Hajian-Tilaki K. Sample size estimation in diagnostic test studies of biomedical informatics. J Biomed Inform. 2014;48:193–204.
- Newcombe R.G. Two-sided confidence intervals for the single proportion: comparison of seven methods. Stat Med. 1998;17:857–872.
- Wu L., Wang Y., Li X., et al. Deep attention-based spatially recursive networks for fine-grained visual recognition. IEEE Trans Cybern. 2019;49:1791–1802.
- He J., Chen J., Liu S., et al. A transformer architecture for fine-grained recognition. arXiv. 2022 preprint arXiv:2103.07976 2021.
- Liang-Chieh Chen Y.Z., George P., Schroff F., Adam H. Proceedings of the European Conference on Computer Vision (ECCV) 2018. Encoder-decoder with atrous separable convolution for semantic image segmentation; pp. 801–818.
- Gower J.C. Some distance properties of latent root and vector methods used in multivariate analysis. Biometrika. 1966;53(3–4):325–338.
- Peery A.F., Crockett S.D., Murphy C.C., et al. Burden and cost of gastrointestinal, liver, and pancreatic diseases in the United States: update 2021. Gastroenterology. 2021;162(2):621–644.
- Thomasian N.M., Kamel I.R., Bai H.X. Machine intelligence in non-invasive endocrine cancer diagnostics. Nat Rev Endocrinol. 2021;18:1–15.
- Wu L., Shang R., Sharma P., et al. Effect of a deep learning-based system on the miss rate of gastric neoplasms during upper gastrointestinal endoscopy: a single-centre, tandem, randomised controlled trial. Lancet Gastroenterol Hepatol. 2021;6(9):700–708.
- Dong D., Fang M.J., Tang L., et al. Deep learning radiomic nomogram can predict the number of lymph node metastasis in locally advanced gastric cancer: an international multicenter study. Ann Oncol. 2020;31:912–920.
- Ren Z., Wang H., Cui G., et al. Alterations in the human oral and gut microbiomes and lipidomics in COVID-19. Gut. 2021;70:1253–1265.
- Kang X., Lu B., Xiao P., et al. Microbial characteristics of common tongue coatings in patients with precancerous lesions of the upper gastrointestinal tract. J Healthc Eng. 2022;2022
- Xu S., Xiang C., Wu J., et al. Tongue coating bacteria as a potential stable biomarker for gastric cancer independent of lifestyle. Dig Dis Sci. 2021;66:2964–2980.
- Vogtmann E., Hua X., Yu G., et al. The oral microbiome and lung cancer risk: an analysis of 3 prospective cohort studies. J Natl Cancer Inst. 2022;114(11):1501–1510.
- Wen G., Ma J., Hu Y., et al. Grouping attributes zero-shot learning for tongue constitution recognition. Artif Intell Med. 2020;109
- Matos L.C., Machado J.P., Monteiro F.J., et al. Can traditional Chinese medicine diagnosis Be parameterized and standardized? A narrative review. Healthcare (Basel) 2021;9:177.
- Tania M.H., Lwin K., Hossain M.A. Advances in automated tongue diagnosis techniques. Integr Med Res. 2019;8:42–56.
- Xu Q., Zeng Y., Tang W., et al. Multi-task joint learning model for segmenting and classifying tongue images using a deep neural network. IEEE J Biomed Health Inform. 2020;24:2481–2489.
- Meng D., Cao G., Duan Y., et al. Tongue images classification based on constrained high dispersal network. Evid Based Complement Alternat Med. 2017;2017
- Wang Z.C., Zhang S.P., Yuen P.C., et al. Intra-rater and inter-rater reliability of tongue coating diagnosis in traditional Chinese medicine using smartphones: quasi-delphi study. JMIR Mhealth Uhealth. 2020;8
- Kitamoto S., Nagao-Kitamoto H., Jiao Y., et al. The intermucosal connection between the mouth and gut in commensal pathobiont-driven colitis. Cell. 2020;182:447–462.e14.
- Li Y., Cui J., Liu Y., et al. Oral, tongue-coating microbiota, and metabolic disorders: a novel area of interactive research. Front Cardiovasc Med. 2021;8
- Xu J., Xiang C., Zhang C., et al. Microbial biomarkers of common tongue coatings in patients with gastric cancer. Microb Pathog. 2019;127:97–105.
- Ye J., Cai X., Yang J., et al. Bacillus as a potential diagnostic marker for yellow tongue coating. Sci Rep. 2016;6
Source: PubMed