Using Artificial Intelligence to Detect COVID-19 and Community-acquired Pneumonia Based on Pulmonary CT: Evaluation of the Diagnostic Accuracy
Lin Li, Lixin Qin, Zeguo Xu, Youbing Yin, Xin Wang, Bin Kong, Junjie Bai, Yi Lu, Zhenghan Fang, Qi Song, Kunlin Cao, Daliang Liu, Guisheng Wang, Qizhong Xu, Xisheng Fang, Shiqin Zhang, Juan Xia, Jun Xia, Lin Li, Lixin Qin, Zeguo Xu, Youbing Yin, Xin Wang, Bin Kong, Junjie Bai, Yi Lu, Zhenghan Fang, Qi Song, Kunlin Cao, Daliang Liu, Guisheng Wang, Qizhong Xu, Xisheng Fang, Shiqin Zhang, Juan Xia, Jun Xia
Abstract
Background Coronavirus disease 2019 (COVID-19) has widely spread all over the world since the beginning of 2020. It is desirable to develop automatic and accurate detection of COVID-19 using chest CT. Purpose To develop a fully automatic framework to detect COVID-19 using chest CT and evaluate its performance. Materials and Methods In this retrospective and multicenter study, a deep learning model, the COVID-19 detection neural network (COVNet), was developed to extract visual features from volumetric chest CT scans for the detection of COVID-19. CT scans of community-acquired pneumonia (CAP) and other non-pneumonia abnormalities were included to test the robustness of the model. The datasets were collected from six hospitals between August 2016 and February 2020. Diagnostic performance was assessed with the area under the receiver operating characteristic curve, sensitivity, and specificity. Results The collected dataset consisted of 4352 chest CT scans from 3322 patients. The average patient age (±standard deviation) was 49 years ± 15, and there were slightly more men than women (1838 vs 1484, respectively; P = .29). The per-scan sensitivity and specificity for detecting COVID-19 in the independent test set was 90% (95% confidence interval [CI]: 83%, 94%; 114 of 127 scans) and 96% (95% CI: 93%, 98%; 294 of 307 scans), respectively, with an area under the receiver operating characteristic curve of 0.96 (P < .001). The per-scan sensitivity and specificity for detecting CAP in the independent test set was 87% (152 of 175 scans) and 92% (239 of 259 scans), respectively, with an area under the receiver operating characteristic curve of 0.95 (95% CI: 0.93, 0.97). Conclusion A deep learning model can accurately detect coronavirus 2019 and differentiate it from community-acquired pneumonia and other lung conditions. © RSNA, 2020 Online supplemental material is available for this article.
Figures
![Fig 1.](https://www.ncbi.nlm.nih.gov/pmc/articles/instance/7233473/bin/radiol.2020200905.fig1.jpg)
![Fig 2.](https://www.ncbi.nlm.nih.gov/pmc/articles/instance/7233473/bin/radiol.2020200905.fig2.jpg)
![Fig 3a.](https://www.ncbi.nlm.nih.gov/pmc/articles/instance/7233473/bin/radiol.2020200905.fig3a.jpg)
Fig 3b.
Receiver operating characteristic curves of…
Fig 3b.
Receiver operating characteristic curves of the model. Each plot illustrates the receiver operating…
Fig 3c.
Receiver operating characteristic curves of…
Fig 3c.
Receiver operating characteristic curves of the model. Each plot illustrates the receiver operating…
Fig 4a.
Representative examples of the attention…
Fig 4a.
Representative examples of the attention heatmaps generated using Grad-CAM method for (a) COVID-19,…
Fig 4b.
Representative examples of the attention…
Fig 4b.
Representative examples of the attention heatmaps generated using Grad-CAM method for (a) COVID-19,…
Fig 4c.
Representative examples of the attention…
Fig 4c.
Representative examples of the attention heatmaps generated using Grad-CAM method for (a) COVID-19,…
Fig 5.
A representative example of community…
Fig 5.
A representative example of community acquired pneumonia case that is misclassified as COVID-19.…
Fig 6.
A representative example of COVID-19…
Fig 6.
A representative example of COVID-19 case that is misclassified as community acquired pneumonia.…
- Coronavirus Disease 2019 Deep Learning Models: Methodologic Considerations.Dadário AMV, de Paiva JPQ, Chate RC, Machado BS, Szarf G. Dadário AMV, et al. Radiology. 2020 Sep;296(3):E192. doi: 10.1148/radiol.2020201178. Epub 2020 Apr 3. Radiology. 2020. PMID: 32243239 Free PMC article. No abstract available.
- Artificial Intelligence Augmentation of Radiologist Performance in Distinguishing COVID-19 from Pneumonia of Other Origin at Chest CT.Bai HX, Wang R, Xiong Z, Hsieh B, Chang K, Halsey K, Tran TML, Choi JW, Wang DC, Shi LB, Mei J, Jiang XL, Pan I, Zeng QH, Hu PF, Li YH, Fu FX, Huang RY, Sebro R, Yu QZ, Atalay MK, Liao WH. Bai HX, et al. Radiology. 2020 Sep;296(3):E156-E165. doi: 10.1148/radiol.2020201491. Epub 2020 Apr 27. Radiology. 2020. PMID: 32339081 Free PMC article.
- Artificial intelligence for the detection of COVID-19 pneumonia on chest CT using multinational datasets.Harmon SA, Sanford TH, Xu S, Turkbey EB, Roth H, Xu Z, Yang D, Myronenko A, Anderson V, Amalou A, Blain M, Kassin M, Long D, Varble N, Walker SM, Bagci U, Ierardi AM, Stellato E, Plensich GG, Franceschelli G, Girlando C, Irmici G, Labella D, Hammoud D, Malayeri A, Jones E, Summers RM, Choyke PL, Xu D, Flores M, Tamura K, Obinata H, Mori H, Patella F, Cariati M, Carrafiello G, An P, Wood BJ, Turkbey B. Harmon SA, et al. Nat Commun. 2020 Aug 14;11(1):4080. doi: 10.1038/s41467-020-17971-2. Nat Commun. 2020. PMID: 32796848 Free PMC article.
- From community-acquired pneumonia to COVID-19: a deep learning-based method for quantitative analysis of COVID-19 on thick-section CT scans.Li Z, Zhong Z, Li Y, Zhang T, Gao L, Jin D, Sun Y, Ye X, Yu L, Hu Z, Xiao J, Huang L, Tang Y. Li Z, et al. Eur Radiol. 2020 Dec;30(12):6828-6837. doi: 10.1007/s00330-020-07042-x. Epub 2020 Jul 18. Eur Radiol. 2020. PMID: 32683550 Free PMC article.
- Performance of Radiologists in Differentiating COVID-19 from Non-COVID-19 Viral Pneumonia at Chest CT.Bai HX, Hsieh B, Xiong Z, Halsey K, Choi JW, Tran TML, Pan I, Shi LB, Wang DC, Mei J, Jiang XL, Zeng QH, Egglin TK, Hu PF, Agarwal S, Xie FF, Li S, Healey T, Atalay MK, Liao WH. Bai HX, et al. Radiology. 2020 Aug;296(2):E46-E54. doi: 10.1148/radiol.2020200823. Epub 2020 Mar 10. Radiology. 2020. PMID: 32155105 Free PMC article.
- Review on Diagnosis of COVID-19 from Chest CT Images Using Artificial Intelligence.Ozsahin I, Sekeroglu B, Musa MS, Mustapha MT, Uzun Ozsahin D. Ozsahin I, et al. Comput Math Methods Med. 2020 Sep 26;2020:9756518. doi: 10.1155/2020/9756518. eCollection 2020. Comput Math Methods Med. 2020. PMID: 33014121 Free PMC article. Review.
- Deep SVDD and Transfer Learning for COVID-19 Diagnosis Using CT Images.Alhadad AA, Mostafa RR, El-Bakry HM. Alhadad AA, et al. Comput Intell Neurosci. 2023 Mar 7;2023:6070970. doi: 10.1155/2023/6070970. eCollection 2023. Comput Intell Neurosci. 2023. PMID: 36926185 Free PMC article.
- Artificial intelligence for assistance of radiology residents in chest CT evaluation for COVID-19 pneumonia: a comparative diagnostic accuracy study.Mlynska L, Malouhi A, Ingwersen M, Güttler F, Gräger S, Teichgräber U. Mlynska L, et al. Acta Radiol. 2023 Mar 8:2841851231162085. doi: 10.1177/02841851231162085. Online ahead of print. Acta Radiol. 2023. PMID: 36890698 Free PMC article.
- Maxillofacial fracture detection and classification in computed tomography images using convolutional neural network-based models.Warin K, Limprasert W, Suebnukarn S, Paipongna T, Jantana P, Vicharueang S. Warin K, et al. Sci Rep. 2023 Mar 1;13(1):3434. doi: 10.1038/s41598-023-30640-w. Sci Rep. 2023. PMID: 36859660 Free PMC article.
- Comparison of the Diagnostic Performance of Deep Learning Algorithms for Reducing the Time Required for COVID-19 RT-PCR Testing.Lee Y, Kim YS, Lee DI, Jeong S, Kang GH, Jang YS, Kim W, Choi HY, Kim JG. Lee Y, et al. Viruses. 2023 Jan 22;15(2):304. doi: 10.3390/v15020304. Viruses. 2023. PMID: 36851519 Free PMC article.
- Evaluation of EfficientNet models for COVID-19 detection using lung parenchyma.Kurt Z, Işık Ş, Kaya Z, Anagün Y, Koca N, Çiçek S. Kurt Z, et al. Neural Comput Appl. 2023 Feb 20:1-12. doi: 10.1007/s00521-023-08344-z. Online ahead of print. Neural Comput Appl. 2023. PMID: 36843903 Free PMC article.
-
- Chen N, Zhou M, Dong X, et al. . Epidemiological and clinical characteristics of 99 cases of 2019 novel coronavirus pneumonia in Wuhan, China: a descriptive study. The Lancet 2020 Jan 30. doi: 10.1016/S0140-6736(20)30211-7.
-
- Holshue ML, DeBolt C, Lindquist S, et al. . First Case of 2019 Novel Coronavirus in the United States. New England Journal of Medicine 2020 Jan 31. doi:10.1056/NEJMoa2001191.
- Evaluation Study
- Multicenter Study
- Adult
- Aged
- Artificial Intelligence*
- Betacoronavirus*
- COVID-19
- COVID-19 Testing
- Clinical Laboratory Techniques / methods
- Community-Acquired Infections / diagnostic imaging
- Coronavirus Infections / diagnosis
- Coronavirus Infections / diagnostic imaging*
- Deep Learning
- Diagnosis, Differential
- Female
- Humans
- Imaging, Three-Dimensional / methods
- Male
- Middle Aged
- Pandemics
- Pneumonia, Viral / diagnostic imaging*
- ROC Curve
- Radiographic Image Interpretation, Computer-Assisted / methods
- Retrospective Studies
- SARS-CoV-2
- Sensitivity and Specificity
- Tomography, X-Ray Computed / methods
- Full Text Sources
- Other Literature Sources
- Medical
- Miscellaneous
NCBI Literature Resources
The PubMed wordmark and PubMed logo are registered trademarks of the U.S. Department of Health and Human Services (HHS). Unauthorized use of these marks is strictly prohibited.
National Library of Medicine
8600 Rockville Pike
Bethesda, MD 20894
![Fig 3b.](https://www.ncbi.nlm.nih.gov/pmc/articles/instance/7233473/bin/radiol.2020200905.fig3b.jpg)
Fig 3c.
Receiver operating characteristic curves of…
Fig 3c.
Receiver operating characteristic curves of the model. Each plot illustrates the receiver operating…
Fig 4a.
Representative examples of the attention…
Fig 4a.
Representative examples of the attention heatmaps generated using Grad-CAM method for (a) COVID-19,…
Fig 4b.
Representative examples of the attention…
Fig 4b.
Representative examples of the attention heatmaps generated using Grad-CAM method for (a) COVID-19,…
Fig 4c.
Representative examples of the attention…
Fig 4c.
Representative examples of the attention heatmaps generated using Grad-CAM method for (a) COVID-19,…
Fig 5.
A representative example of community…
Fig 5.
A representative example of community acquired pneumonia case that is misclassified as COVID-19.…
Fig 6.
A representative example of COVID-19…
Fig 6.
A representative example of COVID-19 case that is misclassified as community acquired pneumonia.…
- Coronavirus Disease 2019 Deep Learning Models: Methodologic Considerations.Dadário AMV, de Paiva JPQ, Chate RC, Machado BS, Szarf G. Dadário AMV, et al. Radiology. 2020 Sep;296(3):E192. doi: 10.1148/radiol.2020201178. Epub 2020 Apr 3. Radiology. 2020. PMID: 32243239 Free PMC article. No abstract available.
- Artificial Intelligence Augmentation of Radiologist Performance in Distinguishing COVID-19 from Pneumonia of Other Origin at Chest CT.Bai HX, Wang R, Xiong Z, Hsieh B, Chang K, Halsey K, Tran TML, Choi JW, Wang DC, Shi LB, Mei J, Jiang XL, Pan I, Zeng QH, Hu PF, Li YH, Fu FX, Huang RY, Sebro R, Yu QZ, Atalay MK, Liao WH. Bai HX, et al. Radiology. 2020 Sep;296(3):E156-E165. doi: 10.1148/radiol.2020201491. Epub 2020 Apr 27. Radiology. 2020. PMID: 32339081 Free PMC article.
- Artificial intelligence for the detection of COVID-19 pneumonia on chest CT using multinational datasets.Harmon SA, Sanford TH, Xu S, Turkbey EB, Roth H, Xu Z, Yang D, Myronenko A, Anderson V, Amalou A, Blain M, Kassin M, Long D, Varble N, Walker SM, Bagci U, Ierardi AM, Stellato E, Plensich GG, Franceschelli G, Girlando C, Irmici G, Labella D, Hammoud D, Malayeri A, Jones E, Summers RM, Choyke PL, Xu D, Flores M, Tamura K, Obinata H, Mori H, Patella F, Cariati M, Carrafiello G, An P, Wood BJ, Turkbey B. Harmon SA, et al. Nat Commun. 2020 Aug 14;11(1):4080. doi: 10.1038/s41467-020-17971-2. Nat Commun. 2020. PMID: 32796848 Free PMC article.
- From community-acquired pneumonia to COVID-19: a deep learning-based method for quantitative analysis of COVID-19 on thick-section CT scans.Li Z, Zhong Z, Li Y, Zhang T, Gao L, Jin D, Sun Y, Ye X, Yu L, Hu Z, Xiao J, Huang L, Tang Y. Li Z, et al. Eur Radiol. 2020 Dec;30(12):6828-6837. doi: 10.1007/s00330-020-07042-x. Epub 2020 Jul 18. Eur Radiol. 2020. PMID: 32683550 Free PMC article.
- Performance of Radiologists in Differentiating COVID-19 from Non-COVID-19 Viral Pneumonia at Chest CT.Bai HX, Hsieh B, Xiong Z, Halsey K, Choi JW, Tran TML, Pan I, Shi LB, Wang DC, Mei J, Jiang XL, Zeng QH, Egglin TK, Hu PF, Agarwal S, Xie FF, Li S, Healey T, Atalay MK, Liao WH. Bai HX, et al. Radiology. 2020 Aug;296(2):E46-E54. doi: 10.1148/radiol.2020200823. Epub 2020 Mar 10. Radiology. 2020. PMID: 32155105 Free PMC article.
- Review on Diagnosis of COVID-19 from Chest CT Images Using Artificial Intelligence.Ozsahin I, Sekeroglu B, Musa MS, Mustapha MT, Uzun Ozsahin D. Ozsahin I, et al. Comput Math Methods Med. 2020 Sep 26;2020:9756518. doi: 10.1155/2020/9756518. eCollection 2020. Comput Math Methods Med. 2020. PMID: 33014121 Free PMC article. Review.
- Deep SVDD and Transfer Learning for COVID-19 Diagnosis Using CT Images.Alhadad AA, Mostafa RR, El-Bakry HM. Alhadad AA, et al. Comput Intell Neurosci. 2023 Mar 7;2023:6070970. doi: 10.1155/2023/6070970. eCollection 2023. Comput Intell Neurosci. 2023. PMID: 36926185 Free PMC article.
- Artificial intelligence for assistance of radiology residents in chest CT evaluation for COVID-19 pneumonia: a comparative diagnostic accuracy study.Mlynska L, Malouhi A, Ingwersen M, Güttler F, Gräger S, Teichgräber U. Mlynska L, et al. Acta Radiol. 2023 Mar 8:2841851231162085. doi: 10.1177/02841851231162085. Online ahead of print. Acta Radiol. 2023. PMID: 36890698 Free PMC article.
- Maxillofacial fracture detection and classification in computed tomography images using convolutional neural network-based models.Warin K, Limprasert W, Suebnukarn S, Paipongna T, Jantana P, Vicharueang S. Warin K, et al. Sci Rep. 2023 Mar 1;13(1):3434. doi: 10.1038/s41598-023-30640-w. Sci Rep. 2023. PMID: 36859660 Free PMC article.
- Comparison of the Diagnostic Performance of Deep Learning Algorithms for Reducing the Time Required for COVID-19 RT-PCR Testing.Lee Y, Kim YS, Lee DI, Jeong S, Kang GH, Jang YS, Kim W, Choi HY, Kim JG. Lee Y, et al. Viruses. 2023 Jan 22;15(2):304. doi: 10.3390/v15020304. Viruses. 2023. PMID: 36851519 Free PMC article.
- Evaluation of EfficientNet models for COVID-19 detection using lung parenchyma.Kurt Z, Işık Ş, Kaya Z, Anagün Y, Koca N, Çiçek S. Kurt Z, et al. Neural Comput Appl. 2023 Feb 20:1-12. doi: 10.1007/s00521-023-08344-z. Online ahead of print. Neural Comput Appl. 2023. PMID: 36843903 Free PMC article.
-
- Chen N, Zhou M, Dong X, et al. . Epidemiological and clinical characteristics of 99 cases of 2019 novel coronavirus pneumonia in Wuhan, China: a descriptive study. The Lancet 2020 Jan 30. doi: 10.1016/S0140-6736(20)30211-7.
-
- Holshue ML, DeBolt C, Lindquist S, et al. . First Case of 2019 Novel Coronavirus in the United States. New England Journal of Medicine 2020 Jan 31. doi:10.1056/NEJMoa2001191.
- Evaluation Study
- Multicenter Study
- Adult
- Aged
- Artificial Intelligence*
- Betacoronavirus*
- COVID-19
- COVID-19 Testing
- Clinical Laboratory Techniques / methods
- Community-Acquired Infections / diagnostic imaging
- Coronavirus Infections / diagnosis
- Coronavirus Infections / diagnostic imaging*
- Deep Learning
- Diagnosis, Differential
- Female
- Humans
- Imaging, Three-Dimensional / methods
- Male
- Middle Aged
- Pandemics
- Pneumonia, Viral / diagnostic imaging*
- ROC Curve
- Radiographic Image Interpretation, Computer-Assisted / methods
- Retrospective Studies
- SARS-CoV-2
- Sensitivity and Specificity
- Tomography, X-Ray Computed / methods
- Full Text Sources
- Other Literature Sources
- Medical
- Miscellaneous
![Fig 3c.](https://www.ncbi.nlm.nih.gov/pmc/articles/instance/7233473/bin/radiol.2020200905.fig3c.jpg)
Fig 4a.
Representative examples of the attention…
Fig 4a.
Representative examples of the attention heatmaps generated using Grad-CAM method for (a) COVID-19,…
Fig 4b.
Representative examples of the attention…
Fig 4b.
Representative examples of the attention heatmaps generated using Grad-CAM method for (a) COVID-19,…
Fig 4c.
Representative examples of the attention…
Fig 4c.
Representative examples of the attention heatmaps generated using Grad-CAM method for (a) COVID-19,…
Fig 5.
A representative example of community…
Fig 5.
A representative example of community acquired pneumonia case that is misclassified as COVID-19.…
Fig 6.
A representative example of COVID-19…
Fig 6.
A representative example of COVID-19 case that is misclassified as community acquired pneumonia.…
![Fig 4a.](https://www.ncbi.nlm.nih.gov/pmc/articles/instance/7233473/bin/radiol.2020200905.fig4a.jpg)
![Fig 4b.](https://www.ncbi.nlm.nih.gov/pmc/articles/instance/7233473/bin/radiol.2020200905.fig4b.jpg)
![Fig 4c.](https://www.ncbi.nlm.nih.gov/pmc/articles/instance/7233473/bin/radiol.2020200905.fig4c.jpg)
![Fig 5.](https://www.ncbi.nlm.nih.gov/pmc/articles/instance/7233473/bin/radiol.2020200905.fig5.jpg)
![Fig 6.](https://www.ncbi.nlm.nih.gov/pmc/articles/instance/7233473/bin/radiol.2020200905.fig6.jpg)
References
- Chen N, Zhou M, Dong X, et al. . Epidemiological and clinical characteristics of 99 cases of 2019 novel coronavirus pneumonia in Wuhan, China: a descriptive study. The Lancet 2020 Jan 30. doi: 10.1016/S0140-6736(20)30211-7.
- Wang D, Hu B, Hu C, et al. . Clinical Characteristics of 138 Hospitalized Patients With 2019 Novel Coronavirus-Infected Pneumonia in Wuhan, China. Jama 2020.
- Li Q, Guan X, Wu P, et al. . Early Transmission Dynamics in Wuhan, China, of Novel Coronavirus-Infected Pneumonia. The New England Journal of Medicine 2020.
- Holshue ML, DeBolt C, Lindquist S, et al. . First Case of 2019 Novel Coronavirus in the United States. New England Journal of Medicine 2020 Jan 31. doi:10.1056/NEJMoa2001191.
- Ai T, Yang Z, Hou H, et al. . Correlation of Chest CT and RT-PCR Testing in Coronavirus Disease 2019 (COVID-19) in China: A Report of 1014 Cases. Radiology 2020. 10.1148/radiol.2020200642.
- Fang Y, Zhang H, Xie J, et al. . Sensitivity of Chest CT for COVID-19: Comparison to RT-PCR. Radiology. 2020. 10.1148/radiol.2020200432
- Chung M, Bernheim A, Mei X, et al. . CT Imaging Features of 2019 Novel Coronavirus (2019-nCoV). Radiology 2020: 200230.
- Huang C, Wang Y, Li X, et al. . Clinical features of patients infected with 2019 novel coronavirus in Wuhan, China. The Lancet 2020 Jan 24. doi: 10.1016/S0140-6736(20)30183-5.
- Xia C, Li X, Wang X, et al. . A Multi-modality Network for Cardiomyopathy Death Risk Prediction with CMR Images and Clinical Information. International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, Cham, 2019.
- Kong B, Wang X, Bai J, et al. . Learning tree-structured representation for 3D coronary artery segmentation. Computerized Medical Imaging and Graphics 2020, 80: 101688.
- Ye H, Gao F, Yin Y, et al. . Precise diagnosis of intracranial hemorrhage and subtypes using a three-dimensional joint convolutional and recurrent neural network. European Radiology, 2019, 29: 6191–6201.
- Kermany DS, Goldbaum M, Cai W, et al. . Identifying Medical Diagnoses and Treatable Diseases by Image-Based Deep Learning. Cell 2018, 172, 1122–1131.
- Rajaraman S, Candemir S, Kim I, et al. . Visualization and interpretation of convolutional neural network predictions in detecting pneumonia in pediatric chest radiographs. Applied Sciences 8.10 (2018): 1715.
- Depeursinge A, Chin AS, Leung AN, et al. . Automated classification of usual interstitial pneumonia using regional volumetric texture analysis in high-resolution CT. Investigative Radiology 50.4 (2015): 261.
- Anthimopoulos M, Christodoulidis S, Ebner L, et al. . Lung pattern classification for interstitial lung diseases using a deep convolutional neural network. IEEE Transactions on Medical Imaging 35.5 (2016): 1207-1216.
- He K, Zhang X, Ren S, et al. . Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016.
- Ronneberger O, Philipp F, Thomas B. U-net: Convolutional networks for biomedical image segmentation. International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, Cham, 2015.
- DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics 1988 (44), 837–845.
- Selvaraju RR, Cogswell M, Das A, et al. . Grad-cam: Visual explanations from deep networks via gradient-based localization. Proceedings of the IEEE International Conference on Computer Vision. 2017.
Source: PubMed