Machine Learning Augmented Interpretation of Chest X-rays: A Systematic Review

Hassan K Ahmad, Michael R Milne, Quinlan D Buchlak, Nalan Ektas, Georgina Sanderson, Hadi Chamtie, Sajith Karunasena, Jason Chiang, Xavier Holt, Cyril H M Tang, Jarrel C Y Seah, Georgina Bottrell, Nazanin Esmaili, Peter Brotchie, Catherine Jones, Hassan K Ahmad, Michael R Milne, Quinlan D Buchlak, Nalan Ektas, Georgina Sanderson, Hadi Chamtie, Sajith Karunasena, Jason Chiang, Xavier Holt, Cyril H M Tang, Jarrel C Y Seah, Georgina Bottrell, Nazanin Esmaili, Peter Brotchie, Catherine Jones

Abstract

Limitations of the chest X-ray (CXR) have resulted in attempts to create machine learning systems to assist clinicians and improve interpretation accuracy. An understanding of the capabilities and limitations of modern machine learning systems is necessary for clinicians as these tools begin to permeate practice. This systematic review aimed to provide an overview of machine learning applications designed to facilitate CXR interpretation. A systematic search strategy was executed to identify research into machine learning algorithms capable of detecting >2 radiographic findings on CXRs published between January 2020 and September 2022. Model details and study characteristics, including risk of bias and quality, were summarized. Initially, 2248 articles were retrieved, with 46 included in the final review. Published models demonstrated strong standalone performance and were typically as accurate, or more accurate, than radiologists or non-radiologist clinicians. Multiple studies demonstrated an improvement in the clinical finding classification performance of clinicians when models acted as a diagnostic assistance device. Device performance was compared with that of clinicians in 30% of studies, while effects on clinical perception and diagnosis were evaluated in 19%. Only one study was prospectively run. On average, 128,662 images were used to train and validate models. Most classified less than eight clinical findings, while the three most comprehensive models classified 54, 72, and 124 findings. This review suggests that machine learning devices designed to facilitate CXR interpretation perform strongly, improve the detection performance of clinicians, and improve the efficiency of radiology workflow. Several limitations were identified, and clinician involvement and expertise will be key to driving the safe implementation of quality CXR machine learning systems.

Keywords: chest X-ray; deep learning; machine learning; radiology.

Conflict of interest statement

Employees of the funder (Annalise.ai) were involved in study design, data collection, data analysis, data interpretation, and writing of the report.

Figures

Figure 1
Figure 1
PRISMA flow diagram indicating study identification, selection, and inclusion.
Figure 2
Figure 2
Proportion of studies with low, high, or unclear risk of bias as assessed across the four PROBAST domains.
Figure 3
Figure 3
Number of findings detected across included studies.
Figure 4
Figure 4
Dataset use prevalence amongst included articles.

References

    1. Mould R.F. The Early History of X-ray Diagnosis with Emphasis on the Contributions of Physics 1895–1915. Phys. Med. Biol. 1995;40:1741–1787. doi: 10.1088/0031-9155/40/11/001.
    1. United Nations . Sources and Effects of Ionizing Radiation. United Nations; New York, NY, USA: 2011.
    1. Lee C.S., Nagy P.G., Weaver S.J., Newman-Toker D.E. Cognitive and System Factors Contributing to Diagnostic Errors in Radiology. AJR Am. J. Roentgenol. 2013;201:611–617. doi: 10.2214/AJR.12.10375.
    1. Del Ciello A., Franchi P., Contegiacomo A., Cicchetti G., Bonomo L., Larici A.R. Missed Lung Cancer: When, Where, and Why? Diagn. Interv. Radiol. 2017;23:118–126. doi: 10.5152/dir.2016.16187.
    1. Brady A.P. Error and Discrepancy in Radiology: Inevitable or Avoidable? Insights Imaging. 2017;8:171–182. doi: 10.1007/s13244-016-0534-1.
    1. Nagarsheth K., Kurek S. Ultrasound Detection of Pneumothorax Compared with Chest X-ray and Computed Tomography Scan. Am. Surg. 2011;77:480–484. doi: 10.1177/000313481107700427.
    1. Hayden G.E., Wrenn K.W. Chest Radiograph vs. Computed Tomography Scan in the Evaluation for Pneumonia. J. Emerg. Med. 2009;36:266–270. doi: 10.1016/j.jemermed.2007.11.042.
    1. Vikgren J., Zachrisson S., Svalkvist A., Johnsson A.A., Boijsen M., Flinck A., Kheddache S., Båth M. Comparison of Chest Tomosynthesis and Chest Radiography for Detection of Pulmonary Nodules: Human Observer Study of Clinical Cases. Radiology. 2008;249:1034–1041. doi: 10.1148/radiol.2492080304.
    1. Jones C.M., Buchlak Q.D., Oakden-Rayner L., Milne M., Seah J., Esmaili N., Hachey B. Chest Radiographs and Machine Learning—Past, Present and Future. J. Med. Imaging Radiat. Oncol. 2021;65:538–544. doi: 10.1111/1754-9485.13274.
    1. Rajpurkar P., Irvin J., Zhu K., Yang B., Mehta H., Duan T., Ding D., Bagul A., Langlotz C., Shpanskaya K., et al. CheXNet: Radiologist-Level Pneumonia Detection on Chest X-rays with Deep Learning. arXiv. 20171711.05225
    1. Wu J.T., Wong K.C.L., Gur Y., Ansari N., Karargyris A., Sharma A., Morris M., Saboury B., Ahmad H., Boyko O., et al. Comparison of Chest Radiograph Interpretations by Artificial Intelligence Algorithm vs Radiology Residents. JAMA Netw. Open. 2020;3:e2022779. doi: 10.1001/jamanetworkopen.2020.22779.
    1. Buchlak Q.D., Esmaili N., Leveque J.-C., Farrokhi F., Bennett C., Piccardi M., Sethi R.K. Machine Learning Applications to Clinical Decision Support in Neurosurgery: An Artificial Intelligence Augmented Systematic Review. Neurosurg. Rev. 2020;43:1235–1253. doi: 10.1007/s10143-019-01163-8.
    1. Buchlak Q.D., Esmaili N., Leveque J.-C., Bennett C., Farrokhi F., Piccardi M. Machine Learning Applications to Neuroimaging for Glioma Detection and Classification: An Artificial Intelligence Augmented Systematic Review. J. Clin. Neurosci. 2021;89:177–198. doi: 10.1016/j.jocn.2021.04.043.
    1. Ben-Israel D., Jacobs W.B., Casha S., Lang S., Ryu W.H.A., de Lotbiniere-Bassett M., Cadotte D.W. The Impact of Machine Learning on Patient Care: A Systematic Review. Artif. Intell. Med. 2020;103:101785. doi: 10.1016/j.artmed.2019.101785.
    1. Tschandl P., Codella N., Akay B.N., Argenziano G., Braun R.P., Cabo H., Gutman D., Halpern A., Helba B., Hofmann-Wellenhof R., et al. Comparison of the Accuracy of Human Readers versus Machine-Learning Algorithms for Pigmented Skin Lesion Classification: An Open, Web-Based, International, Diagnostic Study. Lancet Oncol. 2019;20:938–947. doi: 10.1016/S1470-2045(19)30333-X.
    1. Buchlak Q.D., Milne M.R., Seah J., Johnson A., Samarasinghe G., Hachey B., Esmaili N., Tran A., Leveque J.-C., Farrokhi F., et al. Charting the Potential of Brain Computed Tomography Deep Learning Systems. J. Clin. Neurosci. 2022;99:217–223. doi: 10.1016/j.jocn.2022.03.014.
    1. Hwang E.J., Hong J.H., Lee K.H., Kim J.I., Nam J.G., Kim D.S., Choi H., Yoo S.J., Goo J.M., Park C.M. Deep Learning Algorithm for Surveillance of Pneumothorax after Lung Biopsy: A Multicenter Diagnostic Cohort Study. Eur. Radiol. 2020;30:3660–3671. doi: 10.1007/s00330-020-06771-3.
    1. Kim J.H., Kim J.Y., Kim G.H., Kang D., Kim I.J., Seo J., Andrews J.R., Park C.M. Clinical Validation of a Deep Learning Algorithm for Detection of Pneumonia on Chest Radiographs in Emergency Department Patients with Acute Febrile Respiratory Illness. J. Clin. Med. 2020;9:1981. doi: 10.3390/jcm9061981.
    1. Khasawneh N., Fraiwan M., Fraiwan L., Khassawneh B., Ibnian A. Detection of COVID-19 from Chest X-ray Images Using Deep Convolutional Neural Networks. Sensors. 2021;21:5940. doi: 10.3390/s21175940.
    1. Nasiri H., Hasani S. Automated Detection of COVID-19 Cases from Chest X-ray Images Using Deep Neural Network and XGBoost. Radiography (London) 2022;28:732–738. doi: 10.1016/j.radi.2022.03.011.
    1. Khan S.H., Sohail A., Khan A., Lee Y.-S. COVID-19 Detection in Chest X-ray Images Using a New Channel Boosted CNN. Diagnostics. 2022;12:267. doi: 10.3390/diagnostics12020267.
    1. Aboutalebi H., Pavlova M., Shafiee M.J., Sabri A., Alaref A., Wong A. COVID-Net CXR-S: Deep Convolutional Neural Network for Severity Assessment of COVID-19 Cases from Chest X-ray Images. Res. Sq. 2021;12:25. doi: 10.3390/diagnostics12010025.
    1. Ezzoddin M., Nasiri H., Dorrigiv M. Diagnosis of COVID-19 Cases from Chest X-ray Images Using Deep Neural Network and LightGBM. arXiv. 20222203.14275
    1. Nasiri H., Kheyroddin G., Dorrigiv M., Esmaeili M., Nafchi A.R., Ghorbani M.H., Zarkesh-Ha P. Classification of COVID-19 in Chest X-ray Images Using Fusion of Deep Features and LightGBM. arXiv. 20222206.04548
    1. Wang X., Yu J., Zhu Q., Li S., Zhao Z., Yang B., Pu J. Potential of Deep Learning in Assessing Pneumoconiosis Depicted on Digital Chest Radiography. Occup. Environ. Med. 2020;77:597–602. doi: 10.1136/oemed-2019-106386.
    1. Qin Z.Z., Sander M.S., Rai B., Titahong C.N., Sudrungrot S., Laah S.N., Adhikari L.M., Carter E.J., Puri L., Codlin A.J., et al. Using Artificial Intelligence to Read Chest Radiographs for Tuberculosis Detection: A Multi-Site Evaluation of the Diagnostic Accuracy of Three Deep Learning Systems. Sci. Rep. 2019;9:15000. doi: 10.1038/s41598-019-51503-3.
    1. Jang S., Song H., Shin Y.J., Kim J., Kim J., Lee K.W., Lee S.S., Lee W., Lee S., Lee K.H. Deep Learning-Based Automatic Detection Algorithm for Reducing Overlooked Lung Cancers on Chest Radiographs. Radiology. 2020;296:652–661. doi: 10.1148/radiol.2020200165.
    1. Gordienko Y., Gang P., Hui J., Zeng W., Kochura Y., Alienin O., Rokovyi O., Stirenko S. Advances in Intelligent Systems and Computing. Springer International Publishing; Cham, Switzerland: 2019. Deep Learning with Lung Segmentation and Bone Shadow Exclusion Techniques for Chest X-ray Analysis of Lung Cancer; pp. 638–647. Advances in intelligent systems and computing.
    1. Singh V., Danda V., Gorniak R., Flanders A., Lakhani P. Assessment of Critical Feeding Tube Malpositions on Radiographs Using Deep Learning. J. Digit. Imaging. 2019;32:651–655. doi: 10.1007/s10278-019-00229-9.
    1. Singh R., Kalra M.K., Nitiwarangkul C., Patti J.A., Homayounieh F., Padole A., Rao P., Putha P., Muse V.V., Sharma A., et al. Deep Learning in Chest Radiography: Detection of Findings and Presence of Change. PLoS ONE. 2018;13:e0204155. doi: 10.1371/journal.pone.0204155.
    1. Nam J.G., Park S., Hwang E.J., Lee J.H., Jin K.-N., Lim K.Y., Vu T.H., Sohn J.H., Hwang S., Goo J.M., et al. Development and Validation of Deep Learning-Based Automatic Detection Algorithm for Malignant Pulmonary Nodules on Chest Radiographs. Radiology. 2019;290:218–228. doi: 10.1148/radiol.2018180237.
    1. Cicero M., Bilbily A., Colak E., Dowdell T., Gray B., Perampaladas K., Barfett J. Training and Validating a Deep Convolutional Neural Network for Computer-Aided Detection and Classification of Abnormalities on Frontal Chest Radiographs. Investig. Radiol. 2017;52:281–287. doi: 10.1097/RLI.0000000000000341.
    1. Tschandl P., Rinner C., Apalla Z., Argenziano G., Codella N., Halpern A., Janda M., Lallas A., Longo C., Malvehy J., et al. Human-Computer Collaboration for Skin Cancer Recognition. Nat. Med. 2020;26:1229–1234. doi: 10.1038/s41591-020-0942-0.
    1. Wang H., Xia Y. ChestNet: A Deep Neural Network for Classification of Thoracic Diseases on Chest Radiography. arXiv. 20181807.03058
    1. Kelly C.J., Karthikesalingam A., Suleyman M., Corrado G., King D. Key Challenges for Delivering Clinical Impact with Artificial Intelligence. BMC Med. 2019;17:195. doi: 10.1186/s12916-019-1426-2.
    1. Geis J.R., Brady A.P., Wu C.C., Spencer J., Ranschaert E., Jaremko J.L., Langer S.G., Borondy Kitts A., Birch J., Shields W.F., et al. Ethics of Artificial Intelligence in Radiology: Summary of the Joint European and North American Multisociety Statement. Radiology. 2019;293:436–440. doi: 10.1148/radiol.2019191586.
    1. DeGrave A.J., Janizek J.D., Lee S.-I. AI for Radiographic COVID-19 Detection Selects Shortcuts over Signal. Nat. Mach. Intell. 2021;3:610–619. doi: 10.1038/s42256-021-00338-7.
    1. Eden J., Levit L., Berg A., Morton S., editors. Finding What Works in Health Care: Standards for Systematic Reviews. National Academies Press; Washington, DC, USA: 2011.
    1. Moher D., Liberati A., Tetzlaff J., Altman D.G. PRISMA Group Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. Int. J. Surg. 2010;8:336–341. doi: 10.1016/j.ijsu.2010.02.007.
    1. Wolff R.F., Moons K.G.M., Riley R.D., Whiting P.F., Westwood M., Collins G.S., Reitsma J.B., Kleijnen J., Mallett S. PROBAST: A Tool to Assess the Risk of Bias and Applicability of Prediction Model Studies. Ann. Intern. Med. 2019;170:51–58. doi: 10.7326/M18-1376.
    1. Page M.J., Moher D., Bossuyt P.M., Boutron I., Hoffmann T.C., Mulrow C.D., Shamseer L., Tetzlaff J.M., Akl E.A., Brennan S.E., et al. PRISMA 2020 Explanation and Elaboration: Updated Guidance and Exemplars for Reporting Systematic Reviews. BMJ. 2021;372:n160. doi: 10.1136/bmj.n160.
    1. Jones C.M., Danaher L., Milne M.R., Tang C., Seah J., Oakden-Rayner L., Johnson A., Buchlak Q.D., Esmaili N. Assessment of the Effect of a Comprehensive Chest Radiograph Deep Learning Model on Radiologist Reports and Patient Outcomes: A Real-World Observational Study. BMJ Open. 2021;11:e052902. doi: 10.1136/bmjopen-2021-052902.
    1. Ahn J.S., Ebrahimian S., McDermott S., Lee S., Naccarato L., Di Capua J.F., Wu M.Y., Zhang E.W., Muse V., Miller B., et al. Association of Artificial Intelligence-Aided Chest Radiograph Interpretation With Reader Performance and Efficiency. JAMA Netw. Open. 2022;5:e2229289. doi: 10.1001/jamanetworkopen.2022.29289.
    1. Albahli S., Yar G.N.A.H. Fast and Accurate Detection of COVID-19 along with 14 Other Chest Pathologies Using a Multi-Level Classification: Algorithm Development and Validation Study. J. Med. Internet Res. 2021;23:e23693. doi: 10.2196/23693.
    1. Cohen J.P., Morrison P., Dao L., Roth K., Duong T.Q., Ghassemi M. COVID-19 Image Data Collection: Prospective Predictions Are the Future. arXiv. 20202006.11988
    1. Altaf F., Islam S.M.S., Janjua N.K. A Novel Augmented Deep Transfer Learning for Classification of COVID-19 and Other Thoracic Diseases from X-rays. Neural Comput. Appl. 2021;33:14037–14048. doi: 10.1007/s00521-021-06044-0.
    1. Wang X., Peng Y., Lu L., Lu Z., Bagheri M., Summers R.M. ChestX-ray8: Hospital-Scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseases; Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR); Honolulu, HI, USA. 21–26 July 2017.
    1. Cohen J.P., Morrison P., Dao L. COVID-19 Image Data Collection. arXiv. 20202003.11597
    1. Baltruschat I., Steinmeister L., Nickisch H., Saalbach A., Grass M., Adam G., Knopp T., Ittrich H. Smart Chest X-ray Worklist Prioritization Using Artificial Intelligence: A Clinical Workflow Simulation. Eur. Radiol. 2021;31:3837–3845. doi: 10.1007/s00330-020-07480-7.
    1. Bharati S., Podder P., Mondal M.R.H. Hybrid Deep Learning for Detecting Lung Diseases from X-ray Images. Inform. Med. Unlocked. 2020;20:100391. doi: 10.1016/j.imu.2020.100391.
    1. Chakravarty A., Sarkar T., Ghosh N., Sethuraman R., Sheet D. Learning Decision Ensemble Using a Graph Neural Network for Comorbidity Aware Chest Radiograph Screening. Annu. Int. Conf. IEEE Eng. Med. Biol. Soc. 2020;2020:1234–1237.
    1. Irvin J., Rajpurkar P., Ko M., Yu Y., Ciurea-Ilcus S., Chute C., Marklund H., Haghgoo B., Ball R., Shpanskaya K., et al. CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison. Proc. Conf. AAAI Artif. Intell. 2019;33:590–597. doi: 10.1609/aaai.v33i01.3301590.
    1. Chen H., Miao S., Xu D., Hager G.D., Harrison A.P. Deep Hiearchical Multi-Label Classification Applied to Chest X-ray Abnormality Taxonomies. Med. Image Anal. 2020;66:101811. doi: 10.1016/j.media.2020.101811.
    1. Gohagan J.K., Prorok P.C., Hayes R.B., Kramer B.-S. The Prostate, Lung, Colorectal and Ovarian (PLCO) Cancer Screening Trial of the National Cancer Institute: History, Organization, and Status. Control. Clin. Trials. 2000;21:251S–272S. doi: 10.1016/S0197-2456(00)00097-0.
    1. Cho Y., Kim Y.-G., Lee S.M., Seo J.B., Kim N. Reproducibility of Abnormality Detection on Chest Radiographs Using Convolutional Neural Network in Paired Radiographs Obtained within a Short-Term Interval. Sci. Rep. 2020;10:17417. doi: 10.1038/s41598-020-74626-4.
    1. Cho Y., Park B., Lee S.M., Lee K.H., Seo J.B., Kim N. Optimal Number of Strong Labels for Curriculum Learning with Convolutional Neural Network to Classify Pulmonary Abnormalities in Chest Radiographs. Comput. Biol. Med. 2021;136:104750. doi: 10.1016/j.compbiomed.2021.104750.
    1. Choi S.Y., Park S., Kim M., Park J., Choi Y.R., Jin K.N. Evaluation of a Deep Learning-Based Computer-Aided Detection Algorithm on Chest Radiographs: Case-Control Study. Medicine (Baltimore) 2021;100:e25663. doi: 10.1097/MD.0000000000025663.
    1. Fang J., Xu Y., Zhao Y., Yan Y., Liu J., Liu J. Weighing Features of Lung and Heart Regions for Thoracic Disease Classification. BMC Med. Imaging. 2021;21:99. doi: 10.1186/s12880-021-00627-y.
    1. Gipson J., Tang V., Seah J., Kavnoudias H., Zia A., Lee R., Mitra B., Clements W. Diagnostic Accuracy of a Commercially Available Deep-Learning Algorithm in Supine Chest Radiographs Following Trauma. Br. J. Radiol. 2022;95:20210979. doi: 10.1259/bjr.20210979.
    1. Gündel S., Setio A.A.A., Ghesu F.C., Grbic S., Georgescu B., Maier A., Comaniciu D. Robust Classification from Noisy Labels: Integrating Additional Knowledge for Chest Radiography Abnormality Assessment. Med. Image Anal. 2021;72:102087. doi: 10.1016/j.media.2021.102087.
    1. Han Y., Chen C., Tang L., Lin M., Jaiswal A., Wang S., Tewfik A., Shih G., Ding Y., Peng Y. Using Radiomics as Prior Knowledge for Thorax Disease Classification and Localization in Chest X-rays. AMIA Annu. Symp. Proc. 2022;2021:546–555.
    1. Hwang E.J., Park J., Hong W., Lee H.-J., Choi H., Kim H., Nam J.G., Goo J.M., Yoon S.H., Lee C.H., et al. Artificial Intelligence System for Identification of False-Negative Interpretations in Chest Radiographs. Eur. Radiol. 2022;32:4468–4478. doi: 10.1007/s00330-022-08593-x.
    1. Jabbour S., Fouhey D., Kazerooni E., Wiens J., Sjoding M.W. Combining Chest X-rays and Electronic Health Record (EHR) Data Using Machine Learning to Diagnose Acute Respiratory Failure. J. Am. Med. Inform. Assoc. 2022;29:1060–1068. doi: 10.1093/jamia/ocac030.
    1. Jadhav A., Wong K.C.L., Wu J.T., Moradi M., Syeda-Mahmood T. Combining Deep Learning and Knowledge-Driven Reasoning for Chest X-ray Findings Detection. AMIA Annu. Symp. Proc. 2020;2020:593–601.
    1. Johnson A.E.W., Pollard T.J., Shen L., Lehman L.-W.H., Feng M., Ghassemi M., Moody B., Szolovits P., Celi L.A., Mark R.G. MIMIC-III, a Freely Accessible Critical Care Database. Sci. Data. 2016;3:160035. doi: 10.1038/sdata.2016.35.
    1. Jin K.N., Kim E.Y., Kim Y.J., Lee G.P., Kim H., Oh S., Kim Y.S., Han J.H., Cho Y.J. Diagnostic Effect of Artificial Intelligence Solution for Referable Thoracic Abnormalities on Chest Radiography: A Multicenter Respiratory Outpatient Diagnostic Cohort Study. Eur. Radiol. 2022;32:3469–3479. doi: 10.1007/s00330-021-08397-5.
    1. Kim E.Y., Kim Y.J., Choi W.-J., Lee G.P., Choi Y.R., Jin K.N., Cho Y.J. Correction: Performance of a Deep-Learning Algorithm for Referable Thoracic Abnormalities on Chest Radiographs: A Multicenter Study of a Health Screening Cohort. PLoS ONE. 2021;16:e0251045. doi: 10.1371/journal.pone.0251045.
    1. Kim E.Y., Kim Y.J., Choi W.-J., Jeon J.S., Kim M.Y., Oh D.H., Jin K.N., Cho Y.J. Concordance Rate of Radiologists and a Commercialized Deep-Learning Solution for Chest X-ray: Real-World Experience with a Multicenter Health Screening Cohort. PLoS ONE. 2022;17:e0264383. doi: 10.1371/journal.pone.0264383.
    1. Kuo P.-C., Tsai C.C., López D.M., Karargyris A., Pollard T.J., Johnson A.E.W., Celi L.A. Recalibration of Deep Learning Models for Abnormality Detection in Smartphone-Captured Chest Radiograph. NPJ Digit. Med. 2021;4:25. doi: 10.1038/s41746-021-00393-9.
    1. Lee M.S., Han S.W. DuETNet: Dual Encoder Based Transfer Network for Thoracic Disease Classification. Pattern Recognit. Lett. 2022;161:143–153. doi: 10.1016/j.patrec.2022.08.007.
    1. Li F., Shi J.-X., Yan L., Wang Y.-G., Zhang X.-D., Jiang M.-S., Wu Z.-Z., Zhou K.-Q. Lesion-Aware Convolutional Neural Network for Chest Radiograph Classification. Clin. Radiol. 2021;76:155.e1–155.e14. doi: 10.1016/j.crad.2020.08.027.
    1. Majkowska A., Mittal S., Steiner D.F., Reicher J.J., McKinney S.M., Duggan G.E., Eswaran K., Cameron Chen P.-H., Liu Y., Kalidindi S.R., et al. Chest Radiograph Interpretation with Deep Learning Models: Assessment with Radiologist-Adjudicated Reference Standards and Population-Adjusted Evaluation. Radiology. 2020;294:421–431. doi: 10.1148/radiol.2019191293.
    1. Mosquera C., Diaz F.N., Binder F., Rabellino J.M., Benitez S.E., Beresñak A.D., Seehaus A., Ducrey G., Ocantos J.A., Luna D.R. Chest X-ray Automated Triage: A Semiologic Approach Designed for Clinical Implementation, Exploiting Different Types of Labels through a Combination of Four Deep Learning Architectures. Comput. Methods Programs Biomed. 2021;206:106130. doi: 10.1016/j.cmpb.2021.106130.
    1. Nam J.G., Kim M., Park J., Hwang E.J., Lee J.H., Hong J.H., Goo J.M., Park C.M. Development and Validation of a Deep Learning Algorithm Detecting 10 Common Abnormalities on Chest Radiographs. Eur. Respir. J. 2021;57:2003061. doi: 10.1183/13993003.03061-2020.
    1. Bustos A., Pertusa A., Salinas J.-M., de la Iglesia-Vayá M. PadChest: A Large Chest X-ray Image Dataset with Multi-Label Annotated Reports. Med. Image Anal. 2020;66:101797. doi: 10.1016/j.media.2020.101797.
    1. Niehues S.M., Adams L.C., Gaudin R.A., Erxleben C., Keller S., Makowski M.R., Vahldiek J.L., Bressem K.K. Deep-Learning-Based Diagnosis of Bedside Chest X-ray in Intensive Care and Emergency Medicine. Investig. Radiol. 2021;56:525–534. doi: 10.1097/RLI.0000000000000771.
    1. Park S., Lee S.M., Lee K.H., Jung K.-H., Bae W., Choe J., Seo J.B. Deep Learning-Based Detection System for Multiclass Lesions on Chest Radiographs: Comparison with Observer Readings. Eur. Radiol. 2020;30:1359–1368. doi: 10.1007/s00330-019-06532-x.
    1. Paul A., Tang Y.-X., Shen T.C., Summers R.M. Discriminative Ensemble Learning for Few-Shot Chest X-ray Diagnosis. Med. Image Anal. 2021;68:101911. doi: 10.1016/j.media.2020.101911.
    1. Demner-Fushman D., Kohli M.D., Rosenman M.B., Shooshan S.E., Rodriguez L., Antani S., Thoma G.R., McDonald C.J. Preparing a Collection of Radiology Examinations for Distribution and Retrieval. J. Am. Med. Inform. Assoc. 2016;23:304–310. doi: 10.1093/jamia/ocv080.
    1. Pham H.H., Le T.T., Tran D.Q., Ngo D.T., Nguyen H.Q. Interpreting Chest X-rays via CNNs That Exploit Hierarchical Disease Dependencies and Uncertainty Labels. Neurocomputing. 2021;437:186–194. doi: 10.1016/j.neucom.2020.03.127.
    1. Rudolph J., Huemmer C., Ghesu F.-C., Mansoor A., Preuhs A., Fieselmann A., Fink N., Dinkel J., Koliogiannis V., Schwarze V., et al. Artificial Intelligence in Chest Radiography Reporting Accuracy: Added Clinical Value in the Emergency Unit Setting Without 24/7 Radiology Coverage. Investig. Radiol. 2022;57:90–98. doi: 10.1097/RLI.0000000000000813.
    1. Rudolph J., Schachtner B., Fink N., Koliogiannis V., Schwarze V., Goller S., Trappmann L., Hoppe B.F., Mansour N., Fischer M., et al. Clinically Focused Multi-Cohort Benchmarking as a Tool for External Validation of Artificial Intelligence Algorithm Performance in Basic Chest Radiography Analysis. Sci. Rep. 2022;12:12764. doi: 10.1038/s41598-022-16514-7.
    1. Seah J.C.Y., Tang C.H.M., Buchlak Q.D., Holt X.G., Wardman J.B., Aimoldin A., Esmaili N., Ahmad H., Pham H., Lambert J.F., et al. Effect of a Comprehensive Deep-Learning Model on the Accuracy of Chest X-ray Interpretation by Radiologists: A Retrospective, Multireader Multicase Study. Lancet Digit. Health. 2021;3:e496–e506. doi: 10.1016/S2589-7500(21)00106-0.
    1. Senan E.M., Alzahrani A., Alzahrani M.Y., Alsharif N., Aldhyani T.H.H. Automated Diagnosis of Chest X-ray for Early Detection of COVID-19 Disease. Comput. Math. Methods Med. 2021;2021:6919483. doi: 10.1155/2021/6919483.
    1. Sharma A., Rani S., Gupta D. Artificial Intelligence-Based Classification of Chest X-ray Images into COVID-19 and Other Infectious Diseases. Int. J. Biomed. Imaging. 2020;2020:8889023. doi: 10.1155/2020/8889023.
    1. Jaeger S., Candemir S., Antani S., Wáng Y.-X.J., Lu P.-X., Thoma G. Two Public Chest X-ray Datasets for Computer-Aided Screening of Pulmonary Diseases. Quant. Imaging Med. Surg. 2014;4:475–477.
    1. Sung J., Park S., Lee S.M., Bae W., Park B., Jung E., Seo J.B., Jung K.-H. Added Value of Deep Learning-Based Detection System for Multiple Major Findings on Chest Radiographs: A Randomized Crossover Study. Radiology. 2021;299:450–459. doi: 10.1148/radiol.2021202818.
    1. van Beek E.J.R., Ahn J.S., Kim M.J., Murchison J.T. Validation Study of Machine-Learning Chest Radiograph Software in Primary and Emergency Medicine. Clin. Radiol. 2022;78:1–7. doi: 10.1016/j.crad.2022.08.129.
    1. Verma D., Bose C., Tufchi N., Pant K., Tripathi V., Thapliyal A. An Efficient Framework for Identification of Tuberculosis and Pneumonia in Chest X-ray Images Using Neural Network. Procedia Comput. Sci. 2020;171:217–224. doi: 10.1016/j.procs.2020.04.023.
    1. Wang B., Zhang W. MARnet: Multi-Scale Adaptive Residual Neural Network for Chest X-ray Images Recognition of Lung Diseases. Math. Biosci. Eng. 2021;19:331–350. doi: 10.3934/mbe.2022017.
    1. Wang H., Jia H., Lu L., Xia Y. Thorax-Net: An Attention Regularized Deep Neural Network for Classification of Thoracic Diseases on Chest Radiography. IEEE J. Biomed. Health Inform. 2020;24:475–485. doi: 10.1109/JBHI.2019.2928369.
    1. Wang H., Yang Y.-Y., Pan Y., Han P., Li Z.-X., Huang H.-G., Zhu S.-Z. Detecting Thoracic Diseases via Representation Learning with Adaptive Sampling. Neurocomputing. 2020;406:354–360. doi: 10.1016/j.neucom.2019.06.113.
    1. Wang H., Wang S., Qin Z., Zhang Y., Li R., Xia Y. Triple Attention Learning for Classification of 14 Thoracic Diseases Using Chest Radiography. Med. Image Anal. 2021;67:101846. doi: 10.1016/j.media.2020.101846.
    1. Xu J., Li H., Li X. MS-ANet: Deep Learning for Automated Multi-Label Thoracic Disease Detection and Classification. PeerJ Comput. Sci. 2021;7:e541. doi: 10.7717/peerj-cs.541.
    1. Zhou L., Yin X., Zhang T., Feng Y., Zhao Y., Jin M., Peng M., Xing C., Li F., Wang Z., et al. Detection and Semiquantitative Analysis of Cardiomegaly, Pneumothorax, and Pleural Effusion on Chest Radiographs. Radiol. Artif. Intell. 2021;3:e200172. doi: 10.1148/ryai.2021200172.
    1. Zoph B., Vasudevan V., Shlens J., Le Q.V. Learning Transferable Architectures for Scalable Image Recognition; Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition; Salt Lake City, UT, USA. 18–23 June 2018.
    1. Huang G., Liu Z., Van Der Maaten L., Weinberger K.Q. Densely Connected Convolutional Networks; Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR); Honolulu, HI, USA. 21–26 July 2017.
    1. He K., Zhang X., Ren S., Sun J. Deep Residual Learning for Image Recognition; Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR); Las Vegas, NV, USA. 27–30 June 2016.
    1. Tan M., Le Q.V. EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks. arXiv. 20191905.11946
    1. Simonyan K., Zisserman A. Very Deep Convolutional Networks for Large-Scale Image Recognition. arXiv. 20141409.1556
    1. Oakden-Rayner L. Exploring Large-Scale Public Medical Image Datasets. Acad. Radiol. 2020;27:106–112. doi: 10.1016/j.acra.2019.10.006.
    1. Goddard K., Roudsari A., Wyatt J.C. Automation Bias: A Systematic Review of Frequency, Effect Mediators, and Mitigators. J. Am. Med. Inform. Assoc. 2012;19:121–127. doi: 10.1136/amiajnl-2011-000089.
    1. Crosby J., Rhines T., Li F., MacMahon H., Giger M. Deep Convolutional Neural Networks in the Classification of Dual-Energy Thoracic Radiographic Views for Efficient Workflow: Analysis on over 6500 Clinical Radiographs. J. Med. Imaging (Bellingham) 2020;7:016501. doi: 10.1117/1.JMI.7.1.016501.
    1. Huang S.-C., Pareek A., Zamanian R., Banerjee I., Lungren M.P. Multimodal Fusion with Deep Neural Networks for Leveraging CT Imaging and Electronic Health Record: A Case-Study in Pulmonary Embolism Detection. Sci. Rep. 2020;10:22147. doi: 10.1038/s41598-020-78888-w.
    1. Morrison A., Polisena J., Husereau D., Moulton K., Clark M., Fiander M., Mierzwinski-Urban M., Clifford T., Hutton B., Rabb D. The Effect of English-Language Restriction on Systematic Review-Based Meta-Analyses: A Systematic Review of Empirical Studies. Int. J. Technol. Assess. Health Care. 2012;28:138–144. doi: 10.1017/S0266462312000086.
    1. Parikh J.R., Wolfman D., Bender C.E., Arleo E. Radiologist Burnout According to Surveyed Radiology Practice Leaders. J. Am. Coll. Radiol. 2020;17:78–81. doi: 10.1016/j.jacr.2019.07.008.
    1. Rosenkrantz A.B., Hughes D.R., Duszak R., Jr. The U.S. Radiologist Workforce: An Analysis of Temporal and Geographic Variation by Using Large National Datasets. Radiology. 2016;279:175–184. doi: 10.1148/radiol.2015150921.
    1. Johnson A.E.W., Pollard T.J., Berkowitz S.J., Greenbaum N.R., Lungren M.P., Deng C.-Y., Mark R.G., Horng S. MIMIC-CXR, a de-Identified Publicly Available Database of Chest Radiographs with Free-Text Reports. Sci. Data. 2019;6:317. doi: 10.1038/s41597-019-0322-0.
    1. Mollura D.J., Azene E.M., Starikovsky A., Thelwell A., Iosifescu S., Kimble C., Polin A., Garra B.S., DeStigter K.K., Short B., et al. White Paper Report of the RAD-AID Conference on International Radiology for Developing Countries: Identifying Challenges, Opportunities, and Strategies for Imaging Services in the Developing World. J. Am. Coll. Radiol. 2010;7:495–500. doi: 10.1016/j.jacr.2010.01.018.
    1. Candemir S., Antani S. A Review on Lung Boundary Detection in Chest X-rays. Int. J. Comput. Assist. Radiol. Surg. 2019;14:563–576. doi: 10.1007/s11548-019-01917-1.
    1. Buchlak Q.D., Esmaili N., Leveque J.-C., Bennett C., Piccardi M., Farrokhi F. Ethical Thinking Machines in Surgery and the Requirement for Clinical Leadership. Am. J. Surg. 2020;220:1372–1374. doi: 10.1016/j.amjsurg.2020.06.073.
    1. He J., Baxter S.L., Xu J., Xu J., Zhou X., Zhang K. The Practical Implementation of Artificial Intelligence Technologies in Medicine. Nat. Med. 2019;25:30–36. doi: 10.1038/s41591-018-0307-0.
    1. Strohm L., Hehakaya C., Ranschaert E.R., Boon W.P.C., Moors E.H.M. Implementation of Artificial Intelligence (AI) Applications in Radiology: Hindering and Facilitating Factors. Eur. Radiol. 2020;30:5525–5532. doi: 10.1007/s00330-020-06946-y.
    1. Moradi M., Madani A., Karargyris A., Syeda-Mahmood T.F. Chest X-ray Generation and Data Augmentation for Cardiovascular Abnormality Classification. In: Angelini E.D., Landman B.A., editors. Proceedings of the Medical Imaging 2018: Image Processing; Houston, TX, USA. 10–15 February 2018; San Francisco, CA, USA: SPIE; 2018.
    1. Liu W.-T., Lin C.-S., Tsao T.-P., Lee C.-C., Cheng C.-C., Chen J.-T., Tsai C.-S., Lin W.-S., Lin C. A Deep-Learning Algorithm-Enhanced System Integrating Electrocardiograms and Chest X-rays for Diagnosing Aortic Dissection. Can. J. Cardiol. 2022;38:160–168. doi: 10.1016/j.cjca.2021.09.028.
    1. Nishimori M., Kiuchi K., Nishimura K., Kusano K., Yoshida A., Adachi K., Hirayama Y., Miyazaki Y., Fujiwara R., Sommer P., et al. Accessory Pathway Analysis Using a Multimodal Deep Learning Model. Sci. Rep. 2021;11:8045. doi: 10.1038/s41598-021-87631-y.
    1. Lewis A., Mahmoodi E., Zhou Y., Coffee M., Sizikova E. Improving Tuberculosis (TB) Prediction Using Synthetically Generated Computed Tomography (CT) Images. arXiv. 20212109.11480

Source: PubMed

Sponsors and Collaborators

Medical Conditions

Drug Interventions

3
Subscribe