Deep Learning System Boosts Radiologist Detection of Intracranial Hemorrhage

Roshan Warman, Anmol Warman, Pranav Warman, Andrew Degnan, Johan Blickman, Varun Chowdhary, Dev Dash, Rohit Sangal, Jason Vadhan, Tulio Bueso, Thomas Windisch, Gabriel Neves, Roshan Warman, Anmol Warman, Pranav Warman, Andrew Degnan, Johan Blickman, Varun Chowdhary, Dev Dash, Rohit Sangal, Jason Vadhan, Tulio Bueso, Thomas Windisch, Gabriel Neves

Abstract

Background: Intracranial hemorrhage (ICH) requires emergent medical treatment for positive outcomes. While previous artificial intelligence (AI) solutions achieved rapid diagnostics, none were shown to improve the performance of radiologists in detecting ICHs. Here, we show that the Caire ICH artificial intelligence system enhances a radiologist's ICH diagnosis performance.

Methods: A dataset of non-contrast-enhanced axial cranial computed tomography (CT) scans (n=532) were labeled for the presence or absence of an ICH. If an ICH was detected, its ICH subtype was identified. After a washout period, the three radiologists reviewed the same dataset with the assistance of the Caire ICH system. Performance was measured with respect to reader agreement, accuracy, sensitivity, and specificity when compared to the ground truth, defined as reader consensus.

Results: Caire ICH improved the inter-reader agreement on average by 5.76% in a dataset with an ICH prevalence of 74.3%. Further, radiologists using Caire ICH detected an average of 18 more ICHs and significantly increased their accuracy by 6.15%, their sensitivity by 4.6%, and their specificity by 10.62%. The Caire ICH system also improved the radiologist's ability to accurately identify the ICH subtypes present.

Conclusion: The Caire ICH device significantly improves the performance of a cohort of radiologists. Such a device has the potential to be a tool that can improve patient outcomes and reduce misdiagnosis of ICH.

Keywords: artificial intelligence; deep learning system; diagnosis; intracranial hemorrhage; radiologist.

Conflict of interest statement

The authors have declared financial relationships, which are detailed in the next section.

Copyright © 2022, Warman et al.

Figures

Figure 1. Study design workflow.
Figure 1. Study design workflow.
Figure 2. Examples of scans initially missed…
Figure 2. Examples of scans initially missed by radiologists that were correctly predicted with the Caire ICH system.
(A) A perimesencephalic subarachnoid hemorrhage (yellow arrow) and (B) a chronic convexity subdural hemorrhage (red arrow).
Figure 3. Examples of scans missed by…
Figure 3. Examples of scans missed by radiologists and the Caire ICH system.
(A) An acute tentorial subdural hemorrhage (yellow arrow) and (B) a chronic convexity subdural hemorrhage (red arrow).

References

    1. Sources, effects, and risks of ionizing radiation: United Nations scientific committee on the effects of atomic radiation. Volume I Scientific Annex A. United Nations. New York. [ Oct; 2022 ]. 2022.
    1. Trends in use of medical imaging in US health care systems and in Ontario, Canada, 2000-2016. Smith-Bindman R, Kwan ML, Marlow EC, et al. JAMA. 2019;322:843–856.
    1. Radiology, psychiatry, and paediatrics posts are put on national shortage list. Rimmer A. BMJ. 2015;350:2.
    1. Stressors contributing to burnout amongst pediatric radiologists: results from a survey of the Society for Pediatric Radiology. Ayyala RS, Ahmed FS, Ruzal-Shapiro C, Taylor GA. Pediatr Radiol. 2019;49:714–722.
    1. The growing issue of burnout in radiology - a survey-based evaluation of driving factors and potential impacts in pediatric radiologists. Ayyala RS, Baird GL, Sze RW, Brown BP, Taylor GA. Pediatr Radiol. 2020;50:1071–1077.
    1. Intracranial hemorrhage. Caceres JA, Goldstein JN. Emerg Med Clin North Am. 2012;30:771–794.
    1. Spontaneous intracerebral hemorrhage. Qureshi AI, Tuhrim S, Broderick JP, Batjer HH, Hondo H, Hanley DF. N Engl J Med. 2001;344:1450–1460.
    1. Intracerebral hemorrhage: an update on diagnosis and treatment. Hostettler IC, Seiffge DJ, Werring DJ. Expert Rev Neurother. 2019;19:679–694.
    1. nterpretable artificial intelligence for COVID-19 diagnosis from chest CT reveals specificity of ground-glass opacities. Warman A, Warman P, Sharma A, et al. medRxiv. 2020
    1. Pivotal trial of an autonomous AI-based diagnostic system for detection of diabetic retinopathy in primary care offices. Abràmoff MD, Lavin PT, Birch M, Shah N, Folk JC. NPJ Digit Med. 2018;1:39.
    1. High-performance medicine: the convergence of human and artificial intelligence. Topol EJ. Nat Med. 2019;25:44–56.
    1. Utility of artificial intelligence tool as a prospective radiology peer reviewer - detection of unreported intracranial hemorrhage. Rao B, Zohrabian V, Cedeno P, Saha A, Pahade J, Davis MA. Acad Radiol. 2021;28:85–93.
    1. Active reprioritization of the reading worklist using artificial intelligence has a beneficial effect on the turnaround time for interpretation of head CT with intracranial hemorrhage. O'Neill TJ, Xi Y, Stehel E, Browning T, Ng YS, Baker C, Peshock RM. Radiol Artif Intell. 2021;3:0.
    1. Critical care management of patients following aneurysmal subarachnoid hemorrhage: recommendations from the Neurocritical Care Society's Multidisciplinary Consensus Conference. Diringer MN, Bleck TP, Claude Hemphill J 3rd, et al. Neurocrit Care. 2011;15:211–240.
    1. Guidelines for the management of spontaneous intracerebral hemorrhage: a guideline for healthcare professionals from the American Heart Association/American. Hemphill JC 3rd, Greenberg SM, Anderson CS, et al. Stroke. 2015;46:2032–2060.
    1. What's the control in studies measuring the effect of computer-aided detection (CAD) on observer performance? Obuchowski NA, Meziane M, Dachman AH, Lieber ML, Mazzone PJ. Acad Radiol. 2010;17:761–767.
    1. Multireader, multicase receiver operating characteristic analysis. Obuchowski NA, Beiden SV, Berbaum KS, Hillis SL, Ishwaran H, Song HH, Wagner RF. Acad Radiol. 2004;11:980–995.
    1. Radiology resident evaluation of head CT scan orders in the emergency department. Erly WK, Berger WG, Krupinski E, Seeger JF, Guisto JA. AJNR Am J Neuroradiol. 2002;23:103–107.
    1. Radiologist shortage leaves patient care at risk, warns royal college. Rimmer A. BMJ. 2017;359:0.
    1. Growing number of emergency cranial CTs in patients with head injury not justified by their clinical need. Lambert L, Foltan O, Briza J, Lambertova A, Harsa P, Banerjee R, Danes J. Wien Klin Wochenschr. 2017;129:159–163.
    1. National trends in CT use in the emergency department: 1995-2007. Larson DB, Johnson LW, Schnell BM, Salisbury SR, Forman HP. Radiology. 2011;258:164–173.
    1. Discrepancy and error in radiology: concepts, causes and consequences. Brady A, Laoide RÓ, McCarthy P, McDermott R. Ulster Med J. 2012;81:3–9.
    1. Missed diagnosis of subarachnoid hemorrhage in the emergency department. Vermeulen MJ, Schull MJ. Stroke. 2007;38:1216–1221.
    1. Malpractice in radiology: what should you worry about? Cannavale A, Santoni M, Mancarella P, Passariello R, Arbarello P. Radiol Res Pract. 2013;2013:219259.
    1. Multi-reader multi-case studies using the area under the receiver operator characteristic curve as a measure of diagnostic accuracy: systematic review with a focus on quality of data reporting. Dendumrongsup T, Plumb AA, Halligan S, Fanshawe TR, Altman DG, Mallett S. PLoS One. 2014;9:0.
    1. Effect of a comprehensive deep-learning model on the accuracy of chest x-ray interpretation by radiologists: a retrospective, multireader multicase study. Seah JCY, Tang CHM, Buchlak QD, et al. Lancet Digit Health. 2021;3:496–506.

Source: PubMed

Sponsors and Collaborators

Medical Conditions

Drug Interventions

3
Subscribe