Diagnostic accuracy of a decision-support software for the detection of intracranial large-vessel occlusion in CT angiography

Abstract

Aim: To investigate the real-world clinical performance of the decision-support software "e-CTA" (e-Stroke Suite, Brainomix Limited, Oxford UK) for the detection of acute intracranial large-vessel occlusion (LVO) on computed tomography (CT) angiography at a UK district general hospital.

Materials and methods: The retrospective study included 300 consecutive CT angiograms of the head and neck performed between 8 March 2021 and 20 May 2021. e-CTA findings were recorded and compared with the radiologist report. Cases in which there was disagreement between e-CTA and the radiologist were reviewed by a sub-specialist vascular radiologist as the reference standard.

Results: The incidence of intracranial LVO was 7%. e-CTA correctly identified 18 of 21 intracranial proximal LVOs (86%). There were 34 false positives. The sensitivity was 0.86 (95% confidence interval [CI], 0.64-0.97), with specificity of 0.88 (95% CI, 0.83-0.91). The positive predictive value was 0.35 (95% CI, 0.27-0.43). The negative predictive value was 0.99 (95% CI, 0.96-1.00).

Conclusion: Sensitivity, specificity, and negative predictive values were similar to those reported in the literature (Seker et al., Int J Stroke. 2021; 17:77-82); however, the positive predictive value for e-CTA was significantly lower. In practice, this meant that over half of all reported occlusions by the software were false positives. Radiologists should be aware of these metrics in order to assign appropriate weight to software findings when formulating a report. Differences in population demographics, scanners, CT protocols, and incidence are all factors potentially influencing software accuracy. Local validation testing may help provide accuracy metrics more relevant to individual institutions.

Copyright © 2023 The Authors. Published by Elsevier Ltd.. All rights reserved.