Validation of Combined Deep Learning Triaging and Computer-Aided Diagnosis in 2901 Breast MRI Examinations From the Second Screening Round of the Dense Tissue and Early Breast Neoplasm Screening Trial

Erik Verburg, Carla H van Gils, Bas H M van der Velden, Marije F Bakker, Ruud M Pijnappel, Wouter B Veldhuis, Kenneth G A Gilhuijs, Erik Verburg, Carla H van Gils, Bas H M van der Velden, Marije F Bakker, Ruud M Pijnappel, Wouter B Veldhuis, Kenneth G A Gilhuijs

Abstract

Objectives: Computer-aided triaging (CAT) and computer-aided diagnosis (CAD) of screening breast magnetic resonance imaging have shown potential to reduce the workload of radiologists in the context of dismissing normal breast scans and dismissing benign disease in women with extremely dense breasts. The aim of this study was to validate the potential of integrating CAT and CAD to reduce workload and workup on benign lesions in the second screening round of the DENSE trial, without missing cancer.

Methods: We included 2901 breast magnetic resonance imaging scans, obtained from 8 hospitals in the Netherlands. Computer-aided triaging and CAD were previously developed on data from the first screening round. Computer-aided triaging dismissed examinations without lesions. Magnetic resonance imaging examinations triaged to radiological reading were counted and subsequently processed by CAD. The number of benign lesions correctly classified by CAD was recorded. The false-positive fraction of the CAD was compared with that of unassisted radiological reading in the second screening round. Receiver operating characteristics (ROC) analysis was performed and the generalizability of CAT and CAD was assessed by comparing results from first and second screening rounds.

Results: Computer-aided triaging dismissed 950 of 2901 (32.7%) examinations with 49 lesions in total; none were malignant. Subsequent CAD classified 132 of 285 (46.3%) lesions as benign without misclassifying any malignant lesion. Together, CAT and CAD yielded significantly fewer false-positive lesions, 53 of 109 (48.6%) and 89 of 109 (78.9%), respectively ( P = 0.001), than radiological reading alone. Computer-aided triaging had a smaller area under the ROC curve in the second screening round compared with the first, 0.83 versus 0.76 ( P = 0.001), but this did not affect the negative predictive value at the 100% sensitivity operating threshold. Computer-aided diagnosis was not associated with significant differences in area under the ROC curve (0.857 vs 0.753, P = 0.08). At the operating thresholds, the specificities of CAT (39.7% vs 41.0%, P = 0.70) and CAD (41.0% vs 38.2%, P = 0.62) were successfully reproduced in the second round.

Conclusion: The combined application of CAT and CAD showed potential to reduce workload of radiologists and to reduce number of biopsies on benign lesions. Computer-aided triaging (CAT) correctly dismissed 950 of 2901 (32.7%) examinations with 49 lesions in total; none were malignant. Subsequent computer-aided diagnosis (CAD) classified 132 of 285 (46.3%) lesions as benign without misclassifying any malignant lesion. Together, CAT and CAD yielded significantly fewer false-positive lesions, 53 of 109 (48.6%) and 89 of 109 (78.9%), respectively ( P = 0.001), than radiological reading alone.

Trial registration: ClinicalTrials.gov NCT01315015.

Conflict of interest statement

Conflicts of interest and sources of funding: This study is financially supported by KWF, grant number UU-2014-7151, and used data acquired during the DENSE trial. The DENSE trial was supported by the regional screening organizations, Volpara Solutions, the Dutch Expert Centre for Screening, and the National Institute for Public Health and the Environment. The DENSE trial is financially supported by the University Medical Center Utrecht (project number: UMCU DENSE), the Netherlands Organization for Health Research and Development (ZonMw, project numbers: ZONMW-200320002-UMCU and ZonMW Preventie 50-53125-98-014), the Dutch Cancer Society (KWF Kankerbestrijding; project numbers: DCS-UU-2009-4348, UU-2014-6859, and UU2014-7151), the Dutch Pink Ribbon/A Sister's Hope (project number: Pink Ribbon-10074), Bayer AG Pharmaceuticals, Radiology (project number: BSP-DENSE), and Stichting Kankerpreventie Midden-West.

Copyright © 2022 The Author(s). Published by Wolters Kluwer Health, Inc.

Figures

FIGURE 1
FIGURE 1
Combination of CAT and CAD applied to the second screening round of the DENSE trial. Breasts with probability of lesions lower than operating threshold T were dismissed by CAT for processing by CAD. If the probability of malignant disease was larger or equal to operating threshold C, the lesion was classified as malignant.
FIGURE 2
FIGURE 2
ROC curves of CAT for the task of distinguishing between examinations with lesions (benign and malignant) and examinations without lesions, applied to first (left) and second (right) screening-round data. The 95% confidence intervals are shown in the legend.
FIGURE 3
FIGURE 3
ROC curves of CAD for the task of distinguishing between benign and malignant lesions, applied to first and second screening round data. The shaded regions represent the 95% confidence intervals.
FIGURE 4
FIGURE 4
A comparison of the workload of the radiologist in terms of reading and workup with and without computerized analysis. Note: Two benign lesions that CAD would classify as probably malignant were dismissed by CAT, which explains why CAD classifies only 73 of the 75 as malignant, as stated in the CAD Results section.

References

    1. Bakker MF de Lange SV Pijnappel RM, et al. . Supplemental MRI screening for women with extremely dense breast tissue. N Engl J Med. 2019;381:2091–2102.
    1. Saadatmand S Geuzinge HA Rutgers EJ, et al. . MRI versus mammography for breast cancer screening in women with familial risk (FaMRIsc): a multicentre, randomised, controlled trial. Lancet Oncol. 2019;20:1136–1147.
    1. Menezes GL Knuttel FM Stehouwer BL, et al. . Magnetic resonance imaging in breast cancer: a literature review and future perspectives. World J Clin Oncol. 2014;5:61–70.
    1. Maicas G Carneiro G Bradley AP, et al. . Deep Reinforcement Learning for Active Breast Lesion Detection from DCE-MRI. Cham, Switzerland: Springer International Publishing; 2017:665–673.
    1. Vignati A Giannini V De Luca M, et al. . Performance of a fully automatic lesion detection system for breast DCE-MRI. J Magn Reson Imaging. 2011;34:1341–1351.
    1. Verburg E van Gils CH van der Velden BHM, et al. . Deep learning for automated triaging of 4581 breast MRI examinations from the DENSE trial. Radiology. 2022;302:29–36.
    1. Gilhuijs KGA, Giger ML, Bick U. Computerized analysis of breast lesions in three dimensions using dynamic magnetic-resonance imaging. Med Phys. 1998;25:1647–1654.
    1. Honda E Nakayama R Koyama H, et al. . Computer-aided diagnosis scheme for distinguishing between benign and malignant masses in breast DCE-MRI. J Digit Imaging. 2016;29:388–393.
    1. Rahbar H, Partridge SC. Multiparametric MR imaging of breast Cancer. Magn Reson Imaging Clin N Am. 2016;24:223–238.
    1. Dalmis MU Gubern-Mérida A Vreemann S, et al. . Artificial intelligence–based classification of breast lesions imaged with a multiparametric breast MRI protocol with ultrafast DCE-MRI, T2, and DWI. Invest Radiol. 2019;54:325–332.
    1. Truhn D Schrading S Haarburger C, et al. . Radiomic versus convolutional neural networks analysis for classification of contrast-enhancing lesions at multiparametric breast MRI. Radiology. 2019;290:290–297.
    1. Gallego-Ortiz C, Martel AL. Using quantitative features extracted from T2-weighted MRI to improve breast MRI computer-aided diagnosis (CAD). Plos One. 2017;12:e0187501.
    1. Bhooshan N Giger M Lan L, et al. . Combined use of T2-weighted MRI and T1-weighted dynamic contrast-enhanced MRI in the automated analysis of breast lesions. Magn Reson Med. 2011;66:555–564.
    1. Verburg E van Gils CH Bakker MF, et al. . Computer-aided diagnosis in multiparametric magnetic resonance imaging screening of women with extremely dense breasts to reduce false-positive diagnoses. Invest Radiol. 2020;55:438–444.
    1. Emaus MJ Bakker MF Peeters PH, et al. . MR imaging as an additional screening modality for the detection of breast Cancer in women aged 50–75 years with extremely dense breasts: the DENSE trial study design. Radiology. 2015;277:527–537.
    1. Veenhuizen SGA de Lange SV Bakker MF, et al. . Supplemental breast MRI for women with extremely dense breasts: results of the second screening round of the DENSE trial. Radiology. 2021;299:278–286.
    1. Morris EA Comstock CE Lee CH, et al. . ACR BI-RADS® magnetic resonance imaging. ACR BI-RADS® Atlas, Breast Imaging Reporting and Data System, 5. 2013.
    1. Alderliesten T Schlief A Peterse J, et al. . Validation of semiautomatic measurement of the extent of breast tumors using contrast-enhanced magnetic resonance imaging. Invest Radiol. 2007;42:42–49.
    1. Intergraal Kankercentrum Nederland . Oncoline. 2017. . Accessed September 1, 2019.
    1. den Dekker BM Bakker MF de Lange SV, et al. . Reducing false-positive screening MRI rate in women with extremely dense breasts using prediction models based on data from the DENSE trial. Radiology. 2021;301:283–292.
    1. Lehman CD Peacock S DeMartini WB, et al. . A new automated software system to evaluate breast MR examinations: improved specificity without decreased sensitivity. Am J Roentgenol. 2006;187:51–56.
    1. Clauser P Marcon M Dietzel M, et al. . A new method to reduce false positive results in breast MRI by evaluation of multiple spectral regions in proton MR-spectroscopy. Eur J Radiol. 2017;92:51–57.
    1. Gubern-Mérida A Martí R Melendez J, et al. . Automated localization of breast cancer in DCE-MRI. Med Image Anal. 2015;20:265–274.
    1. Dalmış MU Vreemann S Kooi T, et al. . Fully automated detection of breast cancer in screening MRI using convolutional neural networks. J Med Imaging. 2018;5:014502.
    1. Renz DM Böttcher J Diekmann F, et al. . Detection and classification of contrast-enhancing masses by a fully automatic computer-assisted diagnosis system for breast MRI. J Magn Reson Imaging. 2012;35:1077–1088.
    1. Chang Y-C Huang Y-H Huang C-S, et al. . Computerized breast lesions detection using kinetic and morphologic analysis for dynamic contrast-enhanced MRI. Magn Reson Imaging. 2014;32:514–522.
    1. Ayatollahi F Shokouhi SB Mann RM, et al. . Automatic breast lesion detection in ultrafast DCE-MRI using deep learning. Med Phys. 2021;48:5897–5907.
    1. Schönenberger C Hejduk P Ciritsis A, et al. . Classification of mammographic breast microcalcifications using a deep convolutional neural network: a BI-RADS–based approach. Invest Radiol. 2021;56:224–231.
    1. Joe BN. AI to dismiss normal breast MRI scans and reduce workload. Radiology. 2022;302:37–38.

Source: PubMed

Sponsoren und Mitarbeiter

Krankheiten

Drogeninterventionen

3
Abonnieren