Deep learning computer-aided detection system for pneumonia in febrile neutropenia patients: a diagnostic cohort study

Eui Jin Hwang, Jong Hyuk Lee, Jae Hyun Kim, Woo Hyeon Lim, Jin Mo Goo, Chang Min Park, Eui Jin Hwang, Jong Hyuk Lee, Jae Hyun Kim, Woo Hyeon Lim, Jin Mo Goo, Chang Min Park

Abstract

Background: Diagnosis of pneumonia is critical in managing patients with febrile neutropenia (FN), however, chest X-ray (CXR) has limited performance in the detection of pneumonia. We aimed to evaluate the performance of a deep learning-based computer-aided detection (CAD) system in pneumonia detection in the CXRs of consecutive FN patients and investigated whether CAD could improve radiologists' diagnostic performance when used as a second reader.

Methods: CXRs of patients with FN (a body temperature ≥ 38.3 °C, or a sustained body temperature ≥ 38.0 °C for an hour; absolute neutrophil count < 500/mm3) obtained between January and December 2017 were consecutively included, from a single tertiary referral hospital. Reference standards for the diagnosis of pneumonia were defined by consensus of two thoracic radiologists after reviewing medical records and CXRs. A commercialized, deep learning-based CAD system was retrospectively applied to detect pulmonary infiltrates on CXRs. For comparing performance, five radiologists independently interpreted CXRs initially without the CAD results (radiologist-alone interpretation), followed by the interpretation with CAD. The sensitivities and specificities for detection of pneumonia were compared between radiologist-alone interpretation and interpretation with CAD. The standalone performance of the CAD was also evaluated, using area under the receiver operating characteristic curve (AUC), sensitivity, and specificity. Moreover, sensitivity and specificity of standalone CAD were compared with those of radiologist-alone interpretation.

Results: Among 525 CXRs from 413 patients (52.3% men; median age 59 years), pneumonia was diagnosed in 128 (24.4%) CXRs. In the interpretation with CAD, average sensitivity of radiologists was significantly improved (75.4% to 79.4%, P = 0.003) while their specificity remained similar (75.4% to 76.8%, P = 0.101), compared to radiologist-alone interpretation. The CAD exhibited AUC, sensitivity, and specificity of 0.895, 88.3%, and 68.3%, respectively. The standalone CAD exhibited higher sensitivity (86.6% vs. 75.2%, P < 0.001) and lower specificity (64.8% vs. 75.4%, P < 0.001) compared to radiologist-alone interpretation.

Conclusions: In patients with FN, the deep learning-based CAD system exhibited radiologist-level performance in detecting pneumonia on CXRs and enhanced radiologists' performance.

Keywords: Artificial intelligence; Deep learning; Febrile neutropenia; Pneumonia; Radiography; Thoracic.

Conflict of interest statement

EJH reports research grant from Lunit Inc., outside the present study. JHL reports no competing interest. JHK reports no competing interest. WHL reports no competing interest. JMG reports no competing interest. CMP reports research grant from Lunit Inc., outside the present study, and holds stock options of Lunit Inc. and Coreline Soft.

© 2021. The Author(s).

Figures

Fig. 1
Fig. 1
Identification of pneumonia on chest X-ray (CXR) using the computer-aided detection (CAD) system
Fig. 2
Fig. 2
Performance of the computer-aided detection (CAD) system in all patients
Fig. 3
Fig. 3
Performance of the computer-aided detection (CAD) system and radiologists in the reader test
Fig. 4
Fig. 4
Performance of the computer-aided detection (CAD) system in different subgroups

References

    1. Klastersky J, de Naurois J, Rolston K, Rapoport B, Maschmeyer G, Aapro M, Herrstedt J, Committee EG. Management of febrile neutropaenia: ESMO Clinical Practice Guidelines. Ann Oncol. 2016;27(suppl 5):v111–v118. doi: 10.1093/annonc/mdw325.
    1. Chindaprasirt J, Wanitpongpun C, Limpawattana P, Thepsuthammarat K, Sripakdee W, Sookprasert A, Wirasorn K. Mortality, length of stay, and cost associated with hospitalized adult cancer patients with febrile neutropenia. Asian Pac J Cancer Prev. 2013;14(2):1115–1119. doi: 10.7314/APJCP.2013.14.2.1115.
    1. Kuderer NM, Dale DC, Crawford J, Cosler LE, Lyman GH. Mortality, morbidity, and cost associated with febrile neutropenia in adult cancer patients. Cancer. 2006;106(10):2258–2266. doi: 10.1002/cncr.21847.
    1. Crawford J, Dale DC, Lyman GH. Chemotherapy-induced neutropenia: risks, consequences, and new directions for its management. Cancer. 2004;100(2):228–237. doi: 10.1002/cncr.11882.
    1. Taplitz RA, Kennedy EB, Bow EJ, Crews J, Gleason C, Hawley DK, Langston AA, Nastoupil LJ, Rajotte M, Rolston K, et al. Outpatient management of fever and neutropenia in adults treated for malignancy: American Society of Clinical Oncology and Infectious Diseases Society of America Clinical Practice Guideline Update. J Clin Oncol. 2018;36(14):1443–1453. doi: 10.1200/JCO.2017.77.6211.
    1. Yoshida M, Akiyama N, Fujita H, Miura K, Miyatake J, Handa H, Kito K, Takahashi M, Shigeno K, Kanda Y, et al. Analysis of bacteremia/fungemia and pneumonia accompanying acute myelogenous leukemia from 1987 to 2001 in the Japan Adult Leukemia Study Group. Int J Hematol. 2011;93(1):66–73. doi: 10.1007/s12185-010-0746-y.
    1. Lanoix JP, Schmit JL, Douadi Y. Bacterial lung sepsis in patients with febrile neutropenia. Curr Opin Pulm Med. 2012;18(3):175–180. doi: 10.1097/MCP.0b013e328351f8e8.
    1. Evans SE, Ost DE. Pneumonia in the neutropenic cancer patient. Curr Opin Pulm Med. 2015;21(3):260–271. doi: 10.1097/MCP.0000000000000156.
    1. Maschmeyer G, Link H, Hiddemann W, Meyer P, Helmerking M, Eisenmann E, Schmitt J, Adam D. Pulmonary infiltrations in febrile patients with neutropenia. Risk factors and outcome under empirical antimicrobial therapy in a randomized multicenter study. Cancer. 1994;73(9):2296–2304. doi: 10.1002/1097-0142(19940501)73:9<2296::AID-CNCR2820730910>;2-7.
    1. Turkoglu M, Mirza E, Tunccan OG, Erdem GU, Dizbay M, Yagci M, Aygencel G, Turkoz Sucak G. Acinetobacter baumannii infection in patients with hematologic malignancies in intensive care unit: risk factors and impact on mortality. J Crit Care. 2011;26(5):460–467. doi: 10.1016/j.jcrc.2011.04.007.
    1. Heussel CP, Kauczor HU, Ullmann AJ. Pneumonia in neutropenic patients. Eur Radiol. 2004;14(2):256–271. doi: 10.1007/s00330-003-1985-6.
    1. Heussel CP. Importance of pulmonary imaging diagnostics in the management of febrile neutropenic patients. Mycoses. 2011;54(Suppl 1):17–26. doi: 10.1111/j.1439-0507.2010.01982.x.
    1. Freifeld AG, Bow EJ, Sepkowitz KA, Boeckh MJ, Ito JI, Mullen CA, Raad II, Rolston KV, Young JA, Wingard JR, et al. Clinical practice guideline for the use of antimicrobial agents in neutropenic patients with cancer: 2010 update by the infectious diseases society of America. Clin Infect Dis. 2011;52(4):e56–93. doi: 10.1093/cid/cir073.
    1. Heussel CP, Kauczor HU, Heussel GE, Fischer B, Begrich M, Mildenberger P, Thelen M. Pneumonia in febrile neutropenic patients and in bone marrow and blood stem-cell transplant recipients: use of high-resolution computed tomography. J Clin Oncol. 1999;17(3):796–805. doi: 10.1200/JCO.1999.17.3.796.
    1. Yolin-Raley DS, Dagogo-Jack I, Niell HB, Soiffer RJ, Antin JH, Alyea EP, 3rd, Glotzbecker BE. The utility of routine chest radiography in the initial evaluation of adult patients with febrile neutropenia patients undergoing HSCT. J Natl Compr Canc Netw. 2015;13(2):184–189. doi: 10.6004/jnccn.2015.0027.
    1. Zornoza J, Goldman AM, Wallace S, Valdivieso M, Bodey GP. Radiologic features of gram-negative pneumonias in the neutropenic patient. AJR Am J Roentgenol. 1976;127(6):989–996. doi: 10.2214/ajr.127.6.989.
    1. Hopstaken RM, Witbraad T, van Engelshoven JM, Dinant GJ. Inter-observer variation in the interpretation of chest radiographs for pneumonia in community-acquired lower respiratory tract infections. Clin Radiol. 2004;59(8):743–752. doi: 10.1016/j.crad.2004.01.011.
    1. Melbye H, Dale K. Interobserver variability in the radiographic diagnosis of adult outpatient pneumonia. Acta Radiol. 1992;33(1):79–81.
    1. Hashmi MF, Katiyar S, Keskar AG, Bokde ND, Geem ZW. Efficient pneumonia detection in chest Xray images using deep transfer learning. Diagnostics (Basel) 2020;10(6):417. doi: 10.3390/diagnostics10060417.
    1. Kermany DS, Goldbaum M, Cai W, Valentim CCS, Liang H, Baxter SL, McKeown A, Yang G, Wu X, Yan F, et al. Identifying medical diagnoses and treatable diseases by image-based deep learning. Cell. 2018;172(5):1122–1131. doi: 10.1016/j.cell.2018.02.010.
    1. Hwang EJ, Park S, Jin KN, Kim JI, Choi SY, Lee JH, Goo JM, Aum J, Yim JJ, Cohen JG, et al. Development and validation of a deep learning-based automated detection algorithm for major thoracic diseases on chest radiographs. JAMA Netw Open. 2019;2(3):e191095. doi: 10.1001/jamanetworkopen.2019.1095.
    1. Zech JR, Badgeley MA, Liu M, Costa AB, Titano JJ, Oermann EK. Variable generalization performance of a deep learning model to detect pneumonia in chest radiographs: a cross-sectional study. PLoS Med. 2018;15(11):e1002683. doi: 10.1371/journal.pmed.1002683.
    1. Metlay JP, Waterer GW, Long AC, Anzueto A, Brozek J, Crothers K, Cooley LA, Dean NC, Fine MJ, Flanders SA, et al. Diagnosis and treatment of adults with community-acquired pneumonia. An official clinical practice guideline of the American Thoracic Society and Infectious Diseases Society of America. Am J Respir Crit Care Med. 2019;200(7):e45–e67. doi: 10.1164/rccm.201908-1581ST.
    1. DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. 1988;44(3):837–845. doi: 10.2307/2531595.
    1. Genders TS, Spronk S, Stijnen T, Steyerberg EW, Lesaffre E, Hunink MG. Methods for calculating sensitivity and specificity of clustered data: a tutorial. Radiology. 2012;265(3):910–916. doi: 10.1148/radiol.12120509.
    1. Maschmeyer G. Pneumonia in febrile neutropenic patients: radiologic diagnosis. Curr Opin Oncol. 2001;13(4):229–235. doi: 10.1097/00001622-200107000-00004.
    1. Park SH, Han K. Methodologic guide for evaluating clinical performance and effect of artificial intelligence technology for medical diagnosis and prediction. Radiology. 2018;286(3):800–809. doi: 10.1148/radiol.2017171920.
    1. Hwang EJ, Kim H, Lee JH, Goo JM, Park CM. Automated identification of chest radiographs with referable abnormality with deep learning: need for recalibration. Eur Radiol. 2020;30(12):6902–6912. doi: 10.1007/s00330-020-07062-7.
    1. Petrick N, Sahiner B, Armato SG, 3rd, Bert A, Correale L, Delsanto S, Freedman MT, Fryd D, Gur D, Hadjiiski L, et al. Evaluation of computer-aided detection and diagnosis systems. Med Phys. 2013;40(8):087001. doi: 10.1118/1.4816310.
    1. Hwang EJ, Park CM. Clinical implementation of deep learning in thoracic radiology: potential applications and challenges. Korean J Radiol. 2020;21(5):511–525. doi: 10.3348/kjr.2019.0821.
    1. Hwang EJ, Kim H, Yoon SH, Goo JM, Park CM. Implementation of a deep learning-based computer-aided detection system for the interpretation of chest radiographs in patients suspected for COVID-19. Korean J Radiol. 2020;21(10):1150–1160. doi: 10.3348/kjr.2020.0536.
    1. Gerritsen MG, Willemink MJ, Pompe E, van der Bruggen T, van Rhenen A, Lammers JW, Wessels F, Sprengers RW, de Jong PA, Minnema MC. Improving early diagnosis of pulmonary infections in patients with febrile neutropenia using low-dose chest computed tomography. PLoS ONE. 2017;12(2):e0172256. doi: 10.1371/journal.pone.0172256.
    1. Patsios D, Maimon N, Chung T, Roberts H, Disperati P, Minden M, Paul N. Chest low-dose computed tomography in neutropenic acute myeloid leukaemia patients. Respir Med. 2010;104(4):600–605. doi: 10.1016/j.rmed.2009.11.003.
    1. Kunz WG, Patzig M, Crispin A, Stahl R, Reiser MF, Notohamiprodjo M. The value of supine chest X-ray in the diagnosis of pneumonia in the basal lung zones. Acad Radiol. 2018;25(10):1252–1256. doi: 10.1016/j.acra.2018.01.027.
    1. Kitazono MT, Lau CT, Parada AN, Renjen P, Miller WT., Jr Differentiation of pleural effusions from parenchymal opacities: accuracy of bedside chest radiography. AJR Am J Roentgenol. 2010;194(2):407–412. doi: 10.2214/AJR.09.2950.

Source: PubMed

Sponsors et collaborateurs

Les conditions médicales

Interventions en matière de drogue

3
S'abonner