An Algorithm Based on Deep Learning for Predicting In-Hospital Cardiac Arrest

Joon-Myoung Kwon, Youngnam Lee, Yeha Lee, Seungwoo Lee, Jinsik Park, Joon-Myoung Kwon, Youngnam Lee, Yeha Lee, Seungwoo Lee, Jinsik Park

Abstract

Background: In-hospital cardiac arrest is a major burden to public health, which affects patient safety. Although traditional track-and-trigger systems are used to predict cardiac arrest early, they have limitations, with low sensitivity and high false-alarm rates. We propose a deep learning-based early warning system that shows higher performance than the existing track-and-trigger systems.

Methods and results: This retrospective cohort study reviewed patients who were admitted to 2 hospitals from June 2010 to July 2017. A total of 52 131 patients were included. Specifically, a recurrent neural network was trained using data from June 2010 to January 2017. The result was tested using the data from February to July 2017. The primary outcome was cardiac arrest, and the secondary outcome was death without attempted resuscitation. As comparative measures, we used the area under the receiver operating characteristic curve (AUROC), the area under the precision-recall curve (AUPRC), and the net reclassification index. Furthermore, we evaluated sensitivity while varying the number of alarms. The deep learning-based early warning system (AUROC: 0.850; AUPRC: 0.044) significantly outperformed a modified early warning score (AUROC: 0.603; AUPRC: 0.003), a random forest algorithm (AUROC: 0.780; AUPRC: 0.014), and logistic regression (AUROC: 0.613; AUPRC: 0.007). Furthermore, the deep learning-based early warning system reduced the number of alarms by 82.2%, 13.5%, and 42.1% compared with the modified early warning system, random forest, and logistic regression, respectively, at the same sensitivity.

Conclusions: An algorithm based on deep learning had high sensitivity and a low false-alarm rate for detection of patients with cardiac arrest in the multicenter study.

Keywords: artificial intelligence; cardiac arrest; deep learning; machine learning; rapid response system; resuscitation.

© 2018 The Authors. Published on behalf of the American Heart Association, Inc., by Wiley.

Figures

Figure 1
Figure 1
The process of the DEWS. DEWS indicates deep learning–based early warning system; HR, heart rate; RNN, recurrent neural network; RR, respiratory rate; SBP, systolic blood pressure; BT, body temperature.
Figure 2
Figure 2
The architecture of the recurrent neural network. X t and h t indicate input and output at time t; W, weights.
Figure 3
Figure 3
Study flow chart. DEWS indicates deep learning–based early warning system.
Figure 4
Figure 4
Accuracy for predicting in‐hospital cardiac arrest and death without attempted resuscitation. AUPRC indicates area under the precision–recall curve; AUROC, area under the receiver operating characteristic curve; CI, confidence interval; DEWS, deep learning–based early warning system; MEWS, modified early warning score; SPTTS, single‐parameter track‐and‐trigger system.
Figure 5
Figure 5
Sensitivity according to MACHP for predicting in‐hospital cardiac arrest. DEWS indicates deep learning–based early warning system; MACHP, mean alarm count per hour per patient; MEWS, modified early warning score; SPTTS, single‐parameter track‐and‐trigger system; TTS, track‐and‐trigger system.
Figure 6
Figure 6
Trend of the DEWS score. A, The change in the mean of the DEWS scores over time as a group of patients. B, Cumulative percentage of cardiac arrest patients on detection time before event. We used the DEWS with sensitivity 70% for this plot. DEWS indicates deep learning–based early warning system.

References

    1. Graham R, McCoy MA, Schultz AM; Committee on the Treatment of Cardiac Arrest: Current Status and Future Directions; Board on Health Sciences Policy; Institute of Medicine Strategies to Improve Cardiac Arrest Survival: A Time to Act. Washington, DC: National Academic Press; 2015.
    1. Chan PS, Nallamothu BK, Krumholz HM, Curtis LH, Li Y, Hammill BG, Spertus JA. Readmission rates and long‐term hospital costs among survivors of an in‐hospital cardiac arrest. Circ Cardiovasc Qual Outcomes. 2014;7:889–895.
    1. Stecker EC, Reinier K, Marijon E, Narayanan K, Teodorescu C, Uy‐Evanado A, Gunson K, Jui J, Chugh SS. Public health burden of sudden cardiac death in the United States. Circ Arrhythm Electrophysiol. 2014;7:212–217.
    1. Benjamin EJ, Blaha MJ, Chiuve SE, Cushman M, Das SR, Deo R, De Ferranti SD, Floyd J, Fornage M, Gillespie C, Isasi CR, Jim'nez MC, Jordan LC, Judd SE, Lackland D, Lichtman JH, Lisabeth L, Liu S, Longenecker CT, MacKey RH, Matsushita K, Mozaffarian D, Mussolino ME, Nasir K, Neumar RW, Palaniappan L, Pandey DK, Thiagarajan RR, Reeves MJ, Ritchey M, Rodriguez CJ, Roth GA, Rosamond WD, Sasson C, Towfghi A, Tsao CW, Turner MB, Virani SS, Voeks JH, Willey JZ, Wilkins JT, Wu JHY, Alger HM, Wong SS, Muntner P. Heart disease and stroke statistics—2017 update: a report from the American Heart Association. Circulation. 2017; 135:e146–e603.
    1. Nadkarni VM. First documented rhythm and clinical outcome from in‐hospital cardiac arrest among children and adults. JAMA. 2006;295:50.
    1. Kause J, Smith G, Prytherch D, Parr M, Flabouris A, Hillman K. A comparison of antecedents to cardiac arrests, deaths and emergency intensive care admissions in Australia and New Zealand, and the United Kingdom—the ACADEMIA study. Resuscitation. 2004;62:275–282.
    1. Schein RMH, Hazday N, Pena M, Ruben BH, Sprung CL. Clinical antecedents to in‐hospital cardiopulmonary arrest. Chest. 1990;98:1388–1392.
    1. Franklin C, Mathew J. Developing strategies to prevent inhospital cardiac arrest: analyzing responses of physicians and nurses in the hours before the event. Crit Care Med. 1994;22:244–247.
    1. Mozaffarian D, Benjamin EJ, Go AS, Arnett DK, Blaha MJ, Cushman M, De Ferranti S, Després JP, Fullerton HJ, Howard VJ, Huffman MD, Judd SE, Kissela BM, Lackland DT, Lichtman JH, Lisabeth LD, Liu S, Mackey RH, Matchar DB, McGuire DK, Mohler ER, Moy CS, Muntner P, Mussolino ME, Nasir K, Neumar RW, Nichol G, Palaniappan L, Pandey DK, Reeves MJ, Rodriguez CJ, Sorlie PD, Stein J, Towfighi A, Turan TN, Virani SS, Willey JZ, Woo D, Yeh RW, Turner MB. Heart disease and stroke statistics—2015 update: a report from the American Heart Association. Circulation. 2015;131:e29–e322.
    1. Merchant RM, Yang L, Becker LB, Berg RA, Nadkarni V, Nichol G, Carr BG, Mitra N, Bradley SM, Abella BS, Peter W. Incidence of treated cardiac arrest in hospitalized patients in the United States. Crit Care Med. 2012;39:2401–2406.
    1. Nolan JP, Soar J, Cariou A, Cronberg T, Moulaert VRM, Deakin CD, Bottiger BW, Friberg H, Sunde K, Sandroni C. European Resuscitation Council and European Society of Intensive Care Medicine guidelines for post‐resuscitation care 2015: section 5 of the European Resuscitation Council Guidelines for Resuscitation 2015. Resuscitation. 2015;95:202–222.
    1. Jones DA, DeVita MA, Bellomo R. Rapid‐response teams. N Engl J Med. 2011;365:139–146.
    1. Kronick SL, Kurz MC, Lin S, Edelson DP, Berg RA, Billi JE, Cabanas JG, Cone DC, Diercks DB, Foster J, Meeks RA, Travers AH, Welsford M. Part 4: systems of care and continuous quality improvement: 2015 American Heart Association guidelines update for cardiopulmonary resuscitation and emergency cardiovascular care. Circulation. 2015;132:S397–S413.
    1. Smith GB, Prytherch DR, Schmidt PE, Featherstone PI, Higgins B. A review, and performance evaluation, of single‐parameter “track and trigger” systems. Resuscitation. 2008;79:11–21.
    1. Smith GB, Prytherch DR, Schmidt PE, Featherstone PI. Review and performance evaluation of aggregate weighted “track and trigger” systems. Resuscitation. 2008;77:170–179.
    1. Subbe CP, Kruger M, Rutherford P, Gemmel L. Validation of a modified Early Warning Score in medical admissions. QJM. 2001;94:521–526.
    1. Romero‐Brufau S, Huddleston JM, Naessens JM, Johnson MG, Hickman J, Morlan BW, Jensen JB, Caples SM, Elmer JL, Schmidt JA, Morgenthaler TI, Santrach PJ. Widely used track and trigger scores: are they ready for automation in practice? Resuscitation. 2014;85:549–552.
    1. Hillman K, Chen J, Cretikos M, Bellomo R, Brown D, Doig G, Finfer S, Flabouris A; MERIT Study Investigators . Introduction of the medical emergency team (MET) system: a cluster‐randomised controlled trial. Lancet. 2005;365:2091–2097.
    1. Trinkle RM, Flabouris A. Documenting Rapid Response System afferent limb failure and associated patient outcomes. Resuscitation. 2011;82:810–814.
    1. Gao H, McDonnell A, Harrison DA, Moore T, Adam S, Daly K, Esmonde L, Goldhill DR, Parry GJ, Rashidian A, Subbe CP, Harvey S. Systematic review and evaluation of physiological track and trigger warning systems for identifying at‐risk patients on the ward. Intensive Care Med. 2007;33:667–679.
    1. Bell MB, Konrad D, Granath F, Ekbom A, Martling CR. Prevalence and sensitivity of MET‐criteria in a Scandinavian University Hospital. Resuscitation. 2006;70:66–73.
    1. Gulshan V, Peng L, Coram M, Stumpe MC, Wu D, Narayanaswamy A, Venugopalan S, Widner K, Madams T, Cuadros J, Kim R, Raman R, Nelson PC, Mega JL, Webster DR. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA. 2016;304:649–656.
    1. Jha S, Topol EJ. Adapting to artificial intelligence. JAMA. 2016;316:2353.
    1. Deo RC. Machine learning in medicine. Circulation. 2015;132:1920–1930.
    1. LeCun Y, Bengio Y, Hinton G. Deep learning. Nature. 2015;521:436–444.
    1. Goldhill DR, McNarry AF. Physiological abnormalities in early warning scores are related to mortality in adult inpatients. Br J Anaesth. 2004;92:882–884.
    1. Rowley G, Fielding K. Reliability and accuracy of the Glasgow Coma Scale. Lancet. 1991;337:535–538.
    1. Smith GB, Prytherch DR, Meredith P, Schmidt PE, Featherstone PI. The ability of the National Early Warning Score (NEWS) to discriminate patients at risk of early cardiac arrest, unanticipated intensive care unit admission, and death. Resuscitation. 2013;84:465–470.
    1. Prytherch DR, Smith GB, Schmidt PE, Featherstone PI. ViEWS—towards a national early warning score for detecting adult inpatient deterioration. Resuscitation. 2010;81:932–937.
    1. Hochreiter S, Urgen Schmidhuber J. Long short‐term memory. Neural Comput. 1997;9:1735–1780.
    1. Kingma DP, Ba J. Adam: a method for stochastic optimization. 2017 IEEE Int Conf Consum Electron ICCE 2017 2014:434–435.
    1. Churpek MM, Yuen TC, Winslow C, Meltzer DO, Kattan MW, Edelson DP. Multicenter comparison of machine learning methods and conventional regression for predicting clinical deterioration on the wards. Crit Care Med. 2016;44:368–374.
    1. Taylor RA, Pare JR, Venkatesh AK, Mowafi H, Melnick ER, Fleischman W, Hall MK. Prediction of in‐hospital mortality in emergency department patients with sepsis: a local big data‐driven, machine learning approach. Acad Emerg Med. 2016;23:269–278.
    1. Ozenne B, Subtil F, Maucort‐Boulch D. The precision‐recall curve overcame the optimism of the receiver operating characteristic curve in rare diseases. J Clin Epidemiol. 2015;68:855–859.
    1. Weng CG, Poon J. A new evaluation measure for imbalanced datasets In:Seventh Australasian data mining conference (Aus DM 2008). South Australia: ACS; 2008;87:27–32.
    1. Leening MJG, Vedder MM, Witteman JCM, Pencina MJ, Steyerberg EW. Net reclassification improvement: computation, interpretation, and controversies. Ann Intern Med. 2014;160:122–131.
    1. Subbe CP, Davies RG, Williams E, Rutherford P, Gemmell L. Effect of introducing the Modified Early Warning score on clinical outcomes, cardio‐pulmonary arrests and intensive care utilisation in acute medical admissions. Anaesthesia. 2003;58:797–802.
    1. Suwanpasu S, Sattayasomboon Y. Accuracy of Modified Early Warning Scores for predicting mortality in hospital: a systematic review and meta‐analysis. J Intensive Crit Care. 2016;02:1–11.
    1. Bengio Y, Courville A, Vincent P. Representation learning: a review and new perspectives. IEEE Trans Pattern Anal Mach Intell. 2013;35:1798–1828.
    1. Mo D. A survey on deep learning: one small step toward AI. Tech Report, Univ New Mex Dept Comput Sci 2012:1–16.
    1. Bengio Y, Lecun Y. Scaling Learning Algorithms towards AI. Large Scale Kernel Mach 2007:321–360.
    1. DeVita MA, Smith GB, Adam SK, Adams‐Pizarro I, Buist M, Bellomo R, Bonello R, Cerchiari E, Farlow B, Goldsmith D, Haskell H, Hillman K, Howell M, Hravnak M, Hunt EA, Hvarfner A, Kellett J, Lighthall GK, Lippert A, Lippert FK, Mahroof R, Myers JS, Rosen M, Reynolds S, Rotondi A, Rubulotta F, Winters B. “Identifying the hospitalised patient in crisis”—a consensus conference on the afferent limb of rapid response systems. Resuscitation. 2010;81:375–382.
    1. Wolpert DH. The supervised learning no‐free‐lunch theorems. Proc 6th Online World Conf Soft Comput Ind Appl 2001:10–24.
    1. Pirracchio R, Petersen ML, Carone M, Rigon MR, Chevret S, van der Laan MJ. Mortality prediction in the ICU: can we do better? Results from the Super ICU Learner Algorithm (SICULA) project, a population‐ based study.. Lancet Respir Med. 2015;3:42–52.
    1. Fong RC, Vedaldi A. Interpretable explanations of black boxes by meaningful perturbation. Proc IEEE Int Conf Comput Vis. 2017;1:3449–3457.
    1. Chen X, Duan Y, Houthooft R, Schulman J, Sutskever I, Abbeel P. InfoGAN: interpretable representation learning by information maximizing generative adversarial nets. Neural Inf Process Syst. 2016;1:2172–2180.

Source: PubMed

Sponsoren und Mitarbeiter

Krankheiten

Drogeninterventionen

3
Abonnieren