Effect of a machine learning-based severe sepsis prediction algorithm on patient survival and hospital length of stay: a randomised clinical trial

David W Shimabukuro, Christopher W Barton, Mitchell D Feldman, Samson J Mataraso, Ritankar Das, David W Shimabukuro, Christopher W Barton, Mitchell D Feldman, Samson J Mataraso, Ritankar Das

Abstract

Introduction: Several methods have been developed to electronically monitor patients for severe sepsis, but few provide predictive capabilities to enable early intervention; furthermore, no severe sepsis prediction systems have been previously validated in a randomised study. We tested the use of a machine learning-based severe sepsis prediction system for reductions in average length of stay and in-hospital mortality rate.

Methods: We conducted a randomised controlled clinical trial at two medical-surgical intensive care units at the University of California, San Francisco Medical Center, evaluating the primary outcome of average length of stay, and secondary outcome of in-hospital mortality rate from December 2016 to February 2017. Adult patients (18+) admitted to participating units were eligible for this factorial, open-label study. Enrolled patients were assigned to a trial arm by a random allocation sequence. In the control group, only the current severe sepsis detector was used; in the experimental group, the machine learning algorithm (MLA) was also used. On receiving an alert, the care team evaluated the patient and initiated the severe sepsis bundle, if appropriate. Although participants were randomly assigned to a trial arm, group assignments were automatically revealed for any patients who received MLA alerts.

Results: Outcomes from 75 patients in the control and 67 patients in the experimental group were analysed. Average length of stay decreased from 13.0 days in the control to 10.3 days in the experimental group (p=0.042). In-hospital mortality decreased by 12.4 percentage points when using the MLA (p=0.018), a relative reduction of 58.0%. No adverse events were reported during this trial.

Conclusion: The MLA was associated with improved patient outcomes. This is the first randomised controlled trial of a sepsis surveillance system to demonstrate statistically significant differences in length of stay and in-hospital mortality.

Trial registration: NCT03015454.

Keywords: alerts; electronic health records; machine learning; patient monitoring; prediction; sepsis; severe sepsis.

Conflict of interest statement

Competing interests: SM and RD are employees of Dascena. CB reports receiving consulting fees from Dascena. CB, DS and MF report receiving research grant funding from Dascena.

Figures

Figure 1
Figure 1
Patients assessed, enrolled, randomised and analysed in each arm of the randomised controlled trial.
Figure 2
Figure 2
Decrease in average hospital and ICU length of stay with the use of the machine learning algorithm. The error bars represent one standard error above and below the mean length of stay. ICU, intensive care unit.
Figure 3
Figure 3
Reduction of in-hospital mortality rate when using the machine learning algorithm. The error bars represent one standard error above and below the average in-hospital mortality rate.

References

    1. Angus DC, Linde-Zwirble WT, Lidicker J, et al. . Epidemiology of severe sepsis in the United States: analysis of incidence, outcome, and associated costs of care. Crit Care Med 2001;29:1303–10.
    1. Chalupka AN, Talmor D. The economics of sepsis. Crit Care Clin 2012;28:57–76.
    1. Singer M, Deutschman CS, Seymour CW, et al. . The third international consensus definitions for Sepsis and Septic Shock (sepsis-3). JAMA 2016;315:801–10.
    1. Henry KE, Hager DN, Pronovost PJ, et al. . A Targeted Real-time Early Warning Score (TREWScore) for septic shock. Sci Transl Med 2015;7:299ra122
    1. Lever A, Mackenzie I. Sepsis: definition, epidemiology, and diagnosis. BMJ 2007;335:879–83.
    1. Stevenson EK, Rubenstein AR, Radin GT, et al. . Two decades of mortality trends among patients with severe sepsis: a comparative meta-analysis*. Crit Care Med 2014;42:625
    1. Kumar A, Roberts D, Wood KE, et al. . Duration of hypotension before initiation of effective antimicrobial therapy is the critical determinant of survival in human septic shock. Crit Care Med 2006;34:1589–96.
    1. Nguyen HB, Corbett SW, Steele R, et al. . Implementation of a bundle of quality indicators for the early management of severe sepsis and septic shock is associated with decreased mortality. Crit Care Med 2007;35:1105–12.
    1. Calvert JS, Price DA, Chettipally UK, et al. . A computational approach to early sepsis detection. Comput Biol Med 2016;74:69–73.
    1. Calvert J, Desautels T, Chettipally U, et al. . High-performance detection and early prediction of septic shock for alcohol-use disorder patients. Ann Med Surg 2016;8:50–5.
    1. Desautels T, Calvert J, Hoffman J, et al. . Prediction of sepsis in the intensive care unit with minimal electronic health record data: a machine learning approach. JMIR Med Inform 2016;4:e28
    1. Vincent JL, Moreno R, Takala J, et al. . The SOFA (Sepsis-related Organ Failure Assessment) score to describe organ dysfunction/failure. On behalf of the working group on sepsis-related problems of the European society of intensive care medicine. Intensive Care Med 1996;22:707–10.
    1. Balk RA. Severe sepsis and septic shock. Definitions, epidemiology, and clinical manifestations. Crit Care Clin 2000;16:179–92.
    1. Subbe CP, Slater A, Menon D, et al. . Validation of physiological scoring systems in the accident and emergency department. Emerg Med J 2006;23:841–5.
    1. McLymont N, Glover GW. Scoring systems for the characterization of sepsis and associated outcomes. Ann Transl Med 2016;4:527
    1. Churpek MM, Snyder A, Han X, et al. . Quick sepsis-related organ failure assessment, systemic inflammatory response syndrome, and early warning scores for detecting clinical deterioration in infected patients outside the intensive care unit. Am J Respir Crit Care Med 2017;195:906–11.
    1. Foster KR, Koprowski R, Skufca JD. Machine learning, medical diagnosis, and biomedical engineering research - commentary. Biomed Eng Online 2014;13:94
    1. Nachimuthu SK, Haug PJ. Early detection of sepsis in the emergency department using dynamic bayesian networks. AMIA Annu Symp Proc 2012;2012:653–62.
    1. Stanculescu I, Williams CK, Freer Y. A Hierarchical Switching Linear Dynamical System Applied to the Detection of Sepsis in Neonatal Condition Monitoring. Proceedings of the Thirtieth Conference on Uncertainty in Artificial Intelligence (UAI) 2014;752–61.
    1. Stanculescu I, Williams CK, Freer Y. Autoregressive hidden Markov models for the early detection of neonatal sepsis. IEEE J Biomed Health Inform 2014;18:1560–70.
    1. Dyagilev K, Saria S. Learning (predictive) risk scores in the presence of censoring due to interventions. Mach Learn 2016;102:323–48.
    1. Sawyer AM, Deal EN, Labelle AJ, et al. . Implementation of a real-time computerized sepsis alert in non-intensive care unit patients. Critical Care Medicine 2011;39.3:469–73.
    1. Rivers E, Nguyen B, Havstad S, et al. . Early goal-directed therapy in the treatment of severe sepsis and septic shock. N Engl J Med 2001;345:1368–77.
    1. Yealy DM, Kellum JA, Huang DT, et al. . A randomized trial of protocol-based care for early septic shock. N Engl J Med 2014;370:1683–93.
    1. Singer M. Antibiotics for sepsis: does each hour really count, or is it incestuous amplification? Am J Respir Crit Care Med 2017;196:800–2.
    1. Chang J, Sullivan M, Shea E, et al. . The effectiveness of a real-time electronic alert to detect severe sepsis in an intensive care unit. Poster 13 presented at: SOCCA 28th Annual Meeting and Critical Care Update 2015. Honolulu HI: (accessed 22 Feb 2017).
    1. Dellinger RP, Levy MM, Rhodes A, et al. . Surviving sepsis campaign: international guidelines for management of severe Sepsis and Septic Shock, 2012. Intensive Care Med 2013;39:165–228.
    1. Calvert J, Mao Q, Rogers AJ, et al. . A computational approach to mortality prediction of alcohol use disorder inpatients. Comput Biol Med 2016;75:74–9.
    1. Calvert J, Mao Q, Hoffman JL, et al. . Using electronic health record collected clinical variables to predict medical intensive care unit mortality. Ann Med Surg 2016;11:52–7.
    1. Hooper MH, Weavind L, Wheeler AP, et al. . Randomized trial of automated, electronic monitoring to facilitate early detection of sepsis in the intensive care unit. Crit Care Med 2012;40:2096–101.
    1. Semler MW, Weavind L, Hooper MH, et al. . An electronic tool for the evaluation and treatment of Sepsis in the ICU: a randomized controlled trial. Crit Care Med 2015;43:1595
    1. Manaktala S, Claypool SR. Evaluating the impact of a computerized surveillance algorithm and decision support system on sepsis mortality. J Am Med Inform Assoc 2017;24:88–95.
    1. Berger T, Birnbaum A, Bijur P, et al. . A computerized alert screening for severe sepsis in emergency department patients increases lactate testing but does not improve inpatient mortality. Appl Clin Inform 2010;1:394–407.
    1. Cvach M. Monitor alarm fatigue: an integrative review. Biomed Instrum Technol 2012;46:268–77.

Source: PubMed

Sponsors and Collaborators

Medical Conditions

Drug Interventions

3
Subscribe