Using Machine Learning to Predict ICU Transfer in Hospitalized COVID-19 Patients

Fu-Yuan Cheng, Himanshu Joshi, Pranai Tandon, Robert Freeman, David L Reich, Madhu Mazumdar, Roopa Kohli-Seth, Matthew Levin, Prem Timsina, Arash Kia, Fu-Yuan Cheng, Himanshu Joshi, Pranai Tandon, Robert Freeman, David L Reich, Madhu Mazumdar, Roopa Kohli-Seth, Matthew Levin, Prem Timsina, Arash Kia

Abstract

Objectives: Approximately 20-30% of patients with COVID-19 require hospitalization, and 5-12% may require critical care in an intensive care unit (ICU). A rapid surge in cases of severe COVID-19 will lead to a corresponding surge in demand for ICU care. Because of constraints on resources, frontline healthcare workers may be unable to provide the frequent monitoring and assessment required for all patients at high risk of clinical deterioration. We developed a machine learning-based risk prioritization tool that predicts ICU transfer within 24 h, seeking to facilitate efficient use of care providers' efforts and help hospitals plan their flow of operations.

Methods: A retrospective cohort was comprised of non-ICU COVID-19 admissions at a large acute care health system between 26 February and 18 April 2020. Time series data, including vital signs, nursing assessments, laboratory data, and electrocardiograms, were used as input variables for training a random forest (RF) model. The cohort was randomly split (70:30) into training and test sets. The RF model was trained using 10-fold cross-validation on the training set, and its predictive performance on the test set was then evaluated.

Results: The cohort consisted of 1987 unique patients diagnosed with COVID-19 and admitted to non-ICU units of the hospital. The median time to ICU transfer was 2.45 days from the time of admission. Compared to actual admissions, the tool had 72.8% (95% CI: 63.2-81.1%) sensitivity, 76.3% (95% CI: 74.7-77.9%) specificity, 76.2% (95% CI: 74.6-77.7%) accuracy, and 79.9% (95% CI: 75.2-84.6%) area under the receiver operating characteristics curve.

Conclusions: A ML-based prediction model can be used as a screening tool to identify patients at risk of imminent ICU transfer within 24 h. This tool could improve the management of hospital resources and patient-throughput planning, thus delivering more effective care to patients hospitalized with COVID-19.

Keywords: COVID-19; critical care; intensive care units; random forest; supervised machine learning.

Conflict of interest statement

The authors declare no conflict of interest.

Figures

Figure 1
Figure 1
Feature vector labeling strategy. (a) Basis for positive labels; (b) and (c) basis for negative labels. V1–3: observations used for creating the feature vector; t0: time of ICU transfer.
Figure 2
Figure 2
Gini importance: top 20 predictive variables.
Figure 3
Figure 3
Receiver operating characteristic curve of the prediction model on training set (left) and test set (right).

References

    1. World Health Organization WHO COVID-19 Dashboard. [(accessed on 9 April 2020)]; Available online:
    1. Fauci A.S., Lane H.C., Redfield R.R. Covid-19—Navigating the Uncharted. N. Engl. J. Med. 2020;382:1268–1269. doi: 10.1056/NEJMe2002387.
    1. CDC COVID-19 Response Team. Severe Outcomes Among Patients with Coronavirus Disease 2019 (COVID-19)—United States, February 12–March 16, 2020. MMWR Morb. Mortal. Wkly. Rep. 2020;69:343–346. doi: 10.15585/mmwr.mm6912e2.
    1. Myers L.C., Parodi S.M., Escobar G.J., Liu V.X. Characteristics of Hospitalized Adults With COVID-19 in an Integrated Health Care System in California. JAMA. 2020 doi: 10.1001/jama.2020.7202.
    1. Richardson S., Hirsch J.S., Narasimhan M., Crawford J.M., McGinn T., Davidson K.W., Cookingham J. Presenting Characteristics, Comorbidities, and Outcomes Among 5700 Patients Hospitalized With COVID-19 in the New York City Area. JAMA. 2020 doi: 10.1001/jama.2020.6775.
    1. Howell E. Active Bed Management by Hospitalists and Emergency Department Throughput. Ann. Intern. Med. 2008;149:804. doi: 10.7326/0003-4819-149-11-200812020-00006.
    1. Moriarty J.P., Schiebel N.E., Johnson M.G., Jensen J.B., Caples S.M., Morlan B.W., Huddleston J.M., Heubner M., Naessens J.M. Evaluating implementation of a rapid response team: Considering alternative outcome measures. Int. J. Qual. Health Care. 2014;26:49–57. doi: 10.1093/intqhc/mzt091.
    1. Verity R., Okell L.C., Dorigatti I., Winskill P., Whittaker C., Imai N., Cori A., Fu H., Baguelin M., Dighe A., et al. Estimates of the severity of coronavirus disease 2019: A model-based analysis. Lancet Infect. Dis. 2020;20:669–677. doi: 10.1016/S1473-3099(20)30243-7.
    1. CDC COVID-19 Response Team Preliminary Estimates of the Prevalence of Selected Underlying Health Conditions Among Patients with Coronavirus Disease 2019—United States, February 12–March 28, 2020. MMWR Morb. Mortal. Wkly. Rep. 2020;69:382–386.
    1. American Hospital Association Fast Facts on U.S. Hospitals, 2020|AHA. 2018 AHA Annual Survey. [(accessed on 9 April 2020)];2020 Available online: .
    1. Halpern N.A., Goldman D.A., Tan K.S., Pastores S.M. Trends in Critical Care Beds and Use Among Population Groups and Medicare and Medicaid Beneficiaries in the United States. Crit. Care Med. 2016;44:1490–1499. doi: 10.1097/CCM.0000000000001722.
    1. Halpern N.A., Tan K.S. United States Resource Availability for COVID-19. [(accessed on 9 April 2020)];2020 Available online: .
    1. National Center for Immunization and Respiratory Diseases (NCIRD) Interim Clinical Guidance for Management of Patients with Confirmed Coronavirus Disease (COVID-19) [(accessed on 9 April 2020)];2020 Available online: .
    1. Tang N., Li D., Wang X., Sun Z. Abnormal coagulation parameters are associated with poor prognosis in patients with novel coronavirus pneumonia. J. Thromb. Haemost. 2020;18:844–847. doi: 10.1111/jth.14768.
    1. Breiman L. Random Forrests. Mach. Learn. 2001;45:5–32. doi: 10.1023/A:1010933404324.
    1. Basu S., Kumbier K., Brown J.B., Yu B. Iterative random forests to discover predictive and stable high-order interactions. Proc. Natl. Acad. Sci. USA. 2018;115:1943–1948. doi: 10.1073/pnas.1711236115.
    1. Batista G.E.A.P.A., Monard M.C. An analysis of four missing data treatment methods for supervised learning. Appl. Artif. Intell. 2003;17:519–533. doi: 10.1080/713827181.
    1. Japkowicz N. The Class Imbalance Problem: Significance and Strategies; Proceedings of the 2000 International Conference on Artificial Intelligence; Las Vegas, NV, USA. 26 June 2000.
    1. Kia A., Timsina P., Joshi H.N., Klang E., Gupta R.R., Freeman R.M., Reich D.L., Tomlinson M.S., Dudley J.T., Mazumdar M., et al. MEWS++: Enhancing the Prediction of Clinical Deterioration in Admitted Patients through a Machine Learning Model. J. Clin. Med. 2020;9:343. doi: 10.3390/jcm9020343.
    1. The Apache Software Foundation. MLlib: Main Guide—Spark 2.3.0 Documentation. [(accessed on 1 June 2020)];2018 Available online: .
    1. Nicastri E., Petrosillo N., Bartoli T.A., Lepore L., Mondi A., Palmieri F., Antinori A. National Institute for the Infectious Diseases “L. Spallanzani” IRCCS. Recommendations for COVID-19 Clinical Management. Infect. Dis. Rep. 2020;12:8543. doi: 10.4081/idr.2020.8543.
    1. Zhou F., Yu T., Du R., Fan G., Liu Y., Liu Z., Xiang J., Wang Y., Song B., Guan L., et al. Clinical course and risk factors for mortality of adult inpatients with COVID-19 in Wuhan, China: A retrospective cohort study. Lancet. 2020;395:1054–1062. doi: 10.1016/S0140-6736(20)30566-3.
    1. Guo T., Fan Y., Chen M., Wu X., Zhang L., He T., Wang H., Wan J., Wang X., Lu Z. Cardiovascular Implications of Fatal Outcomes of Patients With Coronavirus Disease 2019 (COVID-19) JAMA Cardiol. 2020 doi: 10.1001/jamacardio.2020.1017.
    1. Maxim L.D., Niebo R., Utell M.J. Screening tests: A review with examples. Inhal. Toxicol. 2014;26:811–828. doi: 10.3109/08958378.2014.955932.
    1. R Core Team . R: A Language and Environment for Statistical Computing. R Core Team; Vienna, Austria: 2019. [(accessed on 1 June 2020)]. Available online:
    1. Grau J., Grosse I., Keilwagen J. PRROC: Computing and visualizing precision-recall and receiver operating characteristic curves in R. Bioinformatics. 2015;31:2595–2597. doi: 10.1093/bioinformatics/btv153.
    1. Robin X., Turck N., Hainard A., Tiberti N., Lisacek F., Sanchez J.C., Müller M. pROC: An open-source package for R and S+ to analyze and compare ROC curves. BMC Bioinform. 2011;12:77. doi: 10.1186/1471-2105-12-77.
    1. Stevenson M., Nunes T., Sanchez J., Thornton R., Reiczigel J., Robison-Cox J., Sebastiani P. EpiR: An R Package for the Analysis of Epidemiological Data. [(accessed on 1 June 2020)];2013 Available online: .
    1. Cascella M., Rajnik M., Cuomo A., Dulebohn S.C., Di Napoli R. Features, Evaluation and Treatment Coronavirus (COVID-19) StatPearls Publishing; Treasure Island, FL, USA: 2020.
    1. Qin C., Zhou L., Hu Z., Zhang S., Yang S., Tao Y., Xie C., Ma K., Shang K., Tian D.S., et al. Dysregulation of Immune Response in Patients With Coronavirus 2019 (COVID-19) in Wuhan, China. Clin. Infect. Dis. 2020 doi: 10.1093/cid/ciaa248.
    1. Bhatraju P.K., Ghassemieh B.J., Nichols M., Kim R., Jerome K.R., Nalla A.K., Greninger A.L., Pipavath S., Wurfel M.M., Kritek P.A., et al. Covid-19 in Critically Ill Patients in the Seattle Region—Case Series. N. Engl. J. Med. 2020 doi: 10.1056/NEJMoa2004500.
    1. Matthay M.A., Aldrich J.M., Gotts J.E. Treatment for severe acute respiratory distress syndrome from COVID-19. Lancet Respir. Med. 2020 doi: 10.1016/S2213-2600(20)30127-2.
    1. Wang L. C-reactive protein levels in the early stage of COVID-19. Méd. Mal. Infect. 2020 doi: 10.1016/j.medmal.2020.03.007.
    1. Kyriacos U., Jelsma J., Jordan S. Monitoring vital signs using early warning scoring systems: A review of the literature. J. Nurs. Manag. 2011;19:311–330. doi: 10.1111/j.1365-2834.2011.01246.x.
    1. Aird W.C. The Hematologic System as a Marker of Organ Dysfunction in Sepsis. Mayo Clin. Proc. 2003;78:869–881. doi: 10.4065/78.7.869.
    1. Lippi G., South A.M., Henry B.M. Electrolyte Imbalances in Patients with Severe Coronavirus Disease 2019 (COVID-19) Ann. Clin. Biochem. 2020;57:262–265. doi: 10.1177/0004563220922255.
    1. Goyal P., Choi J.J., Pinheiro L.C., Schenck E.J., Chen R., Jabri A., Satlin M.J., Nahid M., Nahid J.B., Hoffman K.L., et al. Clinical Characteristics of Covid-19 in New York City. N. Engl. J. Med. 2020 doi: 10.1056/NEJMc2010419.
    1. Lippi G., Plebani M. Laboratory abnormalities in patients with COVID-2019 infection. Clin. Chem. Lab. Med. 2020 doi: 10.1515/cclm-2020-0198.
    1. Sun Q., Qiu H., Huang M., Yang Y. Lower mortality of COVID-19 by early recognition and intervention: Experience from Jiangsu Province. Ann. Intensive Care. 2020;10:33. doi: 10.1186/s13613-020-00650-2.
    1. Churpek M.M., Wendlandt B., Zadravecz F.J., Adhikari R., Winslow C., Edelson D.P. Association between intensive care unit transfer delay and hospital mortality: A multicenter investigation. J. Hosp. Med. 2016 doi: 10.1002/jhm.2630.
    1. Hu W., Chan C.W., Zubizarreta J.R., Escobar G.J. An examination of early transfers to the ICU based on a physiologic risk score. Manuf. Serv. Oper. Manag. 2018 doi: 10.1287/msom.2017.0658.
    1. Medrzycka-Dabrowska W., Lewandowska K., Kwiecień-Jagus K., Czyz-Szypenbajl K. Sleep deprivation in Intensive Care Unit-systematic review. Open Med. 2018 doi: 10.1515/med-2018-0057.
    1. Kotfis K., Williams Roberson S., Wilson J.E., Dabrowski W., Pun B.T., Ely E.W. COVID-19: ICU delirium management during SARS-CoV-2 pandemic. Crit. Care. 2020 doi: 10.1186/s13054-020-02882-x.

Source: PubMed

スポンサーと協力者

医学的状態

薬物療法

3
購読する