The CCEDRRN COVID-19 Mortality Score to predict death among nonpalliative patients with COVID-19 presenting to emergency departments: a derivation and validation study

Corinne M Hohl, Rhonda J Rosychuk, Patrick M Archambault, Fiona O'Sullivan, Murdoch Leeies, Éric Mercier, Gregory Clark, Grant D Innes, Steven C Brooks, Jake Hayward, Vi Ho, Tomislav Jelic, Michelle Welsford, Marco L A Sivilotti, Laurie J Morrison, Jeffrey J Perry, Canadian COVID-19 Emergency Department Rapid Response Network (CCEDRRN) investigators for the Network of Canadian Emergency Researchers and the Canadian Critical Care Trials Group, Corinne M Hohl, Rhonda J Rosychuk, Patrick M Archambault, Fiona O'Sullivan, Murdoch Leeies, Éric Mercier, Gregory Clark, Grant D Innes, Steven C Brooks, Jake Hayward, Vi Ho, Tomislav Jelic, Michelle Welsford, Marco L A Sivilotti, Laurie J Morrison, Jeffrey J Perry, Canadian COVID-19 Emergency Department Rapid Response Network (CCEDRRN) investigators for the Network of Canadian Emergency Researchers and the Canadian Critical Care Trials Group

Abstract

Background: Predicting mortality from COVID-19 using information available when patients present to the emergency department can inform goals-of-care decisions and assist with ethical allocation of critical care resources. The study objective was to develop and validate a clinical score to predict emergency department and in-hospital mortality among consecutive nonpalliative patients with COVID-19; in this study, we define palliative patients as those who do not want resuscitative measures, such as intubation, intensive care unit care or cardiopulmonary resuscitation.

Methods: This derivation and validation study used observational cohort data recruited from 46 hospitals in 8 Canadian provinces participating in the Canadian COVID-19 Emergency Department Rapid Response Network (CCEDRRN). We included adult (age ≥ 18 yr) nonpalliative patients with confirmed COVID-19 who presented to the emergency department of a participating site between Mar. 1, 2020, and Jan. 31, 2021. We randomly assigned hospitals to derivation or validation, and prespecified clinical variables as candidate predictors. We used logistic regression to develop the score in a derivation cohort and examined its performance in predicting emergency department and in-hospital mortality in a validation cohort.

Results: Of 8761 eligible patients, 618 (7.0%) died. The CCEDRRN COVID-19 Mortality Score included age, sex, type of residence, arrival mode, chest pain, severe liver disease, respiratory rate and level of respiratory support. The area under the curve was 0.92 (95% confidence interval [CI] 0.90-0.93) in derivation and 0.92 (95% CI 0.90-0.93) in validation. The score had excellent calibration. These results suggest that scores of 6 or less would categorize patients as being at low risk for in-hospital death, with a negative predictive value of 99.9%. Patients in the low-risk group had an in-hospital mortality rate of 0.1%. Patients with a score of 15 or higher had an observed mortality rate of 81.0%.

Interpretation: The CCEDRRN COVID-19 Mortality Score is a simple score that can be used for level-of-care discussions with patients and in situations of critical care resource constraints to accurately predict death using variables available on emergency department arrival. The score was derived and validated mostly in unvaccinated patients, and before variants of concern were circulating widely and newer treatment regimens implemented in Canada.

Study registration: ClinicalTrials.gov, no. NCT04702945.

Conflict of interest statement

Competing interests: None declared.

© 2022 CMA Impact Inc. or its licensors.

Figures

Figure 1:
Figure 1:
Flow diagram showing included and excluded emergency department (ED) visits. *Numbers total more than 8761 because 51 patients made visits to multiple EDs, some of which were derivation EDs and some of which were validation EDs.
Figure 2:
Figure 2:
Distribution and performance of the CCEDRRN COVID-19 Mortality Score in the validation cohort (left panel) and combined derivation and validation cohorts (right panel): (A) distribution of the score, (B) observed in-hospital mortality rates across the range of the score, (C) predicted versus observed risk of in-hospital death (dashed line represents line of no difference between predicted and observed risk) and (D) receiver-operating characteristic curve with area under the curve and associated 95% confidence interval.

References

    1. Tan E, Song J, Deane AM, et al. Global impact of coronavirus disease 2019 infection requiring admission to the ICU: a systematic review and meta-analysis. Chest. 2021;159:524–36.
    1. Rosenbaum L. Facing COVID-19 in Italy: ethics, logistics, and therapeutics on the epidemic’s front line. N Engl J Med. 2020;382:1873–5.
    1. Satomi E, Souza PMR, Thomé BDC, et al. Fair allocation of scarce medical resources during COVID-19 pandemic: ethical considerations. Einstein (Sao Paulo) 2020;18:eAE5775.
    1. Arya A, Buchman S, Gagnon B, et al. Pandemic palliative care: beyond ventilators and saving lives. CMAJ. 2020;192:E400–4.
    1. Maves RC, Downar J, Dichter JR, et al. ACCP Task Force for Mass Critical Care. Triage of scarce critical care resources in COVID-19 an implementation guide for regional allocation: an expert panel report of the Task Force for Mass Critical Care and the American College of Chest Physicians. Chest. 2020;158:212–25.
    1. Wynants L, Van Calster B, Collins GS, et al. Prediction models for diagnosis and prognosis of COVID-19 infection: systematic review and critical appraisal. BMJ. 2020;369:m1328.
    1. Update to living systematic review on prediction models for diagnosis and prognosis of COVID-19. BMJ. 2021;372:n236.
    1. Gupta RK, Marks M, Samuels THA, et al. UCLH COVID-19 Reporting Group. Systematic evaluation and external validation of 22 prognostic models among hospitalised adults with COVID-19: an observational cohort study. Eur Respir J. 2020;56:2003498.
    1. Knight SR, Ho A, Pius R, et al. ISARIC4C investigators. Risk stratification of patients admitted to hospital with COVID-19 using the ISARIC WHO Clinical Characterisation Protocol: development and validation of the 4C Mortality Score. BMJ. 2020;370:m3339.
    1. Cho SY, Park SS, Song MK, et al. Prognosis score system to predict survival for COVID-19 cases: a Korean nationwide cohort study. J Med Internet Res. 2021;23:e26257.
    1. Berenguer J, Borobia AM, Ryan P, et al. COVID-19@Spain and COVID@ HULP Study Groups. Development and validation of a prediction model for 30-day mortality in hospitalised patients with COVID-19: the COVID-19 SEIMC score. Thorax. 2021;76:920–9.
    1. Nicholson CJ, Wooster L, Sigurslid HH, et al. Estimating risk of mechanical ventilation and in-hospital mortality among adult COVID-19 patients admitted to Mass General Brigham: the VICE and DICE scores. EClinicalMedicine. 2021;33:100765.
    1. Liang W, Liang H, Ou L, et al. Development and validation of a clinical risk score to predict the occurrence of critical illness in hospitalized patients with COVID-19. JAMA Intern Med. 2020;180:1081–9.
    1. Magro B, Zuccaro V, Novelli L, et al. Predicting in-hospital mortality from coronavirus disease 2019: a simple validated app for clinical use. PLoS One. 2021;16:e0245281.
    1. Moons KGM, Altman DG, Reitsma JB, et al. Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): explanation and elaboration. Ann Intern Med. 2015;162:W1–73.
    1. Hohl CM, Rosychuk RJ, McRae AD, et al. Canadian COVID-19 Emergency Department Rapid Response Network investigators and for the Network of Canadian Emergency Researchers and the Canadian Critical Care Trials Group. Development of the Canadian COVID-19 Emergency Department Rapid Response Network population-based registry: a methodology study. CMAJ Open. 2021;9:E261–70.
    1. Hohl CM, Rosychuk RJ, Archambault PM, et al. Derivation and validation of a clinical score to predict death among non-palliative COVID-19 patients presenting to emergency departments: the CCEDRRN COVID Mortality Score. medRxiv. 2021 July 31; doi: 10.1101/2021.07.28.21261283.
    1. McRae AD, Hohl CM, Rosychuk RJ, et al. Canadian COVID-19 Emergency Department Rapid Response Network (CCEDRRN) investigators for the Network of Canadian Emergency Researchers and the Canadian Critical Care Trials Group. Development and validation of a clinical risk score to predict SARS-CoV-2 infection in emergency department patients: the CCEDRRN COVID-19 Infection Score (CCIS) BMJ Open. 2021;11:e055832.
    1. Davis P, Rosychuk RJ, Hau JP, et al. Diagnostic yield of screening for SARS-CoV-2 among patients admitted for alternate diagnoses. medRxiv. 2021 Sept 27; doi: 10.1101/2021.09.23.21264036.
    1. Hohl CM, Rosychuk RJ, Hau JP, et al. Treatments, resource utilization, and outcomes of COVID-19 patients presenting to emergency departments across pandemic waves: an observational study by the Canadian COVID-19 Emergency Department Rapid Response Network (CCEDRRN) medRxiv. 2021 Aug 1; doi: 10.1101/2021.07.30.21261288.
    1. Dugdale CM, Anahtar MN, Chiosi JJ, et al. Clinical, laboratory, and radiologic characteristics of patients with initial false-negative severe acute respiratory syndrome coronavirus 2 nucleic acid amplification test results. Open Forum Infect Dis. 2020;8:ofaa559.
    1. Turgeon AF, Lauzier F, Simard JF, et al. Canadian Critical Care Trials Group. Mortality associated with withdrawal of life-sustaining therapy for patients with severe traumatic brain injury: a Canadian multicentre cohort study. CMAJ. 2011;183:1581–8.
    1. Creutzfeldt CJ, Becker KJ, Weinstein JR, et al. Do-not-attempt-resuscitation orders and prognostic models for intraparenchymal hemorrhage. Crit Care Med. 2011;39:158–62.
    1. Wilkinson D. The self-fulfilling prophecy in intensive care. Theor Med Bioeth. 2009;30:401–10.
    1. Knight SR, Ho A, Pius R, et al. ISARIC4C investigators. Risk stratification of patients admitted to hospital with COVID-19 using the ISARIC WHO Clinical Characterisation Protocol: development and validation of the 4C Mortality Score. BMJ. 2020;370:m3339.
    1. Haimovich AD, Ravindra NG, Stoytchev S, et al. Development and validation of the quick COVID-19 Severity Index: a prognostic took for early clinical decompensation. Ann Emerg Med. 2020;76:442–53.
    1. Chua F, Vancheeswaran R, Draper A, et al. Early prognostication of COVID-19 to guide hospitalisation versus outpatient monitoring using a point-of-test risk prediction score. Thorax. 2021;76:696–703.
    1. Goodacre S, Thomas B, Sutton L, et al. Derivation and validation of a clinical severity score for acutely ill adults with suspected COVID-19: the PRIEST observational cohort study. PLoS One. 2021;16:e0245840.
    1. Magro B, Zuccaro V, Novelli L, et al. Predicting in-hospital mortality from coronavirus disease 2019: a simple validated app for clinical use. PLoS One. 2021;16:e0245281.
    1. Riley RD, Snell KI, Ensor J, et al. Minimum sample size for developing a multivariable prediction model: Part II — binary and time-to-event outcomes. Stat Med. 2019;38:1276–96.
    1. Marshall A, Altman DG, Holder RL, et al. Combining estimates of interest in prognostic modelling studies after multiple imputation: current practice and guidelines. BMC Med Res Methodol. 2009;9:57.
    1. Wood AM, Royston P, White IR. The estimation and use of predictions for the assessment of model performance using large samples with multiply imputed data. Biom J. 2015;57:614–32.
    1. King JT, Jr, Yoon JS, Rentsch CT, et al. Development and validation of a 30-day mortality index based on pre-existing medical administrative data from 13,323 COVID-19 patients: the Veterans Health Administration COVID-19 (VACO) Index. PLoS One. 2020;15:e0241825.
    1. DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. 1988;44:837–45.
    1. Haimovich AD, Ravindra NG, Stoytchev S, et al. Development and validation of the quick COVID-19 Severity Index: a prognostic tool for early clinical decompensation. Ann Emerg Med. 2020;76:442–53.
    1. Gupta RK, Harrison EM, Ho A, et al. ISARIC4C Investigators. Development and validation of the ISARIC 4C Deterioration model for adults hospitalised with COVID-19: a prospective cohort study. Lancet Respir Med. 2021;9:349–59.
    1. Demirtas CO, Keklikkiran C, Ergenc I, et al. Liver stiffness is associated with disease severity and worse clinical scenarios in coronavirus disease 2019: a prospective transient elastography study. Int J Clin Pract. 2021;75:e14363.
    1. COVID-19 vaccination in Canada. Ottawa: Government of Canada; [accessed 2022 Jan. 24]. updated 2022 Jan. 21. Available:

Source: PubMed

3
구독하다