CCEDRRN COVID-19 Infection Score (CCIS): development and validation in a Canadian cohort of a clinical risk score to predict SARS-CoV-2 infection in patients presenting to the emergency department with suspected COVID-19

Andrew D McRae, Corinne M Hohl, Rhonda Rosychuk, Shabnam Vatanpour, Gelareh Ghaderi, Patrick M Archambault, Steven C Brooks, Ivy Cheng, Philip Davis, Jake Hayward, Eddy Lang, Robert Ohle, Brian Rowe, Michelle Welsford, Krishan Yadav, Laurie J Morrison, Jeffrey Perry, Canadian COVID-19 Emergency Department Rapid Response Network (CCEDRRN) investigators for the Network of Canadian Emergency Researchers and the Canadian Critical Care Trials Group, Patrick Fok, Hana Wiemer, Samuel Campbell, Kory Arsenault, Tara Dahn, Kavish Chandra, Patrick Archambault, Joel Turner, Éric Mercier, Greg Clark, Sébastien Robert, Raoul Daoust, Laurie Morrison, Steven Brooks, Ivy Cheng, Krishan Yadav, Michelle Welsford, Rob Ohle, Justin Yan, Rohit Mohindra, Megan Landes, Tomislav Jelic, Ankit Kapur, Phil Davis, Andrew McRae, Brian Rowe, Katie Lin, Stephanie VandenBerg, Jake Hayward, Jaspreet Khangura, Corinne Hohl, Daniel Ting, Maja Stachura, Frank Scheuermeyer, Balijeet Braar, Craig Murray, John Taylor, Ian Martin, Sean Wormsbecker, Matt Bouchard, Lee Graham, Andrew D McRae, Corinne M Hohl, Rhonda Rosychuk, Shabnam Vatanpour, Gelareh Ghaderi, Patrick M Archambault, Steven C Brooks, Ivy Cheng, Philip Davis, Jake Hayward, Eddy Lang, Robert Ohle, Brian Rowe, Michelle Welsford, Krishan Yadav, Laurie J Morrison, Jeffrey Perry, Canadian COVID-19 Emergency Department Rapid Response Network (CCEDRRN) investigators for the Network of Canadian Emergency Researchers and the Canadian Critical Care Trials Group, Patrick Fok, Hana Wiemer, Samuel Campbell, Kory Arsenault, Tara Dahn, Kavish Chandra, Patrick Archambault, Joel Turner, Éric Mercier, Greg Clark, Sébastien Robert, Raoul Daoust, Laurie Morrison, Steven Brooks, Ivy Cheng, Krishan Yadav, Michelle Welsford, Rob Ohle, Justin Yan, Rohit Mohindra, Megan Landes, Tomislav Jelic, Ankit Kapur, Phil Davis, Andrew McRae, Brian Rowe, Katie Lin, Stephanie VandenBerg, Jake Hayward, Jaspreet Khangura, Corinne Hohl, Daniel Ting, Maja Stachura, Frank Scheuermeyer, Balijeet Braar, Craig Murray, John Taylor, Ian Martin, Sean Wormsbecker, Matt Bouchard, Lee Graham

Abstract

Objectives: To develop and validate a clinical risk score that can accurately quantify the probability of SARS-CoV-2 infection in patients presenting to an emergency department without the need for laboratory testing.

Design: Cohort study of participants in the Canadian COVID-19 Emergency Department Rapid Response Network (CCEDRRN) registry. Regression models were fitted to predict a positive SARS-CoV-2 test result using clinical and demographic predictors, as well as an indicator of local SARS-CoV-2 incidence.

Setting: 32 emergency departments in eight Canadian provinces.

Participants: 27 665 consecutively enrolled patients who were tested for SARS-CoV-2 in participating emergency departments between 1 March and 30 October 2020.

Main outcome measures: Positive SARS-CoV-2 nucleic acid test result within 14 days of an index emergency department encounter for suspected COVID-19 disease.

Results: We derived a 10-item CCEDRRN COVID-19 Infection Score using data from 21 743 patients. This score included variables from history and physical examination and an indicator of local disease incidence. The score had a c-statistic of 0.838 with excellent calibration. We externally validated the rule in 5295 patients. The score maintained excellent discrimination and calibration and had superior performance compared with another previously published risk score. Score cut-offs were identified that can rule-in or rule-out SARS-CoV-2 infection without the need for nucleic acid testing with 97.4% sensitivity (95% CI 96.4 to 98.3) and 95.9% specificity (95% CI 95.5 to 96.0).

Conclusions: The CCEDRRN COVID-19 Infection Score uses clinical characteristics and publicly available indicators of disease incidence to quantify a patient's probability of SARS-CoV-2 infection. The score can identify patients at sufficiently high risk of SARS-CoV-2 infection to warrant isolation and empirical therapy prior to test confirmation while also identifying patients at sufficiently low risk of infection that they may not need testing.

Trial registration number: NCT04702945.

Keywords: COVID-19; accident & emergency medicine; epidemiology.

Conflict of interest statement

Competing interests: ADM, CMH, RR, PMA, SCB, IC, PD, JH, EL, RO, BR, MW, KY, LJM and JP are coinvestigators on the funding sources listed in the funding statement and have no additional competing interests. GG and SV have no competing interests.

© Author(s) (or their employer(s)) 2021. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Figures

Figure 1
Figure 1
Flow diagram of patients through the .
Figure 2
Figure 2
Distribution and performance of the CCEDRRN COVID-19 infection score in the derivation cohort (left panel) and validation cohorts (right panel): (A) distribution of the score, (B) observed in-hospital mortality across the range of the score, (C) predicted versus observed probability of in-hospital mortality and (D) receiver operating characteristic curve with area under the curve and associated 95% CI. CCEDRRN, Canadian COVID-19 Emergency Department Rapid Response Network.

References

    1. World Health Organization . WHO Coronavirus Dashboard, 2021. Available: [Accessed 19 Nov 2021].
    1. Update to living systematic review on prediction models for diagnosis and prognosis of covid-19. BMJ 2021;372:n236. 10.1136/bmj.n236
    1. Kline JA, Camargo CA, Courtney DM, et al. . Clinical prediction rule for SARS-CoV-2 infection from 116 U.S. emergency departments 2-22-2021. PLoS One 2021;16:e0248438. 10.1371/journal.pone.0248438
    1. McDonald SA, Medford RJ, Basit MA, et al. . Derivation with internal validation of a multivariable predictive model to predict COVID-19 test results in emergency department patients. Acad Emerg Med 2021;28:206–14. 10.1111/acem.14182
    1. Kurstjens S, van der Horst A, Herpers R, et al. . Rapid identification of SARS-CoV-2-infected patients at the emergency department using routine testing. Clin Chem Lab Med 2020;58:1587–93. 10.1515/cclm-2020-0593
    1. Lippi G, Henry BM, Hoehn J, et al. . Validation of the Corona-Score for rapid identification of SARS-CoV-2 infections in patients seeking emergency department care in the United States. Clin Chem Lab Med 2020;58:e311–3. 10.1515/cclm-2020-1121
    1. Sung J, Choudry N, Bachour R. Development and validation of a simple risk score for diagnosing COVID-19 in the emergency room. Epidemiol Infect 2020;148:e273. 10.1017/S0950268820002769
    1. Schwab P, DuMont Schütte A, Dietz B, et al. . Clinical predictive models for COVID-19: systematic study. J Med Internet Res 2020;22:e21439. 10.2196/21439
    1. Plante TB, Blau AM, Berg AN, et al. . Development and external validation of a machine learning tool to rule out COVID-19 among adults in the emergency department using routine blood tests: a large, multicenter, real-world study. J Med Internet Res 2020;22:e24048. 10.2196/24048
    1. Soltan AAS, Kouchaki S, Zhu T, et al. . Rapid triage for COVID-19 using routine clinical data for patients attending Hospital: development and prospective validation of an artificial intelligence screening test. Lancet Digit Health 2021;3:e78–87. 10.1016/S2589-7500(20)30274-0
    1. Cabitza F, Campagner A, Ferrari D, et al. . Development, evaluation, and validation of machine learning models for COVID-19 detection based on routine blood tests. Clin Chem Lab Med 2020;59:421–31. 10.1515/cclm-2020-1294
    1. Jehi L, Ji X, Milinovich A, et al. . Individualizing risk prediction for positive coronavirus disease 2019 testing: results from 11,672 patients. Chest 2020;158:1364–75. 10.1016/j.chest.2020.05.580
    1. Joshi RP, Pejaver V, Hammarlund NE, et al. . A predictive tool for identification of SARS-CoV-2 PCR-negative emergency department patients using routine test results. J Clin Virol 2020;129:104502. 10.1016/j.jcv.2020.104502
    1. Formica V, Minieri M, Bernardini S, et al. . Complete blood count might help to identify subjects with high probability of testing positive to SARS-CoV-2. Clin Med 2020;20:e114–9. 10.7861/clinmed.2020-0373
    1. Hohl CM, Rosychuk RJ, McRae AD, et al. . Development of the Canadian COVID-19 emergency department rapid response network population-based registry: a methodology study. CMAJ Open 2021;9:E261–70. 10.9778/cmajo.20200290
    1. Moons KGM, Altman DG, Reitsma JB, et al. . Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): explanation and elaboration. Ann Intern Med 2015;162:W1–73. 10.7326/M14-0698
    1. WHO . Clinical characterisation protocol (CCP), version 3.1/3.2. Oxford University, 2020.
    1. ISARIC/WHO Clinical Characterisation Protocol for Severe Emerging Infections [COVID-19] [UPH]. Health Res. Available: [Accessed 4 Jun 2021].
    1. Health regional Archive (public view), 2019. Available: [Accessed 4 Jun 2021].
    1. Visit 6 manual 17: ARIC neurocognitive exam. Available:
    1. Leening MJG, Vedder MM, Witteman JCM, et al. . Net reclassification improvement: computation, interpretation, and controversies: a literature review and clinician's guide. Ann Intern Med 2014;160:122–31. 10.7326/M13-1522
    1. Kerr KF, Wang Z, Janes H, et al. . Net reclassification indices for evaluating risk prediction instruments: a critical review. Epidemiology 2014;25:114–21. 10.1097/EDE.0000000000000018
    1. R Core Team . R: a language and environment for statistical computing. Vienna, Austria: R foundation for statistical computing, 2020.
    1. Harrell . rms: regression modeling Strategies., version 6.2-0, 2021. Available:
    1. Nevel AE, Kline JA. Inter-Rater reliability and prospective validation of a clinical prediction rule for SARS-CoV-2 infection. Acad Emerg Med 2021;28:Epub ahead of print:761–7. 10.1111/acem.14309
    1. Vyas DA, Eisenstein LG, Jones DS. Hidden in Plain Sight - Reconsidering the Use of Race Correction in Clinical Algorithms. N Engl J Med 2020;383:874–82. 10.1056/NEJMms2004740

Source: PubMed

3
구독하다