Assessment of Clinical Criteria for Sepsis: For the Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3)

Abstract

Importance: The Third International Consensus Definitions Task Force defined sepsis as "life-threatening organ dysfunction due to a dysregulated host response to infection." The performance of clinical criteria for this sepsis definition is unknown.

Objective: To evaluate the validity of clinical criteria to identify patients with suspected infection who are at risk of sepsis.

Design, settings, and population: Among 1.3 million electronic health record encounters from January 1, 2010, to December 31, 2012, at 12 hospitals in southwestern Pennsylvania, we identified those with suspected infection in whom to compare criteria. Confirmatory analyses were performed in 4 data sets of 706,399 out-of-hospital and hospital encounters at 165 US and non-US hospitals ranging from January 1, 2008, until December 31, 2013.

Exposures: Sequential [Sepsis-related] Organ Failure Assessment (SOFA) score, systemic inflammatory response syndrome (SIRS) criteria, Logistic Organ Dysfunction System (LODS) score, and a new model derived using multivariable logistic regression in a split sample, the quick Sequential [Sepsis-related] Organ Failure Assessment (qSOFA) score (range, 0-3 points, with 1 point each for systolic hypotension [≤100 mm Hg], tachypnea [≥22/min], or altered mentation).

Main outcomes and measures: For construct validity, pairwise agreement was assessed. For predictive validity, the discrimination for outcomes (primary: in-hospital mortality; secondary: in-hospital mortality or intensive care unit [ICU] length of stay ≥3 days) more common in sepsis than uncomplicated infection was determined. Results were expressed as the fold change in outcome over deciles of baseline risk of death and area under the receiver operating characteristic curve (AUROC).

Results: In the primary cohort, 148,907 encounters had suspected infection (n = 74,453 derivation; n = 74,454 validation), of whom 6347 (4%) died. Among ICU encounters in the validation cohort (n = 7932 with suspected infection, of whom 1289 [16%] died), the predictive validity for in-hospital mortality was lower for SIRS (AUROC = 0.64; 95% CI, 0.62-0.66) and qSOFA (AUROC = 0.66; 95% CI, 0.64-0.68) vs SOFA (AUROC = 0.74; 95% CI, 0.73-0.76; P < .001 for both) or LODS (AUROC = 0.75; 95% CI, 0.73-0.76; P < .001 for both). Among non-ICU encounters in the validation cohort (n = 66 522 with suspected infection, of whom 1886 [3%] died), qSOFA had predictive validity (AUROC = 0.81; 95% CI, 0.80-0.82) that was greater than SOFA (AUROC = 0.79; 95% CI, 0.78-0.80; P < .001) and SIRS (AUROC = 0.76; 95% CI, 0.75-0.77; P < .001). Relative to qSOFA scores lower than 2, encounters with qSOFA scores of 2 or higher had a 3- to 14-fold increase in hospital mortality across baseline risk deciles. Findings were similar in external data sets and for the secondary outcome.

Conclusions and relevance: Among ICU encounters with suspected infection, the predictive validity for in-hospital mortality of SOFA was not significantly different than the more complex LODS but was statistically greater than SIRS and qSOFA, supporting its use in clinical criteria for sepsis. Among encounters with suspected infection outside of the ICU, the predictive validity for in-hospital mortality of qSOFA was statistically greater than SOFA and SIRS, supporting its use as a prompt to consider possible sepsis.

Conflict of interest statement

Conflict of Interest Disclosures: All authors have completed and submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. No other disclosures were reported.

Figures

Figure 1. Accrual of Encounters for Primary Cohort

ED indicates emergency department; ICU, intensive care unit; PACU, postanesthesia care unit.

Figure 2. Distribution of Patient Encounters Over SIRS Criteria and SOFA, LODS, and qSOFA Scores Among ICU Patients and Non-ICU Patients With Suspected Infection in the UPMC Validation Cohort (N = 74 454)

ICU indicates intensive care unit; LODS, Logistic Organ Dysfunction System; qSOFA, quick Sequential [Sepsis-related] Organ Function Assessment; SIRS, systemic inflammatory response syndrome; SOFA, Sequential [Sepsis-related] Organ Function Assessment. The x-axis is the score range, with LODS truncated at 14 points (of 22 points) and SOFA truncated at 16 points (of 24 points) for illustration.

Figure 3. Area Under the Receiver Operating Characteristic Curve and 95% Confidence Intervals for In-Hospital Mortality of Candidate Criteria (SIRS, SOFA, LODS, and qSOFA) Among Suspected Infection Encounters in the UPMC Validation Cohort (N = 74 454)

ICU indicates intensive care unit; LODS, Logistic Organ Dysfunction System; qSOFA, quick Sequential [Sepsis-related] Organ Function Assessment; SIRS, systemic inflammatory response syndrome; SOFA, Sequential [Sepsis-related] Organ Function Assessment. The area under the receiver operating characteristic curve (AUROC) data in the blue-shaded diagonal cells derive from models that include baseline variables plus candidate criteria. For comparison, the AUROC of the baseline model alone is 0.58 (95% CI, 0.57–0.60) in the ICU and 0.69 (95% CI, 0.68–0.70) outside of the ICU. Below the AUROC data cells are P values for comparisons between criteria, while above the AUROC data cells are Cronbach α data (with bootstrap 95% confidence intervals), a measure of agreement.

Figure 4. Fold Change in Rate of In-Hospital Mortality (Log Scale) Comparing Encounters With ≥2 vs

ICU indicates intensive care unit; LODS, Logistic Organ Dysfunction System; qSOFA, quick Sequential [Sepsis-related] Organ Function Assessment; SIRS, systemic inflammatory response syndrome; SOFA, Sequential [Sepsis-related] Organ Function Assessment. Panel A shows ICU encounters comparing fold change for SIRS, SOFA, LODS, and qSOFA. Panel B shows non-ICU encounters. Medians and ranges of baseline risk of in-hospital mortality within decile shown are below the x-axis. Interpretive example: The x-axis divides the cohort into deciles of baseline risk, determined by age, sex, comorbidities, and race/ethnicity. For a young woman with no comorbidities (panel A, decile 2) admitted to the ICU with pneumonia, her chance of dying in the hospital is 10-fold greater if she has 3 SOFA points compared with 1 SOFA point. On the other hand, she has only a small increase in the chance of dying if she has 3 SIRS criteria compared with 1 SIRS criterion. For an older woman with chronic obstructive pulmonary disease admitted to the ward with pneumonia (panel B, decile 6), her chance of dying in the hospital is 7-fold higher if she has 3 qSOFA points compared with 1 qSOFA point. On the other hand, she has only a 3-fold increase in odds of dying if she has 3 SIRS criteria compared with 1 SIRS criterion.