Measuring the modified early warning score and the Rothman index: advantages of utilizing the electronic medical record in an early warning system

G Duncan Finlay, Michael J Rothman, Robert A Smith, G Duncan Finlay, Michael J Rothman, Robert A Smith

Abstract

Early detection of an impending cardiac or pulmonary arrest is an important focus for hospitals trying to improve quality of care. Unfortunately, all current early warning systems suffer from high false-alarm rates. Most systems are based on the Modified Early Warning Score (MEWS); 4 of its 5 inputs are vital signs. The purpose of this study was to compare the accuracy of MEWS against the Rothman Index (RI), a patient acuity score based upon summation of excess risk functions that utilize additional data from the electronic medical record (EMR). MEWS and RI scores were computed retrospectively for 32,472 patient visits. Nursing assessments, a category of EMR inputs only used by the RI, showed sharp differences 24 hours before death. Receiver operating characteristic curves for 24-hour mortality demonstrated superior RI performance with c-statistics, 0.82 and 0.93, respectively. At the point where MEWS triggers an alarm, we identified the RI point corresponding to equal sensitivity and found the positive likelihood ratio (LR+) for MEWS was 7.8, and for the RI was 16.9 with false alarms reduced by 53%. At the RI point corresponding to equal LR+, the sensitivity for MEWS was 49% and 77% for RI, capturing 54% more of those patients who will die within 24 hours.

Published 2013. The Authors Journal of Hospital Medicine published by Wiley Periodicals, Inc. on behalf of Society of Hospital Medicine.

Figures

FIG 1
FIG 1
Modified Early Warning Score (MEWS) and Rothman Index (RI). Shown are receiver operating characteristic curves for 24-hour hospital mortality of general medical-surgical unit patients (N = 32,472); area under the curve is MEWS = 0.82, RI = 0.93. (A) An alarm at MEWS = 4 corresponds to the cut point of RI = 16 for similar sensitivity (49.8%, 48.9%), resulting in 1 true positive for 18 false positives by MEWS, and for 8 false positives by RI. (B) Cut point at RI = 30 provides a positive predictive value (PPV) similar to MEWS = 4; these points of PPV (5.3%, 5.2%) result in 49% sensitivity by MEWS and 77% sensitivity by RI.

References

    1. Subbe CP, Kruger M, Rutherford P, Gemmel L. Validation of a modified Early Warning Score in medical admissions. QJM Mon J Assoc Physicians. 2001;94:521–526.
    1. Kyriacos U, Jelsma J, Jordan S. Monitoring vital signs using early warning scoring systems: a review of the literature. J Nurs Manag. 2011;19:311–330.
    1. Kirkland LL, Malinchoc M, O'Byrne M. A clinical deterioration prediction tool for internal medicine patients. Am J Med Qual. 2013;28:135–142.
    1. Prytherch DR, Smith GB, Schmidt PE, Featherstone PI. ViEWS—towards a national early warning score for detecting adult inpatient deterioration. Resuscitation. 2010;81:932–937.
    1. Escobar GJ, LaGuardia JC, Turk BJ, Ragins A, Kipnis P, Draper D. Early detection of impending physiologic deterioration among patients who are not in intensive care: development of predictive models using data from an automated electronic medical record. J Hosp Med. 2012;7:388–395.
    1. Alvarez CA, Clark CA, Zhang S. Predicting out of intensive care unit cardiopulmonary arrest or death using electronic medical record data. BMC Med Inform Decis Mak. 2013;13:28.
    1. Bergstrom N, Braden BJ, Laguzza A, Holman V. The Braden Scale for predicting pressure sore risk. Nurs Res. 1987;36:205–210.
    1. Rothman MJ, Rothman SI, Beals J., IV Development and validation of a continuous measure of patient condition using the electronic medical record. J Biomed Inform. 2013;46:837–848.
    1. Bradley EH, Yakusheva O, Horwitz LI, Sipsma H, Fletcher J. Identifying patients at increased risk for unplanned readmission. Med Care. 2013;51:761–766.
    1. Kelly CA, Upex A, Bateman DN. Comparison of consciousness level assessment in the poisoned patient using the alert/verbal/painful/unresponsive scale and the Glasgow Coma Scale. Ann Emerg Med. 2004;44:108–113.
    1. Cochran W, Cox GM. Experimental Design. New York, NY: John Wiley & Sons; 1950.
    1. Lynn LA, Curry JP. Patterns of unexpected in-hospital deaths: a root cause analysis. Patient Saf Surg. 2011;5:3.
    1. Rothman MJ, Solinger AB, Rothman SI, Finlay GD. Clinical implications and validity of nursing assessments: a longitudinal measure of patient condition from analysis of the Electronic Medical Record. BMJ Open. 2012;2(4):e000646. pii.
    1. Tepas JJ, III, Rimar JM, Hsiao AL, Nussbaum MS. Automated analysis of electronic medical record data reflects the pathophysiology of operative complications. Surgery. 2013;154:918–926.
    1. Subbe CP, Thorpe CM, Hancock C. Not getting better means getting worse—trends in Early Warning Scores suggest that there might only be a short time span to rescue those threatening to fall off a “physiological” cliff? Resuscitation. 2013;84:409–410.

Source: PubMed

Sponsors and Collaborators

Medical Conditions

Drug Interventions

3
Subscribe