Prediction of Clinical Deterioration in Hospitalized Adult Patients with Hematologic Malignancies Using a Neural Network Model

Scott B Hu, Deborah J L Wong, Aditi Correa, Ning Li, Jane C Deng, Scott B Hu, Deborah J L Wong, Aditi Correa, Ning Li, Jane C Deng

Abstract

Introduction: Clinical deterioration (ICU transfer and cardiac arrest) occurs during approximately 5-10% of hospital admissions. Existing prediction models have a high false positive rate, leading to multiple false alarms and alarm fatigue. We used routine vital signs and laboratory values obtained from the electronic medical record (EMR) along with a machine learning algorithm called a neural network to develop a prediction model that would increase the predictive accuracy and decrease false alarm rates.

Design: Retrospective cohort study.

Setting: The hematologic malignancy unit in an academic medical center in the United States.

Patient population: Adult patients admitted to the hematologic malignancy unit from 2009 to 2010.

Intervention: None.

Measurements and main results: Vital signs and laboratory values were obtained from the electronic medical record system and then used as predictors (features). A neural network was used to build a model to predict clinical deterioration events (ICU transfer and cardiac arrest). The performance of the neural network model was compared to the VitalPac Early Warning Score (ViEWS). Five hundred sixty five consecutive total admissions were available with 43 admissions resulting in clinical deterioration. Using simulation, the neural network outperformed the ViEWS model with a positive predictive value of 82% compared to 24%, respectively.

Conclusion: We developed and tested a neural network-based prediction model for clinical deterioration in patients hospitalized in the hematologic malignancy unit. Our neural network model outperformed an existing model, substantially increasing the positive predictive value, allowing the clinician to be confident in the alarm raised. This system can be readily implemented in a real-time fashion in existing EMR systems.

Conflict of interest statement

Competing Interests: The authors have declared that no competing interests exist.

Figures

Fig 1. Representative neural network model demonstrating…
Fig 1. Representative neural network model demonstrating a simplified version of the neural network used to predict clinical deterioration in hematologic malignancy patients.
The features (predictors) are listed on the left and represented by the circles which are the input nodes. The middle layer of circles represent the hidden layer with the circles representing the hidden nodes. The far right single circle represents the output node that serves to predict clinical deterioration from the neural network.

References

    1. Churpek MM, Yuen TC, Park SY, Gibbons R, Edelson DP. Using Electronic Health Record Data to Develop and Validate a Prediction Model for Adverse Outcomes in the Wards. Critical care medicine. 2013. Epub 2013/11/20. 10.1097/CCM.0000000000000038
    1. Liu V, Kipnis P, Rizk NW, Escobar GJ. Adverse outcomes associated with delayed intensive care unit transfers in an integrated healthcare system. Journal of hospital medicine: an official publication of the Society of Hospital Medicine. 2012;7(3):224–30. Epub 2011/11/01. 10.1002/jhm.964 .
    1. Mardini L, Lipes J, Jayaraman D. Adverse outcomes associated with delayed intensive care consultation in medical and surgical inpatients. Journal of critical care. 2012;27(6):688–93. Epub 2012/06/16. 10.1016/j.jcrc.2012.04.011 .
    1. Young MP, Gooder VJ, McBride K, James B, Fisher ES. Inpatient transfers to the intensive care unit: delays are associated with increased mortality and morbidity. Journal of general internal medicine. 2003;18(2):77–83. Epub 2003/01/25.
    1. Mokart D, Lambert J, Schnell D, Fouche L, Rabbat A, Kouatchet A, et al. Delayed intensive care unit admission is associated with increased mortality in patients with cancer with acute respiratory failure. Leukemia & lymphoma. 2013;54(8):1724–9. Epub 2012/11/29. 10.3109/10428194.2012.753446 .
    1. Churpek MM, Yuen TC, Edelson DP. Risk stratification of hospitalized patients on the wards. Chest. 2013;143(6):1758–65. Epub 2013/06/05. 10.1378/chest.12-1605
    1. Churpek MM, Yuen TC, Edelson DP. Predicting clinical deterioration in the hospital: the impact of outcome selection. Resuscitation. 2013;84(5):564–8. Epub 2012/10/02. 10.1016/j.resuscitation.2012.09.024
    1. Prytherch DR, Smith GB, Schmidt PE, Featherstone PI. ViEWS—Towards a national early warning score for detecting adult inpatient deterioration. Resuscitation. 2010;81(8):932–7. Epub 2010/07/20. 10.1016/j.resuscitation.2010.04.014 .
    1. Smith GB, Prytherch DR, Meredith P, Schmidt PE, Featherstone PI. The ability of the National Early Warning Score (NEWS) to discriminate patients at risk of early cardiac arrest, unanticipated intensive care unit admission, and death. Resuscitation. 2013;84(4):465–70. Epub 2013/01/09. 10.1016/j.resuscitation.2012.12.016 .
    1. Subbe CP, Kruger M, Rutherford P, Gemmel L. Validation of a modified Early Warning Score in medical admissions. QJM: monthly journal of the Association of Physicians. 2001;94(10):521–6. Epub 2001/10/06. .
    1. Finlay GD, Rothman MJ, Smith RA. Measuring the modified early warning score and the Rothman Index: Advantages of utilizing the electronic medical record in an early warning system. Journal of hospital medicine: an official publication of the Society of Hospital Medicine. 2013. Epub 2013/12/21. 10.1002/jhm.2132 .
    1. Morgan R WF, Wright M. An early warning scoring system for detecting developing critical illness. Clinical Intensive Care. 1997;8:100.
    1. Smith GB, Prytherch DR, Schmidt PE, Featherstone PI. Review and performance evaluation of aggregate weighted 'track and trigger' systems. Resuscitation. 2008;77(2):170–9. Epub 2008/02/06. 10.1016/j.resuscitation.2007.12.004 .
    1. Cvach M. Monitor alarm fatigue: an integrative review. Biomedical instrumentation & technology / Association for the Advancement of Medical Instrumentation. 2012;46(4):268–77. Epub 2012/07/31. 10.2345/0899-8205-46.4.268 .
    1. Rothman MJ, Rothman SI, Beals Jt. Development and validation of a continuous measure of patient condition using the Electronic Medical Record. Journal of biomedical informatics. 2013;46(5):837–48. Epub 2013/07/09. 10.1016/j.jbi.2013.06.011 .
    1. Tsien CL, Fackler JC. Poor prognosis for existing monitors in the intensive care unit. Critical care medicine. 1997;25(4):614–9. Epub 1997/04/01. .
    1. Sendelbach S, Funk M. Alarm fatigue: a patient safety concern. AACN advanced critical care. 2013;24(4):378–86; quiz 87–8. Epub 2013/10/25. 10.1097/NCI.0b013e3182a903f9 .
    1. Alam N HE, van Tienhoven AJ, van de Ven PM, Jansma EP, Nanayakkara PW. The impact of the use of the Early Warning Score (EWS) on patient outcomes: a systematic review. Resuscitation. 2014;85(5):587–94. 10.1016/j.resuscitation.2014.01.013
    1. Manning T, Sleator RD, Walsh P. Biologically inspired intelligent decision making: A commentary on the use of artificial neural networks in bioinformatics. Bioengineered. 2013;5(2). Epub 2013/12/18. .
    1. Penny W, Frost D. Neural networks in clinical medicine. Medical decision making: an international journal of the Society for Medical Decision Making. 1996;16(4):386–98. Epub 1996/10/01. .
    1. Motta S, Pappalardo F. Mathematical modeling of biological systems. Brief Bioinform. 2013;14(4):411–22. 10.1093/bib/bbs061 .
    1. Pappalardo AP F., Pennisi M., Castiglione F., Motta S.. Mathematical and Computational Models in Tumor Immunology. Mathematical Modelling of Natural Phenomena. 2012;7(03). Epub 6/6/2012.
    1. Lengline E, Raffoux E, Lemiale V, Darmon M, Canet E, Boissel N, et al. Intensive care unit management of patients with newly diagnosed acute myeloid leukemia with no organ failure. Leukemia & lymphoma. 2012;53(7):1352–9. Epub 2012/01/12. 10.3109/10428194.2011.649752 .
    1. Bergmeir C, Benitez JM. Neural Networks in R Using the Stuttgart Neural Network Simulator: RSNNS. Journal of Statistical Software. 2012;46(7):1–26.
    1. T. S, O. S, N. B, T. L. ROCR: visualizing classifier performance in R. Bioinformatics. 2005;21(20):7881.
    1. Thiel SW, Rosini JM, Shannon W, Doherty JA, Micek ST, Kollef MH. Early prediction of septic shock in hospitalized patients. Journal of hospital medicine: an official publication of the Society of Hospital Medicine. 2010;5(1):19–25. Epub 2010/01/12. 10.1002/jhm.530 .
    1. Kellett J, Kim A. Validation of an abbreviated Vitalpac Early Warning Score (ViEWS) in 75,419 consecutive admissions to a Canadian regional hospital. Resuscitation. 2012;83(3):297–302. Epub 2011/09/13. 10.1016/j.resuscitation.2011.08.022 .
    1. Opio MO, Nansubuga G, Kellett J. Validation of the VitalPAC Early Warning Score (ViEWS) in acutely ill medical patients attending a resource-poor hospital in sub-Saharan Africa. Resuscitation. 2013;84(6):743–6. Epub 2013/02/27. 10.1016/j.resuscitation.2013.02.007 .
    1. Lawless ST. Crying wolf: false alarms in a pediatric intensive care unit. Critical care medicine. 1994;22(6):981–5. Epub 1994/06/01. .
    1. Siebig S, Kuhls S, Imhoff M, Gather U, Scholmerich J, Wrede CE. Intensive care unit alarms—how many do we need? Critical care medicine. 2010;38(2):451–6. Epub 2009/12/18. 10.1097/CCM.0b013e3181cb0888 .
    1. Chambrin MC, Ravaux P, Calvelo-Aros D, Jaborska A, Chopin C, Boniface B. Multicentric study of monitoring alarms in the adult intensive care unit (ICU): a descriptive analysis. Intensive care medicine. 1999;25(12):1360–6. Epub 2000/02/08. .
    1. Bliss JP, Dunn MC. Behavioural implications of alarm mistrust as a function of task workload. Ergonomics. 2000;43(9):1283–300. Epub 2000/10/03. 10.1080/001401300421743 .
    1. Bliss JP, Fallon CK, Nica N. The role of alarm signal duration as a cue for alarm validity. Applied ergonomics. 2007;38(2):191–9. Epub 2006/06/03. 10.1016/j.apergo.2006.03.004 .
    1. Edworthy J, Hellier E. Fewer but better auditory alarms will improve patient safety. Quality & safety in health care. 2005;14(3):212–5. Epub 2005/06/04. 10.1136/qshc.2004.013052
    1. Hravnak M, Devita MA, Clontz A, Edwards L, Valenta C, Pinsky MR. Cardiorespiratory instability before and after implementing an integrated monitoring system. Critical care medicine. 2011;39(1):65–72. Epub 2010/10/12. 10.1097/CCM.0b013e3181fb7b1c
    1. Hravnak M, Edwards L, Clontz A, Valenta C, Devita MA, Pinsky MR. Defining the incidence of cardiorespiratory instability in patients in step-down units using an electronic integrated monitoring system. Archives of internal medicine. 2008;168(12):1300–8. Epub 2008/06/25. 10.1001/archinte.168.12.1300
    1. Yousef K, Pinsky MR, DeVita MA, Sereika S, Hravnak M. Characteristics of patients with cardiorespiratory instability in a step-down unit. American journal of critical care: an official publication, American Association of Critical-Care Nurses. 2012;21(5):344–50. Epub 2012/09/04. 10.4037/ajcc2012797

Source: PubMed

3
S'abonner