Delirium prediction in the intensive care unit: comparison of two delirium prediction models

Annelies Wassenaar, Lisette Schoonhoven, John W Devlin, Frank M P van Haren, Arjen J C Slooter, Philippe G Jorens, Mathieu van der Jagt, Koen S Simons, Ingrid Egerod, Lisa D Burry, Albertus Beishuizen, Joaquim Matos, A Rogier T Donders, Peter Pickkers, Mark van den Boogaard, Annelies Wassenaar, Lisette Schoonhoven, John W Devlin, Frank M P van Haren, Arjen J C Slooter, Philippe G Jorens, Mathieu van der Jagt, Koen S Simons, Ingrid Egerod, Lisa D Burry, Albertus Beishuizen, Joaquim Matos, A Rogier T Donders, Peter Pickkers, Mark van den Boogaard

Abstract

Background: Accurate prediction of delirium in the intensive care unit (ICU) may facilitate efficient use of early preventive strategies and stratification of ICU patients by delirium risk in clinical research, but the optimal delirium prediction model to use is unclear. We compared the predictive performance and user convenience of the prediction model for delirium (PRE-DELIRIC) and early prediction model for delirium (E-PRE-DELIRIC) in ICU patients and determined the value of a two-stage calculation.

Methods: This 7-country, 11-hospital, prospective cohort study evaluated consecutive adults admitted to the ICU who could be reliably assessed for delirium using the Confusion Assessment Method-ICU or the Intensive Care Delirium Screening Checklist. The predictive performance of the models was measured using the area under the receiver operating characteristic curve. Calibration was assessed graphically. A physician questionnaire evaluated user convenience. For the two-stage calculation we used E-PRE-DELIRIC immediately after ICU admission and updated the prediction using PRE-DELIRIC after 24 h.

Results: In total 2178 patients were included. The area under the receiver operating characteristic curve was significantly greater for PRE-DELIRIC (0.74 (95% confidence interval 0.71-0.76)) compared to E-PRE-DELIRIC (0.68 (95% confidence interval 0.66-0.71)) (z score of - 2.73 (p < 0.01)). Both models were well-calibrated. The sensitivity improved when using the two-stage calculation in low-risk patients. Compared to PRE-DELIRIC, ICU physicians (n = 68) rated the E-PRE-DELIRIC model more feasible.

Conclusions: While both ICU delirium prediction models have moderate-to-good performance, the PRE-DELIRIC model predicts delirium better. However, ICU physicians rated the user convenience of E-PRE-DELIRIC superior to PRE-DELIRIC. In low-risk patients the delirium prediction further improves after an update with the PRE-DELIRIC model after 24 h.

Trial registration: ClinicalTrials.gov, NCT02518646 . Registered on 21 July 2015.

Keywords: Adult; Clinical prediction; Critical illness; Delirium; Intensive care unit.

Conflict of interest statement

Ethics approval and consent to participate

The study was reviewed by the Medical Research Ethics Committee (MREC) Arnhem-Nijmegen region, The Netherlands (CMO Region Arnhem-Nijmegen, no. 2015–1782) and the local MRECs/ Institutional Review Boards (IRBs)/ Research Ethics Boards (REBs) of the participating ICUs. Each institutional MREC/IRB/REB waived the need for informed consent. Only de-identified data were entered into the study database and used for analysis.

Competing interests

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Figures

Fig. 1
Fig. 1
Study flowchart. E-PRE-DELIRIC, early prediction model for delirium in ICU patients
Fig. 2
Fig. 2
AUROC for the early prediction model for delirium in ICU patients (E-PRE-DELIRIC) and the prediction model for delirium in ICU patients (PRE-DELIRIC)
Fig. 3
Fig. 3
Calibration plot for the early prediction model for delirium in ICU patients (E-PRE-DELIRIC) and the prediction model for delirium in ICU patients (PRE-DELIRIC)

References

    1. APA . Diagnostic and statistical manual of mental disorders, fifth edition. 5. Washington, D.C.: American Psychiatric Association (APA); 2013.
    1. Milbrandt EB, Deppen S, Harrison PL, et al. Costs associated with delirium in mechanically ventilated patients. Crit Care Med. 2004;32(4):955–962. doi: 10.1097/01.CCM.0000119429.16055.92.
    1. Salluh JI, Wang H, Schneider EB, et al. Outcome of delirium in critically ill patients: systematic review and meta-analysis. BMJ. 2015;350:h2538. doi: 10.1136/bmj.h2538.
    1. Mistraletti G, Pelosi P, Mantovani ES, Berardino M, Gregoretti C. Delirium: clinical approach and prevention. Best Pract Res Clin Anaesthesiol. 2012;26(3):311–326. doi: 10.1016/j.bpa.2012.07.001.
    1. Altman DG, Royston P. What do we mean by validating a prognostic model? Stat Med. 2000;19(4):453–473. doi: 10.1002/(SICI)1097-0258(20000229)19:4<453::AID-SIM350>;2-5.
    1. Giannini A, Garrouste-Orgeas M, Latour JM. What’s new in ICU visiting policies: can we continue to keep the doors closed? Intensive Care Med. 2014;40(5):730–733. doi: 10.1007/s00134-014-3267-y.
    1. Ely EW. The ABCDEF bundle: science and philosophy of how ICU liberation serves patients and families. Crit Care Med. 2017;45(2):321–330. doi: 10.1097/CCM.0000000000002175.
    1. Morandi A, Piva S, Ely EW, et al. Worldwide survey of the “Assessing Pain, Both Spontaneous Awakening and Breathing Trials, Choice of Drugs, Delirium Monitoring/Management, Early Exercise/Mobility, and Family Empowerment” (ABCDEF) bundle. Crit Care Med. 2017;45(11):e1111–e1122. doi: 10.1097/CCM.0000000000002640.
    1. van den Boogaard M, Pickkers P, Slooter AJ, et al. Development and validation of PRE-DELIRIC (PREdiction of DELIRium in ICu patients) delirium prediction model for intensive care patients: observational multicentre study. BMJ. 2012;344:e420. doi: 10.1136/bmj.e420.
    1. van den Boogaard M, Schoonhoven L, Maseda E, et al. Recalibration of the delirium prediction model for ICU patients (PRE-DELIRIC): a multinational observational study. Intensive Care Med. 2014;40(3):361–369. doi: 10.1007/s00134-013-3202-7.
    1. Wassenaar A, van den Boogaard M, van Achterberg T, et al. Multinational development and validation of an early prediction model for delirium in ICU patients. Intensive Care Med. 2015;41(6):1048–1056. doi: 10.1007/s00134-015-3777-2.
    1. Ely EW, Shintani A, Truman B, et al. Delirium as a predictor of mortality in mechanically ventilated patients in the intensive care unit. JAMA. 2004;291(14):1753–1762. doi: 10.1001/jama.291.14.1753.
    1. Serafim RB, Dutra MF, Saddy F, et al. Delirium in postoperative nonventilated intensive care patients: risk factors and outcomes. Ann Intensive Care. 2012;2(1):51. doi: 10.1186/2110-5820-2-51.
    1. Moons KG, Royston P, Vergouwe Y, Grobbee DE, Altman DG. Prognosis and prognostic research: what, why, and how? BMJ. 2009;338:b375. doi: 10.1136/bmj.b375.
    1. Castor Electronic Data Capture. Amsterdam, the Netherlands: Ciwit BV; 2017.
    1. Knaus WA, Draper EA, Wagner DP, Zimmerman JE. APACHE II: a severity of disease classification system. Crit Care Med. 1985;13(10):818–829. doi: 10.1097/00003246-198510000-00009.
    1. Vincent JL, Moreno R, Takala J, et al. The SOFA (Sepsis-related Organ Failure Assessment) score to describe organ dysfunction/failure. On behalf of the Working Group on Sepsis-Related Problems of the European Society of Intensive Care Medicine. Intensive Care Med. 1996;22(7):707–710. doi: 10.1007/BF01709751.
    1. Ely EW, Inouye SK, Bernard GR, et al. Delirium in mechanically ventilated patients: validity and reliability of the confusion assessment method for the intensive care unit (CAM-ICU) JAMA. 2001;286(21):2703–2710. doi: 10.1001/jama.286.21.2703.
    1. Bergeron N, Dubois MJ, Dumont M, Dial S, Skrobik Y. Intensive Care Delirium Screening Checklist: evaluation of a new screening tool. Intensive Care Med. 2001;27(5):859–864. doi: 10.1007/s001340100909.
    1. Moons KG, Kengne AP, Woodward M, et al. Risk prediction models: I. Development, internal validation, and assessing the incremental value of a new (bio)marker. Heart. 2012;98(9):683–690. doi: 10.1136/heartjnl-2011-301246.
    1. Riker RR, Picard JT, Fraser GL. Prospective evaluation of the Sedation-Agitation Scale for adult critically ill patients. Crit Care Med. 1999;27(7):1325–1329. doi: 10.1097/00003246-199907000-00022.
    1. Sessler CN, Gosnell MS, Grap MJ, et al. The Richmond Agitation-Sedation Scale: validity and reliability in adult intensive care unit patients. Am J Respir Crit Care Med. 2002;166(10):1338–1344. doi: 10.1164/rccm.2107138.
    1. van Eijk MM, van den Boogaard M, van Marum RJ, et al. Routine use of the confusion assessment method for the intensive care unit: a multicenter study. Am J Respir Crit Care Med. 2011;184(3):340–344. doi: 10.1164/rccm.201101-0065OC.
    1. Collins GS, Ogundimu EO, Altman DG. Sample size considerations for the external validation of a multivariable prognostic model: a resampling study. Stat Med. 2016;35(2):214–226. doi: 10.1002/sim.6787.
    1. Steyerberg EW. Clinical prediction models: a practical approach to development, validation, and updating. New York: Springer Science+Business Media; 2009.
    1. Hanley JA, McNeil BJ. A method of comparing the areas under receiver operating characteristic curves derived from the same cases. Radiology. 1983;148(3):839–843. doi: 10.1148/radiology.148.3.6878708.
    1. R Core Team. R: A language and environment for statistical computing. Vienna: 2017. /
    1. Grol R, Wensing M. What drives change? Barriers to and incentives for achieving evidence-based practice. Med J Aust. 2004;180(6 Suppl):S57–S60.
    1. Grol R, Wensing M: Implementation; Effective improvement of patient care. (In Dutch: Implementatie; Effectieve Verbeteringen Van Patiëntenzorg. Amsterdam: Bohn Stafleu en van Loghum; 2016.
    1. Barr J, Fraser GL, Puntillo K, et al. Clinical practice guidelines for the management of pain, agitation, and delirium in adult patients in the intensive care unit. Crit Care Med. 2013;41(1):263–306. doi: 10.1097/CCM.0b013e3182783b72.
    1. NICE . DELIRIUM: diagnosis, prevention and management. London: The National Institute for Health and Clinical Excellence; 2010.
    1. Siddiqi N. Predicting delirium: time to use delirium risk scores in routine practice? Age Ageing. 2016;45(1):9–10. doi: 10.1093/ageing/afv183.
    1. Altman DG, Vergouwe Y, Royston P, Moons KG. Prognosis and prognostic research: validating a prognostic model. BMJ. 2009;338:b605. doi: 10.1136/bmj.b605.
    1. Siontis GC, Tzoulaki I, Castaldi PJ, Ioannidis JP. External validation of new risk prediction models is infrequent and reveals worse prognostic discrimination. J Clin Epidemiol. 2015;68(1):25–34. doi: 10.1016/j.jclinepi.2014.09.007.
    1. Lee A, Mu JL, Joynt GM, et al. Risk prediction models for delirium in the intensive care unit after cardiac surgery: a systematic review and independent external validation. Br J Anaesth. 2017;118(3):391–399. doi: 10.1093/bja/aew476.
    1. van den Boogaard M, Schoonhoven L, van der Hoeven JG, van Achterberg T, Pickkers P. Incidence and short-term consequences of delirium in critically ill patients: a prospective observational cohort study. Int J Nurs Stud. 2012;49(7):775–783. doi: 10.1016/j.ijnurstu.2011.11.016.
    1. Peterson JF, Pun BT, Dittus RS, et al. Delirium and its motoric subtypes: a study of 614 critically ill patients. J Am Geriatr Soc. 2006;54(3):479–484. doi: 10.1111/j.1532-5415.2005.00621.x.
    1. Woien H, Balsliemke S, Stubhaug A. The incidence of delirium in Norwegian intensive care units; deep sedation makes assessment difficult. Acta Anaesthesiol Scand. 2013;57(3):294–302. doi: 10.1111/j.1399-6576.2012.02793.x.
    1. Simon R, Altman DG. Statistical aspects of prognostic factor studies in oncology. Br J Cancer. 1994;69(6):979–985. doi: 10.1038/bjc.1994.192.
    1. Dykema J, Jones NR, Piche T, Stevenson J. Surveying clinicians by web: current issues in design and administration. Eval Health Prof. 2013;36(3):352–381. doi: 10.1177/0163278713496630.
    1. Moons KG, Altman DG, Vergouwe Y, Royston P. Prognosis and prognostic research: application and impact of prognostic models in clinical practice. BMJ. 2009;338:b606. doi: 10.1136/bmj.b606.
    1. Gusmao-Flores D, Salluh JI, Chalhub RA, Quarantini LC. The Confusion Assessment Method for the Intensive Care Unit (CAM-ICU) and Intensive Care Delirium Screening Checklist (ICDSC) for the diagnosis of delirium: a systematic review and meta-analysis of clinical studies. Crit Care. 2012;16(4):R115. doi: 10.1186/cc11407.

Source: PubMed

スポンサーと協力者

医学的状態

薬物療法

3
購読する