Risk Modelling for Quality Improvement in the Critically Ill: Making Best Use of Routinely Available Data

The aim of the proposed study is to better understand the epidemiology of, risk factors for and consequences of critical illness leading to improvements in the risk models used to underpin national clinical audits for adult general critical care, cardiothoracic critical care and in-hospital cardiac arrest using data linkage with other routinely collected data sources.

Study Overview

• Status: Completed
• Conditions: Critical Illness; Heart Arrest

Detailed Description

Aim: To improve risk models used to underpin national clinical audits for adult general critical care, cardiothoracic critical care and in-hospital cardiac arrest using data linkage with other routinely collected data sources.

Specific objectives are:

1. To improve risk models for adult general critical care by: (1a) developing risk models for mortality at fixed time-points and time-to event outcomes (by data linkage between the CMP and death registrations from ONS); developing risk models for longer term chronic health outcomes of (1b) diabetes (by data linkage between the CMP and the National Diabetes Audit) and (1c) end-stage renal disease (by data linkage between the CMP and the UK Renal Registry); and (1d) developing risk models for subsequent health care utilisation and costs (by data linkage between the CMP and HES)
2. To improve risk models for cardiothoracic critical care by: (2a) enhancing risk factor data (by data linkage with the National Adult Cardiac Surgery Database); (2b) developing risk models for longer term mortality (by data linkage between the CMP and death registrations from ONS); and (2c) developing risk models for subsequent health care utilisation and costs (by data linkage between the CMP and HES)
3. .To improve risk models for in-hospital cardiac arrest by: (3a) enhancing risk factor data (by data linkage between NCAA and HES); (3b) developing risk models for longer term mortality, health care utilisation and costs (by data linkage between NCAA and ONS); (3c) developing risk models for subsequent critical care utilisation (by data linkage between NCAA and CMP); and (3d) developing risk models for subsequent health care utilisation and costs (by data linkage between NCAA, ONS and HES)
4. Immediate translation of the improved risk models into practice through: (4a) adoption into routine comparative outcome reporting for the national clinical audits; and (4b) communication of research output to providers, managers, commissioners, policy makers and academics in critical care

Data collection: The project will utilise high quality clinical data collected for the Case Mix Programme (CMP) and National Cardiac Arrest Audit (NCAA) - the national clinical audits for adult critical care and in-hospital cardiac arrest. These data will be linked with data from the National Diabetes Audit, UK Renal Registry and National Adult Cardiac Surgery Audit, routine administrative data from Hospital Episode Statistics (HES) and death registrations from the Office for National Statistics (ONS).

Data linkage will be undertaken by the HSCIC Data Linkage and Extract Service (DLES) acting as a trusted third party. Identifiers (with no associated clinical data) will be uploaded from each national clinical audit to secure servers at HSCIC. DLES will perform the data linkage and will return a common key that can be used to link all records of the same patient across the datasets. The three national clinical audits external to ICNARC will extract an agreed, pseudonymised dataset for linked records and DLES will extract data from HES and ONS and these datasets will be passed to ICNARC. ICNARC will produce pseudonymised data extracts from the CMP and NCAA and these will be linked to the datasets provided by the national clinical audits and DLES using the common key. In this way, only pseudonymised data will be linked between the multiple data sources.

Data analysis: The following approaches for model development will be applied depending on the outcome and objectives of the analysis:

• Modelling of mortality at fixed time-points (30 days, 90 days, 1 year) using logistic regression.
• Modelling of time-to-event outcomes using standard survival regression methods such as Weibull and Cox regression.
• To handle interval-censored data: Cox proportional hazards models, complementary log-log models using partial likelihood estimation (to permit interval censoring) and discrete-time hazard models.
• To account for both interval censoring of the time-to-onset and competition with death: Cause-specific Cox proportional hazards models and illness-death models will be considered.
• For Objective 3, return of spontaneous circulation (ROSC) for greater than 20 minutes and survival to hospital discharge outcome, multilevel logistic regression with random effects of hospital will be applied.
• To model Hospital resource use and costs post-critical care: multilevel regression model and Log linear regression model.

Risk prediction models will be validated for their discrimination, calibration and overall fit using a panel of measures including: c index; plots of observed against predicted risk; Hosmer-Lemeshow goodness-of-fit statistic; Cox's calibration regression; Shapiro's R, Brier's score and corresponding approximate R2 measures; and reclassification.

• Study Type: Observational
• Enrollment (Actual): 1007147

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: Child; Adult; Older Adult
• Accepts Healthy Volunteers: No
• Genders Eligible for Study: All

• Sampling Method: Non-Probability Sample

Study Population

Patients admitted to an adult critical care unit or cardiothoracic critical care unit or experiencing an in-hospital cardiac arrest in an NHS acute hospital in England or Wales.

Description

Inclusion Criteria:

• Admission to an adult critical care unit or cardiothoracic critical care unit or in-hospital cardiac arrest

Exclusion Criteria:

• None

Study Plan

How is the study designed?

Design Details

• Observational Models: Cohort
• Time Perspectives: Retrospective

Number of groups / cohorts

2

Cohorts and Interventions

Group / Cohort
Admission to ICU; Patients admitted to an adult critical care unit or cardiothoracic critical care unit
In-hospital cardiac arrest; Patients experiencing an in-hospital cardiac arrest

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Time Frame
All cause mortality	30-days
All cause mortality	90-days
All cause mortality	1-year

Other Outcome Measures

Outcome Measure	Measure Description	Time Frame
diabetes	new diagnosis of diabetes post-critical care	up to 5 year
end-stage renal disease	new diagnosis of end-stage renal disease post-critical care	up to 5 year
ROSC	return of spontaneous circulation (ROSC) for greater than 20 minutes	at time of reanimation
survival to hospital discharge		30 days, 90 days and 1 year

Collaborators and Investigators

• Sponsor: Intensive Care National Audit & Research Centre
• Investigators: Principal Investigator: David Harrison, MA PhD, Head Statistician, ICNARC

Publications and helpful links

General Publications

• Ferrando-Vivas P, Shankar-Hari M, Thomas K, Doidge JC, Caskey FJ, Forni L, Harris S, Ostermann M, Gornik I, Holman N, Lone N, Young B, Jenkins D, Webb S, Nolan JP, Soar J, Rowan KM, Harrison DA. Improving risk prediction model quality in the critically ill: data linkage study [Internet]. Southampton (UK): National Institute for Health and Care Research; 2022 Dec. Available from http://www.ncbi.nlm.nih.gov/books/NBK587779/

Helpful Links

• Funding details

Study record dates

Study Major Dates

• Study Start (Actual): August 1, 2015
• Primary Completion (Actual): March 31, 2016
• Study Completion (Actual): December 23, 2022

Study Registration Dates

• First Submitted: May 20, 2015
• First Submitted That Met QC Criteria: May 21, 2015
• First Posted (Estimate): May 27, 2015

Study Record Updates

• Last Update Posted (Estimate): January 9, 2023
• Last Update Submitted That Met QC Criteria: January 6, 2023
• Last Verified: January 1, 2023

More Information

Terms related to this study

• Additional Relevant MeSH Terms: Pathologic Processes; Heart Diseases; Cardiovascular Diseases; Disease Attributes; Critical Illness; Heart Arrest
• Other Study ID Numbers: ICNARC/02/07/15

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: Yes
• IPD Plan Description: An anonymised study dataset will be available on request from the Chief Investigator, subject to any necessary approvals