Prediction of outcome after moderate and severe traumatic brain injury: external validation of the International Mission on Prognosis and Analysis of Clinical Trials (IMPACT) and Corticoid Randomisation After Significant Head injury (CRASH) prognostic models

Abstract

Objective: The International Mission on Prognosis and Analysis of Clinical Trials and Corticoid Randomisation After Significant Head injury prognostic models predict outcome after traumatic brain injury but have not been compared in large datasets. The objective of this is study is to validate externally and compare the International Mission on Prognosis and Analysis of Clinical Trials and Corticoid Randomisation after Significant Head injury prognostic models for prediction of outcome after moderate or severe traumatic brain injury.

Design: External validation study.

Patients: We considered five new datasets with a total of 9,036 patients, comprising three randomized trials and two observational series, containing prospectively collected individual traumatic brain injury patient data.

Measurements and main results: Outcomes were mortality and unfavorable outcome, based on the Glasgow Outcome Score at 6 months after injury. To assess performance, we studied the discrimination of the models (by area under the receiver operating characteristic curves), and calibration (by comparison of the mean observed to predicted outcomes and calibration slopes). The highest discrimination was found in the Trauma Audit and Research Network trauma registry (area under the receiver operating characteristic curves between 0.83 and 0.87), and the lowest discrimination in the Pharmos trial (area under the receiver operating characteristic curves between 0.65 and 0.71). Although differences in predictor effects between development and validation populations were found (calibration slopes varying between 0.58 and 1.53), the differences in discrimination were largely explained by differences in case mix in the validation studies. Calibration was good, the fraction of observed outcomes generally agreed well with the mean predicted outcome. No meaningful differences were noted in performance between the International Mission on Prognosis and Analysis of Clinical Trials and Corticoid Randomisation After Significant Head injury models. More complex models discriminated slightly better than simpler variants.

Conclusions: Since both the International Mission on Prognosis and Analysis of Clinical Trials and the Corticoid Randomisation After Significant Head injury prognostic models show good generalizability to more recent data, they are valid instruments to quantify prognosis in traumatic brain injury.

Figures

Figure 1

Figure 1a. Six-month Glasgow Outcome Scale in development (IMPACT and CRASH) and validation datasets (NABIS Hypothermia, Cerestat, APOE and Pharmos). Figure 1b. In-hospital mortality in the TARN-TBI registry.

Figure 1

Figure 1a. Six-month Glasgow Outcome Scale in development (IMPACT and CRASH) and validation datasets (NABIS Hypothermia, Cerestat, APOE and Pharmos). Figure 1b. In-hospital mortality in the TARN-TBI registry.

Figure 2

Figure 2a. Calibration plots for external validation of the IMPACT and CRASH models for prediction of 6-month unfavourable outcome in the Pharmos trial data. Predicted probabilities are on the x-axis and observed outcomes on the y-axis. The triangles indicate the observed frequencies by quantiles of predicted probability with 95% confidence intervals (vertical lines). The distribution of the predicted probabilities is shown at the bottom of the graphs, separate for those with and without the outcome of interest. Figure 2b. Calibration plots for external validation of the IMPACT and CRASH models for prediction of mortality in the Pharmos trial data.

Figure 2

Figure 2a. Calibration plots for external validation of the IMPACT and CRASH models for prediction of 6-month unfavourable outcome in the Pharmos trial data. Predicted probabilities are on the x-axis and observed outcomes on the y-axis. The triangles indicate the observed frequencies by quantiles of predicted probability with 95% confidence intervals (vertical lines). The distribution of the predicted probabilities is shown at the bottom of the graphs, separate for those with and without the outcome of interest. Figure 2b. Calibration plots for external validation of the IMPACT and CRASH models for prediction of mortality in the Pharmos trial data.

Figure 3

Calibration plots for external validation of the IMPACT and CRASH models for prediction of mortality in the TARN TBI dataset.