Predicting outcome on admission and post-admission for acetaminophen-induced acute liver failure using classification and regression tree models

Abstract

Background/aim: Assessing prognosis for acetaminophen-induced acute liver failure (APAP-ALF) patients often presents significant challenges. King's College (KCC) has been validated on hospital admission, but little has been published on later phases of illness. We aimed to improve determinations of prognosis both at the time of and following admission for APAP-ALF using Classification and Regression Tree (CART) models.

Methods: CART models were applied to US ALFSG registry data to predict 21-day death or liver transplant early (on admission) and post-admission (days 3-7) for 803 APAP-ALF patients enrolled 01/1998-09/2013. Accuracy in prediction of outcome (AC), sensitivity (SN), specificity (SP), and area under receiver-operating curve (AUROC) were compared between 3 models: KCC (INR, creatinine, coma grade, pH), CART analysis using only KCC variables (KCC-CART) and a CART model using new variables (NEW-CART).

Results: Traditional KCC yielded 69% AC, 90% SP, 27% SN, and 0.58 AUROC on admission, with similar performance post-admission. KCC-CART at admission offered predictive 66% AC, 65% SP, 67% SN, and 0.74 AUROC. Post-admission, KCC-CART had predictive 82% AC, 86% SP, 46% SN and 0.81 AUROC. NEW-CART models using MELD (Model for end stage liver disease), lactate and mechanical ventilation on admission yielded predictive 72% AC, 71% SP, 77% SN and AUROC 0.79. For later stages, NEW-CART (MELD, lactate, coma grade) offered predictive AC 86%, SP 91%, SN 46%, AUROC 0.73.

Conclusion: CARTs offer simple prognostic models for APAP-ALF patients, which have higher AUROC and SN than KCC, with similar AC and negligibly worse SP. Admission and post-admission predictions were developed.

Key points: • Prognostication in acetaminophen-induced acute liver failure (APAP-ALF) is challenging beyond admission • Little has been published regarding the use of King's College Criteria (KCC) beyond admission and KCC has shown limited sensitivity in subsequent studies • Classification and Regression Tree (CART) methodology allows the development of predictive models using binary splits and offers an intuitive method for predicting outcome, using processes familiar to clinicians • Data from the ALFSG registry suggested that CART prognosis models for the APAP population offer improved sensitivity and model performance over traditional regression-based KCC, while maintaining similar accuracy and negligibly worse specificity • KCC-CART models offered modest improvement over traditional KCC, with NEW-CART models performing better than KCC-CART particularly at late time points.

Trial registration: ClinicalTrials.gov NCT00518440.

Conflict of interest statement

Competing Interests: The authors have declared that no competing interests exist.

Figures

Fig 1. ALFSG Subjects in this Study.

There were 588 APAP-ALF subjects who spontaneously survived, and 215 subjects who had a LT or died by day 21.

Fig 2. Admission CART Models.

The admission KCC-CART (left panel) has three decision rules and consists of six total nodes. Each node provides the total number of subjects within the node, as well as the number of survivors and dead/LT patients with the respective rates. Node 6 represents high risk of dead/LT outcome, nodes 1 and 3 are low risk of dead/LT outcome, and node 5 is moderate risk of dead/LT outcome. To calculate performance measures for the model, all subjects in nodes 5 and 6 were predicted as dead/LT outcomes, and all subjects in nodes 1 and 3 were predicted as spontaneous survivors. The admission NEW-CART (right panel) also has three decision rules and consists of six total nodes. Node 6 patients were considered high risk for dead/LT outcome and were predicted as such, whereas nodes 1, 3 and 5 were predicted as survivors.

Fig 3. Post-Admission CART Models.

The post-admission KCC-CART (left panel) has two decision rules and consists of four total nodes. Each node provides the total number of subjects within the node, as well as the number of survivors and dead/LT patients with the respective rates. Node 4 represents high risk of dead/LT outcome, and nodes 1 and 3 are low risk of dead/LT outcome. To calculate performance measures for the model, all subjects in node 4 were predicted as dead/LT outcomes, and all subjects in nodes 1 and 3 were predicted as spontaneous survivors. The post-admission NEW-CART (right panel) has three decision rules and consists of six total nodes. Node 6 patients were considered high risk for dead/LT outcome and were predicted as such, whereas nodes 1, 3 and 5 were predicted as survivors.

Fig 4. Plots of Confidence Intervals of Accuracy, AUROC, Sensitivity and Specificity for Admission and Post-Admission Models.

Plots display confidence intervals for accuracy (AC), area under the receiver operating curve (AUROC), sensitivity (SN) and specificity (SP) from Table 2. The admission plot (top panel) illustrates non-overlapping, higher confidence intervals for both KCC-CART and NEW-CART compared to KCC for AUROC and sensitivity, and lower confidence intervals for specificity. This indicates that the CART models had significantly better AUROC and sensitivity than KCC, but had significantly worse specificity compared to KCC. Confidence intervals for accuracy for KCC, KCC-CART and NEW-CART all overlap, indicating no significant differences between the models. The post-admission plot (bottom panel) again indicated no significant differences between the models in terms of accuracy; however, the AUROC and sensitivity of KCC-CART was significantly higher than that of NEW-CART and KCC. The specificity of KCC was highest, but did no differ significantly from that of the NEW-CART. KCC demonstrated significantly higher specificity than the KCC-CART, but the difference was not significant compared to the NEW-CART.