Derivation and validation of cutoffs for clinical use of cell cycle arrest biomarkers

Eric A J Hoste, Peter A McCullough, Kianoush Kashani, Lakhmir S Chawla, Michael Joannidis, Andrew D Shaw, Thorsten Feldkamp, Denise L Uettwiller-Geiger, Paul McCarthy, Jing Shi, Michael G Walker, John A Kellum, Sapphire Investigators, Eric A J Hoste, Peter A McCullough, Kianoush Kashani, Lakhmir S Chawla, Michael Joannidis, Andrew D Shaw, Thorsten Feldkamp, Denise L Uettwiller-Geiger, Paul McCarthy, Jing Shi, Michael G Walker, John A Kellum, Sapphire Investigators

Abstract

Background: Acute kidney injury (AKI) remains a deadly condition. Tissue inhibitor of metalloproteinases (TIMP)-2 and insulin-like growth factor binding protein (IGFBP)7 are two recently discovered urinary biomarkers for AKI. We now report on the development, and diagnostic accuracy of two clinical cutoffs for a test using these markers.

Methods: We derived cutoffs based on sensitivity and specificity for prediction of Kidney Disease: Improving Global Outcomes Stages 2-3 AKI within 12 h using data from a previously published multicenter cohort (Sapphire). Next, we verified these cutoffs in a new study (Opal) enrolling 154 critically ill adults from six sites in the USA.

Results: One hundred subjects (14%) in Sapphire and 27 (18%) in Opal met the primary end point. The results of the Opal study replicated those of Sapphire. Relative risk (95% CI) in both studies for subjects testing at ≤0.3 versus >0.3-2 were 4.7 (1.5-16) and 4.4 (2.5-8.7), or 12 (4.2-40) and 18 (10-37) for ≤0.3 versus >2. For the 0.3 cutoff, sensitivity was 89% in both studies, and specificity 50 and 53%. For 2.0, sensitivity was 42 and 44%, and specificity 95 and 90%.

Conclusions: Urinary [TIMP-2]•[IGFBP7] values of 0.3 or greater identify patients at high risk and those >2 at highest risk for AKI and provide new information to support clinical decision-making.

Clinical trials registration: Clintrials.gov # NCT01209169 (Sapphire) and NCT01846884 (Opal).

Keywords: acute kidney injury; acute renal failure; biomarkers; insulin-like growth factor binding protein (IGFBP)7 and tissue inhibitor of metalloproteinases (TIMP)-2; sensitivity and specificity (MeSH).

© The Author 2014. Published by Oxford University Press on behalf of ERA-EDTA.

Figures

FIGURE 1:
FIGURE 1:
Study design (Opal) and number of subjects.
FIGURE 2:
FIGURE 2:
[TIMP-2]•[IGFBP7] ROC curves and operating characteristics for the Sapphire (solid) and Opal (dash) cohorts. Closed triangles and circles indicate [TIMP-2]•[IGFBP7] cutoffs of 0.3 and 2.0, respectively. End point was AKI Stages 2 or 3 within 12 h of sample collection. Area under the ROC curve [95% confidence interval (CI)] = 0.80 (0.74–0.84) and 0.79 (0.69–0.88) for Sapphire and Opal, respectively. NPV and PPVs are presented in Supplementary Data Figure S4 of the supplement.
FIGURE 3:
FIGURE 3:
Relative risk of AKI Stage 2 or 3 within 12 h in the Opal (light gray), Sapphire (medium gray) and combined Opal and Sapphire (dark gray) cohort. Samples were collected within 18 h of enrollment. Risk for each [TIMP-2]•[IGFBP7] range is shown relative to the lowest [TIMP-2]•[IGFBP7] range (≤0.3). Raw risk in lowest stratum = 4.3, 2.7 and 2.9%, respectively, for the Opal, Sapphire and combined cohorts. Error bars indicate the 95% CI. For both cohorts together 700 (46%) of patients had values ≤0.3; 675 (44%) had values between 0.3 and 2 (raw risk of AKI 12.6%); and 154 (10%) had values >2.0 (raw risk of AKI 49%). Cochran–Armitage test for significant trend: P

FIGURE 4:

Relative risk of MAKE 30…

FIGURE 4:

Relative risk of MAKE 30 in the Sapphire cohort. Samples were collected within…

FIGURE 4:
Relative risk of MAKE30 in the Sapphire cohort. Samples were collected within 18 h of enrollment. Risk for each [TIMP-2]•[IGFBP7] range is shown relative to the lowest [TIMP-2]•[IGFBP7] range (≤0.3). Raw risk in lowest stratum = 18%. Error bars indicate the 95% CI. Cochran–Armitage test for significant trend: P < 0.001. *P = 0.036; **P < 0.001.

FIGURE 5:

Prevalence adjusted PPV ( A…

FIGURE 5:

Prevalence adjusted PPV ( A ) and NPV ( B ) for [TIMP-2]•[IGFBP7]…

FIGURE 5:
Prevalence adjusted PPV (A) and NPV (B) for [TIMP-2]•[IGFBP7] cutoff values of 0.3 and 2.0 in the Opal (light gray), Sapphire (medium gray) and combined Opal and Sapphire (dark gray) cohort. Samples were collected within 18 h of enrollment. End point is AKI Stage 2 or 3 within the time window for prediction of AKI indicated along the abscissa (zero time = time of sample collection). Prevalence was adjusted to match the AKI distribution from Joannidis et al. [13] as described in the text [13]. Error bars indicate the 95% CI. Median time from a positive test result to a positive end point was 12.5 h [interquartile range (IQR) 2.7–26] for the Sapphire study and 8 h (IQR 0–15.5) for Opal.
FIGURE 4:
FIGURE 4:
Relative risk of MAKE30 in the Sapphire cohort. Samples were collected within 18 h of enrollment. Risk for each [TIMP-2]•[IGFBP7] range is shown relative to the lowest [TIMP-2]•[IGFBP7] range (≤0.3). Raw risk in lowest stratum = 18%. Error bars indicate the 95% CI. Cochran–Armitage test for significant trend: P < 0.001. *P = 0.036; **P < 0.001.
FIGURE 5:
FIGURE 5:
Prevalence adjusted PPV (A) and NPV (B) for [TIMP-2]•[IGFBP7] cutoff values of 0.3 and 2.0 in the Opal (light gray), Sapphire (medium gray) and combined Opal and Sapphire (dark gray) cohort. Samples were collected within 18 h of enrollment. End point is AKI Stage 2 or 3 within the time window for prediction of AKI indicated along the abscissa (zero time = time of sample collection). Prevalence was adjusted to match the AKI distribution from Joannidis et al. [13] as described in the text [13]. Error bars indicate the 95% CI. Median time from a positive test result to a positive end point was 12.5 h [interquartile range (IQR) 2.7–26] for the Sapphire study and 8 h (IQR 0–15.5) for Opal.

References

    1. Hoste EA, Clermont G, Kersten A, et al. RIFLE criteria for acute kidney injury are associated with hospital mortality in critically ill patients: a cohort analysis. Crit Care. 2006;10:R73.
    1. Murugan R, Karajala-Subramanyam V, Lee M, et al. Acute kidney injury in non-severe pneumonia is associated with an increased immune response and lower survival. Kidney Int. 2010;77:527–535.
    1. Bellomo R, Kellum JA, Ronco C. Acute kidney injury. Lancet. 2012;380:756–766.
    1. Lameire NH, Bagga A, Cruz D, et al. Acute kidney injury: an increasing global concern. Lancet. 2013;382:170–179.
    1. Hobson CE, Yavas S, Segal MS, et al. Acute kidney injury is associated with increased long-term mortality after cardiothoracic surgery. Circulation. 2009;119:2444–2453.
    1. Bihorac A, Yavas S, Subbiah S, et al. Long-term risk of mortality and acute kidney injury during hospitalization after major surgery. Ann Surg. 2009;249:851–858.
    1. Kidney Disease: Improving Global Outcomes (KDIGO) Acute Kidney Injury Work Group. KDIGO clinical practice guideline for acute kidney injury. Kidney Int Suppl. 2012;2:1–138.
    1. Kashani K, Al-Khafaji A, Ardiles T, et al. Discovery and validation of cell cycle arrest biomarkers in human acute kidney injury. Crit Care. 2013;17:R25.
    1. Bossuyt PM, Reitsma JB, Bruns DE, et al. Towards complete and accurate reporting of studies of diagnostic accuracy: the STARD initiative. Clin Chem. 2003;49:1–6.
    1. Palevsky PM, Molitoris BA, Okusa MD, et al. Design of clinical trials in acute kidney injury: report from an NIDDK workshop on trial methodology. Clin J Am Soc Nephrol. 2012;7:844–850.
    1. Cruz DN, Bagshaw SM, Maisel A, et al. Use of biomarkers to assess prognosis and guide management for patients with acute kidney injury. Contrib Nephrol. 2013;182:45–64.
    1. Pepe MS. The Statistical Evaluation of Medical Tests for Classification and Prediction. Oxford: Oxford University Press; 2003.
    1. Joannidis M, Metnitz B, Bauer P, et al. Acute kidney injury in critically ill patients classified by AKIN versus RIFLE using the SAPS 3 database. Intensive Care Med. 2009;35:1692–1702.
    1. The R Project for Statistical Computing . (1 September 2004, date last accessed)
    1. Zhou Xh, McClish DK, Obuchowski NA. Statistical Methods in Diagnostic Medicine. 2nd edn. Hoboken: Wiley; 2011.
    1. Agresti A. Categorical Data Analysis. Hoben, NJ: Wiley; 2002.
    1. Pencina MJ, D'Agostino RB, Sr., D'Agostino RB, Jr., et al. Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond. Stat Med. 2008;27:157–172.
    1. Kundu S, Aulchenko YS, van Duijn CM, et al. PredictABEL: an R package for the assessment of risk prediction models. Eur J Epidemiol. 2011;26:261–264.
    1. Robin X, Turck N, Hainard A, et al. pROC: an open-source package for R and S+ to analyze and compare ROC curves. BMC Bioinformatics. 2011;12:77.
    1. DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. 1988;44:837–845.
    1. NCEPOD. Adding insult to injury 2009 . (1 September 2004, date last accessed)
    1. Balasubramanian G, Al-Aly Z, Moiz A, et al. Early nephrologist involvement in hospital-acquired acute kidney injury: a pilot study. Am J Kidney Dis. 2011;57:228–234.
    1. Colpaert K, Hoste EA, Steurbaut K, et al. Impact of real-time electronic alerting of acute kidney injury on therapeutic intervention and progression of RIFLE class. Crit Care Med. 2012;40:1164–1170.
    1. Maisel AS, Krishnaswamy P, Nowak RM, et al. Rapid measurement of B-type natriuretic peptide in the emergency diagnosis of heart failure. N Engl J Med. 2002;347:161–167.
    1. Keller T, Zeller T, Ojeda F, et al. Serial changes in highly sensitive troponin I assay and early diagnosis of myocardial infarction. JAMA. 2011;306:2684–2693.
    1. Januzzi JL, Jr, Bamberg F, Lee H, et al. High-sensitivity troponin T concentrations in acute chest pain patients evaluated with cardiac computed tomography. Circulation. 2010;121:1227–1234.
    1. Bihorac A, Chawla LS, Shaw AD, et al. Validation of cell-cycle arrest biomarkers for acute kidney injury using clinical adjudication. Am J Respir Crit Care Med. 2014;189:932–939.
    1. Brienza N, Giglio MT, Marucci M, et al. Does perioperative hemodynamic optimization protect renal function in surgical patients? A meta-analytic study. Crit Care Med. 2009;37:2079–2090.
    1. Brar SS, Aharonian V, Mansukhani P, et al. Haemodynamic-guided fluid administration for the prevention of contrast-induced acute kidney injury: the POSEIDON randomised controlled trial. Lancet. 2014;383:1814–1823.
    1. Goldstein SL, Kirkendall E, Nguyen H, et al. Electronic health record identification of nephrotoxin exposure and associated acute kidney injury. Pediatrics. 2013;132:756–767.
    1. Perner A, Haase N, Guttormsen AB, et al. Hydroxyethyl starch 130/0.42 versus Ringer's acetate in severe sepsis. N Engl J Med. 2012;367:124–134.
    1. Hirschberg R, Kopple J, Lipsett P, et al. Multicenter clinical trial of recombinant human insulin-like growth factor I in patient with acute renal failure. Kidney Int. 1999;55:2423–2432.
    1. Siew ED, Matheny ME, Ikizler TA, et al. Commonly used surrogates for baseline renal function affect the classification and prognosis of acute kidney injury. Kidney Int. 2010;77:536–542.
    1. Sapphire investigators. . (1 September 2004, date last accessed)

Source: PubMed

Sponsors and Collaborators

Medical Conditions

Drug Interventions

3
Subscribe