Acute Kidney Injury Predictor Validation Study (AKIpredValid)

Prospective Validation of the AKIpredictor Through Comparison With Predictions of Acute Kidney Injury by ICU Physicians

Purpose: To evaluate the performance of AKIpredictor, a computer-based algorithm that predicts the development of AKI in the 7 days following ICU admission, by comparing it with similar predictions made by attending physicians.

Primary objective: To compare the performances of AKIpredictor and physicians in predicting AKI stage 2 or 3 in the 7 days following ICU admission Secondary objective(s): To investigate the influence of the level of seniority of the physician on the accuracy of the predictions; feasibility of making predictions within a 3 hour window for physicians Trial Design: Monocentric, prospective, longitudinal, non-interventional Endpoints: Primary: comparing the area under the ROC curves of the AKIpredictor and physicians.

Secondary: estimation of PPV, NPV, sensitivity and specificity of both predictors at different thresholds; evaluation of alternative negative endpoints (ICU readmission after discharge, death); subgroup analyses.

Sample Size: This is a pilot study. Sample size calculations to obtain sufficient power are not feasible due to lack of previous studies. The investigation will be conducted with a preset end time on June 30th. The investigators expect to include approximately 150 patients.

Summary of eligibility criteria: All adult patients admitted to UZ Leuven's surgical ICU in the period of the study, with the exclusion of those with end-stage renal disease or AKI already present at the time of admission

Study Overview

• Status: Completed
• Conditions: Critical Illness; Acute Kidney Injury
• Intervention / Treatment: Other: Computer model predictions of acute kidney injury

Detailed Description

Inclusion and exclusion criteria:

All adult patients admitted to the UZ Leuven's ICU after the approval of the study by the local Ethical Committee and before June 30th 2018 will be included in the study. The only exclusion criteria will be the presence of end-stage renal disease before or of AKI on ICU admission.

Computer predictions:

Elements from laboratory reports, clinical history and physiological monitoring will be used both by clinicians and by the computer algorithm to estimate the risk of AKI. Such measurements are part of the routine standard care in the ICU: no additional exam will be required for this study. These values are regularly stored in the electronic health care system (KWS, Klinisch Werkstation and iMDsoft's MetaVision). They will be retrieved in read-only mode when needed for formulating the predictions. The retrieval will take place regularly on a weekly basis in order to limit the amount of time that patients' identification data will be stored for the study.

In detail, the following parameters will be used by the computer model:

• upon admission: age, baseline serum creatinine (lowest value of the previous 3 months; if not available, calculated using the Schwartz formula), surgical or medical category (transplant surgery / cardiovascular surgery / abdominal or pelvic surgery / thoracic surgery / other: medical, trauma, other surgery), planned admission (yes / no), history of diabetes (yes / no), blood glucose, suspected sepsis (yes / no), and hemodynamic support (none / mechanical / pharmacological / both);
• on the first morning after admission: serum creatinine (measured in the morning), APACHE II score, blood lactate (worst value since admission), total bilirubin, and hours of ICU stay;
• at 24h after admission: hourly urine output, doses of inotropes and vasopressors, and continuous arterial blood pressure values.

Physicians predictions:

Physicians' predictions will be collected alternatively through interviews conducted personally by one of the investigators or by means of a questionnaire compiled by the physicians themselves. To parallel the timing of AKIpredictor, clinicians' predictions will be gathered at three time-points:

• admission predictions: at the earliest after admission (up to 3h);
• morning predictions: on the first morning after admission, right before or after the handoffs that take place between 8.30 and 9.00 AM;
• 24h predictions: at 24h after admission (up to 24h+3h).

Interviews to multiple physicians about the same patient in the same time frame will be encouraged, as long as they are not influenced by one another (junior resident, senior resident, staff member).

In every prediction, both a continuous (on a 0-100 % scale) and a binary (yes / no) estimate of AKI risk will be inquired. A self-assessed degree of confidence about the prediction will also be tracked (very confident / medium confident / not confident at all). The exact time at which the human prediction is collected will be saved. Finally, data about the clinician expressing the prediction will be tracked: age, gender, seniority (see above), years of ICU experience.

Endpoint assessment The effective development of AKI will be diagnosed in agreement with the 2012 KDIGO guidelines [1]. To take into account AKI occurring outside the ICU where hourly measurements of urine output are not recorded, only the creatinine criterion will be used. Serum creatinine values will be collected from the electronic health record system KWS (Klinisch WerkStation) each day for 7 days following ICU admission. This biochemical exam is routinely performed on a daily basis in any hospitalized patient unless considered at low risk for complications, and it is always required if there's suspicion of an acute renal insult. For these reasons, potential missing creatinine values will be considered evidence of an improving patient and absence of AKI for that day.

ICU discharge and eventual readmission will also be tracked for secondary analyses. The death of the patient will also be recorded and included among the negative outcomes.

Assessment of efficacy The efficacy of both predictors will be evaluated with the construction of ROC curves. Due to the lack of previous studies on the subject, a current estimate of the accuracy of clinicians' prediction of AKI is not currently available. Appropriate sample size calculations to obtain sufficient power are therefore not feasible at the moment. Statistical analysis will be performed at the end of the data collection as follows. A ROC curve will be plotted both for the computer-based AKIpredictor and the continuous (%) estimate of AKI risk assessed by the clinicians at each of the three time-points (upon admission, on day 1 and at 24 hours). The area under the curve (AUC) of each plot will be calculated. The AUCs will then be compared in pairs by bootstrapping, and a significance value of the comparison will be derived. Subgroup analysis will be performed with the same approach.

Ethics and regulatory approvals:

The trial will be conducted in compliance with the principles of the Declaration of Helsinki (current version) and the principles of good clinical practice. This protocol will be submitted to the ethical committee of the UZ Leuven. Any eventual subsequent protocol amendment will also be submitted to the ethical committee and Regulatory Authorities for approval.

By virtue of its non-interventional nature and that all the data will be stored in anonymized fashion, informed consent will not be required from the subjects if this is deemed appropriate by the local Ethical Committee. The investigators shall treat all information and data relating to the study as confidential and shall not disclose such information to any third parties or use such information for any purpose other than the performance of the study. The collection, processing and disclosure of personal data, such as patient health and medical information is subject to compliance with applicable personal data protection and the processing of personal data (Directive 95/46/EC and Belgian law of December 8, 1992 on the Protection of the Privacy in relation to the Processing of Personal Data).

• Study Type: Observational
• Enrollment (Actual): 252

Contacts and Locations

Study Locations

• Belgium
  • Kessel-Lo, Belgium, 3010
    • Geert Meyfroidt

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: 18 years and older (Adult, Older Adult)
• Accepts Healthy Volunteers: No
• Genders Eligible for Study: All

• Sampling Method: Non-Probability Sample

Study Population

All adult patients admitted to UZ Leuven's surgical ICU in the period of the study, with the exclusion of those with end-stage renal disease or AKI already present at the time of admission

Description

Inclusion Criteria:

• All adult patients admitted to UZ Leuven's surgical ICU in the period of the study

Exclusion Criteria:

• Patients with end-stage renal disease or AKI already present at the time of admission

Study Plan

How is the study designed?

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
AUROC comparison	Comparison of the area under the receiver-operator characteristic (ROC) curves between the predictions made by the AKIpredictor and the ones made by ICU physicians, to predict AKI 2-3.	1 week

Collaborators and Investigators

• Sponsor: Geert Meyfroidt, MD, PhD
• Investigators: Principal Investigator: Geert Meyfroidt, MD, Phd, KU Leuven

Publications and helpful links

General Publications

• Flechet M, Falini S, Bonetti C, Guiza F, Schetz M, Van den Berghe G, Meyfroidt G. Machine learning versus physicians' prediction of acute kidney injury in critically ill adults: a prospective evaluation of the AKIpredictor. Crit Care. 2019 Aug 16;23(1):282. doi: 10.1186/s13054-019-2563-x.

Study record dates

Study Major Dates

• Study Start (Actual): May 2, 2018
• Primary Completion (Actual): June 30, 2018
• Study Completion (Actual): September 1, 2018

Study Registration Dates

• First Submitted: June 21, 2018
• First Submitted That Met QC Criteria: June 29, 2018
• First Posted (Actual): July 2, 2018

Study Record Updates

• Last Update Posted (Actual): September 23, 2022
• Last Update Submitted That Met QC Criteria: September 22, 2022
• Last Verified: May 1, 2019

More Information

Terms related to this study

• Additional Relevant MeSH Terms: Pathologic Processes; Kidney Diseases; Urologic Diseases; Disease Attributes; Renal Insufficiency; Wounds and Injuries; Critical Illness; Acute Kidney Injury
• Other Study ID Numbers: S61388

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: NO

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No