Passive Evaluation in Operational Environment of the AI Clinician Decision Support System for Sepsis Treatment

Sepsis, or systemic infection, is a common reason for ICU admission and death throughout the world. Despite advances in the way we treat this condition, it remains a significant economic and healthcare burden. A key part of the treatment of sepsis is the administration of IV fluids and blood pressure medication. However, there is huge uncertainty around dosing of these drugs in an individual patient. A tool to personalise these medications could improve patient survival. The study team has developed a new method to automatically and continuously review and recommend the correct medication doses to doctors, which was created using artificial intelligence (AI) techniques applied to large medical databases. The researchers' previous work has shown it has the potential to improve patient survival rates. The tool will be capable of processing patient data within the electronic patient record of NHS hospitals in real-time to suggest a course of action. This tool will be evaluated and refined in simulation studies and then be tested in two NHS Trusts in "shadow mode" (results not provided to duty clinicians). This will allow comparison of actual decisions made and recommended decisions from the AI system. The second stage of this clinical evaluation will display the recommendations to clinicians to assess the acceptability of the tool and confirm technical feasibility to inform future clinical trials. The long-term expected benefits of this project are numerous: improved patient survival, reduced use of precious intensive care resources and reduction in healthcare costs.

Study Overview

• Status: Recruiting
• Conditions: Sepsis

Detailed Description

Sepsis is life-threatening organ dysfunction due to severe infection and affects 250,000 patients annually in the UK (pre-COVID-19), of whom 48,000 die. In addition, virtually all COVID-19 intensive care unit (ICU) deaths had sepsis. It is a leading cause of death and the most expensive condition treated in hospitals. It was recognised as a top research priority by the James Lind Alliance, a partnership of patients and clinicians to prioritise the most pressing unanswered questions facing the NHS.

The cornerstone of sepsis resuscitation is the administration of intravenous fluids and/or vasopressors (drugs that squeeze the blood vessels to increase blood pressure) to maintain blood flow to prevent organ failure. However, there is huge uncertainty around the individual dosing of these drugs in an individual patient, partially due to high sepsis heterogeneity. The current guidelines provide recommendations at a population-level but fail to individualise the decisions. Wrong decisions lead to poorer outcomes and increased ICU-resource use. A tool to personalise these medications could improve patient survival.

The study team has developed a new method to automatically and continuously review and recommend the correct dose of these medications to doctors, which was created using artificial intelligence (AI) techniques applied to large medical databases. The method used is called reinforcement learning. In this framework, the study models patients with sepsis in the ICU as belonging to a large number of possible disease states, and analyses what interventions are likely to help them transition to healthier states, and eventually to survival. The researchers demonstrated in their initial publication that the value of the AI selected strategy was on average reliably higher than human clinicians. In a large validation cohort independent from the training data, mortality was lowest in patients where clinicians' actual doses matched the AI decisions: mortality rates rose, in a dose dependent manner, as the clinicians' actual decisions diverged from the AI decisions. The study team has estimated that their AI algorithm could reduce mortality by 10% (in relative terms), which represents over 1,000 lives saved annually in the UK and would scale to hundreds of thousands of lives worldwide. Now, the study team intends to start clinical testing of this AI technology in the UK.

The envisioned end-product will be a piece of software that will be accessible by clinicians (ICU doctors initially, then eventually to ICU nurses as well) at the bedside in intensive care. This software will be connected to the electronic patient record, which will be fed to the AI algorithm. In return, the AI will identify where the patient sits in the array of possible disease states, and which actions (a dose of intravenous fluids and vasopressors) are most likely to be beneficial.

First, the study team will develop this software tool, capable of processing patient data within the electronic patient record of NHS hospitals in real-time to suggest a course of action. The study will start by evaluating and refining this tool in simulation studies. The study team will then test the AI tool in two NHS Trusts in a "shadow mode" when the result is not provided to duty clinicians in charge of patient care. This will allow comparison of actual decisions made and recommended decisions from the AI system. In the second stage of the clinical evaluation, the study team will display the recommendations to clinicians to assess the acceptability of the tool to clinicians and also confirm the technical feasibility to inform future large scale clinical trials.

The long-term expected benefits of this project are numerous: improved patient survival, reduced use of precious intensive care resources and reduction in healthcare costs.

• Study Type: Observational
• Enrollment (Estimated): 15

Contacts and Locations

• Study Contact: Name: Anthony C Gordon; Phone Number: 02033126328; Email: anthony.gordon@imperial.ac.uk

Study Locations

• United Kingdom
  • London, United Kingdom, NW1 2PG
    • Not yet recruiting
    • Univeristy College London Hospitals NHS Foundation Trust
    • Principal Investigator: David Brealey
    • Contact: Novin Fard
  • London, United Kingdom, W2 1BL
    • Recruiting
    • Imperial College Hospitals NHS Trust
    • Contact: Robyn Kullar
    • Principal Investigator: Anthony C Gordon, FMedSci

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: 18 years and older (Adult, Older Adult)
• Accepts Healthy Volunteers: No

• Sampling Method: Non-Probability Sample

Study Population

• Patients: adult septic patients in intensive care.
• Human Evaluators: senior registrars, ICU fellows and consultants.

Description

Inclusion Criteria:

For patients:

• Adult patient > 18 years old
• Admitted to an intensive care unit
• Likely or confirmed diagnosis of sepsis as per sepsis-3 definition (as defined in the glossary)
• ICU length of stay > 24h

For Evaluators:

- ICU doctors at the senior registrar, ICU fellow or consultant level

Exclusion Criteria:

For patients:

• Not for full active care, e.g. not for vasopressors
• Not expected to survive more than 24h
• Elective surgical admission (these patients are regularly on antibiotics but given as a prophylaxis, with no sepsis)
• Opted-out for use of their data for research (NHS and NHS-X website)

For both patients and evaluators:

Declined participation No patient consent is required

Study Plan

How is the study designed?

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Data Availability	Data availability: what percentage of essential and optional data fields are available 24/7.	18 months
Anonymised patients' data	Patient demographics (age in years, gender, primary diagnosis) Outcomes: organ function (hourly SOFA), ICU and hospital mortality	18 months
Rate of intravenous fluids administered to patients	Rate of intravenous fluids (millilitres per hour) and vasopressors (noradrenaline equivalent, in micrograms per kilogram per minute) administered to patients.	18 months
Evaluators' data (the doctors assessing the AI in the background)	The evaluator's grade and seniority (i.e. A custom-made interface linked to a database will capture and record the following: What rate of intravenous fluids (millilitres per hour) and vasopressors (noradrenaline equivalent, in micrograms per kilogram per minute) they would recommend, before and after seeing the AI suggested dose.; Agreement with the AI suggested dose on a Likert scale including the following labels: "the AI suggested dose is:" "certainly too low", "possibly too low","likely appropriate","possibly too high", "certainly too high". After collecting this data, we will report the median difference (with interquartile range) between the AI suggested doses and the evaluator suggested doses, as well as the proportion of AI decisions in each agreement category ("certainly too low", "possibly too low","likely appropriate","possibly too high", "certainly too high").	18 months
System Availability	System availability: delays in generating response 24/7. Number and nature of technical issues (drop-outs, freezes).	18 months

Collaborators and Investigators

• Sponsor: Imperial College London
• Collaborators: National Institute for Health Research, United Kingdom

Study record dates

Study Major Dates

• Study Start (Actual): March 24, 2022
• Primary Completion (Estimated): August 31, 2024
• Study Completion (Estimated): August 31, 2024

Study Registration Dates

• First Submitted: March 1, 2022
• First Submitted That Met QC Criteria: March 10, 2022
• First Posted (Actual): March 18, 2022

Study Record Updates

• Last Update Posted (Actual): October 27, 2023
• Last Update Submitted That Met QC Criteria: October 26, 2023
• Last Verified: October 1, 2023

More Information

Terms related to this study

• Keywords: Artificial Intelligence
• Additional Relevant MeSH Terms: Pathologic Processes; Infections; Systemic Inflammatory Response Syndrome; Inflammation; Sepsis; Toxemia
• Other Study ID Numbers: 20HH6297

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: NO

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No