Machine Learning for Risk Stratification in the Emergency Department (MARS-ED) (MARS-ED)

Machine Learning for Risk Stratification in the Emergency Department: a Pilot Clinical Trial

Rationale

Identifying emergency department (ED) patients at high and low risk shortly after admission could help decision-making regarding patient care. Several clinical risk scores and triage systems for stratification of patients have been developed, but often underperform in clinical practice. Moreover, most of these risk scores only have been diagnostically validated in an observational cohort, but never have been evaluated for their actual clinical impact. In a recent retrospective study that was conducted in the Maastricht University Medical Center (MUMC+), a novel clinical risk score, the RISKINDEX, was introduced that predicted 31-day mortality of sepsis patients presenting to an ED. The RISKINDEX hereby also outperformed internal medicine specialists. Observational follow-up studies underlined the potential of the risk score. However, it remains unknown to what extent these models have any beneficial value when it is actually implemented in clinical practice.

Objective

To determine the diagnostic accuracy, policy changes and clinical impact of the RISKINDEX as basis to conduct a large scale, multi-center randomised trial.

Study design

The MARS-ED study is designed as a multi-center, randomized, open-label, non-inferiority pilot clinical trial.

Study population

Adult patients who are assessed and treated by an internal medicine specialist in the ED of whom a minimum of 4 different laboratory results (hematology or clinical chemistry, required for calculation of ML risk score) are available within the first two hours of the ED visit.

Intervention

Physicians will be presented with the ML risk score (the RISKINDEX) of the patients they are actively treating, directly after assessment of regular diagnostics has taken place.

Main study parameters

Primary

- Diagnostic accuracy, policy changes and clinical impact of a novel clinical risk score (the RISKINDEX)

Secondary

• Policy changes due to presentation of ML score (treatment policy, requesting ancillary investigations, treatment restrictions (i.e., no intubation or resuscitation)
• Intensive care (ICU) and medium care (MC) admission
• Length of admission
• Mortality within 31 days
• Readmission
• Patient preference
• Feasibility of novel clinical risk score

Study Overview

• Status: Completed
• Conditions: Emergencies; Acute Pain
• Intervention / Treatment: Other: RISK-INDEX

Detailed Description

See our protocol paper, PMID 38263188

• Study Type: Interventional
• Enrollment (Actual): 1300
• Phase: Not Applicable

Contacts and Locations

Study Locations

• Netherlands
  • Limburg
    • Maastricht, Limburg, Netherlands, 6229 HX
      • Maastricht University Medical Centre

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: 18 years and older (Adult, Older Adult)
• Accepts Healthy Volunteers: Yes

Description

Inclusion Criteria:

• Adult, defined as ≥ 18 years of age
• Assessed and treated by an internal medicine specialist (gastroenterologists included) in the ED
• Willing to give written consent, either directly or after deferred consent procedure (see section 11.2).

Exclusion Criteria:

• <4 different laboratory results available (hematology or clinical chemistry) within the first two hours of the ED visit (calculation ML prediction score otherwise not possible)
• Unwilling to provide written consent, either directly or after deferred consent procedure (see section 11.2).

Study Plan

How is the study designed?

Design Details

• Primary Purpose: Diagnostic
• Allocation: Randomized
• Interventional Model: Parallel Assignment
• Masking: None (Open Label)

Number of Arms

2

Arms and Interventions

Participant Group / Arm	Intervention / Treatment
No Intervention: Standard care; Routine clinical care. Physicians will actively be asked to self-report their clinical impression of each included patient and policy will be monitored.
Experimental: RISKINDEX; Routine clinical care. Physicians will actively be asked to self-report their clinical impression of each included patient and policy will be monitored. In the intervention group, physicians will be presented with the RISKINDEX. Subsequently, self-report will again be initiated to evaluate the physicians' response to the ML score and possible policy changes due to the intervention.	Other: RISK-INDEX; Presentation of RISKINDEX to the physician after approximately 2 hours. The ML RISKINDEX is a prediction model based on laboratory data from the ED. It is based on date of birth, sex and at least four laboratory data which are sampled within the first two hours of the ED visit. Laboratory data that are used as input include samples that are commonly drawn in patients that require treatment from an internal medicine physician, such as urea, albumin, C-reactive protein (CRP), lactate and bilirubin.

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
RISK-INDEX performance	Discriminatory performance of ML risk score to predict 31-day mortality. This will be calculated using an area under the receiver operating characteristic curves (AUC).	31 days
Policy changes	Policy changes after presentation of RISK-INDEX. This will be assessed by a filled out questionnaire by the physician where they state whether a policy change has been made as a result of the RISK-INDEX outcome.	As soon as RISK-INDEX score is presented

Collaborators and Investigators

• Sponsor: Maastricht University Medical Center
• Investigators: Principal Investigator: Steven Meex, PhD, Maastricht University Medical Center

Publications and helpful links

General Publications

• van Dam PMEL, van Doorn WPTM, van Gils F, Sevenich L, Lambriks L, Meex SJR, Cals JWL, Stassen PM. Machine learning for risk stratification in the emergency department (MARS-ED) study protocol for a randomized controlled pilot trial on the implementation of a prediction model based on machine learning technology predicting 31-day mortality in the emergency department. Scand J Trauma Resusc Emerg Med. 2024 Jan 23;32(1):5. doi: 10.1186/s13049-024-01177-2.

Helpful Links

• Explainable Machine Learning models for Rapid Risk Stratification in the Emergency Department: A multi-center study

Study record dates

Study Major Dates

• Study Start (Actual): September 12, 2022
• Primary Completion (Actual): November 1, 2024
• Study Completion (Actual): November 1, 2024

Study Registration Dates

• First Submitted: July 19, 2022
• First Submitted That Met QC Criteria: August 9, 2022
• First Posted (Actual): August 11, 2022

Study Record Updates

• Last Update Posted (Estimated): November 26, 2024
• Last Update Submitted That Met QC Criteria: November 22, 2024
• Last Verified: November 1, 2024

More Information

Terms related to this study

• Keywords: machine learning; risk stratification; pilot
• Additional Relevant MeSH Terms: Pain; Neurologic Manifestations; Pathologic Processes; Disease Attributes; Emergencies; Acute Pain
• Other Study ID Numbers: NL78478.068.21; METC 21-068 (Other Identifier: METC)

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: NO

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No
• product manufactured in and exported from the U.S.: No