Machine Learning Assisted Differentiation of Low Acuity Patients at Dispatch (MADLAD)

Machine Learning Assisted Differentiation of Low Acuity Patients at Dispatch: A Randomized Controlled Trial

BACKGROUND:

At Emergency Medical Dispatch (EMD) centers, Resource Constrained Situations (RCS) where there are more callers requiring an ambulance than there are available ambulances are common. At the EMD centers in Uppsala and Västmanland, patients experiencing these situations are typically assigned a low-priority response, are often elderly, and have non-specific symptoms. Machine learning techniques offer a promising but largely untested approach to assessing risks among these patients.

OBJECTIVES:

To establish whether the provision of machine learning-based risk scores improves the ability of dispatchers to identify patients at high risk for deterioration in RCS.

DESIGN:

Multi-centre, parallel-grouped, randomized, analyst-blinded trial.

POPULATION:

Adult patients contacting the national emergency line (112), assessed by a dispatch nurse in Uppsala or Västmanland as requiring a low-priority ambulance response, and experiencing an RCS.

OUTCOMES:

Primary:

1. Proportion of RCS where the first available ambulance was dispatched to the patient with the highest National Early Warning Score (NEWS) score

Secondary:

• Difference in composite risk score consisting of ambulance interventions, emergent transport, hospital admission, intensive care, and mortality between patients receiving immediate vs. delayed ambulance response during RCS.
• Difference in NEWS between patients receiving immediate vs. delayed ambulance response during RCS.

INTERVENTION:

A machine learning model will estimate the risk associated with each patient involved in the RCS, and propose a patient to receive the available ambulance. In the intervention arm only, the assessment will be displayed in a user interface integrated into the dispatching system.

TRIAL SIZE:

1500 RCS each consisting of multiple patients randomized 1:1 to control and intervention arms

Study Overview

• Status: Completed
• Conditions: Emergencies
• Intervention / Treatment: Diagnostic test: openTriage - Alitis algorithm

• Study Type: Interventional
• Enrollment (Actual): 2499
• Phase: Not Applicable

Contacts and Locations

Study Locations

• Sweden
  • Uppsala, Sweden
    • Uppsala University Hospital
  • Västmanland
    • Västerås, Västmanland, Sweden
      • Västmanland hospital Västerås

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: 14 years and older (Adult, Older Adult)
• Accepts Healthy Volunteers: No

Description

Inclusion Criteria:

• Identification of a resource constrained situation by ambulance director (i.e., 2 or more patients awaiting an ambulance response)
• Assigned priority 2A or 2B (Low-priority ambulance response) by dispatch nurse call-taker
• Valid Swedish personal identification number collected at dispatch
• Age >= 18 years

Exclusion Criteria:

• Relevant calls received more than 30 minutes apart
• Logistical factors (eg. the patients' geographical locations) affect the ambulance assignment decision
• On scene risk factors (eg. a patient is outdoors and risks hypothermia) or risk mitigators (eg. healthcare staff already on-scene with a patient) affect the ambulance assignment decision

Study Plan

How is the study designed?

Design Details

• Primary Purpose: Health Services Research
• Allocation: Randomized
• Interventional Model: Parallel Assignment
• Masking: Single

Number of Arms

2

Arms and Interventions

Participant Group / Arm	Intervention / Treatment
Experimental: Intervention; Calculation of risk assessment score by machine learning algorithm and display of risk assessment information to dispatch nurses. Staff encouraged but not required to comply with suggested ranking.	Diagnostic test: openTriage - Alitis algorithm; A machine learning algorithm (Gradient boosting) applied to structured data collected in the Alitis Clinical Decision Support system, patient demographics, and free-text notes.
No Intervention: Control; Ambulance dispatch per standard of care

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Proportion of RCS where the first available ambulance was dispatched to the patient with the highest National Early Warning Score (NEWS).	NEWS is a widely used and well-validated scoring algorithm based on objective patient vital signs, which are not causally dependent on the outcomes used to train the machine learning models investigated in this study. NEWS values will be based on the first set of vital signs obtained by ambulance nurses upon making contact with the patient. NEWS is measured on a 0-21 scale, with higher values corresponding to patients at higher risk for deterioration.	Upon ambulance response (Within 8 hours of dispatch)

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Difference in composite outcome measure score between patients with immediate vs. delayed response.	This measure investigates a composite score consisting of the outcomes used to train the machine learning models. The composite score is generated by identifying the following patient outcomes and assigning the corresponding weights: Abnormal intitial Arway/Breathing/Circulation findings by ambulance crew (4) Emergent (lights and sirens) transport to the hospital (2) Provision of prehospital interventions (1) Admission to in-patient care or mortality within 30 days (1) This results in a score from 0-8, with higher scores representing more	Up to 30 days
Difference in National Early Warning Score (NEWS) between patients with immediate vs. delayed response.	Per primary outcome	Upon ambulance response (Within 8 hours of dispatch)

Collaborators and Investigators

• Sponsor: Uppsala University Hospital
• Collaborators: Region Västmanland
• Investigators: Principal Investigator: Hans Blomberg, MD, PhD, Uppsala University Hospital

Publications and helpful links

General Publications

• Spangler D, Hermansson T, Smekal D, Blomberg H. A validation of machine learning-based risk scores in the prehospital setting. PLoS One. 2019 Dec 13;14(12):e0226518. doi: 10.1371/journal.pone.0226518. eCollection 2019.
• Spangler D, Edmark L, Winblad U, Collden-Benneck J, Borg H, Blomberg H. Using trigger tools to identify triage errors by ambulance dispatch nurses in Sweden: an observational study. BMJ Open. 2020 Mar 19;10(3):e035004. doi: 10.1136/bmjopen-2019-035004.

Helpful Links

• Source code for risk assessment tool used in intervention

Study record dates

Study Major Dates

• Study Start (Actual): February 1, 2021
• Primary Completion (Actual): November 30, 2024
• Study Completion (Actual): November 30, 2024

Study Registration Dates

• First Submitted: February 3, 2021
• First Submitted That Met QC Criteria: February 11, 2021
• First Posted (Actual): February 17, 2021

Study Record Updates

• Last Update Posted (Actual): March 25, 2025
• Last Update Submitted That Met QC Criteria: January 7, 2025
• Last Verified: January 1, 2025

More Information

Terms related to this study

• Additional Relevant MeSH Terms: Pathologic Processes; Disease Attributes; Emergencies
• Other Study ID Numbers: SVLC001

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: YES
• IPD Plan Description: Individual level data available upon reasonable request to authors after publication
• IPD Sharing Time Frame: Upon publication
• IPD Sharing Access Criteria: Researchers with ethics board approved research plan
• IPD Sharing Supporting Information Type: STUDY_PROTOCOL; ANALYTIC_CODE

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No