Artificial Intelligence/Machine Learning Modeling on Time to Palliative Care Review in an Inpatient Hospital Population

The Impact of Artificial Intelligence/Machine Learning (AI/ML) on Time to Palliative Care Review in an Inpatient Hospital Population

Investigators are testing whether machine learning prediction models integrated into a health care model will accurately identify participants who may benefit from a comprehensive review by a palliative care specialist, and decrease time to receiving a palliative care consult in an inpatient setting.

Study Overview

• Status: Completed
• Conditions: Palliative Care
• Intervention / Treatment: Other: Control Tower

Detailed Description

The need for timely palliative care is crucial. Aging patient populations are becoming more complex, often needing care from multiple specialties. There has been a growing mismatch between clinical care and patient preferences particularly with regards to services near end-of-life. Research has shown that that most people prefer to die at home despite the majority dying outside of the home (nursing home or hospital). Given the current model of care and incentives palliative care is considered the care of last resort after all attempts at cure have been exhausted. This delay can lead to sub-optimal symptom management for pain and lower quality of life. As the demand for palliative care increases, policy initiatives and referral triage tools to that lead to quality palliative care services are needed.

In 2018 the Mayo Clinic developed a fully integrated information technology (IT) solution focusing on the identification of patients who may benefit from early palliative care review. The tool, known as Control Tower, pulls disparate data sources centered on a machine learning algorithm which predicts the need for palliative care in hospital. This algorithm was put into production as of December 2018 into a silent mode. The algorithm along with other key patient indicators are integrated into a graphical user interface (GUI) which allows a human operator to review the algorithm predictions and subsequently record the operator's assessment. The tool is expected to enhance risk assessment and create a healthcare model in which palliative care can pro-actively and effectively screen for patient need. Anticipated benefits of the approach include improved symptom control and patient satisfaction as well as a measurable impact on inpatient hospital mortality.

The overall objective of this study is to assess the effectiveness and implementation of the Control Tower palliative care algorithm into hospital practice by creating a stepped wedge cluster randomized trial in 16 inpatient units. By creating an algorithm that automatically screens and monitors patient health status during inpatient hospitalization, the investigators hypothesize that participants will receive needed palliative care earlier than under the usual course of care. In addition to testing clinical effectiveness study members will also collect data for process measures to assess the algorithm and healthcare performance after translation of the prediction algorithm from a research domain to a practice setting.

• Study Type: Interventional
• Enrollment (Actual): 2231
• Phase: Not Applicable

Contacts and Locations

Study Locations

• United States
  • Minnesota
    • Rochester, Minnesota, United States, 55906
      • Mayo Clinic

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: 18 years and older (Adult, Older Adult)
• Accepts Healthy Volunteers: No
• Genders Eligible for Study: All

Description

Inclusion Criteria:

• Admitted to Mayo Clinic St. Mary's Hospital and Methodist Hospital during August 19, 2019 - August 19, 2020.
• Once a day Monday through Friday, the CT operator selects 12 patients from all of the nursing units that are participating in the trial (whether or not they are currently in the intervention group) with palliative scores of at least 7 (out of 100), i.e., those that are high risk and displayed as red in the CT GUI (unless they are already being seen by palliative care.)
• The CT operator chooses the selected patients by looking at the patients in sorted order starting with the highest score and proceeding down the list, evaluating each patient for exclusion criteria.
• Once the CT operator identifies 12 appropriate patients or once they reaches the end of the high-risk patients (score of 7 or higher) they stop.

Exclusion Criteria:

• We will exclude all patients who do not provide research authorization to review their medical records for general research studies in accordance with Minnesota Statute 144.335.
• We will exclude patients under the age of 18 years of age.
• We will exclude patients previously seen by Palliative care during the index hospital visit (i.e., green icon within CT user interface regardless of score)
• We will exclude patient who no longer have an active encounter (patients who have died or patients who have transferred to another facility are excluded) at the time of the review
• We will exclude patients currently enrolled with the Hospice service at Mayo
• We will exclude patients currently enrolled in the Palliative Homebound program (an alternative healthcare model at Mayo)
• We will exclude patients who are about to be discharged in the next 24 hours through indication of note

Study Plan

How is the study designed?

Design Details

• Primary Purpose: Health Services Research
• Allocation: Randomized
• Interventional Model: Crossover Assignment
• Masking: None (Open Label)

Number of Arms

2

Arms and Interventions

Participant Group / Arm	Intervention / Treatment
Experimental: Control Tower Intervention; For participants in the intervention arm, the results of the prediction model will be presented through a GUI interface hereby known as the Control Tower. Participants receive scores from Control Tower (0-100; higher score indicating increased need) for palliative care and are subsequently ranked from highest to lowest. Red (7 or greater) is considered high risk. The intervention will include a Control Tower operator who will interact with the inpatient palliative care consult service. The operator will monitor the Control Tower during weekday normal business hours and select daily a cohort of participants in the intervention units with the highest need of palliative care review. The final list of participants will then be sent to palliative care. The palliative care team who is on service will also assess the need for each participants, and those participants which they agree could benefit they will approach the attending clinical team to suggest a palliative care referral.	Other: Control Tower; A workstation and software tool that extracts medical data from Mayo's data mart and electronic health record, and processes it through a prediction model that determines whether a patient is suited for a palliative care consult.
No Intervention: Standard of Care; For participants who are not in an intervention period they will receive the standard of care commensurate with their clinical unit. This is feasible given that this is a pragmatic clinical trial where the investigators can easily control the communication between the control tower operator and palliative care team to prevent any contamination between clusters. In addition to the usual source of care control the investigators intentionally have calibrated the prediction model and the Control Tower review to match the average capacity of the palliative care service, knowing that that the team will still receive palliative care consults through the traditional pathway i.e. the attending care team consulting palliative care directly.

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Timely identification for need of palliative care	Measured as time in hours to the electronic record of consult by the palliative care team in the inpatient setting.	12 months

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
The number of inpatient palliative care consults	Measured by the rate of palliative care consults in the inpatient units of interest	12 months
Timely identification for need of palliative care per unit	Measured as time in hours to the electronic record of consult by the palliative care team in the inpatient setting for each of the 16 nursing units.	12 months
Transition time to hospice-designated bed	For all patients with Medicare insurance the time until transferred to a hospice-designated bed from admission.	12 months
Time to hospice designation	Measured as time in hours to the electronic record of consult by the hospice care team in the inpatient setting.	12 months
Emergency Department visit within 30 days of discharge	Measured by the number of study participants who upon discharge from the inpatient setting are readmitted to the Emergency Department at any Mayo Clinic facility within 30 days.	12 months
Hospitalization or readmission within 30 days of discharge	Measured by the number of study participants who upon discharge from the inpatient setting are readmitted to an inpatient unit at any Mayo Clinic facility within 30 days (excluding transfers and planned readmits).	12 months
ICU transfers	Measured by the number of study participants who transferred to a intensive care unit during their inpatient stay.	12 months
Ratio of inpatient hospice death to non-hospice hospital deaths	Measured by the number of deaths of study participants in hospice designated beds by the number of deaths in non-hospice beds.	12 months
Rate of discharge to external hospice	Measured by the number of participants whose electronic health record indicates discharge to external hospice.	12 months
Inpatient length of stay	Measured by the difference between admission to first unit to discharge from hospital for all study participants.	12 months

Collaborators and Investigators

• Sponsor: Mayo Clinic

Publications and helpful links

General Publications

• Wilson PM, Philpot LM, Ramar P, Storlie CB, Strand J, Morgan AA, Asai SW, Ebbert JO, Herasevich VD, Soleimani J, Pickering BW. Improving time to palliative care review with predictive modeling in an inpatient adult population: study protocol for a stepped-wedge, pragmatic randomized controlled trial. Trials. 2021 Sep 16;22(1):635. doi: 10.1186/s13063-021-05546-5.

Helpful Links

• Mayo Clinic Clinical Trials

Study record dates

Study Major Dates

• Study Start (Actual): August 19, 2019
• Primary Completion (Actual): November 18, 2020
• Study Completion (Actual): December 20, 2020

Study Registration Dates

• First Submitted: June 3, 2019
• First Submitted That Met QC Criteria: June 3, 2019
• First Posted (Actual): June 6, 2019

Study Record Updates

• Last Update Posted (Actual): December 30, 2020
• Last Update Submitted That Met QC Criteria: December 29, 2020
• Last Verified: December 1, 2020

More Information

Terms related to this study

• Keywords: Artificial Intelligence; Healthcare Delivery; Machine learning; Pragmatic Clinical Trial
• Other Study ID Numbers: 19-002315

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: NO

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No