Nudging Flu Vaccination in Patients at Moderately High Risk for Flu and Flu-related Complications

This study will test the relative efficacy of high-risk messages in increasing flu shot rates in patients at moderately high risk for flu and complications (those in the top 11-20% of risk). It will also examine whether informing patients that their high-risk status was determined by analyzing their medical records or by an artificial intelligence (AI) / machine-learning (ML) algorithm analyzing their medical records will affect the likelihood of receiving a flu vaccine.

Study Overview

• Status: Completed
• Conditions: Influenza; Health Promotion; Health Behavior; Risk Reduction; Vaccination
• Intervention / Treatment: Behavioral: Risk Reduction

Detailed Description

Almost everyone age 6 months or older can benefit from the vaccine, which can reduce illnesses, missed work, hospitalizations, and death by reducing the likelihood of contracting influenza. Flu shots are particularly important for patients at high risk of experiencing severe outcomes.

In the 2020-21 and 2021-22 flu seasons, the study team sent messages to Geisinger patients in the top 10% of risk for flu and complications according to an artificial intelligence algorithm. Messages that disclosed patients' risk status significantly increased flu vaccination rates. Additionally, messages that included risk information were most effective in patients at relatively lower risk (those in the top 4-10%) compared with those at the highest risk (top 3%).

The present work will test the effectiveness of high-risk messages in patients who are in the top 11-20% of risk, at high risk but lower than previous studies. These communications will inform patients they are at high risk with either (a) no additional explanation, (b) an explanation that this determination comes from an analysis of their medical records, or (c) the additional explanation that an AI or ML algorithm made this determination.

• Study Type: Interventional
• Enrollment (Actual): 40671
• Phase: Not Applicable

Contacts and Locations

Study Locations

• United States
  • Pennsylvania
    • Danville, Pennsylvania, United States, 17822
      • Geisinger

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: 18 years and older (Adult, Older Adult)
• Accepts Healthy Volunteers: No
• Genders Eligible for Study: All

Description

Inclusion Criteria:

• Included on a list of active Geisinger patients (all patients on this list attended at least one primary care appointment at Geisinger between 10/1/2008 and 4/13/2022, and either had a Geisinger primary care provider assigned as of April 2022, or were in the Electronic Health Record [EHR] since at least September 2021 and had at least one encounter in 2020-2022)
• Aged 18 or older
• In the top 11-20% of risk for flu and flu complications, according to Medial's flu complications machine learning algorithm (which operates on coded EHR data)
• Has a Geisinger PCP assigned as of August 2022
• Has had an encounter in the last 2 years as of August 2022

Exclusion criteria:

- Cannot be contacted via any of the communication modalities (e.g., letter, patient portal, SMS) being used in the study, either due to insufficient/missing contact information in the EHR or because they opted out of all modalities

Study Plan

How is the study designed?

Design Details

• Primary Purpose: Prevention
• Allocation: Randomized
• Interventional Model: Parallel Assignment
• Masking: Single

Number of Arms

5

Arms and Interventions

Participant Group / Arm	Intervention / Treatment
No Intervention: Passive control; Patients in the passive control arm will receive no additional pro-vaccination intervention beyond the health system's normal efforts. Although some patients are currently targeted for flu vaccination encouragement due to a conventional non-ML assessment that they are at high risk for complications, these patients are not told that they are at high risk or that they have been targeted.
Experimental: Active control; Patients in the active control arm will receive messages reminding them to get a flu shot without being advised of their risk status.	Behavioral: Risk Reduction; Letter, patient portal, SMS and/or another modality
Experimental: High risk only; Patients in this treatment arm will receive messages telling them they have been identified to be at high risk for flu complications, without specifying how or why the health system believes this to be the case.	Behavioral: Risk Reduction; Letter, patient portal, SMS and/or another modality
Experimental: Risk based on medical records; Patients in this treatment arm will receive messages telling them they have been identified to be at high risk for flu complications via review of their medical records.	Behavioral: Risk Reduction; Letter, patient portal, SMS and/or another modality
Experimental: High risk based on algorithm; Patients in this treatment arm will receive messages telling them they have been identified to be at high risk for flu complications via analysis of their medical records by a computer algorithm.	Behavioral: Risk Reduction; Letter, patient portal, SMS and/or another modality

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Flu vaccination	Received a flu vaccination within within 6 weeks of the patient's study start date	Within 6 weeks of the patient's study start date

Other Outcome Measures

Outcome Measure	Measure Description	Time Frame
High confidence flu diagnosis	Patient received a flu diagnosis via a positive polymerase chain reaction (PCR)/antigen/molecular test (yes/no) during the 2022-23 flu season (from the patient's study start date through April 30, 2023).	Up to 8 months
"Likely flu" diagnosis	Received a "high confidence flu" diagnosis (with positive PCR/antigen/molecular test) and/or "likely flu" diagnosis (as assessed via International Classification of Disease [ICD] codes or Tamiflu administration or positive PCR/antigen/molecular test) (yes/no) during the 2022-23 flu season (from the patient's study start date through April 30, 2023). Note that "likely flu" is a superset of the "high confidence flu" diagnoses.	Up to 8 months
Flu complications	Diagnosed with flu-related complications (yes/no) from the patient's study start date through July 31, 2023.	Up to 11 months
ER visits	Number of ER visits from the patient's study start date through July 31, 2023.	Up to 11 months
Hospitalizations	Number of hospitalizations from the patient's study start date through July 31, 2023.	Up to 11 months
COVID-19 vaccination rates	Received at least one COVID-19 vaccination (yes/no) during the 2022-23 flu season (from the patient's study start date through April 30, 2023).	Up to 8 months

Collaborators and Investigators

• Sponsor: Geisinger Clinic
• Collaborators: Massachusetts Institute of Technology; National Bureau of Economic Research, Inc.
• Investigators: Principal Investigator: Christopher Chabris, PhD, Geisinger Clinic

Study record dates

Study Major Dates

• Study Start (Actual): September 13, 2022
• Primary Completion (Actual): October 26, 2022
• Study Completion (Actual): October 26, 2022

Study Registration Dates

• First Submitted: August 18, 2022
• First Submitted That Met QC Criteria: August 19, 2022
• First Posted (Actual): August 22, 2022

Study Record Updates

• Last Update Posted (Actual): December 5, 2022
• Last Update Submitted That Met QC Criteria: December 2, 2022
• Last Verified: December 1, 2022

More Information

Terms related to this study

• Keywords: Choice Architecture; Flu Vaccine; Machine Learning; Perceived Credibility
• Other Study ID Numbers: 2022-0410

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: Yes
• IPD Plan Description: Data with no personally identifiable information will be made available to other researchers on the Open Science Framework for transparency. This will include the essential data and code needed to replicate the analysis that yielded reported findings.
• IPD Sharing Time Frame: The data will become available after publication of study results in a scientific journal and will be available as long as the Open Science Framework hosts the data.
• IPD Sharing Access Criteria: The data on the Open Science Framework will be open to anyone requesting that information.
• IPD Sharing Supporting Information Type: Study Protocol

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No