Personalized Nudging to Increase Influenza Vaccinations

A Prospective Randomized Trial of Personalized Nudges to Increase Influenza Vaccinations

The purpose of this study is to prospectively test whether personalized, message-based nudges can increase flu vaccination compared with nudges that are not personalized or no nudges.

Study Overview

• Status: Completed
• Conditions: Influenza, Human
• Intervention / Treatment: Behavioral: Reminder

Detailed Description

On average, 8% of the US population gets sick from influenza each flu season. Since 2010, the annual disease burden of influenza in the U.S. has included 9-41 million illnesses, 140,000-710,000 hospitalizations, and 12,000-52,000 deaths. The Centers for Disease Control and Prevention (CDC) recommends flu vaccination to everyone aged 6 months and older, with rare exceptions; almost anyone can benefit from the vaccine, which can reduce illnesses, missed work, hospitalizations, and death.

Successful efforts to get patients vaccinated against influenza have included text message reminders timed to precede upcoming flu shot-eligible appointments by up to 3 days. For example, the Roybal-funded flu shot megastudy conducted with Penn Medicine and Geisinger patients assessed the effectiveness of numerous types of messages in increasing vaccination, relative to standard communications by the respective health systems; an average 2.1 percentage point absolute increase (or 5% relative increase) in flu shots occurred due to the messages. Similarly, follow-up analysis of the megastudy using machine learning revealed that interventions differed in relative effectiveness across groups of patients as a function of overlapping covariates (e.g., age, sex, insurance type, and comorbidities). This analysis found that nudges optimally targeted to subgroups who responded most strongly to those nudges in the megastudy would have resulted in up to three times the increases in vaccination observed when simply randomly assigning patients to messages.

The present study aims to prospectively test the efficacy of a patient-facing, message-based nudge via short message service (SMS) texts that is predicted by this retrospective machine learning algorithm to be most effective for them (Personalized Nudge) relative to Passive Control (no messages), Active Control (simple reminder message), and Best Nudge (best performing message from the 2020 megastudy). Patients will be randomized to one of these four arms.

Of the 19 original messages from the megastudy, 12 can be carried out at Geisinger in 2024; the other 7 are either no longer relevant (e.g., those that refer to an ongoing coronavirus pandemic) or cannot be carried out due to a technical limitation (e.g., Geisinger is unable to send pictures, so nudges with pictures are excluded). A treatment assignment tree based on the algorithm described above will be applied to this subset of nudges to generate rules for assigning patients to personalized messages based on observed covariates.

The last patients will be enrolled on December 28th for appointments scheduled on December 31st. At least 90,000 patients are expected to be enrolled.

• Study Type: Interventional
• Enrollment (Actual): 77482
• Phase: Not Applicable

Contacts and Locations

Study Locations

• United States
  • Pennsylvania
    • Danville, Pennsylvania, United States, 17822
      • Geisinger Clinic

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: Adult; Older Adult
• Accepts Healthy Volunteers: No

Description

Inclusion Criteria:

• Age 18+
• Has not received the 2024 flu vaccine according to the Geisinger electronic health record (EHR) prior to randomization
• Has a non-acute, flu-shot eligible, in-person Geisinger appointment scheduled with enough time to be randomized
• Has a Geisinger primary care provider

Exclusion Criteria:

• Cannot be contacted by SMS (e.g., due to insufficient/missing contact information in the EHR or because they opted out)
• Appointment type or department not approved for outreach by Geisinger leadership at the time of outreach
• Has an allergy to flu vaccines according to any EHR allergy table known to the study team
• Has a health maintenance modifier indicating they are permanently discontinued from receiving a seasonal flu shot
• Shares a phone number with someone who has already been enrolled

Study Plan

How is the study designed?

Design Details

• Primary Purpose: Prevention
• Allocation: Randomized
• Interventional Model: Parallel Assignment
• Masking: None (Open Label)

Number of Arms

4

Arms and Interventions

Participant Group / Arm	Intervention / Treatment
No Intervention: Passive Control; Patients randomized to this arm will receive no special communications, beyond what Geisinger sends out as standard practice.
Experimental: Active Control; Patients will receive a simple message encouraging them to get a flu shot at their appointment.	Behavioral: Reminder; Flu shot messages via SMS
Experimental: Best Nudge; Patients will receive the nudge found to be numerically most effective in the megastudy, including language that a flu vaccine is "reserved" for them at their upcoming appointment.	Behavioral: Reminder; Flu shot messages via SMS
Experimental: Personalized Nudge; Patients will receive the nudge predicted to be most effective for them on the basis of the machine learning-derived treatment assignment trees.	Behavioral: Reminder; Flu shot messages via SMS

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Number of Patients With Flu Shot Receipt Between Enrollment Date and Target Appointment Date	Our field experiment will be conducted with Geisinger Health patients via SMS messages sent prior to their first flu shot-eligible appointment during the study period, referred to as the "target appointment." The key dependent variable is whether patients receive a flu shot at or before their target appointment (as recorded in their electronic health records). If patients cancel or do not show up for their target appointment after they have been randomized to a treatment and then schedule a new appointment during the study period, their new flu-shot eligible appointment becomes the target appointment and the outcome window extends from three days prior to the original appointment through the date of the appointment. Patients who miss their target appointment and do not reschedule within the study period will still be included in the analysis. Their outcome window is from three days prior to the original appointment through the date of the original canceled appointment.	Between the enrollment date and target appointment date (at least 4 days and up to 4 months)

Other Outcome Measures

Outcome Measure	Measure Description	Time Frame
Number of Patients With Flu Shot Receipt On or Before December 31, 2024	Received a flu shot on or before December 31, 2024	Up to 4 months after randomization
Number of Patients with Flu Shot Receipt Between Enrollment Date and First Eligible Appointment	Patients received the flu shot at or before their first eligible appointment	4 Days
Number of Patients with Flu Diagnosis (encounter diagnosis or flu test)	Patients received a flu diagnosis via encounter diagnosis or flu test between enrollment and April 30, 2025.	Up to 8 months after randomization
Number of Patients with Flu-related Complications	Patients experienced flu-related complications before as defined by relevant diagnosis, hospitalization, or death, between enrollment and July 31, 2025 as recorded in the electronic health record	Up to 11 months after randomization

Collaborators and Investigators

• Sponsor: Geisinger Clinic
• Collaborators: University of Michigan; Massachusetts Institute of Technology; Abdul Latif Jameel Poverty Action Lab
• Investigators: Principal Investigator: Christopher F Chabris, PhD, Geisinger Clinic

Publications and helpful links

General Publications

• Milkman KL, Patel MS, Gandhi L, Graci HN, Gromet DM, Ho H, Kay JS, Lee TW, Akinola M, Beshears J, Bogard JE, Buttenheim A, Chabris CF, Chapman GB, Choi JJ, Dai H, Fox CR, Goren A, Hilchey MD, Hmurovic J, John LK, Karlan D, Kim M, Laibson D, Lamberton C, Madrian BC, Meyer MN, Modanu M, Nam J, Rogers T, Rondina R, Saccardo S, Shermohammed M, Soman D, Sparks J, Warren C, Weber M, Berman R, Evans CN, Snider CK, Tsukayama E, Van den Bulte C, Volpp KG, Duckworth AL. A megastudy of text-based nudges encouraging patients to get vaccinated at an upcoming doctor's appointment. Proc Natl Acad Sci U S A. 2021 May 18;118(20):e2101165118. doi: 10.1073/pnas.2101165118.
• Patel MS, Milkman KL, Gandhi L, Graci HN, Gromet D, Ho H, Kay JS, Lee TW, Rothschild J, Akinola M, Beshears J, Bogard JE, Buttenheim A, Chabris C, Chapman GB, Choi JJ, Dai H, Fox CR, Goren A, Hilchey MD, Hmurovic J, John LK, Karlan D, Kim M, Laibson D, Lamberton C, Madrian BC, Meyer MN, Modanu M, Nam J, Rogers T, Rondina R, Saccardo S, Shermohammed M, Soman D, Sparks J, Warren C, Weber M, Berman R, Evans CN, Lee SH, Snider CK, Tsukayama E, Van den Bulte C, Volpp KG, Duckworth AL. A Randomized Trial of Behavioral Nudges Delivered Through Text Messages to Increase Influenza Vaccination Among Patients With an Upcoming Primary Care Visit. Am J Health Promot. 2023 Mar;37(3):324-332. doi: 10.1177/08901171221131021. Epub 2022 Oct 4.

Helpful Links

• Centers for Disease Control and Prevention. (2020). Disease Burden of Influenza.

Study record dates

Study Major Dates

• Study Start (Actual): September 9, 2024
• Primary Completion (Actual): December 31, 2024
• Study Completion (Actual): December 31, 2024

Study Registration Dates

• First Submitted: August 20, 2024
• First Submitted That Met QC Criteria: August 20, 2024
• First Posted (Actual): August 22, 2024

Study Record Updates

• Last Update Posted (Actual): March 25, 2025
• Last Update Submitted That Met QC Criteria: January 13, 2025
• Last Verified: January 1, 2025

More Information

Terms related to this study

• Keywords: machine learning; vaccination; health promotion; health behavior; behavioral economics; nudges; reminder systems; personalization; behavioral risk assessment
• Additional Relevant MeSH Terms: Respiratory Tract Infections; Infections; Orthomyxoviridae Infections; RNA Virus Infections; Virus Diseases; Respiratory Tract Diseases; Influenza, Human
• Other Study ID Numbers: 2024-0561

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: YES
• IPD Plan Description: Data with no personally identifiable information will be made available to other researchers on the Open Science Framework for transparency. This will include the essential data and code needed to replicate the analysis that yielded reported findings.
• IPD Sharing Time Frame: By the paper's online publication date. Data will remain available for as long as the Open Science Framework hosts it.
• IPD Sharing Access Criteria: The data on the Open Science Framework will be open to anyone requesting that information.
• IPD Sharing Supporting Information Type: STUDY_PROTOCOL; SAP; ANALYTIC_CODE

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No
• product manufactured in and exported from the U.S.: No