The Doctors for Coronavirus Prevention Project Thanksgiving / Christmas Messaging Campaign

Facebook ads with physician-delivered videos were shown before the Thanksgiving and Christmas holidays and focused on staying safe during the COVID pandemic by limiting travel and mask-wearing.

Study Overview

• Status: Completed
• Conditions: Covid19
• Intervention / Treatment: Behavioral: Facebook Ads on the importance of staying safe during the Thanksgiving / Christmas holiday

Detailed Description

The investigators used Facebook ads to show a 20 second video clip recorded by Massachusetts General Hospital, Harvard and Lynn Community health center doctors (6 people in all) to approximately 20,000,000 Facebook users. The ads will be shown before the Thanksgiving and Christmas holiday and will focus on staying safe - limiting travel and mask-wearing. The investigators will randomize exposure to the ad campaign at the ZIP code or county level to ask:

Do the videos change mobility and Thanksgiving and Christmas holiday travel? Do they reduce the spread of COVID-19?

Are there spillover impacts from the video messages? For example, if individuals decide to stay home, then do the geographical regions that tend to be visited by people from treated areas experience any effects, either through information spillovers or through a reduction in travel?

• Study Type: Interventional
• Enrollment (Actual): 820
• Phase: Not Applicable

Contacts and Locations

Study Locations

• United States
  • Massachusetts
    • Cambridge, Massachusetts, United States, 02138
      • JPAL North America

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: 18 years and older (Adult, Older Adult)
• Accepts Healthy Volunteers: No
• Genders Eligible for Study: All

Description

Inclusion Criteria:

- Individuals viewing the ads must be Facebook users, 18 years and older. Facebook will decide specifically which users receive the messages within a target geographical area (zip or county).

Exclusion Criteria:

-

Study Plan

How is the study designed?

Design Details

• Primary Purpose: Prevention
• Allocation: Randomized
• Interventional Model: Sequential Assignment
• Masking: None (Open Label)

Number of Arms

2

Arms and Interventions

Participant Group / Arm	Intervention / Treatment
Experimental: High-Intensity Treatment Arm; Treatment: Individuals received approximately 3 Facebook ads over a 2-week period. Each ad contained a short video recorded by a physician using a script that discusses staying safe during the holidays by considering not traveling and using a mask when appropriate. Randomization to treatment: The 13 states which centrally report COVID-19 cases at the ZCTA level contained 829 counties. At Thanksgiving, 9 counties were excluded due to data limitations. At Christmas, 62 counties were excluded due to data limitations and potential negative impacts in rural, conservative areas resulting from polarization in the wake of the 2020 presidential election. Approximately half of the counties were randomized to high-intensity treatment with the remaining randomized to low-intensity treatment. In high-intensity counties, 3/4 of ZCTAs were treated (i.e., Facebook users in those ZCTAs received ads) and 1/4 were not. In low-intensity counties, 1/4 of ZCTAs were treated and 3/4 were not.	Behavioral: Facebook Ads on the importance of staying safe during the Thanksgiving / Christmas holiday; Facebook ads to show a 15 second video clip recorded by MGH, Harvard and Lynn Community health center and other health care professionals. The ads will be shown before the Thanksgiving/Christmas holiday and will focus on staying safe - limiting travel and mask-wearing. The investigators will randomize exposure to the ad campaign at the ZIP code or county level.
No Intervention: Low-Intensity Control Arm; Control: Individuals did not receive ads containing short physician-recorded videos. Randomization to treatment: The 13 states which centrally report COVID-19 cases at the ZCTA level contained 829 counties. At Thanksgiving, 9 counties were excluded due to data limitations. At Christmas, 62 counties were excluded due to data limitations and potential negative impacts in rural, conservative areas resulting from polarization in the wake of the 2020 presidential election. Approximately half of the counties were randomized to high-intensity treatment with the remaining randomized to low-intensity treatment. In high-intensity counties, 3/4 of ZCTAs were treated (i.e., Facebook users in those ZCTAs received ads) and 1/4 were not. In low-intensity counties, 1/4 of ZCTAs were treated and 3/4 were not.

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Facebook Movement Metric	The change in movement metric is the percent change in distance covered in a day compared to the same day of the week in the benchmark period of February 2-29, 2020, by people who started the day in a particular location. We define holiday travel as travel during the three days preceding each holiday, since the available data does not allow us to compute the impact of the intervention on the return travel (after the holiday). The reason is that the mobility data describes the behavior throughout the day, for people who were in each county that morning. Since the campaign was targeted based on home location, we can only capture its impact on travel away from home, not back home.	November 26, 2020 (Thanksgiving); December 24-25, 2020 (Christmas); February 2-29, 2020 (Baseline benchmark)
Percentage Leaving Home on Day of Holiday	The Facebook stay put metric is the percentage of people who stay within a small geographical area (a "tile" of 600m*600m in which they started the day). We use it to compute the share of people leaving home on the day of the holiday (i.e. this variable = 1 - stay put on the day of the holiday).	November 26, 2020 (Thanksgiving); December 24-25, 2020 (Christmas)
Inverse Hyperbolic Sine of COVID-19 Cases	Inverse hyperbolic sine of COVID-19 cases during a 14 day period starting 5 days after each holiday	December 1-14 (Thanksgiving arms) and January 1-14 (Christmas arms)

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Knowledge of COVID-19 Prevention Message (Recall of ad, Intent to Travel, Mask Wearing and Beliefs About Travel)	Facebook will ask a set of users in treated zip codes to answer four survey questions (each user answers only one question each). A small within-zip control group will be held from being treated, enabling comparisons between those who see the videos and those who don't. These questions will be asked a few days after the ad is shown on the User's feed and will include 1) recall of the ad; 2) intention of traveling over holiday; 3) intention of wearing a mask; and 4) beliefs about whether people should travel over holiday.	up to one month
Average Number of Tiles People Occupy (Mobility Measure)	this measures each day "how much people move around by counting the number of level-16 Bing tiles they are seen in within a day. People seen in more tiles are probably moving around more, while people seen in fewer are probably moving around less. Each day take eligible people in a given region and compute the number of distinct tiles they were seen in." This is aggregated to the county level.	up to one month
Percentage of Eligible Participants Only Observed in a Single Level-16 Bing Tile (no Change in Movement)	Facebook "Stay-put" data: this measures each day "the percentage of eligible people who are only observed in a single level-16 Bing tile (600m x 600m) during the course of a day" aggregated to they county level.	up to one month

Collaborators and Investigators

• Sponsor: National Bureau of Economic Research, Inc.
• Collaborators: Massachusetts General Hospital; Yale University; Massachusetts Institute of Technology; Stanford University; Harvard University

Publications and helpful links

General Publications

• Breza E, Stanford FC, Alsan M, Alsan B, Banerjee A, Chandrasekhar AG, Eichmeyer S, Glushko T, Goldsmith-Pinkham P, Holland K, Hoppe E, Karnani M, Liegl S, Loisel T, Ogbu-Nwobodo L, Olken BA, Torres C, Vautrey PL, Warner ET, Wootton S, Duflo E. Effects of a large-scale social media advertising campaign on holiday travel and COVID-19 infections: a cluster randomized controlled trial. Nat Med. 2021 Sep;27(9):1622-1628. doi: 10.1038/s41591-021-01487-3. Epub 2021 Aug 19.

Study record dates

Study Major Dates

• Study Start (Actual): November 12, 2020
• Primary Completion (Actual): February 11, 2021
• Study Completion (Actual): February 11, 2021

Study Registration Dates

• First Submitted: November 13, 2020
• First Submitted That Met QC Criteria: November 20, 2020
• First Posted (Actual): November 25, 2020

Study Record Updates

• Last Update Posted (Actual): December 21, 2021
• Last Update Submitted That Met QC Criteria: December 15, 2021
• Last Verified: December 1, 2021

More Information

Terms related to this study

• Additional Relevant MeSH Terms: Coronaviridae Infections; Nidovirales Infections; RNA Virus Infections; Virus Diseases; Infections; Coronavirus Infections
• Other Study ID Numbers: 2003000118A010

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: Undecided
• IPD Plan Description: The pre-analysis plan will be posted publicly. The ability to share IPD will be based on discussions with our partner organization.

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No