Conversational Agent Vaccine Promotion RCT

Community-based Design and Evaluation of A Conversational Agent to Promote SARS-COV2 Vaccination in Black Churches

The objective of this study is to assess the use of and satisfaction with the ECA intervention over a 12-month period, its ability to increase SARS-CoV-2 and influenza vaccination in the BMATP community, and the comparative effectiveness of proactive engagement strategies and cultural tailoring on these factors. The Investigators will assess app use and satisfaction among these participants, along with self-reported vaccination attitudes and behavior, at 6 months and 12 months.

During this time the investigators will conduct a 2x2 factorial RCT (with the individual the unit of random assignment and measurement) to assess the impact of two app design features on engagement and outcomes: (1) the investigators will manipulate engagement mechanisms (ENGAGEMENT), including reminder notifications and trust-building dialogue by the ECA and, (2) independently manipulate cultural tailoring of vaccination promotion counseling language used by the agent (TAILORING) to either adaptive religiosity (tailored) or secular (non-tailored). The investigators' primary hypotheses are that participants with have significantly greater vaccination completion rates in the high engagement and tailored conditions at 6 months (H1) and 12 months (H2) compared to other conditions. The investigators' secondary hypotheses are that participants will use the app more, be more satisfied with the ECA, be more advanced in their stage of change, and have greater knowledge, self-efficacy, and decisional-balance from baseline to 6 and 12 months, in the high engagement and tailored conditions. In addition to the RCT participants, all members of the 12 participating BMATP churches will be invited to use the app (via snowball recruitment), to increase the utility of the social networking functions of the app by having as many users as possible.

Study Overview

• Status: Recruiting
• Conditions: Pandemics
• Intervention / Treatment: Behavioral: ECA-ACE Vaccination Promotion Intervention

Detailed Description

4.1.a.1. Randomization. Study participants will be randomized into one of the four study conditions (ENGAGEMENT x TAILORING) using blocked randomization with randomly permuted block sizes of 2 and 4, by software on the central server referenced when participants first activate the app on their phone.

Participants will continue using the app at home, and will be called at 6 months and 12 months to assess outcome measures during a videoconference call with a research assistant.

4.1.a.2. Qualitative Interviews. The investigators will conduct interviews with a sample of participants completing the study (12), as well as with Church and Health Ministry leadership (12) to understand overall acceptance of ECA-ACE and barriers to deployment and use.

4.1.a.3. Qualitative Methods for all Project Focus Groups and Interviews. Videoconference-based focus groups and interviews will be facilitated using a semi-structured interview guide, audio-recorded, and transcribed verbatim by a professional transcription service. The investigators will conduct a thematic analysis of these transcripts guided by the investigators' research questions with the aim of identifying design insights. The investigators will use grounded theory analysis, including constant comparisons, axial and selective coding, and memo writing. Using NVivo 12.5.0 software, two researchers will inductively code transcripts separately, labeling emergent phenomena in the data to arrive at a codebook. Two researchers will then independently apply the codebook to participant transcripts, meeting regularly to discuss discrepancies in the applications of the codes, updates to the codebook, and contradictory data. Sample sizes specified should be adequate to reach thematic saturation relative to the investigators' research questions.

• Study Type: Interventional
• Enrollment (Estimated): 600
• Phase: Not Applicable

Contacts and Locations

• Study Contact: Name: Madison Blain, OTD; Phone Number: (617) 858-6727‬; Email: m.blain@northeastern.edu
• Study Contact Backup: Name: Timothy W Bickmore, PhD; Phone Number: 617-373-5477; Email: t.bickmore@northeastern.edu

Study Locations

• United States
  • Massachusetts
    • Boston, Massachusetts, United States, 02115
      • Recruiting
      • Northeastern University
      • Contact: Madison Blain, OTD; Phone Number: 6178586727‬; Email: m.blain@northeastern.edu

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: 16 years and older (Adult, Older Adult)
• Accepts Healthy Volunteers: Yes

Description

Inclusion Criteria:

1. over 18 years old;
2. speaks English fluently;
3. is able to independently consent;
4. has adequate corrected vision to use the ECA system (based on a 1 minute ECA functional screener deployed over the web);
5. has adequate hearing to use the ECA system;
6. owns a recent model iPhone or Android smartphone.
7. do not meet current Boston Public Health commission guidelines for SARS-CoV-2 vaccination OR do not meet current Boston Public Health commission guidelines for Influenza vaccination.

Exclusion Criteria:

1. they are not able demonstrate comprehension of the research study;
2. they are unable to use the screener

Study Plan

How is the study designed?

Design Details

• Primary Purpose: Prevention
• Allocation: Randomized
• Interventional Model: Factorial Assignment
• Masking: Triple

Number of Arms

4

Arms and Interventions

Participant Group / Arm	Intervention / Treatment
Experimental: High engagement mechanisms, tailed content; During this time the investigators will conduct a 2x2 factorial RCT (with the individual the unit of random assignment and measurement) to assess the impact of two app design features on engagement and outcomes: (1) the investigators will manipulate engagement mechanisms (ENGAGEMENT), including reminder notifications and trust-building dialogue by the ECA and, (2) independently manipulate cultural tailoring of vaccination promotion counseling language used by the agent (TAILORING) to either adaptive religiosity (tailored) or secular (non-tailored). The investigators' primary hypotheses are that participants with have significantly greater vaccination completion rates in the high engagement and tailored conditions at 6 months (H1) and 12 months (H2) compared to other conditions.	Behavioral: ECA-ACE Vaccination Promotion Intervention; A smartphone-based embodied conversational agent that educates users and motivates them to obtain vaccinations for SARS-CoV-2 and Influenza, according to Boston Public Health Commission guidelines.
Experimental: Low engagement mechanisms, tailed content; manipulate low engagement mechanisms while provide adaptive religiosity (tailored) content	Behavioral: ECA-ACE Vaccination Promotion Intervention; A smartphone-based embodied conversational agent that educates users and motivates them to obtain vaccinations for SARS-CoV-2 and Influenza, according to Boston Public Health Commission guidelines.
Experimental: High engagement mechanism, non-tailed content; manipulate high engagement mechanisms while provide secular (non-tailored) content	Behavioral: ECA-ACE Vaccination Promotion Intervention; A smartphone-based embodied conversational agent that educates users and motivates them to obtain vaccinations for SARS-CoV-2 and Influenza, according to Boston Public Health Commission guidelines.
Experimental: Low engagement mechanism, non-tailed content; manipulate low engagement mechanisms and provide secular (non-tailored) content	Behavioral: ECA-ACE Vaccination Promotion Intervention; A smartphone-based embodied conversational agent that educates users and motivates them to obtain vaccinations for SARS-CoV-2 and Influenza, according to Boston Public Health Commission guidelines.

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Change of SARS-CoV-2 vaccination status	Vaccination completion will be assessed via self-report by asking whether participants are up-to-date with Boston Public Health Commission vaccination guidelines (yes or no) for SARS-CoV-2	baseline, 6months, 12months
Change of Influenza vaccination status Influenza vaccination status	Vaccination completion will be assessed via self-report by asking whether participants are up-to-date with Boston Public Health Commission vaccination guidelines (yes or no) for Influenza	baseline, 6months, 12months

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Change of Satisfaction Status	Self report System Usability Scale (self report), plus system use from log file analysis.	6 months, 12 months
Stage of Change for Vaccination	Self report of Precontemplation, Contemplation, Preparation, or Action/Maintenance for vaccination	baseline,6months,12months
Self-Efficacy for Vaccination	Self report of confidence in ability to get vaccinated in light of barriers	baseline,6months,12months
Decisional Balance for Vaccination	Self report of Pros and Cons of vaccination	baseline,6months,12months
Knowledge	Knowledge of COVID-19 and Influenza assessed via 10-item true/false knowledge tests.	baseline,6months,12months

Collaborators and Investigators

• Sponsor: Northeastern University
• Collaborators: Boston University; Boston Medical Center; Georgia Institute of Technology

Study record dates

Study Major Dates

• Study Start (Actual): December 12, 2022
• Primary Completion (Estimated): June 30, 2025
• Study Completion (Estimated): December 1, 2025

Study Registration Dates

• First Submitted: May 5, 2021
• First Submitted That Met QC Criteria: May 10, 2021
• First Posted (Actual): May 13, 2021

Study Record Updates

• Last Update Posted (Actual): December 22, 2023
• Last Update Submitted That Met QC Criteria: December 21, 2023
• Last Verified: December 1, 2023

More Information

Terms related to this study

• Other Study ID Numbers: IRB#18-07-02

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: YES

IPD Plan Description

The dissemination plan has two overall goals:

1. Scientific Dissemination of Study Findings

   For our scientific stakeholders, the investigators will publish findings in the peer-reviewed scientific literature, and also present findings at scientific conferences and through invited seminars to share these findings. In addition, the investigators plan to make data available for collaborative analyses with colleagues around the world.

2. Community Dissemination of Intervention

In the final two years of the effort, intervention materials will be revised based on feedback and lessons learned from the RCTs. The smartphone app will be opened so that anyone can download and use it, and extended to automatically collect basic sociodemographics, primary outcome measures, and usage directly from users if they electronically consent upon initiation of the app and saved to the central database, so that dissemination, use, and outcomes can be tracked and evaluated during dissemination.

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No