Effectiveness of Interactive Voice Response for COVID-19 Vaccination Training in the Democratic Republic of the Congo

Effectiveness of Interactive Voice Response (IVR) for COVID-19 Vaccination Training of Frontline Health Workers: Experimental Evidence From the Democratic Republic of the Congo

This study seeks to evaluate the effectiveness of a remote training course delivered via interactive voice response (IVR) to the mobile phones of frontline providers in the Democratic Republic of the Congo (DRC) during the COVID-19 pandemic. The randomized trial will examine the impact of the training on health workers' knowledge, beliefs and intended behaviors related to COVID-19 vaccines and vaccine administration. Potential enrollees will be consented first, after which the participants assigned to the treatment group will be invited to participate in the training over a one month period, while the control group will receive the intervention following a month delay, which will allow for the comparison of knowledge and beliefs with a group that is controlled for bias introduced by time. We hypothesize that the training will lead to increased knowledge about vaccines, lower vaccine hesitancy, and increased preparedness to administer vaccines among trained health workers.

Study Overview

• Status: Completed
• Conditions: COVID-19 Vaccine Knowledge; COVID-19 Vaccine Beliefs
• Intervention / Treatment: Behavioral: COVID-19 Vaccine IVR Training; Behavioral: Control Condition

Detailed Description

Background:

As COVID-19 continues to pose a significant threat worldwide, training health workers to administer the COVID-19 vaccine and encourage uptake of the vaccine is essential to limiting the number of deaths and ultimately ending the pandemic. Providing adequate training to health workers in low- and middle-income countries (LMICs) had already been logistically and economically challenging, but the onset of the pandemic has intensified the challenge of providing critical and rapidly evolving health information to health workers at a time when it is needed most, particularly in low bandwidth areas where access to accurate sources of information are limited.

A growing body of research suggests that effective and efficient training can be provided through mobile health (mHealth) solutions. The rapid expansion of mobile phone technology available throughout LMICs has enabled the scaling of health education delivery via mobile phone. However, while app-based learning has grown in popularity, drawbacks include limited access to smartphones and the data costs to access the internet, if available. Interactive voice response (IVR) systems pose an alternative solution by employing pre-recorded audio files in local languages, allowing users to receive information from mobile phones without the need for internet access.

Aims:

In this study, we investigate the effectiveness of an IVR training intervention for improving knowledge of COVID-19 vaccines and vaccine administration among health workers in the Democratic Republic of the Congo (DRC). We will also examine differences in effectiveness across subgroups including by geographic region, professional roles (e.g., nurses, physicians) and type of health worker facility.

Methods:

We are collaborating with Viamo, a global social enterprise that has established IVR systems in more than 23 countries, to conduct a randomized controlled trial to measure the effects of the IVR remote training on health workers' knowledge and beliefs about COVID-19 vaccines and preparedness to administer vaccines. In coordination with the DRC Ministry of Health, Viamo has identified roughly 9,000 facility-based health workers prioritized for participation. Participants will be randomized 1:1 into a treatment and a control group, stratified by geographic region and professional roles to ensure equal representation across regions and types of health workers.

The participants in the treatment group will receive the intervention at the start of the study while the participants in the control group will receive the intervention following a one-month delay, which will allow for the comparison of knowledge, attitudes and practices with a group that is controlled for bias introduced by time. The duration of the study for participants will be roughly one month, which is the time it takes to complete the 5 required IVR training modules and a baseline and endline survey.

Intervention:

Training content is being developed by Viamo in collaboration with the Stanford Center for Health Education and the DRC Ministry of Public Health. The intervention consists of 5 IVR-based modules conveying information about COVID-19 vaccines and vaccine administration best practices. Module topics include the following: 1) the importance of COVID-19 vaccines and recipient eligibility, 2) overview of and differences between approved COVID-19 vaccines , 3) surveillance of adverse vaccine events, 4) organizing a vaccination site, and safe storage and handling COVID-19 vaccines, and 5) micro planning and communication strategies for COVID-19 vaccination efforts. The training will be offered in five local languages: French, Lingala, Swahili, Kikongo, and Tshiluba.

Sample size calculation:

The DRC Ministry of Health identified roughly 9,000 health workers located in 13 focal provinces for prioritization of the training. With a total sample size of 9,000 (including both arms), we estimate a minimum detectable effect size (MDES) of 5.6 percentage points (at 80 percent power and a 5 percent significance level). Because of the voluntary nature of the IVR training participation, we anticipate a response rate of lower than 50 percent. At 80 percent power and 5 percent significance, a sample size of 2900 participants (over both arms) is needed to detect a 10 percent difference.

Data collection and training:

Data will be collected via the IVR platform. Treatment and control groups will complete a baseline and endline survey administered at the same time. The baseline survey will include a brief knowledge assessment, questions about participants' vaccination status, beliefs about COVID-19 vaccines, and preparedness to administer vaccines. The baseline also includes demographic and background questions about participants' prior work experience. The endline survey includes a more extensive knowledge assessment and questions about participants' vaccination status, beliefs about COVID-19 vaccines, and preparedness to administer vaccines.

All surveys will be completed directly by participants via IVR calls direct to participants' mobile phones. Surveys and training content will include audio prompts to overcome literacy barriers and will be offered in one of five local languages, selected by participants directly.

Participants will be trained on how to use the IVR technology on their mobile devices at the start of the study. Callers interact with pre-recorded audio files in their local languages and can choose what information they want to receive using their mobile phone keypads ("For more information on prevention, please press 1").

While the intervention group will be invited to begin the IVR training immediately after completing the baseline survey, the control arm will have a roughly one month delay before receiving the intervention. Besides the short delay, every aspect for treatment and control arms will remain the same.

Data Analysis:

The primary analysis will be based on intention-to-treat at the unit of the individual CHW. The analysis will include randomization strata fixed effects and control for baseline covariates to improve precision of estimates. Because we will have granular by-module completion data across participants, we will also conduct a treatment-on-the-treated instrumental variables analysis to examine the impact of the training on outcomes, among participants completing the training.

• Study Type: Interventional
• Enrollment (Actual): 8959
• Phase: Not Applicable

Contacts and Locations

Study Locations

• United States
  • California
    • Stanford, California, United States, 94305
      • Stanford University

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: 18 years and older (Adult, Older Adult)
• Accepts Healthy Volunteers: No
• Genders Eligible for Study: All

Description

Inclusion Criteria:

- Identified by the DRC Ministry of Health as priority health worker to receive the training

Exclusion Criteria:

- Not identified by the DRC Ministry of Health as priority health worker to receive the training

Study Plan

How is the study designed?

Design Details

• Primary Purpose: Treatment
• Allocation: Randomized
• Interventional Model: Parallel Assignment
• Masking: Double

Number of Arms

2

Arms and Interventions

Participant Group / Arm	Intervention / Treatment
Experimental: Treatment; The participants in the treatment group will be invited to participate in a 5-module IVR training covering an overview of available COVID-19 vaccines and best practices for vaccine administration.	Behavioral: COVID-19 Vaccine IVR Training; The intervention consists of 5 interactive voice response (IVR) modules conveying information about COVID-19 vaccines and vaccine administration best practices. Module topics include the following: 1) the importance of COVID-19 vaccines and recipient eligibility, 2) overview of and differences between approved COVID-19 vaccines , 3) surveillance of adverse vaccine events, 4) organizing a vaccination site, and safe storage and handling COVID-19 vaccines, and 5) micro planning and communication strategies for COVID-19 vaccination efforts. The training will be offered in five local languages: French, Lingala, Swahili, Kikongo, and Tshiluba.
Active Comparator: Control; Participants in the control group will receive the intervention following a one-month delay allowing for a comparison of outcomes with the treatment group.	Behavioral: Control Condition; The control arm will receive the COVID-19 Vaccine IVR Training after an estimated month-long delay. The control arm will be told upon completion of the baseline survey that the training will commence at a later estimated date and will not receive any additional information until the treatment group completes training.

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
COVID-19 vaccine knowledge	Standardized score on an assessment of 10 knowledge questions about COVID-19 vaccines and vaccine administration, drawn directly from the IVR curriculum	Collected after the estimated 1 month time period to complete the IVR training
COVID-19 vaccination status	Current vaccination status (i.e., having received at least one dose of a COVID-19 vaccine)	Collected after the estimated 1 month time period to complete the IVR training
COVID-19 vaccine promotion	Whether participant has ever encouraged someone to receive a COVID-19 vaccine	Collected after the estimated 1 month time period to complete the IVR training

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
COVID-19 vaccine administration preparedness	Agreement (on 3 point likert scale) with statement that participant feels confident and prepared to administer COVID-19 vaccines. Scale options: 1-disagree (minimum), 2-unsure, 3-agree (maximum), with maximum representing the better outcome.	Collected after the estimated 1 month time period to complete the IVR training
Belief about COVID-19 vaccination safety	Agreement (on 3 point likert scale) with statement that COVID-19 vaccines are safe. Scale options: 1-disagree (minimum), 2-unsure, 3-agree (maximum), with maximum representing the better outcome.	Collected after the estimated 1 month time period to complete the IVR training
Belief about COVID-19 vaccination efficacy	Agreement (on 3 point likert scale) with statement that COVID-19 vaccines are effective. Scale options: 1-disagree (minimum), 2-unsure, 3-agree (maximum), with maximum representing the better outcome.	Collected after the estimated 1 month time period to complete the IVR training

Collaborators and Investigators

• Sponsor: Stanford University
• Collaborators: Viamo

Publications and helpful links

General Publications

• Arie S. Can mobile phones transform healthcare in low and middle income countries? BMJ. 2015 Apr 22;350:h1975. doi: 10.1136/bmj.h1975. No abstract available.
• Agarwal S, Labrique A. Newborn health on the line: the potential mHealth applications. JAMA. 2014 Jul 16;312(3):229-30. doi: 10.1001/jama.2014.6371. No abstract available.

Study record dates

Study Major Dates

• Study Start (Actual): November 9, 2021
• Primary Completion (Actual): February 10, 2022
• Study Completion (Actual): February 10, 2022

Study Registration Dates

• First Submitted: October 11, 2021
• First Submitted That Met QC Criteria: November 2, 2021
• First Posted (Actual): November 4, 2021

Study Record Updates

• Last Update Posted (Actual): February 21, 2022
• Last Update Submitted That Met QC Criteria: February 17, 2022
• Last Verified: February 1, 2022

More Information

Terms related to this study

• Additional Relevant MeSH Terms: Coronavirus Infections; Coronaviridae Infections; Nidovirales Infections; RNA Virus Infections; Virus Diseases; Infections; Respiratory Tract Infections; Respiratory Tract Diseases; Pneumonia, Viral; Pneumonia; Lung Diseases; COVID-19
• Other Study ID Numbers: 61246

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: Undecided

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No
• product manufactured in and exported from the U.S.: No