Effectiveness of chatbots on COVID vaccine confidence and acceptance in Thailand, Hong Kong, and Singapore

Abstract

Chatbots have become an increasingly popular tool in the field of health services and communications. Despite chatbots' significance amid the COVID-19 pandemic, few studies have performed a rigorous evaluation of the effectiveness of chatbots in improving vaccine confidence and acceptance. In Thailand, Hong Kong, and Singapore, from February 11th to June 30th, 2022, we conducted multisite randomised controlled trials (RCT) on 2,045 adult guardians of children and seniors who were unvaccinated or had delayed vaccinations. After a week of using COVID-19 vaccine chatbots, the differences in vaccine confidence and acceptance were compared between the intervention and control groups. Compared to non-users, fewer chatbot users reported decreased confidence in vaccine effectiveness in the Thailand child group [Intervention: 4.3 % vs. Control: 17%, P = 0.023]. However, more chatbot users reported decreased vaccine acceptance [26% vs. 12%, P = 0.028] in Hong Kong child group and decreased vaccine confidence in safety [29% vs. 10%, P = 0.041] in Singapore child group. There was no statistically significant change in vaccine confidence or acceptance in the Hong Kong senior group. Employing the RE-AIM framework, process evaluation indicated strong acceptance and implementation support for vaccine chatbots from stakeholders, with high levels of sustainability and scalability. This multisite, parallel RCT study on vaccine chatbots found mixed success in improving vaccine confidence and acceptance among unvaccinated Asian subpopulations. Further studies that link chatbot usage and real-world vaccine uptake are needed to augment evidence for employing vaccine chatbots to advance vaccine confidence and acceptance.

Conflict of interest statement

There are no competing interests for any author. JAE’s institutional affiliation is provided for identification purpose only and does not constitute institutional endorsement. Any views and opinions expressed are personal and belong solely to the individual and do not represent any people, institutions, or organizations that the individual may be associated with in a personal or professional capacity unless explicitly stated.

© 2023. The Author(s).

Figures

Fig. 1. Flow diagram of the randomised controlled trial in Thailand, Hong Kong, and Singapore.

Flowchart presenting the number of participants assessed, enrolled, randomized, lost to follow-up, and analysed in each child group and senior group.

Fig. 2. Associations between vaccine confidence and acceptance and sociodemographic factors, misinformation, risk perception, and chatbot use in Thailand child group.

A proportional odds logistic regression model adjusted for respondent’s sex, age, and employment status. Reference groups for a: respondent’s ethnicity: Thai (number of participants:125, percentage of the total participants: 93%); respondent’s education level: below college level (66, 49%); respondent is a healthcare worker?: yes (17, 13%); financial situation: low (29, 22%); geographical location: urban (51, 38%); child’s gender: female (64, 48%); chatbot use: no (65, 49%). *p-value < 0.05; **p-value < 0.01; ***p-value < 0.001. b Refers to the association between pre-intervention misinformation awareness and risk perception with primary outcomes, for example, higher misinformation awareness (“COVID-19 vaccines cause death”) in the pre-intervention questionnaire is positively associated with increase in vaccine confidence (“Vaccines are effective against all variants”) (OR = 1.41 (95% CI: 1.07–1.84)).

Fig. 3. Associations between vaccine confidence and acceptance and sociodemographic factors, misinformation, risk perception, and chatbot use in Thailand senior group.

A proportional odds logistic regression model adjusted for respondent’s employment status. Reference groups for a: respondent’s ethnicity: Thai (number of participants: 119, percentage of the total participants: 90%); respondent’s sex: female (74, 56%); respondent’s age: 35 and under (84, 64%); respondent’s education level: below college level (66, 50%); respondent is a healthcare worker?: yes (24, 18%); financial situation: low (24, 18%); geographical location: urban (53, 40%); senior’s gender: female (76, 58%); senior’s age: 60–80 (105, 80%); chatbot use: no (73, 55%). *p-value < 0.05; **p-value < 0.01; ***p-value < 0.001. b Refers to the association between pre-intervention misinformation awareness and risk perception with primary outcomes, for example, higher misinformation awareness (“COVID-19 vaccines cause death”) in the pre-intervention questionnaire is positively associated with increase in vaccine confidence (“Vaccines are effective against all variants”) (OR = 1.68 (95% CI: 1.13–2.49)).

Fig. 4. Associations between vaccine confidence and acceptance and sociodemographic factors, misinformation, risk perception, and chatbot use in Hong Kong child group.

A proportional odds logistic regression model adjusted for respondent’s sex, age, and employment status. Reference groups for a: respondent’s ethnicity: Chinese (number of participants: 87, percentage of the total participants: 44%; respondent’s education level: below college level (99, 50%); respondent is a healthcare worker?: yes (10, 5.0%); family income: under 30 K HKD (127, 64%); child’s gender: female (86, 43%); chatbot use: no (109, 54.8%)). *p-value < 0.05; **p-value < 0.01; ***p-value < 0.001. b Refers to the association between pre-intervention misinformation awareness and risk perception with primary outcomes, for example, higher misinformation awareness (“COVID-19 vaccines cause infection”) in the pre-intervention questionnaire is positively associated with increase in vaccine confidence (“Vaccines are important”) (OR = 2.24 (95% CI: 1.39–3.62)).

Fig. 5. Associations between vaccine confidence and acceptance and sociodemographic factors, misinformation, risk perception, and chatbot use in Hong Kong senior group.

A proportional odds logistic regression model adjusted for respondent’s employment status. Reference groups for a: respondent’s ethnicity: Chinese (number of participants: 113, percentage of the total participants: 60%); respondent’s sex: female (116, 62%); respondent’s age: 35 and under (78, 42%); respondent’s education level: below college level (86, 46%); respondent is a healthcare worker?: yes (11, 5.9%); family income: under 30 K HKD (113, 60%); senior’s gender: female (107, 57%); senior’s age: 60 to 80 (138, 73%); chatbot use: no (106, 56%). *p-value < 0.05; **p-value < 0.01; ***p-value < 0.001. b Refers to the association between pre-intervention misinformation awareness and risk perception with primary outcomes, for example, higher misinformation awareness (“COVID-19 vaccines cause death”) in the pre-intervention questionnaire is positively associated with increase in vaccine confidence (“Vaccines are effective”) (OR = 1.84 (95% CI: 1.28–2.64)).

Fig. 6. Associations between vaccine confidence and acceptance and sociodemographic factors, misinformation, risk perception, and chatbot use in Singapore child group.

A proportional odds logistic regression model adjusted for respondent’s sex and employment status, and housing type. Reference groups for a: respondent’s ethnicity: Chinese (number of participants: 44, percentage of the total participants: 46%); respondent’s age: 35 and under (40, 42%); respondent’s education level: below college level (49, 52%); is respondent a healthcare worker?: yes (15, 16%); child’s gender: female (33, 35%); chatbot use: no (50, 52.6%). *p-value < 0.05; **p-value < 0.01; ***p-value < 0.001. b Refers to the association between pre-intervention misinformation awareness and risk perception with primary outcomes, for example, higher misinformation awareness (“COVID-19 vaccines cause genetic change”) in the pre-intervention questionnaire is positively associated with increase in vaccine confidence (“Vaccines are important”) (OR = 7.28 (95% CI: 2.35–22.54)).