Usability Evaluation of Gen AI-based Nutrition Chatbot for Pregnant Women

Usability Evaluation of Gen AI-based Nutrition Chatbot for Pregnant Women: A Pilot Quasi-experimental Study

Background: Pregnancy imposes significant physical demands, with complications like gestational diabetes (GDM) and pre-eclampsia posing serious risks. Nutrition is crucial for mitigation, but accessing reliable guidance remains challenging. This study evaluates the feasibility of an AI chatbot providing nutritional guidance for managing these conditions.

Methods: In a quasi-experimental design, 100 pregnant women will self-select into either the intervention group (n=50, using an AI chatbot) or control group (n=50, receiving standard care). The primary outcome is usability measured by the System Usability Scale (SUS) at 12 weeks, with an expected mean difference of ≥13 points. Secondary outcomes include technology acceptance (Technology Acceptance Model), user engagement, information accuracy, and changes in dietary knowledge/behaviors. Quantitative data will be analyzed using intention-to-treat and t-tests. Semi-structured interviews with 20 participants will explore user experiences through thematic analysis.

Expected Results: The AI chatbot is anticipated to demonstrate superior usability and high user acceptance (TAM >5.0/7), with improvements in dietary knowledge and behavior. Qualitative findings will provide insights into benefits, barriers, and engagement factors.

Conclusion: This study will establish an evidence base on AI chatbot feasibility and acceptance for prenatal nutrition, informing tool optimization and future large-scale trials.

Study Overview

• Status: Not yet recruiting
• Conditions: Diabetes, Gestational; Pre-eclampsia; AI Chatbot for Prenatal Nutrition Guidance
• Intervention / Treatment: Behavioral: a culturally tailored nutrition AI chatbot for pregnant women

Detailed Description

Objectives: This study primarily aims to evaluate the usability of a nutrition AI chatbot for pregnant women by comparing System Usability Scale (SUS) scores between intervention and control groups. Secondary objectives include assessing technology acceptance (Technology Acceptance Model), engagement patterns, information quality (accuracy, comprehensibility, consistency), and changes in nutritional knowledge.

Design: A quasi-experimental design with two parallel groups (n=50 each) will be employed. Using self-selection, participants will choose to enroll in the intervention group (access to an AI chatbot plus routine care) or the control group (access to a standardized WeChat information service plus routine care). Routine care for all participants includes standard prenatal clinic visits and printed nutritional materials.

The WeChat service for the control group will be operated by a trained research assistant using a pre-defined script during two scheduled windows daily, providing information quoted from the official nutritional leaflets. This isolates the mode of information delivery (AI versus human-facilitated messaging) as the primary variable.

Participants: Inclusion criteria: pregnant women aged ≥18 years, able to consent, owning a smartphone with internet access. Exclusion criteria: enrollment in other nutrition interventions or severe mental health conditions impairing technology use. A purposive subsample of 20 participants (10 per group) will complete qualitative interviews.

Sample Size: Based on an expected mean SUS score of 78 (SD=12) in the intervention group and 65 (SD=15) in the control group (Cohen's d=0.95), 23 participants per group are required for 90% power at alpha=0.05. Accounting for 50% attrition, 50 participants per group will be recruited. Propensity score matching will be applied to reduce selection bias using variables including age, gestational age, parity, education, and baseline technology use.

Measurements: The primary outcome, usability, will be measured using the System Usability Scale (SUS) and the Chatbot Usability Questionnaire (CUQ) at 12 weeks. Technology acceptance will be assessed using the Technology Acceptance Model (TAM). Nutritional knowledge will be evaluated at baseline and 12 weeks using a 15-item questionnaire and the FIGO Nutrition Checklist. Information accuracy and consistency will be assessed by an expert panel rating 150 chatbot responses and repeated submission of 20 test questions. Engagement will be analyzed via application usage logs measuring adherence, intensity, and persistence. Semi-structured interviews will explore user experiences in depth.

Data Analysis: Quantitative data will be analyzed using intention-to-treat principles. Primary analysis will compare mean SUS scores between groups using independent samples t-tests, with effect sizes calculated as Cohen's d. Secondary outcomes will be analyzed using similar approaches, with chi-square tests for proportions and linear mixed models for nutritional knowledge change scores. Missing data will be addressed through multiple imputation. Qualitative interview transcripts will be analyzed using thematic analysis with dual independent coding.

Study Timeline: Participants will be enrolled over a 6-month period, with each participant completing a 12-week intervention and follow-up period.

• Study Type: Interventional
• Enrollment (Estimated): 100
• Phase: Not Applicable

Contacts and Locations

• Study Contact: Name: Bronya Luk, DHSc; Phone Number: +852 39708758 +852 39708758; Email: bluk@hkmu.edu.hk

Study Locations

• Hong Kong
  • Hong Kong, Hong Kong
    • Hong Kong Metropolitan University
    • Contact: Bronya Luk, DHSc; Phone Number: +852 39708758; Email: bluk@hkmu.edu.hk

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: Adult; Older Adult
• Accepts Healthy Volunteers: No

Description

Inclusion Criteria:

• Pregnant women aged 18 years or older
• Able to provide informed consent in the study language
• Own a smartphone with internet access and the WeChat application

Exclusion Criteria:

• Current enrollment in other nutrition intervention studies
• Severe mental health conditions that may impair technology use or ability to provide informed consent

Study Plan

How is the study designed?

Design Details

• Primary Purpose: Supportive Care
• Allocation: Non-Randomized
• Interventional Model: Parallel Assignment
• Masking: None (Open Label)

Number of Arms

2

Arms and Interventions

Participant Group / Arm	Intervention / Treatment
Experimental: The intervention group; The intervention group will access the the nutrition AI chatbot.	Behavioral: a culturally tailored nutrition AI chatbot for pregnant women; A culturally tailored nutrition AI chatbot for pregnant women , and the AI chatbot support will be available 24/7
No Intervention: The control group; The control group will receive routine care along with access to a standardized WeChat information service. To ensure a fair comparison, the WeChat service for the control group will be operated by a trained research assistant using a pre-defined script and protocol. The assistant will respond to enquiries during two pre-scheduled windows per day (e.g., 10:00-12:00 and 14:00-16:00) by providing information directly quoted or paraphrased from the official nutritional leaflets.

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
System Usability Scale (SUS)	Usability will be assessed using the System Usability Scale (SUS), a 10-item questionnaire with five-point Likert responses. SUS yields a total score ranging from 0 to 100, with higher scores indicating better perceived usability. Scores will be compared between groups at 12 weeks.	12weeks

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Mean Score on the Technology Acceptance Model (TAM) Questionnaire	Technology acceptance will be assessed using the Technology Acceptance Model (TAM) questionnaire. This 12-item instrument measures two domains: perceived usefulness (6 items) and perceived ease of use (6 items). Each item is rated on a 7-point Likert scale ranging from 1 (strongly disagree) to 7 (strongly agree). Domain scores are calculated as the mean of items within each domain, with higher scores indicating greater perceived usefulness or ease of use.	12 weeks
Proportion of Participants Achieving Adequate Engagement Adherence	Engagement adherence will be measured using application usage logs. Adequate adherence is defined as using the platform at least 3 days per week for at least 10 out of the 12-week intervention period. The proportion of participants meeting this threshold will be reported.	12 weeks
Mean Number of Platform Logins per Week	Engagement intensity will be measured using application usage logs. The average number of logins per week over the 12-week intervention period will be calculated for each participant and reported as a group mean.	12 weeks
Mean Number of Queries Submitted per Participant	Engagement intensity will also be assessed by the total number of queries (questions or requests) submitted by each participant to the platform over the 12-week intervention period, reported as a group mean.	12 weeks
Proportion of Chatbot Responses Rated as Accurate by Clinical Expert Panel	A panel of clinical experts will rate a sample of 150 chatbot responses for accuracy. Responses will be rated as accurate or inaccurate based on alignment with current clinical guidelines. The proportion of responses rated as accurate will be reported.	12 weeks

Collaborators and Investigators

• Sponsor: Hong Kong Metropolitan University
• Investigators: Principal Investigator: Bronya Luk, DHSc, School of Nursing and Health Sciences, Hong Kong Metropolitan University

Publications and helpful links

Helpful Links

• Caropreso, L., de Azevedo Cardoso, T., Eltayebani, M., & Frey, B. N. (2020). Preeclampsia as a risk factor for postpartum depression and psychosis: a systematic review and meta-analysis. Archives of Women's Mental Health, 23(4), 493-505.
• Charlton, M. (2016). The evolving management of gestational diabetes. The Hong Kong Practitioner, 38(2).
• Feghali, M. N., Abebe, K. Z., Comer, D. M., Caritis, S., Catov, J. M., & Scifres, C. M. (2018). Pregnancy outcomes in women with an early diagnosis of gestational diabetes mellitus. Diabetes Research and Clinical Practice, 138, 177-186.
• Hyzy, M., Bond, R., Mulvenna, M., Bai, L., Dix, A., Leigh, S., & Hunt, S. (2022). System Usability Scale Benchmarking for Digital Health Apps: Meta-analysis. JMIR MHealth and UHealth, 10(8).

Study record dates

Study Major Dates

• Study Start (Estimated): June 1, 2026
• Primary Completion (Estimated): January 31, 2027
• Study Completion (Estimated): January 31, 2027

Study Registration Dates

• First Submitted: February 24, 2026
• First Submitted That Met QC Criteria: March 4, 2026
• First Posted (Actual): March 9, 2026

Study Record Updates

• Last Update Posted (Actual): March 9, 2026
• Last Update Submitted That Met QC Criteria: March 4, 2026
• Last Verified: March 1, 2026

More Information

Terms related to this study

• Keywords: feasibility; gestational diabetes; pre-eclampsia; mixed methods; usability; technology acceptance; AI chatbot; prenatal nutrition
• Additional Relevant MeSH Terms: Urogenital Diseases; Endocrine System Diseases; Female Urogenital Diseases and Pregnancy Complications; Metabolic Diseases; Pregnancy Complications; Glucose Metabolism Disorders; Diabetes Mellitus; Hypertension, Pregnancy-Induced; Nutritional and Metabolic Diseases; Diabetes, Gestational; Pre-Eclampsia
• Other Study ID Numbers: HE-RD/2025/2.34 (Other Identifier: Hong Kong Metropolitan University)

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: NO
• IPD Plan Description: The informed consent form signed by participants did not include provisions for the public sharing of individual-level data. Furthermore, the ethical approval granted by the [Name of Your Ethics Committee/IRB] imposes restrictions on the dissemination of data to protect participant confidentiality. Therefore, only aggregated and anonymized results will be published.

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No