Evaluating AI for the Translation of KM Resources

June 29, 2026 updated by: University of Alberta

Evaluating Artificial Intelligence for Language Translation of Health-related Knowledge Mobilization Resources: A Randomized Controlled Trial and Qualitative Study

Knowledge mobilization resources for parents provide information to help make health decisions for their children. These resources can include videos, infographics, and plain language summaries. They are often created in English. This can make them hard to understand for people whose first language is not English. Translating these resources into other languages may be helpful. However, professional translation can take time and be expensive.

The study will compare a resource that is in English with the same resource translated into another language. The investigators will use both high and low resource languages commonly spoken in Alberta: Mandarin, Punjabi, Tagalog, Urdu. Parents who speak these languages will be asked to answer questions about how easy it is to understand and use the information. The investigators will also see if artificial intelligence can be used to translate the resource. To do this, parents will be asked to look at a resource that was translated by a professional and the same resource that was translated using artificial intelligence. Then, parents will answer questions about how clear the translations are. Parents will be asked to participate in the study using the internet. They will answer questions using an online questionnaire. Parents will also be asked if they are interested in taking part in an online interview. This will help us understand their thoughts about the resources in more detail. The plan is to involve 576 parents with 144 parents per language group.

This project will help to better understand whether parents prefer a resource in their own language compared to an English version. It will also help to understand whether artificial intelligence can be used to translate resources so that they are easier for parents to access. This work is very important so that all parents and their children have access to high quality health information. This can help all families make the best decisions for their children's health.

Study Overview

Detailed Description

Background

Knowledge mobilization (KM) aims to ensure that research and evidence are effectively shared, applied and integrated into decision-making processes [1, 2]. KM resources serve different end-users (e.g., policymakers, healthcare providers, patients, caregivers) and can take various forms depending on their target audience (e.g., resources for patients include plain language summaries, videos, or infographics). By translating complex health information into user-friendly formats and evaluating dissemination strategies, this approach enhances awareness, engagement, and accessibility. In the context of child health, KM helps parents and caregivers access, understand, and use research findings to make informed healthcare decisions [3].

Our team has successfully co-created KM resources with families, integrating research, lived experience, and art to resonate with parents [4-10]. Our resources reflect diverse identities, including gender, family structure, ability, appearance, and socioeconomic status. The investigators have extensively evaluated accessible formats (e.g., video, interactive infographics) and effective dissemination strategies (e.g., social media) to maximize engagement [11-13]. Interactive infographics offer broad accessibility, easily shared across various platforms (e.g., websites, social media) to reach diverse audiences, regardless of geographic location (urban, rural, remote) or socioeconomic background (e.g., gender, race, culture, education, income) [14].

Communication challenges due to language barriers (specifically understanding or accessing healthcare information) have been shown to impact quality of care and lead to poor health outcomes [15]. While KM resources are meant to be widely accessible, they can be challenging for those whose first language is not English. According to the 2021 Canadian census, immigrants make up 23% of the population. This proportion is growing and is expected to reach 29-34% by 2041 [16, 17]. The proportion in Alberta is consistent with Canada overall at 23%, with the top places of birth of immigrants being the Philippines, India, and China. Therefore, the communities of interest in this proposal also represent common non-English languages spoken at home in Alberta: Mandarin, Punjabi, Tagalog, Urdu [18].

With increasing diversity in Alberta, the complexities associated with supporting groups with different ethnocultural backgrounds also increases. In situations where immigrants may face challenges in adjusting to new sociopolitical environments and health systems, KM in health is critical to address their knowledge and information needs [19, 20]. Translating KM resources into multiple languages can mitigate disparities in healthcare access and quality, ensuring that immigrants receive the information they need to navigate and engage with the health system effectively [15]. However, language translation of KM resources by a professional (i.e., human) translator can be resource intensive, costly, and time consuming. Artificial intelligence (AI) applications (e.g., ChatGPT, DeepL, Google Translate) have the potential for more efficient translation [21]. Yet, there may be disadvantages with using AI for translation. For instance, translations involve not only the literal meaning of words but also the cultural implications and social connotations attached to them [22, 23]. AI may lack the ability to recognize cultural subtleties, which could result in insensitive or inappropriate interpretations [22]. Nevertheless, as technology rapidly advances, AI is likely to be used for healthcare communications to enhance access and quality of care for those who are not proficient in English [24]. Empirical assessments of AI translation are therefore essential for understanding its capabilities and limitations.

Our previous work has demonstrated that parent-informed and parent-mediated KM resources can increase knowledge and influence decision-making and expectations regarding healthcare seeking; [25, 26] in turn, expectations can impact the care delivered by health professionals. A key tenet of KM is that resources are accessible to the intended end users. An important barrier to accessing KM resources is difficulty understanding the information which may arise due to language limitations. With nearly 25% of Alberta's population reporting an immigrant mother tongue, it is essential to assess the utility of English versus non-English resources to promote equitable access to research and support informed healthcare decisions. Evaluating professional versus AI translated resources will provide crucial information to support efficiencies in KM. This project has the potential for substantial impact in dissemination and uptake of research.

Objective

The study objective is to explore two key research questions (RQs) regarding the language translation of child health KM resources:

RQ1: How does language translation of a pediatric KM resource influence the understanding, accessibility, usability, satisfaction, and preferences of parents whose first language is not English?

RQ2: How do outcomes (understanding, usability, readability, and acceptability) compare between professional and AI translated versions?

Methods

Trial design: This study will use a multi-methods approach, combining both quantitative (non-pharmacological randomized controlled trial [RCT]) and qualitative (interview) data to provide a comprehensive understanding of our research questions [27]. RCTs are the most rigorous design to compare interventions and will provide estimates of differences between the versions of a KM resource. The qualitative interviews will help us gain a greater understanding of any observed differences and parent preferences.

Trial setting: The trial will be conducted entirely online within Canada. Participants will complete the study remotely using their personal electronic devices (e.g., computers, tablets) with internet access. All materials (study link, instructions, questionnaires, interviews) will be delivered electronically.

Patient and public involvement: The investigators will involve parents as partners in this work to help refine the study design, interpret results, and disseminate study findings. This will consist of seeking feedback from our Pediatric Parents' Advisory Group (P-PAG) that was established by our research team in 2016 [28].

Reporting guidelines: The study protocol is reported using the SPIRIT (Standard Protocol Items: Recommendations for Interventional Trials) [29]. The investigators will follow established guidance for reporting RCTs and qualitative studies: CONSORT (Consolidated Standards of Reporting Trials) [30] and SRQR (Standards for Reporting Qualitative Research) [31].

Recruitment: The recruitment approach will be through convenience sampling. The investigators will co-develop a recruitment strategy with our target end users through our established P-PAG. Parents will be recruited from the communities of interest using traditional and non-traditional approaches (e.g., newsletters, posters, list serv, social media) in partnership with our collaborators (Our Kids Health, Translating Emergency Knowledge for Kids).

Study screening, enrollment and consent: Parents will begin the screening and study enrollment process by clicking the study link (hosted by the REDCap platform) provided on the recruitment materials [32, 33]. After accessing the study link, parents will complete a short set of eligibility questions to self-identify as eligible to participate in the study, and review the study information letter. Parents can read the letter and will have the option to download a copy. They will then be stratified by language, randomized and directed within the same study link to complete the online study tasks (e.g., review KM resource[s], questionnaire). Implied consent by overt action will be sought for this portion of the study.

Participants will self-identify as being interested in participating in the optional one-on-one interview at the end of the questionnaire. Interested participants will then be contacted to arrange an interview. Verbal consent will be obtained on Zoom prior to starting the interview and before any data collection. This will be documented in the consent log.

Design and study flow: After enrollment (as described above), participants will be stratified by language (Mandarin, Punjabi, Tagalog, Urdu). Within each language stratum, they will be randomized (computer-generated sequence developed by a statistician) in a 2:1 ratio into one of two groups aligned with the study's research questions:

  • RQ1 Group: English vs. Professional Translation
  • RQ2 Group: Professional Translation vs. AI Translation

Within each RQ group, participants will undergo a second randomization (1:1 ratio, using a random number generator) to determine which version of the KM resource they receive.

RQ1 Group: Participants will be shown either the English or the professionally translated online KM resource in their language via REDCap. They will then be directed to complete a series of 16 questions that will be used to measure the outcomes: understandability, accessibility, usability, satisfaction. After this, they will be shown the alternate version (English or professionally translated) and asked a series of 3 questions about their preference, followed by a series of 10-12 demographic questions to describe the study population (age, education level, country of birth, time in Canada, including self-reported health literacy).

RQ2 Group: Participants will receive either the professionally translated or the AI-translated online KM resource in their language via REDCap. They will then be directed to complete a series of 20 questions that will be used to measure the outcomes: understandability, readability, acceptability, followed by a series of 10-12 demographic questions to describe the study population (age, education level, country of birth, time in Canada, including self-reported health literacy).

Reviewing the resources and completing the questionnaire is expected to take 10-20 minutes via REDCap, depending on whether participants are randomized to RQ1 or RQ2. Data will be collected and managed using the REDCap survey platform for this portion of the study.

Blinding: Participants will be blinded to the study hypothesis and to their group allocation. Team members involved in data collection and analysis, as well as the Investigators, will remain blinded during data interpretation and manuscript drafting. General group labels will be used until group allocation is revealed.

Intervention (KM resource): For the KM resource, the investigators selected an interactive infographic that was originally created in English with funding from Women and Children's Health Research Institute (WCHRI) and the Stollery Children's Hospital Foundation (SCHF). The topic was identified as a priority by local interest holders at the Stollery Children's Hospital. Further, the investigators have already been asked by the Multicultural Health Brokers Cooperative in Edmonton to translate this resource into Ukrainian and Russian due to interest from these communities (translations completed in 2024). The infographic provides information about what to expect when taking an ill or injured child to the emergency department (ED). The infographic contains 750 words and takes on average 3 minutes to read. For this study, the infographic will be professionally and AI-translated into Mandarin, Punjabi, Tagalog and Urdu. ChatGPT (the latest version available at time of translation) will be used, as it is widely used and easily accessible, increasing the generalizability of our results. ChatGPT has also demonstrated strong performance metrics, process efficiency, and cost-effectiveness [34].

Interviews (optional): Participants will be invited to take part in a one-on-one semi-structured online (optional, via Zoom, 30-60 minutes) qualitative interview with a member of the research team to provide deeper insights, exploring their experiences, perceptions, barriers, trust and language preference related to the resources (e.g., what about it being in their language was helpful or unhelpful) [27]. The interviews will occur concurrently with the online portion so that there is minimal time lapse between when the participant completes the questionnaire and participates in an interview. The investigators will purposively select for the interviews to ensure representation across languages and health literacy levels. Interviews will follow a semi-structured guide (8-10 questions) to collect additional information about the main quantitative variables and explore reasons for responses (25, 35). The interview questions will be tailored based on how participants were randomized and their questionnaire responses. Interviews will be conducted in English, professionally transcribed, de-identified, and imported into NVivo for analysis.

Quantitative analysis: Descriptive statistics (count, frequencies, means with standard deviations) will be used for demographic data. The investigators will determine overall understanding by calculating a score (ranging from 0 to 7) for each participant based on the number of questions answered correctly. The investigators will calculate a mean score for each group and compare groups (English vs. non-English and professional vs. AI translated) using a two-sample t-test assuming equal variances. Percentage of parents who preferred English vs. non-English versions will be compared using a chi-squared test. Other secondary outcomes will be analyzed based on differences in mean scores and two-sample t-tests. Statistical uncertainties will be expressed with 95% confidence intervals; p< 0.05 will indicate statistical significance. Analyses will be conducted based on intention-to-treat [36].

Quantitative sample size: The investigators aim to recruit 144 parents per language group across four language groups (total = 576). Participants will be randomized in a 2:1 ratio, with 376 allocated to RQ1 (English vs. non English) and 200 to RQ2 (professional vs. AI translated).

The sample size for RQ1 is based on our primary outcome of understanding. Assuming a correct response rate of 60% for the English version and 75% for the non English version, with α = 0.05 and β = 0.8, 240 participants (120 per arm) will be required to reject the null hypothesis of no difference in understanding between versions. To account for a 15% non completion rate, the investigators plan to recruit 376 participants, which corresponds to 94 participants from each of the four language groups.

The sample size for RQ2 follows guidance for exploratory and pilot studies, which do not require a formal sample size calculation [37, 38]. Based on previous research, the investigators plan to recruit 200 parents (50 per language group). This will provide estimates of potential differences while allowing us to explore variation across languages.

Qualitative analysis: Thematic content analysis will be used to analyze qualitative interview data. The process underpinning data analysis will be constant comparison, specifically pattern-matching and explanation building within, across, and between cases [39]. The phases of inductive analysis that will be employed are coding, categorizing, and developing themes [40].

Qualitative sample size: The investigators aim to conduct interviews with 12-15 parents who were randomized to each RQ (total n=24-30). This sample size is based on qualitative research methodology for one-on-one interviews, [41] and is often determined by data saturation which will be assessed concurrently throughout data collection and analysis to assess data comprehensiveness, variation, richness, and redundancy [42].

Expected outcomes: This study will assess how translation influences understanding and preferences for a child health KM resource among linguistically diverse parents. By exploring the question, 'Is translation enough?' the findings will help shape future KM resource development for culturally and linguistically diverse communities. While translation is a crucial step in making health information accessible, it may not be sufficient on its own to ensure effective communication, understanding, and uptake. If parents struggle to grasp key health concepts due to inadequate translation, their ability to make informed decisions about their child's care may be compromised. This innovative study will also provide empirical data on the use of AI to translate a child health KM resource. By evaluating translation methods, especially the role of AI, this study will provide insights into efficient approaches for scaling KM resources to increase accessibility and address health inequities.

Timelines: A realistic 24-month timeline has been set based on our experience and understanding of the required resources and activities (25,26,28). The first 6 months will involve: finalizing the study protocol and submitting for ethics approval; developing and refining with end user input the data collection instruments (questionnaires, interview guides) and recruitment strategy; and developing and pilot testing the online REDCap database. The investigators will allow 12 months for recruitment and data collection with quantitative and qualitative data collection occurring simultaneously. The final 6 months will involve data cleaning, analysis, interpretation, and report preparation.

Compensation: Participants will be offered a $5 CDN electronic gift card for completing the online questionnaire and a $25 CDN electronic gift card for completing an interview.

Trial monitoring: No on-site monitoring is planned, as all data collection and oversight will occur remotely. The research team will review REDCap data weekly and data will be downloaded regularly (e.g., weekly) for quality control, cleaning and analysis. Recruitment and completion rates will also be monitored to inform outreach/recruitment strategies.

Ancillary and Post-Trial Care: Although this trial involves minimal risk to participants, any concerns or adverse effects identified during the study will be addressed promptly. Participants will be provided with contact information for the study team to report any issues.

Dissemination: Our team plans to publish this trial in a high-impact peer-reviewed journal and present the results and national and international meetings.

Confidentiality: This study is designed to maintain participant confidentiality at all times. All data will be collected through a secure online platform (i.e., REDCap) and stored on password-protected systems accessible only to the research team. No identifying information will be collected unless a participant voluntarily provides it (e.g., to be contacted for an interview, to receive a gift card), in which case this information will be separated and stored separately from the questionnaire responses. All data on the secure online platform will be deleted upon study completion. De-identified data will be kept for a minimum of 5 years and stored securely on an encrypted password protected network drive at the University of Alberta.

Potential harms: This study involves minimal risk in which the magnitude of possible harms implied by participation is no greater than those encountered by participants in those aspects of their everyday life that relate to the research.

Study Type

Interventional

Enrollment (Estimated)

576

Phase

  • Not Applicable

Contacts and Locations

This section provides the contact details for those conducting the study, and information on where this study is being conducted.

Study Contact

Study Contact Backup

  • Name: Sarah A Elliott, PhD
  • Phone Number: 587-341-5520
  • Email: se2@ualberta.ca

Study Locations

    • Alberta
      • Edmonton, Alberta, Canada, T5K2H6
        • University of Alberta
        • Contact:
        • Contact:
        • Principal Investigator:
          • Lisa Hartling, PhD

Participation Criteria

Researchers look for people who fit a certain description, called eligibility criteria. Some examples of these criteria are a person's general health condition or prior treatments.

Eligibility Criteria

Ages Eligible for Study

  • Adult
  • Older Adult

Accepts Healthy Volunteers

Yes

Description

Inclusion Criteria:

  • Adult (18 years or older)
  • Currently a parent or familial caregiver of a child (<18 years)
  • Lives in Canada
  • First language (native language, language learned from birth) is Mandarin, Punjabi, Tagalog, or Urdu
  • Able to communicate (read and respond to questions) in English
  • Access to an electronic device (e.g., computer), Internet, and email

Exclusion Criteria:

  • Under <18 years
  • Does not currently identify as a parent or familial caregiver of a child (<18 years)
  • Does not live in Canada
  • Does not identify their first language as Mandarin, Punjabi, Tagalog, or Urdu
  • Unable to communicate (read and respond to questions) in English
  • No access to an electronic device (e.g., computer), Internet, or email

Study Plan

This section provides details of the study plan, including how the study is designed and what the study is measuring.

How is the study designed?

Design Details

  • Primary Purpose: Other
  • Allocation: Randomized
  • Interventional Model: Crossover Assignment
  • Masking: Double

Arms and Interventions

Participant Group / Arm
Intervention / Treatment
Experimental: RQ1 Group A - English (non-translated)
The interactive infographic was originally created in English by our team with funding from the Women and Children's Health Research Institute and the Stollery Children's Hospital Foundation. The infographic provides information about what to expect when taking an ill or injured child to the emergency department. The topic was identified as a priority by local interest holders at the Stollery Children's Hospital. The infographic contains 750 words and takes on average 3 minutes to read.
Experimental: RQ1 Group B - Professional Translation
Human Translator
The interactive infographic was originally created in English by our team with funding from the Women and Children's Health Research Institute and the Stollery Children's Hospital Foundation. The infographic provides information about what to expect when taking an ill or injured child to the emergency department. The topic was identified as a priority by local interest holders at the Stollery Children's Hospital. The infographic contains 750 words and takes on average 3 minutes to read. The resource will be professionally translated into Mandarin, Punjabi, Tagalog and Urdu. The communities of interest represent common non-English languages spoken at home in Alberta.
Experimental: RQ2 Group A - Professional Translation
Human Translator
The interactive infographic was originally created in English by our team with funding from the Women and Children's Health Research Institute and the Stollery Children's Hospital Foundation. The infographic provides information about what to expect when taking an ill or injured child to the emergency department. The topic was identified as a priority by local interest holders at the Stollery Children's Hospital. The infographic contains 750 words and takes on average 3 minutes to read. The resource will be professionally translated into Mandarin, Punjabi, Tagalog and Urdu. The communities of interest represent common non-English languages spoken at home in Alberta.
Experimental: RQ2 Group B - AI Translation
ChatGPT
The interactive infographic was originally created in English by our team with funding from the Women and Children's Health Research Institute and the Stollery Children's Hospital Foundation. The infographic provides information about what to expect when taking an ill or injured child to the emergency department. The topic was identified as a priority by local interest holders at the Stollery Children's Hospital. The infographic contains 750 words and takes on average 3 minutes to read. The investigators will use ChatGPT to translate the resource (the latest version available at time of translation), as it is widely used and easily accessible, increasing the generalizability of our results. The communities of interest represent common non-English languages spoken at home in Alberta.

What is the study measuring?

Primary Outcome Measures

Outcome Measure
Measure Description
Time Frame
Understanding
Time Frame: Baseline
RQ1, RQ2; Measured by participant responses to a set of 7 multiple-choice questions specifically designed to evaluate key content knowledge about the infographic content. The number of correct answers will serve as an objective indicator of understanding, assessing the ability of each translation method to preserve critical concepts.
Baseline

Secondary Outcome Measures

Outcome Measure
Measure Description
Time Frame
Acceptability/Usability
Time Frame: Baseline
RQ1, RQ2; Measured by participant responses to a set of 6 questions using a 7-point Likert scale (7 'strongly agree', 4 'neutral' and 1 'strongly disagree') assessing whether everyone can use it, in addition to whether it is easy to use and interact with.
Baseline
Satisfaction Level
Time Frame: Baseline
RQ1; Measured by participant responses to a set of 3 questions using a 7-point Likert scale (7 'strongly agree', 4 'neutral' and 1 'strongly disagree') assessing whether it meets the needs and expectation of the user.
Baseline
Preference
Time Frame: Baseline
RQ1; Measured by participant responses to a set of 3 questions using a combination of multiple choice, yes/no, and open comment boxes assessing which version do they prefer to receive.
Baseline
Readability
Time Frame: Baseline
RQ2; Measured by participant responses to a set of 5 questions using a combination of yes/no, open comment boxes assessing how easily a resource can be understood by the end user.
Baseline
Method of Translation and Acceptability
Time Frame: Baseline
RQ2; Measured by participant responses to a set of 2 questions using a combination of multiple choice, yes/no and open comment boxes assessing whether they think the resource was professionally or AI-translated, and if the version is acceptable.
Baseline

Collaborators and Investigators

This is where you will find people and organizations involved with this study.

Investigators

  • Principal Investigator: Lisa Hartling, PhD, University of Alberta

Publications and helpful links

The person responsible for entering information about the study voluntarily provides these publications. These may be about anything related to the study.

General Publications

Study record dates

These dates track the progress of study record and summary results submissions to ClinicalTrials.gov. Study records and reported results are reviewed by the National Library of Medicine (NLM) to make sure they meet specific quality control standards before being posted on the public website.

Study Major Dates

Study Start (Estimated)

September 1, 2026

Primary Completion (Estimated)

September 1, 2027

Study Completion (Estimated)

September 1, 2027

Study Registration Dates

First Submitted

June 18, 2026

First Submitted That Met QC Criteria

June 18, 2026

First Posted (Actual)

June 24, 2026

Study Record Updates

Last Update Posted (Actual)

July 1, 2026

Last Update Submitted That Met QC Criteria

June 29, 2026

Last Verified

June 1, 2026

More Information

Terms related to this study

Plan for Individual participant data (IPD)

Plan to Share Individual Participant Data (IPD)?

NO

Drug and device information, study documents

Studies a U.S. FDA-regulated drug product

No

Studies a U.S. FDA-regulated device product

No

This information was retrieved directly from the website clinicaltrials.gov without any changes. If you have any requests to change, remove or update your study details, please contact register@clinicaltrials.gov. As soon as a change is implemented on clinicaltrials.gov, this will be updated automatically on our website as well.

Clinical Trials on Communication Research

Clinical Trials on KM Resource - English

3
Subscribe