Evaluation of a Chatbot With CALMO (Chatbot-Assisted Learning Model) to Enhance Clinical Learning in Nursing Students (CALMO)

January 15, 2025 updated by: Winnie Lai Sheung CHENG, Caritas Institute of Higher Education

Development and Evaluation of a Chatbot With CALMO to Enhance Clinical Learning in Nursing Students: A Quasi-Experimental Mixed-Methods Study

The present proposed study will design and develop a chatbot to support graduating nursing students' clinical learning. The efficacy of this chatbot technology will be evaluated, using a mixed methods quasi-experimental design. the proposed study hypothesizes that the chatbot-assisted clinical education has the potential to promote clinical performance by improving nursing knowledge, general self-efficacy, and self-reflection and insight.

Study Overview

Status

Status

Indicates the current recruitment status or the expanded access status.
Not yet recruiting

Conditions

Conditions

The disease, disorder, syndrome, illness, or injury that is being studied. On ClinicalTrials.gov, conditions may also include other health-related issues, such as lifespan, quality of life, and health risks.

Intervention / Treatment

Intervention / Treatment

A process or action that is the focus of a clinical study. Interventions include drugs, medical devices, procedures, vaccines, and other products that are either investigational or already available. Interventions can also include noninvasive approaches, such as education or modifying diet and exercise.

Detailed Description

The present proposed study will design and develop a chatbot to support graduating nursing students' clinical learning. The objectives of the proposed study are: (1) Design and develop a chatbot prototype that can simulate conversations, provide clinical information, and offer guidance relevant to various clinical scenarios; (2) Evaluate the impact of the chatbot intervention on nursing students' nursing knowledge, general self-efficacy, and self-reflection and insight; (3) Explore students' perceptions, satisfaction, and acceptance of the chatbot as an educational tool.The efficacy of this chatbot technology will be evaluated, using a mixed methods quasi-experimental design. the proposed study hypothesizes that the chatbot-assisted clinical education has the potential to promote clinical performance by improving nursing knowledge, general self-efficacy, and self-reflection and insight. The proposed study will be conducted in two phases. Phase 1: Design and develop a chatbot prototype which can provide evidence-based clinical information, engaging in interactive dialogue, and offering learning resources tailored to different clinical scenarios. The chatbot will be accessible through a web-based platform or mobile application. Phase 2: Conduct a mixed-methods quasi-experimental design and focus group interviews to evaluate the impact of the chatbot intervention on clinical learning. A convenience sample of 174 nursing students enrolled in a nursing program from two institutions will be recruited and assigned to either the experimental group, which will receive chatbot-supported clinical learning, or the control group, which will receive traditional clinical education without chatbot integration. Quantitative data will be collected from the two groups at three-time points: before-intervention at baseline (T0), 4 weeks after baseline (week 4, T1) and after intervention (week 12; T2) to assess students' nursing knowledge, general self-efficacy, and self-reflection and insight. Qualitative data will be collected through focus group interviews to explore students' perceptions, satisfaction, and experiences with chatbot-assisted learning. Quantitative data will be analyzed using MANOVA to compare the outcomes between and within groups. Thematic analysis will be conducted for qualitative data to identify emerging themes and patterns related to student perceptions and experiences using chatbot. Participants in the Experimental Group (EG) will receive an access code for the self-developed web-based chatbot platform called CALMO, a gender-neutral name as an acronym for Chatbot-Assisted Learning Model, which is planned to be developed during the first phase of this proposed study. The intervention will be available throughout the 12-week study.Participants in the Control Group (CG) will receive their usual clinical education support during the intervention period.A project management team will be formed to evaluate the intervention and ensure the quality and consistency of study intervention via regular face-to-face or online meetings. The fidelity of the study will be ensured by developing mechanisms for tracking participant logins, session completion rates, and engagement with intervention materials. Regular checks of the chatbot functionality will be performed by the platform developer with provision of regular service reports. Communication channels will be established for participants to provide feedback, report technical issues, or seek support from the research team during the intervention. Data will be analyzed using a mixed-methods approach, combining both quantitative and qualitative data. The statistical analysis using Statistical Package for the Social Sciences (SPSS), Version 26. The baseline characteristics of the participants in the two arms will be compared to identify any clinically meaningful differences using Chi-square test (for nominal data), Mann-Witney U test (for ordinal or non-normality data) or t-test (for normality data). Any variables considered to be significantly different will be further evaluated (e.g., included as covariates in the statistical analyses) and adjusted to account for their potential confounding effects on the outcomes. To test the hypothesis and to cater for the clustering effect with repeated measures of the outcome variables, a multivariate approach, such as a Generalized Estimating Equation (GEE), with adjustment of the baseline outcome variables to test for between-group differences will be employed. If students drop out between baseline and final data collection, intention to treat analysis will be applied. Both the Time x Group effect will be tested using a two-sided type I error rate of 5% for all tests of significance. Thematic analysis will be conducted for qualitative data to identify emerging themes and patterns related to students' perceptions and experiences.

Study Type

Study Type

Describes the nature of a clinical study. Study types include interventional studies (also called clinical trials), observational studies (including patient registries), and expanded access.
Interventional

Enrollment (Estimated)

Enrollment

The number of participants in a clinical study. The "anticipated" enrollment is the target number of participants that the researchers need for the study.
174

Phase

Phase

The stage of a clinical trial studying a drug or biological product, based on definitions developed by the U.S. Food and Drug Administration (FDA). The phase is based on the study's objective, the number of participants, and other characteristics. There are five phases: Early Phase 1 (formerly listed as Phase 0), Phase 1, Phase 2, Phase 3, and Phase 4. Not Applicable is used to describe trials without FDA-defined phases, including trials of devices or behavioral interventions.
  • Not Applicable

Contacts and Locations

This section provides the contact details for those conducting the study, and information on where this study is being conducted.

Study Contact

Study Contact

The name and contact information for the person who can answer enrollment questions for a clinical study. Each location where the study is being conducted may also have a specific contact, who may be better able to answer those questions.

Participation Criteria

Researchers look for people who fit a certain description, called eligibility criteria. Some examples of these criteria are a person's general health condition or prior treatments.

Eligibility Criteria

Eligibility Criteria

The key requirements that people who want to participate in a clinical study must meet or the characteristics they must have. Eligibility criteria consist of both inclusion criteria (which are required for a person to participate in the study) and exclusion criteria (which prevent a person from participating). Types of eligibility criteria include whether a study accepts healthy volunteers, has age or age group requirements, or is limited by sex.

Ages Eligible for Study

  • Adult
  • Older Adult

Accepts Healthy Volunteers

Yes

Description

Inclusion Criteria:

  • Baccalaureate nursing students participating in their final clinical placement
  • have sufficient knowledge of English language
  • own a smartphone with internet access
  • possess a valid Hong Kong phone number
  • possess a valid email address

Exclusion Criteria:

  • those who have been involved in the pilot test of the proposed chatbot intervention

Study Plan

This section provides details of the study plan, including how the study is designed and what the study is measuring.

How is the study designed?

Design Details

  • Primary Purpose: Supportive Care
  • Allocation: Non-Randomized
  • Interventional Model: Parallel Assignment
  • Masking: Single

Arms and Interventions

Participant Group / Arm

Participant Group / Arm

A group or subgroup of participants in a clinical trial that receives a specific intervention/treatment, or no intervention, according to the trial's protocol.
Intervention / Treatment

Intervention / Treatment

A process or action that is the focus of a clinical study. Interventions include drugs, medical devices, procedures, vaccines, and other products that are either investigational or already available. Interventions can also include noninvasive approaches, such as education or modifying diet and exercise.
Experimental: Chatbot assisted learning
The participants will recevie a self-developed web-based chatbot platform which is accessible via smartphones. It provides real-time dialogue with the chatbot through texts, visual aids, and audio files. The intervention will be available throughout the 12-week study. the chatbot is characterized by a gender-neutral persona, assumes the role of a friendly and empathetic clinical mentor. Its communication style can be described as calm, tolerant, supportive, and appreciative. Participants will receive weekly Short Message Service (SMS) notifications to remind them to consult with the chatbot.
Behavioral: chatbot-assisted clinical learning
The participants will recevie a self-developed web-based chatbot platform which is accessible via smartphones. It provides real-time dialogue with the chatbot through texts, visual aids, and audio files. The intervention will be available throughout the 12-week study. the chatbot is characterized by a gender-neutral persona, assumes the role of a friendly and empathetic clinical mentor. Its communication style can be described as calm, tolerant, supportive, and appreciative. Participants will receive weekly SMS notifications to remind them to consult with the chatbot.
No Intervention: control group _usual care
Participants in the Control Group will receive their usual clinical education support during the intervention period

What is the study measuring?

Primary Outcome Measures

Primary Outcome Measures

In a clinical study's protocol, the planned outcome measure that is the most important for evaluating the effect of an intervention/treatment. Most clinical studies have one primary outcome measure, but some have more than one.
Outcome Measure
Measure Description
Time Frame
nursing knowledge
Time Frame: before-intervention at baseline (T0), 4 weeks after baseline (week 4, T1) and after intervention (week 12; T2).
multiple-choice questionnaire (MCQ) consisting of 50 items will be administered to evaluate students' nursing knowledge. the score ranges from 0 to 50. the higher the score, the better nursing knowledge.
before-intervention at baseline (T0), 4 weeks after baseline (week 4, T1) and after intervention (week 12; T2).
General self-efficacy
Time Frame: before-intervention at baseline (T0), 4 weeks after baseline (week 4, T1) and after intervention (week 12; T2).
A 10-item General Self-Efficacy Scale to assess a general sense of perceived self-efficacy to manage hassles. It is scored with possible responses, 1=not at all true; 2=hardly true; 3=moderately true; and 4=exactly true, yielding a total score between 10 and 40. The higher the score, the higher the optimistic to cope with daily hassles.
before-intervention at baseline (T0), 4 weeks after baseline (week 4, T1) and after intervention (week 12; T2).

Secondary Outcome Measures

Secondary Outcome Measures

In a clinical study's protocol, a planned outcome measure that is not as important as the primary outcome measure for evaluating the effect of an intervention but is still of interest. Most clinical studies have more than one secondary outcome measure.
Outcome Measure
Measure Description
Time Frame
Self-reflection and insight
Time Frame: : before-intervention at baseline (T0), 4 weeks after baseline (week 4, T1) and after intervention (week 12; T2).
A 12-item Self-Reflection and Insight Scale to assess the need for self-reflection and engagement in self-reflection. The scale is scored from 6 (highly matched) to 1 (least matched), yielding a total score between 12 and 72. Higher scores indicated greater self-reflection and insight.
: before-intervention at baseline (T0), 4 weeks after baseline (week 4, T1) and after intervention (week 12; T2).
Qualitative Comments on the Perceptions of Chatbot-assisted technology
Time Frame: at the end of the 12-week study
it is a narrative feedback from the participants using focus group interviews to explore users' experiences with the chatbot during their clinical learning. A set of semi-structured interview questions encompassing questions to describe the overall experience, identify useful features, and assess the impact on understanding and retention of clinical concepts. Usability and user-friendliness of the chatbot interface will also be evaluated. For example: Can you describe your experience using the chatbot for clinical learning? What specific features or functionalities did you find useful or engaging? How do you think the chatbot has contributed to your understanding and retention of clinical concepts? Are there any limitations or challenges you encountered while using the chatbot? Did you perceive any differences in your learning outcomes when using the chatbot compared to other learning methods or resources? If so, what were those differences?
at the end of the 12-week study

Collaborators and Investigators

This is where you will find people and organizations involved with this study.

Sponsor

Sponsor

The organization or person who initiates the study and who has authority and control over the study.
Caritas Institute of Higher Education

Collaborators

Collaborators

An organization other than the sponsor that provides support for a clinical study. This support may include activities related to funding, design, implementation, data analysis, or reporting.
Tung Wah College

Investigators

Investigators

A researcher involved in a clinical study. Related terms include site principal investigator, site sub-investigator, study chair, study director, and study principal investigator.
  • Principal Investigator: Winnie LS CHENG, PhD, Saint Francis University

Study record dates

These dates track the progress of study record and summary results submissions to ClinicalTrials.gov. Study records and reported results are reviewed by the National Library of Medicine (NLM) to make sure they meet specific quality control standards before being posted on the public website.

Study Major Dates

Study Start (Estimated)

Study Start

The actual date on which the first participant was enrolled in a clinical study. The "anticipated" study start date is the date that the researchers think will be the study start date.
January 11, 2025

Primary Completion (Estimated)

Primary Completion

The date on which the last participant in a clinical study was examined or received an intervention to collect final data for the primary outcome measure. Whether the clinical study ended according to the protocol or was terminated does not affect this date. For clinical studies with more than one primary outcome measure with different completion dates, this term refers to the date on which data collection is completed for all the primary outcome measures. The "anticipated" primary completion date is the date that the researchers think will be the primary completion date for the study.
September 30, 2026

Study Completion (Estimated)

Study Completion

The date on which the last participant in a clinical study was examined or received an intervention/treatment to collect final data for the primary outcome measures, secondary outcome measures, and adverse events (that is, the last participant's last visit). The "anticipated" study completion date is the date that the researchers think will be the study completion date.
December 30, 2026

Study Registration Dates

First Submitted

First Submitted

The date on which the study sponsor or investigator first submitted a study record to ClinicalTrials.gov. There is typically a delay of a few days between the first submitted date and the record's availability on ClinicalTrials.gov (the first posted date).
January 11, 2025

First Submitted That Met QC Criteria

First Submitted That Met QC Criteria

The date on which the study sponsor or investigator first submits a study record that is consistent with National Library of Medicine (NLM) quality control (QC) review criteria. The sponsor or investigator may need to revise and submit a study record one or more times before NLM's QC review criteria are met. It is the responsibility of the sponsor or investigator to ensure that the study record is consistent with the NLM QC review criteria.
January 15, 2025

First Posted (Actual)

First Posted

The date on which the study record was first available on ClinicalTrials.gov after National Library of Medicine (NLM) quality control (QC) review has concluded. There is typically a delay of a few days between the date the study sponsor or investigator submitted the study record and the first posted date.
March 25, 2025

Study Record Updates

Last Update Posted (Actual)

Last Update Posted

The most recent date on which changes to a study record were made available on ClinicalTrials.gov. There may be a delay between when the changes were submitted to ClinicalTrials.gov by the study's sponsor or investigator (the last update submitted date) and the last update posted date.
March 25, 2025

Last Update Submitted That Met QC Criteria

Last Update Submitted That Met QC Criteria

The most recent date on which the study sponsor or investigator submitted changes to a study record that are consistent with National Library of Medicine (NLM) quality control (QC) review criteria. It is the responsibility of the sponsor or investigator to ensure that the study record is consistent with the NLM QC review criteria.
January 15, 2025

Last Verified

Last Verified

The most recent date on which the study sponsor or investigator confirmed the information about a clinical study on ClinicalTrials.gov as accurate and current. If a study with a recruitment status of recruiting; not yet recruiting; or active, not recruiting has not been confirmed within the past 2 years, the study's recruitment status is shown as unknown.
January 1, 2025

More Information

Terms related to this study

Keywords

Other Study ID Numbers

Other Study ID Numbers

Identifiers or ID numbers other than the NCT number that are assigned to a clinical study by the study's sponsor, funders, or others. These numbers may include unique identifiers from other trial registries and National Institutes of Health grant numbers.
  • ISG230111

Plan for Individual participant data (IPD)

Plan to Share Individual Participant Data (IPD)?

NO

IPD Plan Description

I will not publish in Medical Journal Editors

Drug and device information, study documents

Studies a U.S. FDA-regulated drug product

No

Studies a U.S. FDA-regulated device product

No

This information was retrieved directly from the website clinicaltrials.gov without any changes. If you have any requests to change, remove or update your study details, please contact register@clinicaltrials.gov. As soon as a change is implemented on clinicaltrials.gov, this will be updated automatically on our website as well.

Cookie Settings

We use cookies to ensure that we give you the best experience on our website. For more details carefully read our Privacy and Cookies Policy. ...>>>You can change your preferences or withdraw your consent at any time by deleting the cookies from your website or computer as described in the policy. Please accept it and choose your preferences by ticking the corresponding boxes in the "Manage Settings" section below. Please carefully read our Terms of Service before you proceed with using any part of this website.
«Manage Settings