The MARVIN Chatbots to Provide Information for Different Health Conditions

A Master Research Protocol to Adapt and Evaluate an Artificial Intelligence Based Conversational Agent to Provide Information for Different Health Conditions: the MARVIN Chatbots

This research is a continuation of a usability study with the MARVIN chatbot. The investigators aim to adapt the MARVIN chatbot to open it to other health domains (e.g. breast cancer) and populations (e.g. pharmacists). Therefore, this protocol constitutes a master research protocol that will englobe different research projects with individual chatbots. The investigators adopt an adaptive platform trial design, which will allow flexibility in handling multiple interventions adapted to different populations while retaining the characteristics of a platform trial design allowing early withdrawal of ineffective trial arms based on interim data (implementation outcomes) and introduction of new trial arms.

Study Overview

• Status: Recruiting
• Conditions: HIV Infections; Breast Cancer; Pediatric Emergency Medicine
• Intervention / Treatment: Other: MARVIN; Other: MARVIN-Pharma; Other: MARVINA; Other: MARVIN-CHAMP

Detailed Description

From 2020, the Principal Investigator's team has developed an innovative chatbot named Minimal AntiretRoViral INterference (MARVIN), for people with HIV. The primary objectives with this master protocol are to: 1) develop or adapt the MARVIN chatbots for different healthcare contexts through co-construction; 2) evaluate the global usability of individual chatbots in their respective use scenarios with a small sample of research participants; 3) evaluate the implementation and user outcomes (e.g., fidelity, appropriateness) of individual chatbots in their respective use scenarios with a larger sample of research participants. This includes also documentation of factors that inhibit or enhance the utilization of the chatbots and outcomes; and 4) establish and evaluate, through the previous objectives, different partnerships with research participants and research team members to co-construct chatbots that will respond to their needs and encourage continued use of the tool.

• Study Type: Interventional
• Enrollment (Estimated): 400
• Phase: Not Applicable

Contacts and Locations

• Study Contact: Name: Bertrand Lebouché, MD; Phone Number: 514-843-2090; Email: bertrand.lebouche@mcgill.ca

Study Locations

• Canada
  • Quebec
    • Montréal, Quebec, Canada, H2X 3E4
      • Recruiting
      • Centre Hospitalier de l'Universite de Montreal (CHUM)
      • Contact: Marie-Pacale Pomey, MD, PhD; Phone Number: 14302 514 890 8000; Email: marie-pascale.pomey@umontreal.ca
    • Montréal, Quebec, Canada, H4A 3J1
      • Recruiting
      • McGill University Health Centre (MUHC)
      • Contact: Bertrand Lebouché, MD, PhD; Phone Number: 5148432090; Email: bertrand.lecbouche@mcgill.ca
      • Sub-Investigator: Esli Osmanlliu, MDCM, MSc

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: 18 years and older (Child, Adult, Older Adult)
• Accepts Healthy Volunteers: Yes

Description

Inclusion Criteria for all objectives:

• being 14 years or older;
• being fluent in English and/or French;
• being able to understand the requirements of study participation and provide informed consent during the duration of the study;
• having access to a smartphone, tablet, or computer at home/at workplace;
• having access to an internet connection at home or data plan on their device.

Inclusion Criteria Specific to objectives 2 and 3:

• accept to use a Facebook Messenger-based Chatbot;
• accept to use or create a personal Facebook account;
• accept Facebook's privacy and data security policies.

Exclusion Criteria:

• not meeting the inclusion criteria
• any reason, in the opinion of the investigator, which would make the candidate inappropriate for participation in an investigative study involving a chatbot (e.g., cognitive deficit)

Study Plan

How is the study designed?

Design Details

• Primary Purpose: Other
• Allocation: Non-Randomized
• Interventional Model: Parallel Assignment
• Masking: None (Open Label)

Number of Arms

4

Arms and Interventions

Participant Group / Arm	Intervention / Treatment
Other: MARVIN: a Chatbot for HIV patients; Co-construction of the chatbot, Usability study, Implementation, Evaluation of outcomes and Continuous improvements	Other: MARVIN; Chatbot on Meta (Facebook) Messenger for HIV Patients
Other: MARVIN: a Chatbot for Community Pharmacists; Co-construction of the chatbots, Usability study, Implementation, Evaluation of outcomes and Continuous improvements	Other: MARVIN-Pharma; Chatbot on Meta (Facebook) Messenger for community pharmacists
Other: MARVINA: a Chatbot for Breast Cancer Patients; Co-construction of the chatbots, Usability study, Implementation, Evaluation of outcomes and Continuous improvements	Other: MARVINA; Chatbot on Meta (Facebook) Messenger for breast cancer patients
Other: MARVIN CHAMP: a Chatbot for Pediatric Infectious Conditions; Co-construction of the chatbots, Usability study, Implementation, Evaluation of outcomes and Continuous improvements	Other: MARVIN-CHAMP; CHatbot to Assist the Management of Pediatric patients with infectious conditions (CHAMP); Other Names: CHAMP; CHAMPIONS; CHatbot to Assist the Management of Pediatric patients with infectious conditions

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Usability Metric for User eXperience (UMUX-Lite)	The UMUX-Lite is a 2-item questionnaire, answered on a 7-point Likert scale (1 = strongly disagree to 5 = strongly agree), that was deemed appropriate for evaluating medical technology. The items ask whether the chatbot meets the user's needs and the user's perceived ease of use. Scores range from 0 to 100, with more than 68 being considered high usability according to the literature.	Immediately, after 1-month of testing - objective 2
Acceptability E-Scale (AES)	The AES contains 6 items rated with different 5-point Likert scales (e.g., 1=very difficult to 5=very easy). It is a validated measure of acceptability and usability of computer-based interventions for health populations. Scores range from 0 to 30, and according to the literature, scores above 24 are considered high acceptability.	Immediately after 1-month of testing - objective 2
Change in Acceptability E-Scale (AES) over 12 months	The AES contains 6 items rated with different 5-point Likert scales (e.g., 1=very difficult to 5=very easy). It is a validated measure of acceptability and usability of computer-based interventions for health populations. Scores range from 0 to 30, and according to the literature, scores above 24 are considered high acceptability.	Once every 3 months within 12 months
Change in Intervention Appropriateness Measure (IAM) over 12 months	Appropriateness relates to the relevance or compatibility of the innovation to address a particular issue or problem. The compatibility of an information technology innovation is the extent of which it is considered consistent with users' values, needs, and past experiences. The Compatibility Subscale is a validated tool that contains three items and evaluates on 7-points Likert scales (1 = extremely disagree to 7 = extremely agree) how an IT innovation "fits" with the user's work style. A modified version could be used by healthcare professionals, such as pharmacists.	Once every 3 months within 12 months
Change in Compatibility Subscale over 12 months	Appropriateness relates to the relevance or compatibility of the innovation to address a particular issue or problem. The compatibility of an information technology innovation is the extent of which it is considered consistent with users' values, needs, and past experiences. A validated tool that could be used by patients is the Intervention Appropriateness Measure (IAM). It also contains 4 items scored on a five-point Likert scale (1=strongly disagree to 5=strongly agree) of agreement evaluating how the innovation is suitable for users.	Once every 3 months within 12 months

Collaborators and Investigators

• Sponsor: McGill University Health Centre/Research Institute of the McGill University Health Centre
• Collaborators: Centre hospitalier de l'Université de Montréal (CHUM)
• Investigators: Principal Investigator: Bertrand Lebouché, MD, McGill University Health Centre/Research Institute of the McGill University Health Centre

Publications and helpful links

General Publications

• Ma Y, Achiche S, Pomey MP, Paquette J, Adjtoutah N, Vicente S, Engler K; MARVIN chatbots Patient Expert Committee; Laymouna M, Lessard D, Lemire B, Asselah J, Therrien R, Osmanlliu E, Zawati MH, Joly Y, Lebouche B. Adapting and Evaluating an AI-Based Chatbot Through Patient and Stakeholder Engagement to Provide Information for Different Health Conditions: Master Protocol for an Adaptive Platform Trial (the MARVIN Chatbots Study). JMIR Res Protoc. 2024 Feb 13;13:e54668. doi: 10.2196/54668.

Study record dates

Study Major Dates

• Study Start (Actual): March 1, 2024
• Primary Completion (Estimated): December 31, 2033
• Study Completion (Estimated): December 31, 2034

Study Registration Dates

• First Submitted: March 14, 2023
• First Submitted That Met QC Criteria: March 27, 2023
• First Posted (Actual): March 29, 2023

Study Record Updates

• Last Update Posted (Estimated): March 6, 2024
• Last Update Submitted That Met QC Criteria: March 4, 2024
• Last Verified: March 1, 2024

More Information

Terms related to this study

• Keywords: Adherence; Breast Cancer; HIV; Telehealth; mHealth; Self-Management; Artificial Intelligence; Care; Digital Health; Antiretroviral Therapy; Implementation Science; Chatbot; Pediatric Emergency Medicine; Intelligent Conversational Agent; Co-Construction; Adaptive Platform Trial Design; Patient and Stakeholder Engagement; Pediatric Infection; Large Language Models
• Additional Relevant MeSH Terms: Pathologic Processes; Disease Attributes; Emergencies
• Other Study ID Numbers: MP-37-2023-9333

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: NO

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No