Comparing AI Role-Play and Peer Role-Play for Informed Consent Training in Endoscopy: A Randomized Control Trial

COMPARING AI ROLE-PLAY AND PEER ROLE-PLAY FOR INFORMED CONSENT TRAINING IN ENDOSCOPY: A RANDOMIZED CONTROL TRIAL

The goal of this randomized controlled trial is to find out whether AI-based role play is effective in teaching healthcare trainees how to obtain informed consent from patients for endoscopic procedures.

The main question is:

• How does generative AI-based role play compare to traditional peer role play in training healthcare trainees to deliver informed consent for endoscopic procedures?

Researchers will compare AI-based role play with peer role play to see which method is more effective for teaching this skill.

What participants will do:

• Attend a lecture on how to obtain informed consent.
• Take part in simulation-based training using either peer role play or AI-based role play.
• Participate in a simulated clinical encounter to assess their ability to obtain informed consent.

Study Overview

• Status: Recruiting
• Conditions: Informed Consent Process
• Intervention / Treatment: Other: AI-based role play; Other: Peer role play

Detailed Description

BACKGROUND AND RATIONALE Generative artificial intelligence (AI), particularly large language models (LLMs), has recently enabled the development of interactive, customizable virtual patient simulations that support experiential learning in healthcare communication. These AI-driven platforms can simulate realistic patient encounters, incorporating variable demographics, emotional states, and clinical complexity. They also offer scalable access to individualized feedback and repeated practice, circumventing the logistical limitations of traditional peer or actor-based simulations. Preliminary studies suggest that trainees perceive these tools as high in realism and educational value; however, there remains a lack of empirical evidence directly comparing the educational efficacy of generative AI-based simulations to established pedagogical approaches. This is particularly relevant for teaching complex communication tasks such as obtaining informed consent for gastrointestinal endoscopy-an interaction that demands clarity, empathy, and a detailed explanation of procedural risks, benefits, and alternatives.

STUDY AIM The primary objective of this study is to compare the effectiveness of generative AI-based role play and traditional peer role play in training healthcare trainees to obtain informed consent for endoscopic procedures. Specifically, the study will assess differences in trainee's ability to obtain informed, communication skills in obtaining informed consent and self-reported confidence (self-efficacy) following simulation-based training using a randomized controlled trial (RCT) design.

METHODS Study Design This is a single-blinded, parallel-group randomized control trial. A total of 40 medical trainees will be recruited. Participants will be randomized in a 1:1 ratio to either the intervention group (generative AI-based role-play) or the control group (traditional peer role-play). Randomization will be conducted using a computer-generated sequence, with group assignments placed in sealed envelopes distributed by an independent study team member. Due to the nature of the interventions, participant blinding is not feasible; however, outcome assessors will be blinded to group allocation.

Participants Participants will include medical trainees enrolled in a University of Toronto medical training program. Exclusion criteria include prior experience obtaining informed consent for endoscopy 25 or more times in either clinical or simulated settings.

Baseline Assessment All participants will complete a baseline questionnaire collecting demographic data, self-efficacy in obtaining informed consent, and attitudes toward AI in healthcare education.

Educational Intervention Participants will first attend a standardized 20-minute didactic lecture on the principles and components of informed consent in gastrointestinal endoscopy. The session will cover essential topics such as the purpose and risks of upper and lower endoscopy, alternative diagnostic options, addressing patient concerns, and ethical-legal considerations. This lecture is designed to ensure a uniform knowledge base prior to the simulation-based practice sessions.

Simulation-Based Training

Participants will then engage in 30 minutes of simulation-based practice using their assigned modality:

• Control Group - Peer Role Play: Participants will be paired with fellow trainees and alternate between physician and patient roles. Using standardized case prompts, trainees will conduct informed consent discussions. After each interaction, the participant in the patient role will provide peer feedback based on a provided checklist.
• Intervention Group - Generative AI Role Play: Participants will interact individually with an AI-driven virtual patient interface. The simulation will guide a dynamic informed consent conversation in which the AI patient responds in real-time to trainee input. The platform is programmed to emulate realistic patient personas and provides automated, individualized feedback after each scenario, focusing on content completeness, tone, empathy, and clarity.

The role-play scenarios used in both groups will be matched for content, including clinical context (i.e., informed consent for combined upper endoscopy and colonoscopy), relevant patient history, and the same required communication tasks. This design aims to ensure that differences in outcomes can be attributed to the mode of delivery rather than differences in case complexity.

Post-Intervention Assessment After the simulation sessions, all participants will complete a standardized post-test involving a simulated patient encounter. In this assessment, trainees must obtain informed consent for combined upper endoscopy and colonoscopy from a standardized patient (SP, i.e., actor) following a consistent, scripted patient persona. The SP will be trained to provide consistent responses across participants.

All interactions will be video recorded for assessment purposes, ensuring that identifying facial features are not captured to maintain anonymity. Assessors will be blinded to group allocation.

Follow-Up Survey Immediately following the post-test, participants will complete a follow-up survey reassessing self-efficacy in obtaining informed consent. The intervention group will also complete a feedback questionnaire assessing the usability, perceived educational value, and acceptability of the AI-based platform.

OUTCOMES

• Primary Outcomes:

  • Ability to obtain informed consent during the SP post-test, assessed using a structured checklist (assessors blinded to group allocation)
  • Communication performance during the SP post-test, assessed using a structured global rating scale (assessors blinded to group allocation)

• Secondary Outcomes:

  o Change in self-efficacy in obtaining informed consent (pre- vs. post-intervention survey scores).

• Other Outcomes (intervention group only); o Trainee perceptions of the AI-based simulation (intervention group only), including realism, usability, and satisfaction.

STATISTICAL ANALYSIS Quantitative data will be analyzed using standard statistical methods. Between-group differences in informed consent performance scores and communication scores will be assessed using independent samples t-tests or Mann-Whitney U tests, depending on the distribution of the data. A mixed-design (repeated measures) ANOVA will be used to evaluate pre-post changes in self-efficacy and to test for interaction effects between time and group (AI vs. peer role-play). Qualitative data from open-ended survey responses will be analyzed using thematic analysis following an inductive coding approach.

• Study Type: Interventional
• Enrollment (Estimated): 40
• Phase: Not Applicable

Contacts and Locations

Study Locations

• Canada
  • Ontario
    • Toronto, Ontario, Canada, M5G 0A4
      • Recruiting
      • Scarborough Health Network, University of Toronto
      • Contact: Samir Grover, MD, MEd, FRCPC; Phone Number: 416284-8131; Email: sgrover@shn.ca

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: Adult; Older Adult
• Accepts Healthy Volunteers: Yes

Description

Inclusion Criteria:

1) physician trainee enrolled at the University of Toronto.

Exclusion Criteria:

1) have previously obtained informed consent for an endoscopic procedure 25 or more times in either real clinical or simulated settings.

Study Plan

How is the study designed?

Design Details

• Primary Purpose: Other
• Allocation: Randomized
• Interventional Model: Parallel Assignment
• Masking: Single

Number of Arms

2

Arms and Interventions

Participant Group / Arm	Intervention / Treatment
Experimental: AI Role Play; Participants will interact with a generative AI-driven patient simulation designed to mimic a realistic consent discussion for an endoscopic procedure. The AI will respond dynamically to trainee input, prompting key elements of the consent process. After each case, the generative AI platform will provide the trainee with individualized feedback. Training will last 30 minutes. The same patient cases will be used in both groups.	Other: AI-based role play; Participants will role play informed consent scenarios with a generative AI-driven patient simulation designed to mimic a realistic consent discussion for an endoscopic procedure.
Active Comparator: Peer Role Play; Participants will be paired with another trainee (i.e., peer) in this group to conduct role play exercises, alternating between the roles of physician and patient, using standardized prompts and guidelines. After each case, the trainee portraying the patient will be asked to provide feedback to the individual portraying the participant. Training will last 30 minutes. The same patient cases will be used in both groups.	Other: Peer role play; Participants will role play informed consent discussions with a peer (i.e., another participant randomized to this group)

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Informed consent checklist	20-item checklist outlining 19 key steps of the informed consent process, rated on a 3-point scale (yes (2), partial (2), no (0)) and a 10-point global rating scale. The maximum total possible score is 28 on the checklist and 10 on the global rating scale (38 overall), with higher scores representing better performance. Trainee performance during the SP post-test, conducted immediately after training (Day 1), will be evaluated using this scale.	Day 1 (immediately post-training)
Communication Skills	Communication Skills Global Rating Form published by Hodges B, McIlroy JH. Analytic global OSCE ratings are sensitive to level of training. Med Educ 2003; 37: 1012-6. The scale rates overall communication, empathy, coherence, verbal and non-verbal communcation on a 1-5 point scale. The maximum score is 25, with higher scores representing better performance. Trainee performance during the SP post-test, conducted immediately after training (Day 1), will be evaluated using this scale.	Day 1 (immediately post-training)

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Self-efficacy	Change in self-efficacy to perform informed consent will be meausured using a self-efficacy scale developed based on Bandura's framework. The self-efficacy scale contains 9 items, rated on a 0-100 scale (0 = Cannot do at all; 50 = Moderatly certain I can do; 100 = Highly certain I can do), with a total possible score of 900. High scores indicate higher self-efficacy. Trainee will rate their self-efficacy to perform informed consent at baseline and on Day 1 (after the SP post-test).	Baseline and Day 1

Other Outcome Measures

Outcome Measure	Measure Description	Time Frame
Perceptions of the AI platform (AI role play group only)	Participant's ratings of the AI platform including, overall experience, realism, usefulness, acceptability, engagement, and feedback quality, each rated on a 1-5 scale (anchors: very poor, poor, fair, good, excellent). Total possible score 30. Trainee will rate the AI platform on Day 1 (after the SP post-test).	Day 1
AI System Usability (AI role play group only)	Perceived usability evaluation of educational technology of the AI platform, rated using the System Usability (SUS) Scale which assesses 10 domains on a 5-point scale. The total possible score is 50, with higher scores representing better usability. Trainee will complete the SUS scale on Day 1 (after the SP post-test).	Day 1

Collaborators and Investigators

• Sponsor: The Hospital for Sick Children
• Collaborators: University of Toronto; Scarborough General Hospital

Publications and helpful links

General Publications

• Hodges B, McIlroy JH. Analytic global OSCE ratings are sensitive to level of training. Med Educ. 2003 Nov;37(11):1012-6. doi: 10.1046/j.1365-2923.2003.01674.x.

Study record dates

Study Major Dates

• Study Start (Actual): July 2, 2025
• Primary Completion (Estimated): September 1, 2025
• Study Completion (Estimated): December 31, 2026

Study Registration Dates

• First Submitted: June 30, 2025
• First Submitted That Met QC Criteria: July 15, 2025
• First Posted (Actual): July 16, 2025

Study Record Updates

• Last Update Posted (Actual): July 16, 2025
• Last Update Submitted That Met QC Criteria: July 15, 2025
• Last Verified: July 1, 2025

More Information

Terms related to this study

• Keywords: artificial intelligence; education; informed consent; role play; simulation training; virtual patient; simulation-based training
• Other Study ID Numbers: GAS-25-013

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: UNDECIDED

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No