AI Smart Glasses Feedback for Intubation Training in Medical Students

AI Smart Glasses for Automated Scoring and Feedback in Medical Students' Endotracheal Intubation OSCE: A Cluster-Randomized Educational Study

The goal of this educational study is to learn whether AI smart glasses can score and give feedback on endotracheal intubation performed by medical students on a manikin, and when this AI feedback is most useful during anesthesia training. The main questions it aims to answer are:

• Do AI scores of the intubation OSCE (Objective Structured Clinical Examination) agree with scores given by faculty examiners?
• Does practicing with AI smart-glasses feedback before or after the airway lecture lead to better intubation OSCE performance, self-efficacy, and cognitive load?

Medical interns rotating through the Department of Anesthesiology at Tri-Service General Hospital (Taiwan) for 2 weeks will be randomized by rotation group to practice intubation once with the AI smart glasses either before the airway lecture (week 1) or after it (week 2). At the end of the rotation, all participants will take an intubation OSCE while wearing the AI glasses, scored independently by a faculty examiner and by the AI. Participants will also complete short questionnaires on self-efficacy (before and after the rotation) and cognitive load (after AI practice). All procedures are performed on a manikin; no patients are involved.

Study Overview

• Status: Not yet recruiting
• Conditions: Medical Education; Intubation, Intratracheal
• Intervention / Treatment: Behavioral: AI smart-glasses intubation practice with automated feedback

Detailed Description

Background: Endotracheal intubation is one of the core clinical skills required of medical graduates in Taiwan, but intern pass rates in simulated intubation examinations at our hospital are only about 50-60%. Procedural skill learning depends on timely, specific feedback, yet faculty time is limited. AI smart glasses (HTC) record first-person video of the procedure and use artificial intelligence to score it against a standardized checklist and return immediate item-by-item feedback. The agreement between AI and faculty scoring has not been validated, and the optimal timing of AI feedback relative to formal teaching is unknown.

Design: Prospective, single-center, cluster-randomized (by rotation group, 3-4 interns per group), two-arm parallel educational trial. Rotation groups are allocated 1:1 by computer-generated block randomization.

Procedures (2-week anesthesiology rotation):

• Week 1, Day 1: informed consent and pre-test questionnaire.
• Arm 1 (AI before lecture): one AI smart-glasses intubation practice on a manikin during Week 1, Days 2-4, with immediate AI scoring and feedback, followed by a cognitive load questionnaire.
• Week 1, Day 5: airway lecture (about 1 hour) for all interns.
• Arm 2 (AI after lecture): one AI smart-glasses practice during Week 2, Days 1-4, with immediate AI scoring and feedback, followed by a cognitive load questionnaire.
• Last day of Week 2: end-of-rotation intubation OSCE (8 minutes) wearing the AI glasses; a faculty examiner scores the performance in person and the AI scores it independently; post-test questionnaire.

The investigator assisting with the glasses does not teach intubation during practice. OSCE faculty examiners are blinded to group allocation and do not see the AI score before completing their own scoring. Examiners are calibrated before enrollment using at least 10 practice videos.

OSCE checklist: 10 items scored 0/1/2 (total 0-20) plus a global rating scored 1 to 5, pass/fail judgment, and intubation time.

Questionnaires: a 4-item intubation-specific self-efficacy scale adapted from a published smart-glasses intubation training study and a 39-item medical learning self-efficacy questionnaire adapted from a validated multi-dimensional instrument for medical students, both rated on 5-point scales; and a validated 10-item cognitive load questionnaire measuring intrinsic, extraneous, and germane load, rated on a 5-point scale.

Statistical analysis: Agreement between AI and faculty total scores will be assessed with ICC(2,1) (two-way random effects, absolute agreement) with 95% CI and Bland-Altman analysis; item-level agreement with quadratic weighted kappa; pass/fail agreement with Cohen's kappa, sensitivity, and specificity. Between-arm comparisons of OSCE performance, self-efficacy change, and cognitive load will use linear or generalized linear mixed-effects models with rotation group as a random effect, following the intention-to-treat principle.

Sample size: 200 interns (about 100 per arm). With an expected ICC of 0.80, this gives a 95% CI width of no more than 0.10 for the primary agreement analysis; allowing 10% attrition and a design effect of about 1.13, it can detect an effect size of about 0.45 between arms in OSCE total score.

• Study Type: Interventional
• Enrollment (Estimated): 200
• Phase: Not Applicable

Contacts and Locations

• Study Contact: Name: Wei-Hung Chan, MD; Phone Number: 12546 +886-2-87923311; Email: whcken@gmail.com

Study Locations

• Taiwan
  • Taipei, Taiwan, 114202
    • Tri-Service General Hospital

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: 18 years to 30 years (Adult)
• Accepts Healthy Volunteers: Yes
• Genders Eligible for Study: All

Description

Inclusion Criteria:

• Medical interns (medical students in the clinical internship year) rotating through the Department of Anesthesiology, Tri-Service General Hospital, during the study period
• Aged 18 years to 30 years old
• Voluntarily agree to participate and provide written informed consent

Exclusion Criteria:

• Unwilling to participate
• Previously enrolled in this study during an earlier rotation (no repeat enrollment)

Study Plan

How is the study designed?

Design Details

• Primary Purpose: Other
• Allocation: Randomized
• Interventional Model: Parallel Assignment
• Masking: Single

Number of Arms

2

Arms and Interventions

Participant Group / Arm	Intervention / Treatment
Experimental: AI Feedback Before Airway Lecture; Rotation groups randomized to this arm perform one AI smart-glasses intubation practice session on a manikin with immediate AI scoring and feedback during Week 1, Days 2-4 of the 2-week anesthesiology rotation, before the airway lecture (Week 1, Day 5), followed by a cognitive load questionnaire. All participants complete the end-of-rotation intubation OSCE wearing the AI glasses (scored by faculty and AI) and the self-efficacy questionnaires at the start and end of the rotation.	Behavioral: AI smart-glasses intubation practice with automated feedback; One individual practice session (about 15 minutes) of endotracheal intubation on an airway manikin while wearing AI smart glasses (HTC). The glasses record first-person video of the procedure; after upload, the AI scores the performance against the 10-item intubation OSCE checklist (each item 0/1/2, total 0-20, plus a 1-5 global rating) and immediately displays item-by-item scores and written feedback for the student to review. The investigator only assists with operating the glasses and does not teach intubation during the session. The two arms differ only in the timing of this session relative to the airway lecture.
Active Comparator: AI Feedback After Airway Lecture; Rotation groups randomized to this arm attend the airway lecture (Week 1, Day 5) first and then perform one AI smart-glasses intubation practice session on a manikin with immediate AI scoring and feedback during Week 2, Days 1-4 of the 2-week anesthesiology rotation, followed by a cognitive load questionnaire. All participants complete the end-of-rotation intubation OSCE wearing the AI glasses (scored by faculty and AI) and the self-efficacy questionnaires at the start and end of the rotation.	Behavioral: AI smart-glasses intubation practice with automated feedback; One individual practice session (about 15 minutes) of endotracheal intubation on an airway manikin while wearing AI smart glasses (HTC). The glasses record first-person video of the procedure; after upload, the AI scores the performance against the 10-item intubation OSCE checklist (each item 0/1/2, total 0-20, plus a 1-5 global rating) and immediately displays item-by-item scores and written feedback for the student to review. The investigator only assists with operating the glasses and does not teach intubation during the session. The two arms differ only in the timing of this session relative to the airway lecture.

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Agreement Between AI and Faculty Examiner Total Scores on the Intubation OSCE	Intraclass correlation coefficient ICC(2,1) (two-way random effects, absolute agreement, single rater) with 95% confidence interval between the AI-generated total score and the faculty examiner's total score on the 10-item endotracheal intubation OSCE checklist (each item scored 0, 1, or 2; total score range 0-20, higher scores indicate better performance). Bland-Altman mean difference and 95% limits of agreement will also be reported. The faculty examiner is blinded to group allocation and to the AI score.	End of the 2-week anesthesiology rotation (Week 2, last day)

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Faculty-Rated Intubation OSCE Total Score	Total score on the 10-item endotracheal intubation OSCE checklist rated by a faculty examiner blinded to group allocation (each item 0, 1, or 2; range 0-20; higher scores indicate better performance), compared between arms.	End of the 2-week anesthesiology rotation (Week 2, last day)
Faculty-Rated Global Performance Rating on the Intubation OSCE	Faculty examiner's global rating of intubation performance on a 5-point scale (1 = poor, 2 = needs improvement, 3 = fair, 4 = good, 5 = excellent; higher scores indicate better performance), compared between arms.	End of the 2-week anesthesiology rotation (Week 2, last day)
Intubation OSCE Pass Rate	Percentage of participants judged by the faculty examiner to pass the intubation OSCE according to the national graduation competency standard (correct laryngoscope use, correct and appropriate tube placement, and completion of one tube placement within 30 seconds), compared between arms.	End of the 2-week anesthesiology rotation (Week 2, last day)
Intubation Time	Time in seconds from insertion of the laryngoscope blade into the mouth to completion of endotracheal tube placement and removal of the laryngoscope, timed by the faculty examiner during the OSCE, compared between arms.	End of the 2-week anesthesiology rotation (Week 2, last day)
Item-Level and Pass/Fail Agreement Between AI and Faculty Examiner	Agreement between AI and faculty examiner for each of the 10 OSCE checklist items (0/1/2) assessed with quadratic weighted kappa, and for the global rating (scored 1 to 5) assessed with ICC(2,1); agreement on pass/fail assessed with Cohen's kappa, with sensitivity and specificity of the AI pass/fail judgment using the faculty judgment as the reference standard.	End of the 2-week anesthesiology rotation (Week 2, last day)
Change in Intubation-Specific Self-Efficacy Score	Mean score of a 4-item intubation self-efficacy scale adapted from a published smart-glasses intubation training study, each item rated on a 5-point scale (1 = not at all confident, 5 = very confident; range 1-5; higher scores indicate greater self-efficacy). Change from baseline to end of rotation is compared between arms.	Baseline (Week 1, Day 1) and end of the 2-week rotation (Week 2, last day, after the OSCE)
Change in Medical Learning Self-Efficacy Score	Mean score of a 39-item medical learning self-efficacy questionnaire adapted from a validated multi-dimensional medical students' learning self-efficacy questionnaire (subscales: conceptual understanding, practical work, everyday application, higher-order cognitive skills, medical communication, professionalism, and academic self-efficacy), each item rated on a 5-point Likert scale (1 = strongly disagree, 5 = strongly agree; range 1-5; higher scores indicate greater self-efficacy). Change from baseline to end of rotation is compared between arms.	Baseline (Week 1, Day 1) and end of the 2-week rotation (Week 2, last day, after the OSCE)
Cognitive Load During AI Smart-Glasses Practice	Intrinsic (items 1-3), extraneous (items 4-6), and germane (items 7-10) cognitive load subscale mean scores on a validated 10-item cognitive load questionnaire, each item rated on a 5-point scale (1 = not at all applicable, 5 = completely applicable; range 1-5). Higher intrinsic and extraneous scores indicate greater load; higher germane scores indicate greater investment in learning. Compared between arms.	Immediately after the AI smart-glasses practice session (Week 1, Days 2-4 in Arm 1; Week 2, Days 1-4 in Arm 2)

Collaborators and Investigators

• Sponsor: Tri-Service General Hospital (TSGH)
• Investigators: Principal Investigator: Wei-Hung Chan, MD, Tri-Service General Hospital (TSGH)

Publications and helpful links

General Publications

• Leppink J, Paas F, Van der Vleuten CP, Van Gog T, Van Merrienboer JJ. Development of an instrument for measuring different types of cognitive load. Behav Res Methods. 2013 Dec;45(4):1058-72. doi: 10.3758/s13428-013-0334-1.
• Chao CT, Liang JC. The development and validation of a multi-dimensional medical students' learning self-efficacy questionnaire for clinical education. Med Educ Online. 2025 Dec;30(1):2534053. doi: 10.1080/10872981.2025.2534053. Epub 2025 Jul 20.
• Lin YF, Wang CY, Huang YH, Lin SM, Yang YY. Medical students' self-assessed efficacy and satisfaction with training on endotracheal intubation and central venous catheterization with smart glasses in Taiwan: a non-equivalent control-group pre- and post-test study. J Educ Eval Health Prof. 2022;19:25. doi: 10.3352/jeehp.2022.19.25. Epub 2022 Sep 2.

Study record dates

Study Major Dates

• Study Start (Estimated): January 1, 2027
• Primary Completion (Estimated): December 31, 2028
• Study Completion (Estimated): December 31, 2028

Study Registration Dates

• First Submitted: October 20, 2026
• First Submitted That Met QC Criteria: October 20, 2026
• First Posted (Actual): October 27, 2026

Study Record Updates

• Last Update Posted (Actual): November 6, 2026
• Last Update Submitted That Met QC Criteria: November 3, 2026
• Last Verified: November 1, 2026

More Information

Terms related to this study

• Keywords: Artificial Intelligence; Self Efficacy; Cognitive Load; Objective Structured Clinical Examination; Simulation Training; Smart Glasses; Formative Feedback
• Other Study ID Numbers: TSGH-ANES-AIGLASS-2027

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: NO

IPD Plan Description

Individual participant data, including first-person procedure videos, will not be shared. The informed consent specifies that study data are accessible only to the principal investigator and authorized co-investigators and that results will be reported only in aggregate. Aggregate results will be published in peer-reviewed journals.

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No