AI Smart Glasses Feedback for Intubation Training in Medical Students

Rotation groups randomized to this arm perform one AI smart-glasses intubation practice session on a manikin with immediate AI scoring and feedback during Week 1, Days 2-4 of the 2-week anesthesiology rotation, before the airway lecture (Week 1, Day 5), followed by a cognitive load questionnaire. All participants complete the end-of-rotation intubation OSCE wearing the AI glasses (scored by faculty and AI) and the self-efficacy questionnaires at the start and end of the rotation.

One individual practice session (about 15 minutes) of endotracheal intubation on an airway manikin while wearing AI smart glasses (HTC). The glasses record first-person video of the procedure; after upload, the AI scores the performance against the 10-item intubation OSCE checklist (each item 0/1/2, total 0-20, plus a 1-5 global rating) and immediately displays item-by-item scores and written feedback for the student to review. The investigator only assists with operating the glasses and does not teach intubation during the session. The two arms differ only in the timing of this session relative to the airway lecture.

Rotation groups randomized to this arm attend the airway lecture (Week 1, Day 5) first and then perform one AI smart-glasses intubation practice session on a manikin with immediate AI scoring and feedback during Week 2, Days 1-4 of the 2-week anesthesiology rotation, followed by a cognitive load questionnaire. All participants complete the end-of-rotation intubation OSCE wearing the AI glasses (scored by faculty and AI) and the self-efficacy questionnaires at the start and end of the rotation.

One individual practice session (about 15 minutes) of endotracheal intubation on an airway manikin while wearing AI smart glasses (HTC). The glasses record first-person video of the procedure; after upload, the AI scores the performance against the 10-item intubation OSCE checklist (each item 0/1/2, total 0-20, plus a 1-5 global rating) and immediately displays item-by-item scores and written feedback for the student to review. The investigator only assists with operating the glasses and does not teach intubation during the session. The two arms differ only in the timing of this session relative to the airway lecture.

Intraclass correlation coefficient ICC(2,1) (two-way random effects, absolute agreement, single rater) with 95% confidence interval between the AI-generated total score and the faculty examiner's total score on the 10-item endotracheal intubation OSCE checklist (each item scored 0, 1, or 2; total score range 0-20, higher scores indicate better performance). Bland-Altman mean difference and 95% limits of agreement will also be reported. The faculty examiner is blinded to group allocation and to the AI score.

Total score on the 10-item endotracheal intubation OSCE checklist rated by a faculty examiner blinded to group allocation (each item 0, 1, or 2; range 0-20; higher scores indicate better performance), compared between arms.

Faculty examiner's global rating of intubation performance on a 5-point scale (1 = poor, 2 = needs improvement, 3 = fair, 4 = good, 5 = excellent; higher scores indicate better performance), compared between arms.

Percentage of participants judged by the faculty examiner to pass the intubation OSCE according to the national graduation competency standard (correct laryngoscope use, correct and appropriate tube placement, and completion of one tube placement within 30 seconds), compared between arms.

Time in seconds from insertion of the laryngoscope blade into the mouth to completion of endotracheal tube placement and removal of the laryngoscope, timed by the faculty examiner during the OSCE, compared between arms.

Agreement between AI and faculty examiner for each of the 10 OSCE checklist items (0/1/2) assessed with quadratic weighted kappa, and for the global rating (scored 1 to 5) assessed with ICC(2,1); agreement on pass/fail assessed with Cohen's kappa, with sensitivity and specificity of the AI pass/fail judgment using the faculty judgment as the reference standard.

Mean score of a 4-item intubation self-efficacy scale adapted from a published smart-glasses intubation training study, each item rated on a 5-point scale (1 = not at all confident, 5 = very confident; range 1-5; higher scores indicate greater self-efficacy). Change from baseline to end of rotation is compared between arms.

Mean score of a 39-item medical learning self-efficacy questionnaire adapted from a validated multi-dimensional medical students' learning self-efficacy questionnaire (subscales: conceptual understanding, practical work, everyday application, higher-order cognitive skills, medical communication, professionalism, and academic self-efficacy), each item rated on a 5-point Likert scale (1 = strongly disagree, 5 = strongly agree; range 1-5; higher scores indicate greater self-efficacy). Change from baseline to end of rotation is compared between arms.

Intrinsic (items 1-3), extraneous (items 4-6), and germane (items 7-10) cognitive load subscale mean scores on a validated 10-item cognitive load questionnaire, each item rated on a 5-point scale (1 = not at all applicable, 5 = completely applicable; range 1-5). Higher intrinsic and extraneous scores indicate greater load; higher germane scores indicate greater investment in learning. Compared between arms.