Efficacy of educational video game versus traditional educational apps at improving physician decision making in trauma triage: randomized controlled trial

Abstract

Objective: To determine whether a behavioral intervention delivered through a video game can improve the appropriateness of trauma triage decisions in the emergency department of non-trauma centers.

Design: Randomized clinical trial.

Setting: Online intervention in national sample of emergency medicine physicians who make triage decisions at US hospitals.

Participants: 368 emergency medicine physicians primarily working at non-trauma centers. A random sample (n=200) of those with primary outcome data was reassessed at six months.

Interventions: Physicians were randomized in a 1:1 ratio to one hour of exposure to an adventure video game (Night Shift) or apps based on traditional didactic education (myATLS and Trauma Life Support MCQ Review), both on iPads. Night Shift was developed to recalibrate the process of using pattern recognition to recognize moderate-severe injuries (representativeness heuristics) through the use of stories to promote behavior change (narrative engagement). Physicians were randomized with a 2×2 factorial design to intervention (game v traditional education apps) and then to the experimental condition under which they completed the outcome assessment tool (low v high cognitive load). Blinding could not be maintained after allocation but group assignment was masked during the analysis phase.

Main outcome measures: Outcomes of a virtual simulation that included 10 cases; in four of these the patients had severe injuries. Participants completed the simulation within four weeks of their intervention. Decisions to admit, discharge, or transfer were measured. The proportion of patients under-triaged (patients with severe injuries not transferred to a trauma center) was calculated then (primary outcome) and again six months later, with a different set of cases (primary outcome of follow-up study). The secondary outcome was effect of cognitive load on under-triage.

Results: 149 (81%) physicians in the game arm and 148 (80%) in the traditional education arm completed the trial. Of these, 64/100 (64%) and 58/100 (58%), respectively, completed reassessment at six months. The mean age was 40 (SD 8.9), 283 (96%) were trained in emergency medicine, and 207 (70%) were ATLS (advanced trauma life support) certified. Physicians exposed to the game under-triaged fewer severely injured patients than those exposed to didactic education (316/596 (0.53) v 377/592 (0.64), estimated difference 0.11, 95% confidence interval 0.05 to 0.16; P<0.001). Cognitive load did not influence under-triage (161/308 (0.53) v 155/288 (0.54) in the game arm; 197/300 (0.66) v 180/292 (0.62) in the traditional educational apps arm; P=0.66). At six months, physicians exposed to the game remained less likely to under-triage patients (146/256 (0.57) v 172/232 (0.74), estimated difference 0.17, 0.09 to 0.25; P<0.001). No physician reported side effects. The sample might not reflect all emergency medicine physicians, and a small set of cases was used to assess performance.

Conclusions: Compared with apps based on traditional didactic education, exposure of physicians to a theoretically grounded video game improved triage decision making in a validated virtual simulation. Though the observed effect was large, the wide confidence intervals include the possibility of a small benefit, and the real world efficacy of this intervention remains uncertain.

Trial registration: clinicaltrials.gov; NCT02857348 (initial study)/NCT03138304 (follow-up).

Conflict of interest statement

Competing interests: All the authors have completed the ICMJE uniform disclosure form at www.icmje.org/coi_disclosure.pdf and declare no support from any organization for the submitted work other than those listed above; no financial relationships with any organizations that might have an interest in the submitted work in the previous three years, and no relationships or activities that could appear to have influenced the submitted work.

Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://group.bmj.com/group/rights-licensing/permissions.

Figures

Fig 1

Selected screenshots from interaction between Andy and his boss, the department chair. In this instance, Andy has failed to transfer a patient (Benjamin) with a cervical spine fracture to a trauma center, and Benjamin has returned with a central cord syndrome. During the conversation, players can choose how they want to respond to the department chair’s criticism of their performance, either accepting responsibility or arguing that the complication represents the natural evolution of the disease process rather than a diagnostic error

Fig 2

Screenshots from the virtual simulation (outcome assessment tool). A) Each case included a 2D rendering of the patient, a chief complaint, vital signs, a history, and a written description of the physical exam. B) Physicians could select from a prespecified list of 250 medications, studies, and procedures. C) Audiovisual distractors were included, such as nursing requests for help with disruptive patients, to increase the verisimilitude of the experience

Fig 3

Screening, randomization, and analysis. In total, 297 (81%) physicians completed the simulations during the initial trial and 122 (61%) completed the simulations during the follow-up study