A Computer-Based Simulation of DKA Management (DKA)

Integration of a Computer-Based Simulation of DKA Management Into Medical Education: A 2 by 2 Factorial Cluster Randomized Controlled Trial

A 2-by-2 factorial cluster randomized controlled trial was performed to: 1) Compare the effects of supervised (coached) versus unsupervised (no coach) administration of the DKA simulator on trainees' knowledge of DKA management immediately after (primary outcome) using the simulation and 3 and 6 months post-intervention (secondary outcome); 2) Determine whether a preselected number of DKA simulator practice cases or a self-selected number (self-regulated learning) of DKA simulator practice cases will result in superior trainee knowledge with respect to DKA management immediately after (primary outcome) and 3 and 6 months post-intervention (secondary outcome).

Study Overview

• Status: Completed
• Conditions: Diabetes
• Intervention / Treatment: Other: Coach, Preselected number of cases; Other: Coach, Self-selected number of cases; Other: No coach, Preselected number of cases; Other: No coach, Self-selected number of cases

Detailed Description

Diabetic ketoacidosis (DKA) accounts for approximately 115,000 hospital discharges per year in the United States. Appropriate management of this life-threatening clinical presentation requires timely and meticulous intervention, including avoidance of hypokalemia, hypoglycemia and DKA recurrence. However, clinical management continues to be less than ideal; for example, in a recent retrospective chart audit of patients admitted to a large teaching hospital with the primary diagnosis of DKA, 75% of them were placed on an inappropriate insulin regimen. This evidence to practice gap may be bridged with the use of computer simulation training.

In a simulated training environment, the trainee has the opportunity to manage uncommon but important clinical presentations that they may otherwise not experience in their training, without the risk of patient harm. A recent systematic review by Cook et al. analyzed 405 studies that compared a simulation-based intervention to no intervention for health care professionals from various disciplines. The authors found that simulation improved knowledge [pooled effect sizes of 1.20 (95% confidence interval (CI), 1.04-1.35)] and skills [pooled effect sizes of 1.09 (95% CI,1.03-1.16)]. Another systematic review analyzed 50 studies that compared virtual patient simulation with no intervention and found large positive effects of virtual patient simulation compared to no intervention (pooled effect sizes 0.94 (95% CI 0.69-1.19) for knowledge outcomes, 0.80 (95% CI 0.52-1.08) for clinical reasoning and 0.90 (95% CI 0.61-1.19) for other skills).

Although there are many studies comparing simulation to no intervention, very few studies have directly compared different simulation-based interventions. Two systematic reviews of the effectiveness of simulation have demonstrated that repetitive practice is superior to a single-use instructional modality. Few studies included in these reviews reported how much practice is necessary to obtain long-term skill retention. These systematic reviews also demonstrated that training adapted to individualized performance is associated with better learning outcomes.

What remains unclear, however, is who should have the locus of control when defining the parameters of individualized learning. One randomized controlled trial compared self-regulated learning and instructor-regulated learning interventions for resident training using lumbar puncture simulation and revealed that self-regulated learning can lead to superior long-term skill retention at 3 months. As focused repetitive practice is one of the key elements of deliberate practice, one would postulate that self-regulated learners have a higher chance of achieving superior results given that they can optimize their amount of practice; however, this has yet to be assessed for the simulator learning environment.

The effectiveness of unsupervised versus supervised simulation curricula has yielded conflicting results. A systematic review comparing different simulation modalities revealed that group instruction was not associated with better outcomes (pooled effect size -0.22), whereas a previous systematic review of randomized trials comparing simulation to other educational modalities revealed that group instruction was associated with a positive learning effect (pooled effect size 0.72). One could assume that supervised learning is superior to unsupervised learning, as the former provides learners with the opportunity for continuous informative feedback, a key element of deliberate practice, in order to enhance their continued practice. However, this has not been formally tested for simulation-based education.

Self-regulated learning and supervised versus unsupervised learning in simulation education require further exploration. The investigators conducted a 2-by-2 factorial cluster randomized controlled trial, comparing the impact of (1) coached versus non-coached administration, and (2) preselected number of practice cases versus self-selected number of practice cases, on medical student and resident scores on computer-based simulation of DKA management. By utilizing the theory of deliberate practice, the investigators hypothesized that participants who used the simulator in a supervised environment would score superiorly on the simulator. In addition, we hypothesized that participants who were randomized to self regulated learning would score superiorly on the simulator.

• Study Type: Interventional
• Enrollment (Actual): 124
• Phase: Not Applicable

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: 18 years and older (Adult, Older Adult)
• Accepts Healthy Volunteers: No
• Genders Eligible for Study: All

Description

Inclusion Criteria:

First-year internal Medicine trainees (PGY-1) in their first 6 months of training and third-year medical students (CC3) were recruited to participate in this study. Inclusion criteria were: either a CC3 or PGY-1 rotating through their General Internal Medicine (GIM) rotation at one of the 7 teaching sites in Toronto.

Exclusion Criteria:

Excluded were those participants who participated in the DKA study during a previous block and participants who were not rotating through a GIM rotation.

Study Plan

How is the study designed?

Design Details

• Allocation: Randomized
• Interventional Model: Factorial Assignment
• Masking: None (Open Label)

Number of Arms

4

Arms and Interventions

Participant Group / Arm	Intervention / Treatment
Experimental: Cohort A: Coach, Preselected number of cases; Cohort A completed the DKA simulator cases during two one-hour coached sessions. They were assigned a pre-selected number of DKA simulator cases (2 cases with 2 reps each, for 4 reps in total).	Other: Coach, Preselected number of cases
Experimental: Cohort B: Coach, Self-selected number of cases; Cohort B completed the DKA simulator cases during two one-hour coached sessions. They were assigned a self-selected number of DKA simulator cases and were instructed to complete as many cases until they felt comfortable with DKA management.	Other: Coach, Self-selected number of cases
Experimental: Cohort C: No coach, Preselected number of cases; Cohort C completed the DKA simulator cases in a non-coached setting, on their own time. They were assigned a pre-selected number of DKA simulator cases (2 cases with 2 reps each, for 4 reps in total).	Other: No coach, Preselected number of cases
Experimental: Cohort D: No coach, Self-selected number of cases; Cohort D completed the DKA simulator cases in a non-coached setting, on their own time. They were assigned a self-selected number of DKA simulator cases and were instructed to complete as many cases until they felt comfortable with DKA management.	Other: No coach, Self-selected number of cases

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Test 1 scores	Difference in simulator scores between the 4 cohorts immediately post- intervention (Test 1)	Immediately after intervention

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Test 2 and Test 3 scores	Difference in simulator scores between the four cohorts 3 and 6 months post- intervention (Tests 2 and 3).	3 and 6 months after intervention
Self-reported comfort with DKA management (5-point Likert scale)	Differences in comfort level with DKA management between the 4 cohorts after the intervention and throughout the study.	Before, immediately after, 3 months after intervention
Correlation between simulator score and standardized exam score	CC3 participant scores on their standardized DKA-specific Objective Standardized Clinical Exam (OSCE) stations and DKA-specific written examination questions, as well as PGY-1 scores on their standardized DKA-specific OSCE stations were obtained and compared with their scores on Test 1 (primary outcome)	Immediately after intervention, end of rotation (up to 10 months after intervention)
Number of simulations conducted	The average number of simulations per cohort were compared throughout the study.	From intervention to 6 months after intervention
Scores on Standardized exams	Scores on the PGY-1 and CC3 standardized exams were compared in each cohort.	End of rotation (up to 10 months after intervention)

Collaborators and Investigators

• Sponsor: Unity Health Toronto

Study record dates

Study Major Dates

• Study Start: July 1, 2014
• Primary Completion (Actual): June 1, 2015
• Study Completion (Actual): October 1, 2015

Study Registration Dates

• First Submitted: May 24, 2016
• First Submitted That Met QC Criteria: June 9, 2016
• First Posted (Estimate): June 15, 2016

Study Record Updates

• Last Update Posted (Estimate): June 15, 2016
• Last Update Submitted That Met QC Criteria: June 9, 2016
• Last Verified: June 1, 2016

More Information

Terms related to this study

• Other Study ID Numbers: 30043

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: No