Long-terms Effects of Simulation-based Ultrasound Training: A Randomized Trial

The clinical effects of simulation-based ultrasound training has not yet been explored. To examine the long-term effects of training, we plan to conduct a trial, where half of all new residents are randomized to simulation-based training and the other half to traditional clinical training only. The effects are assessed after two months of practice by evaluating two ultrasound scans performed by the residents. These scans are subsequently evaluated by a blinded consultant gynecologist and rated using the Objective Structured Assessment of Ultrasound Skills, which has been validated in previous studies.

Study Overview

• Status: Completed
• Conditions: Ultrasound Training; Ultrasound Simulation; Ultrasound Assessment; Technology-enhanced Learning; Virtual Reality Simulation
• Intervention / Treatment: Other: Simulation-based ultrasound training

Detailed Description

Ultrasonography has become increasingly used in many medical specialties over the last decades as smaller and less expensive ultrasound equipment has become available. Although ultrasound imaging traditionally is considered safe, its use is highly operator dependent (EFSUMB 2010). The lack of sufficient operator skills can lead to diagnostic errors that eventually compromise patient safety due to unnecessary tests or interventions (Moore & Copel 2011). Clinical ultrasound training is challenged with long learning curves and is therefore both time consuming and requires large teacher resources (Jang et al. 2010). Some trainees may therefore never acquire the basic skills and knowledge needed for independent practice (Tolsgaard et al. 2012). In the search for effective training methods, simulation-based training has proven to be an effective adjunct to clinical training, which has been documented in several studies on interventional procedures (Larsen et al. 2009, Zendejas et al 2011, Barsuk et al. 2009). Most comparative studies on simulation-based training have, however, only examined the initial effects of simulation-based training on the first couple of procedures, it is not known if VR simulation training provides sustained improvements trainee performance throughout their learning curves until they reach clinical proficiency. Large resources are currently being allocated to simulation training in many different specialties but the effects of simulation may easily be over-estimated if only evaluating the immediate effects of simulation training. A small difference in the number of supervised ultrasound examinations needed for proficiency may not justify the financial costs associated with investing in simulation equipment, space for simulation training as well as teacher resources during simulation training. Therefore, the research questions of this study was:

In a group of ultrasound novices, what are the implications of simulation-based training compared to clinical training alone on the technical quality of scans performed after two months of training?

Methods

Study design This study is reported according to the CONSORT statement (www.consort-statement.org). A randomized observer-blind study to compare VR ultrasound training with traditional supervised practice on subsequent patient scans (see Figure 1 for flowchart of study design). Ethical approval was obtained from the Regional Ethical Committee of the Capital Region, Denmark. The Danish Data Protection Agency approved the storing of patient relevant information. This randomized trial is reported to clinicaltrials.gov prior to inclusion of participants.

Setting This study is carried out at the Departments of Gynecology at the Juliane Marie Centre, Copenhagen University Hospital, Denmark, Næstved Hospital, and Hillerød Hospital.

Participants Participants are all new OB/GYN residents at three different hospitals in Eastern Denmark. Inclusion criteria are 1) proficiency in written and oral Danish, 2) informed written consent. The exclusion criteria are 1) prior employment at an OB/GYN department in a clinical position, 2) any formal ultrasound training with or without hands-on practice and (3) prior experience with virtual reality simulation.

Randomization A research fellow at The Center for Clinical Education, Rigshospitalet, independently performs randomisation of participants by computer to either intervention group (simulation training) or control group (clinical training only). The allocation of participants is concealed to the primary investigator during the enrolment of participants. Once informed consent has been signed and a participant number has been provided, the allocation sequence is retrieved per telephone.

Intervention Participants in the intervention group receive simulation training using two types of ultrasound simulators (see Figure 2 for flowchart of the simulation programme). Participants are introduced to a high-fidelity Virtual-Reality (VR) simulator (Scantrainer, Medaphor). The VR simulator provides images obtained from real patients and haptic feedback from the ultrasound probe. Participants train on this type of simulator until a predefined proficiency-level has been attained. The selection of modules is based on a previous validation study and only modules and items that discriminate between novice and expert performances are included in the training programme (Madsen et al. 2013). All participants are provided feedback after completing each module. When all modules are passed on the VR simulator, the participants receive proficiency-based training training on the low-fidelity simulator (BluePhantom) to allow participants to review the functions, they just trained, using real ultrasound equipment. The simulator performances are assessed using the Objective Structured Examination of Ultrasound Skills (OSAUS) and feedback is provided using this framework after each trial. Proficiency is set according pass/fail-scores established in a previous study (Tolsgaard et al. 2013).

Clinical training Participants in both groups receive traditional clinical training. This includes supervised practice in terms of apprenticeship learning followed by supervision during independent practice. The new residents do not usually go on calls for the first one to two weeks. Local department rules dictates which types of examinations that always requires a second examination by a supervising clinicians (e.g. fetal demise or pregnancy of unknown location).

Main outcome measure The main outcome measure is the technical proficiency after two months of practice. Two scans from the emergency department are recorded and subsequently assessed by a blinded clinician, who also had access to the ultrasound descriptions that corresponded to each scan. The assessments were performed using the Objective Structured Assessment of Ultrasound Skills (Tolsgaard et al. 2013), which has been validated in a construct validity study including 30 clinicians with different levels of experience prior to this trial.

Sample size calculation Sample size calculations are based on previous studies on clinical performances of ultrasound novices with the performance of medical students, who completed a simulation-based training programme (Tolsgaard et al. 2013). The average difference in OSAUS scores between these groups was 0.85 (2.65-1.8 (mean SD 0.4). It was anticipated that these differences would be diluted according to literature from other areas of simulation-based technical skills training (Smith et al. 2010), where simulation effects persisted after 3 months of training but the differences between groups was diminished to 55% of the initial effects. Assuming a 55% dilution, an alpha-level of 0.05 and a power of 0.80, the total number of participants needed was 26.

• Study Type: Interventional
• Enrollment (Actual): 26
• Phase: Not Applicable

Contacts and Locations

Study Locations

• Denmark
  • Copenhagen, Denmark, 2100
    • Dept. Gynecology, Rigshospitalet.

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: ADULT; OLDER_ADULT; CHILD
• Accepts Healthy Volunteers: Yes
• Genders Eligible for Study: All

Description

Inclusion Criteria:

1) proficiency in written and oral Danish, 2) informed written consent.

Exclusion Criteria:

1) prior employment at an OB/GYN department in a clinical position, 2) any formal ultrasound training with or without hands-on practice and (3) prior experience with virtual reality simulation.

Study Plan

How is the study designed?

Design Details

• Allocation: RANDOMIZED
• Interventional Model: PARALLEL
• Masking: SINGLE

Number of Arms

2

Arms and Interventions

Participant Group / Arm	Intervention / Treatment
ACTIVE_COMPARATOR: Simulation-based training; Participants in the intervention group receive simulation training using two types of ultrasound simulators: The Scantrainer (Medaphor) and the BluePhantom (CAE). All participants are training to pre-established proficiency-criteria.	Other: Simulation-based ultrasound training; Other Names: Scantrainer, Medaphor; BluePhantom, CAE
NO_INTERVENTION: Control; Clinical training alone.

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Technical ultrasound performances	The main outcome measure is the technical proficiency after two months of practice. Two scans from the emergency department are recorded and subsequently assessed by a blinded clinician, who also had access to the ultrasound descriptions that corresponded to each scan. The assessments were performed using the Objective Structured Assessment of Ultrasound Skills (Tolsgaard et al. 2013), which has been validated in a construct validity study including 30 clinicians with different levels of experience prior to this trial.	2 months

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Diagnostic accuracy	Diagnostic accuracy is assessed using the previously described video recordings.	Two months

Other Outcome Measures

Outcome Measure	Measure Description	Time Frame
Number of completed scans	Number of completed scans are recorded after two months	2 months

Collaborators and Investigators

• Sponsor: Rigshospitalet, Denmark

Publications and helpful links

General Publications

• Tolsgaard MG, Ringsted C, Dreisler E, Norgaard LN, Petersen JH, Madsen ME, Freiesleben NL, Sorensen JL, Tabor A. Sustained effect of simulation-based ultrasound training on clinical performance: a randomized trial. Ultrasound Obstet Gynecol. 2015 Sep;46(3):312-8. doi: 10.1002/uog.14780. Epub 2015 Aug 6.

Study record dates

Study Major Dates

• Study Start: July 1, 2013
• Primary Completion (ACTUAL): April 1, 2014

Study Registration Dates

• First Submitted: July 7, 2013
• First Submitted That Met QC Criteria: July 10, 2013
• First Posted (ESTIMATE): July 11, 2013

Study Record Updates

• Last Update Posted (ESTIMATE): April 7, 2014
• Last Update Submitted That Met QC Criteria: April 4, 2014
• Last Verified: April 1, 2014

More Information

Terms related to this study

• Keywords: Ultrasound training; Ultrasound simulation; Ultrasound assessment; Technology-enhanced learning; virtual reality simulation
• Other Study ID Numbers: H-3-2012-154 (OTHER: Ethical board case number)