Proficiency Based Training in Surgery - Employing An Arthroscopic Bankart Repair

The Copernicus Initiative - The Study of Proficiency Based Training in Arthroscopic Surgery

This study examines in a prospective, randomized and blinded study design the impact of a proficiency based training methodology on the acquisition of arthroscopic surgical skills employing both a dry shoulder simulator model, and cadaver specimens for the demonstration of surgical skill proficiency.

Study Overview

• Status: Completed
• Conditions: Study Focus: Simulation and Proficiency Based Training
• Intervention / Treatment: Behavioral: Group B - Simulation Training; Behavioral: Group C - Proficiency Based Progression

Detailed Description

Background: Simulation-based skills training has been proposed as a method of augmenting traditional in vivo training of surgeons.[1] Simulation based training is still in its infancy and there is considerable variability in the United States and around the world on how this type of training is configured.[2] The first successful simulation validation study used a proficiency-based progression methodology.[3] Trainees were required to demonstrate a performance benchmark (based on experienced surgeons performance on a simulator) before completing a laparoscopic cholecystectomy in vivo.[4] This approach has not been used in arthroscopy. In this study we propose to evaluate the utility of proficiency based skills training for learning how to perform an arthroscopic Bankart repair for unidirectional anterior shoulder instability.

Proficiency will be determined using metric based performance levels of surgeons very experienced in performing an arthroscopic Bankart procedure.[5] Metrics have previously been identified, operationally defined and agreed upon at an Arthroscopy Association of North America Delphi panel meeting and then validated. Metrics which have demonstrated construct validity will be used to define a performance benchmark/level of proficiency. Group C will be required to train on a simulator model until they demonstrate this predetermined performance level. At the completion of training all trainees are required to complete a Bankart procedure on a fresh cadaver shoulder specimen. Operative performance will be video recorded and subsequently assessed by two experienced arthroscopic surgeons blinded to subject identity and group. Raters will have been trained to reliably identify predefined procedural steps, errors and sentinel errors to an inter-rater reliability level of r >0.8. [6]

Hypothesis: It is hypothesized that proficiency based progression trainees will perform an arthroscopic Bankart procedure significantly better on a fresh cadaver shoulder than traditionally trained subjects. They will also perform significantly better than trainees who received the exact same curricular material,including the use of a simulator, but will be allowed to progress at their own desired pace with no requirement to demonstrate performance of skills to a proficiency benchmark before progressing in training.

Procedure: Three groups of orthopaedic surgical trainees in postgraduate years four and five will be provided online access to orientation videos demonstrating how to perform a standard arthroscopic Bankart repair. A set of baseline assessments of all 3 groups will be obtained prior to the beginning of training and include visuospatial,[7] perceptual,[8,9] and psychomotor evaluations[10-13] to ensure that the groups are comparable. The standard training group (Group A) will then progress with the training as they would today during a routine resident course. Groups B and C will be prospectively randomized (using an online randomizer) to their respective training conditions. Group C will study the online didactic information until they have demonstrated a predefined proficiency criterion. In addition, successful completion and proficiency demonstration with arthroscopic knot tying will be a requirement for them to proceed to the simulation based training models. The benchmark has been established using knots tied by an experienced group of arthroscopic surgeons. On the simulation models they will continue training until proficiency has been demonstrated. Group C trainees will be supervised and given formative feedback on their performance by an experienced trainer and assessor during these skills training sessions. Group B will complete the same curriculum as Group C with the exception that they will be free to progress through training at their own desired pace with no requirement to demonstrate proficiency in any phase of the training.

Outcome assessment On completion of the course proper all three groups of trainees will be required to complete a Bankart procedure on a fresh cadaver specimen. They will be assisted by an experienced orthopaedic surgeon blinded to the subjects training group and instructed to assist ONLY at the direction of the study participant. No guidance will be provided. Their performance will be video recorded for subsequent analysis. Video raters will be experienced orthopaedic surgeons who will be blinded as to subjects' identity and group. Objective assessment of their intra-operative performance will be measured using the pre-defined performance metrics agreed upon by the Arthroscopy Association of North America Delphi panel and previously demonstrated to show construct validity. These will include correct steps of the procedure completed, time to complete the procedure, intra-procedure errors and sentinel errors.

Power Calculations Based on data derived from the construct validity study on the shoulder model the following power calculation estimates were obtained for Steps completed correctly, Time to perform the procedure, Errors and Sentinel Errors made. Calculations are based on a group size of n = 16, and reference mean and standard deviation scores, an Alpha of 5% which corresponds to a 95% Confidence Interval and a two-tailed statistical test for significance.

In previous studies a transfer effectiveness ration (TER) of 26 - 42% has been shown for novices.[13] A TER of 20% means that the PBP trained group would complete 44.88 steps of the procedure in comparison to the control groups 37.4. The statistical power of this difference is 0.99. A similar magnitude of difference (i.e., 20%) for time and sentinel errors would result in a mean difference with a statistical power > 0.8. Error scores would require a difference between the mean scores of the group ~ 40% to reach an acceptable level of significance. Furthermore, all of these differences would mean that the PBP trainees were still not outperforming the experts.

Statistical Analysis

The following analyses will be conducted:

1. The primary analysis will compare the intra-operative performance of trainees. For each subject, the number or intra-operative steps completed, time taken, errors and sentinel errors will be compared across the three groups with one factor analysis of variance (ANOVA) and specific contrasts between groups will be conducted with Scheffe F-tests.
2. Two-factor ANOVA for repeated measures where factor one is training method and factor two is the anchor number will be conducted on the variables listed. The sphericity assumption, relating to the conditional variances of all pair-wise differences will be evaluated.

Statistical comparisons for repeated measures for the placement of Anchors 1 - 3.

• Anchor Preparation and Insertion steps
• Anchor Preparation and Insertion Errors
• Suture Delivery and Management Steps
• Suture Delivery and Management Errors
• Knot Tying Steps
• Knot Tying Errors

Baseline Aptitude and Performance Pearson Product Moment Correlation Coefficient and Multiple Regression Analysis will be used to assess the strength of the relationship between trainees' performance on individual and combined baseline assessments and objectively assessed performance on the Bankart procedure. The same type of analysis will be used to assess the strength of the relationship between knowledge at the start of the course and Bankart procedure performance.

• Study Type: Interventional
• Enrollment (Actual): 44
• Phase: Not Applicable

Contacts and Locations

Study Locations

• United States
  • Illinois
    • Chicago, Illinois, United States, 60018
      • Orthopedic Learning Center

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: Child; Adult; Older Adult
• Accepts Healthy Volunteers: Yes
• Genders Eligible for Study: All

Description

Inclusion Criteria:

• 4th and 5th year Orthopedic Residents

Exclusion Criteria:

• none

Study Plan

How is the study designed?

Design Details

• Primary Purpose: Health Services Research
• Allocation: Randomized
• Interventional Model: Parallel Assignment
• Masking: Triple

Number of Arms

3

Arms and Interventions

Participant Group / Arm	Intervention / Treatment
No Intervention: Group A; Control: Group of Orthopedic Residents who receive arthroscopic surgery training at the Orthopedic Learning Center during a standard week-end Resident Arthroscopic Training Course
Active Comparator: Group B - Simulation Training; Group B resident training includes the use of a dry model shoulder simulator for practicing the steps and avoiding the errors of an arthroscopic Bankart repair	Behavioral: Group B - Simulation Training; Group B resident training includes the use of a dry model shoulder simulator for practicing the steps and avoiding the errors of an arthroscopic Bankart repair
Experimental: Group C - Proficiency Based Progression; Group C - Proficiency Based Progression Group C residents must test to proficiency on the cognitive material contained in the orientation video, demonstrate arthroscopic knot tying proficiency to a pre-determined benchmark, and perform an arthroscopic Bankart repair on a dry shoulder simulator model to a pre-determined benchmark in order to progress in the training exercise	Behavioral: Group C - Proficiency Based Progression; Group C - Proficiency Based Progression Group C residents must test to proficiency on the cognitive material contained in the orientation video, demonstrate arthroscopic knot tying proficiency to a pre-determined benchmark, and perform an arthroscopic Bankart repair on a dry shoulder simulator model to a pre-determined benchmark in order to progress in the training exercise

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
score on a validated assessment tool using a videotaped performance of an arthroscopic Bankart repair	validated metrics for a reference arthroscopic Bankart repair include 45 steps and 77 potential errors of which 20 are "sentinel errors" (either the error could potentially lead to a significant compromise of the success of the procedure, or introduces significant iatrogenic damage).	9 months

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
time to complete an arthroscopic Bankart repair on a cadaver specimen	videotape duration in minutes and seconds less the time period to create the Bankart lesion	6 months

Collaborators and Investigators

• Sponsor: Arthroscopy Association of North America
• Investigators: Principal Investigator: Richard Angelo, M.D., Arthroscopy Association of North America; Study Chair: Richard Ryu, M.D., Arthroscopy Association of North America; Study Chair: Robert Pedowitz, M. D., PhD., Arthroscopy Association of North America; Study Director: Anthony G Gallagher, PhD., D.Sc, School of Medicine, University College Cork, Ireland.

Publications and helpful links

General Publications

• Angelo RL, Ryu RK, Pedowitz RA, Gallagher AG. The Bankart Performance Metrics Combined With a Cadaveric Shoulder Create a Precise and Accurate Assessment Tool for Measuring Surgeon Skill. Arthroscopy. 2015 Sep;31(9):1655-70. doi: 10.1016/j.arthro.2015.05.006. Epub 2015 Jul 31.
• Angelo RL, Pedowitz RA, Ryu RK, Gallagher AG. The Bankart Performance Metrics Combined With a Shoulder Model Simulator Create a Precise and Accurate Training Tool for Measuring Surgeon Skill. Arthroscopy. 2015 Sep;31(9):1639-54. doi: 10.1016/j.arthro.2015.04.092. Epub 2015 Jun 27.

Study record dates

Study Major Dates

• Study Start: August 1, 2013
• Primary Completion (Actual): February 1, 2014
• Study Completion (Actual): February 1, 2014

Study Registration Dates

• First Submitted: August 7, 2013
• First Submitted That Met QC Criteria: August 12, 2013
• First Posted (Estimate): August 13, 2013

Study Record Updates

• Last Update Posted (Estimate): March 28, 2014
• Last Update Submitted That Met QC Criteria: March 27, 2014
• Last Verified: March 1, 2014

More Information

Terms related to this study

• Keywords: proficiency based progression; surgical skill; metric validation; formative feedback; simulator model
• Other Study ID Numbers: AANA Copernicus Bankart - 1