The Effect of Structured Feedback on Live Surgical Performance (CSATS)

The Effect of Structured Feedback on Live Surgical Performance: A Single-Blind Randomized Control Trial

This study seeks to establish the effect of regular structured feedback on surgical performance using crowd sourced video assessments with validated objective performance scores and time to complete a specific surgical procedure through a randomized controlled trial. The working hypothesis is that regular structured assessment will lead to a greater measurable improvement in performance, as defined by objective performance scores, than will traditional feedback methods.

Study Overview

• Status: Terminated
• Conditions: Surgical Education
• Intervention / Treatment: Other: regular structured assessment of surgical performance

Detailed Description

The American Association of Gynecologic Laparoscopists (AAGL) and Accreditation Council of Graduate Medical Education (ACGME) have endeavored to implement structure and standardization for the fellowship in minimally invasive gynecologic surgical programs (FMIGS) and obstetric and gynecology residency programs respectively. Educational objectives are explicitly delineated. Though the fellowship and residency educational objectives are well rounded, and include the development of skills to become leaders and educators in the field of minimally invasive gynecologic surgery (MIGS) and general obstetrics and gynecology, one of the main objective of the fellowship and residency training is the attainment of surgical skill. This is ensured by a requirement for a breadth of surgical exposure including vaginal surgery, hysteroscopy, traditional laparoscopy, management of surgical complications, minimally invasive hysterectomy, urogynecologic procedures, gastrointestinal surgery, urinary tract surgery, reproductive surgery, and the surgical management of common gynecologic conditions such as endometriosis. Minimum case requirements have been implemented to provide guidance for FMIGS and residency programs. In addition, trainees are required to log surgical cases and this information is utilized for FMIGS and residency program site inspections as well as for feedback of surgical competency. Despite standardization efforts by the AAGL and ACGME, the FMIGS and obstetric and gynecology residency curriculums are still in evolution. There is likely still great variation in surgical exposure between programs. In addition, though regular feedback is a requirement of training programs, the process is likely unstructured and subjective; based on case logs, self-assessments, and feedback by faculty surgeons. There is thus a critical need to develop a standardized, objective, timely, and actionable feedback methodology to facilitate program oversight and ensure FMIGS fellows and obstetrics and gynecology residents are achieving surgical competency throughout their training. In the absence of such structured feedback models, it will remain difficult to optimize FMIGS fellowship and obstetrics and gynecology resident training, and provide objective measurements of the impact of such training on surgical performance and patient outcomes. .

The long-term goal of this study is to assist in developing methods for surgical assessment that facilitate program oversight, allow fellows, residents, and program directors to identify specific skill sets in need of improvement, monitor progress in skill acquisition, and clearly and objectively document achievement of milestones throughout training. The study's overall objective is to determine how consistent structured feedback using crowd sourcing impacts time to complete a task as well as surgical performance as measured by GOALS and GEARS scores. The central hypothesis is that fellows and residents who receive consistent structured feedback over a sustained period of time will have shorter task times, overall improved performance, and greater satisfaction than fellows and residents receiving traditional feedback. The rationale for this work is that it will provide focus and supportive data for large-scale studies of surgical skill acquisition and development.

This study seeks to establish the effect of regular structured feedback on surgical performance using crowd sourced video assessments with GOALS and GEARS through a randomized controlled trial. The study hypothesizes regular structured assessment will lead to a greater measurable improvement in performance, as defined by change and GOALS and GEARS scores, than will traditional feedback methods as well as time to complete a specific procedure.

• Study Type: Interventional
• Enrollment (Actual): 47
• Phase: Not Applicable

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: 18 years to 60 years (Adult)
• Accepts Healthy Volunteers: No
• Genders Eligible for Study: All

Description

Inclusion Criteria:

• current position as a resident in obstetrics and gynecology in a year of training where laparoscopic or robotic hysterectomy is regularly performed -- OR -- FMIGS Fellow regularly performing laparoscopic or robotic hysterectomy
• participants must be between 18 and 60 years of age

Exclusion Criteria:

• persons not in a position as a resident in obstetrics and gynecology in a year of training where laparoscopic or robotic hysterectomy is regularly performed -- OR -- persons not a FMIGS Fellow regularly performing laparoscopic or robotic hysterectomy
• participants younger than 18 and older than 60 years of age

Study Plan

How is the study designed?

Design Details

• Primary Purpose: Other
• Allocation: Randomized
• Interventional Model: Parallel Assignment
• Masking: Single

Number of Arms

2

Arms and Interventions

Participant Group / Arm	Intervention / Treatment
No Intervention: Control Group; Participants randomized to the Control Group will submit a video of surgical performance of vaginal cuff closure during a hysterectomy. Subjects in the control group will receive traditional feedback methods as determined by each participant's training program. Subjects in the control group will repeat the process of video submission of the same procedure using the same surgical approach every 2 weeks for 2 cycles. At 6 weeks, participants will submit a final video performing vaginal cuff closure during hysterectomy.
Experimental: Experimental Group; Participants randomized to the Experimental Group will submit a video of surgical performance of vaginal cuff closure during a hysterectomy. Subjects in the experimental group will then receive feedback within 5 business days of video submission and repeat the process of video submission of the same procedure using the same surgical approach every 2 weeks for 2 cycles. At 6 weeks, participants will submit a final video performing vaginal cuff closure during hysterectomy.	Other: regular structured assessment of surgical performance

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
difference in mean time needed to perform cuff closure during hysterectomy	The difference in mean time (measured in minutes) needed to perform vaginal cuff closure during hysterectomy between subjects in the control versus experimental groups.	6 weeks post study enrollment
difference in mean surgical performance score	The difference in mean surgical performance score of a vaginal cuff closure at the time of hysterectomy between participants in the control versus experimental groups.	6 weeks post study enrollment

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
difference in trainee satisfaction with feedback method	the difference in mean satisfaction score between participants in the control versus experimental groups	6 weeks post study enrollment

Collaborators and Investigators

• Sponsor: George Washington University

Study record dates

Study Major Dates

• Study Start (Actual): December 1, 2016
• Primary Completion (Actual): February 1, 2019
• Study Completion (Actual): February 1, 2019

Study Registration Dates

• First Submitted: December 6, 2016
• First Submitted That Met QC Criteria: December 6, 2016
• First Posted (Estimate): December 8, 2016

Study Record Updates

• Last Update Posted (Actual): October 25, 2022
• Last Update Submitted That Met QC Criteria: October 21, 2022
• Last Verified: October 1, 2022

More Information

Terms related to this study

• Other Study ID Numbers: 101614

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: Undecided