Motion Capture as a Tool for the Assessment of Laparoscopic Performance

The goal of this study is to 1) identify motion capture variables that can be used to differentiate surgical experience level and 2) evaluate if these variables can be used in a surgical education setting to improve resident performance.

Study Overview

• Status: Completed
• Conditions: Healthy; Adult ALL
• Intervention / Treatment: Other: Objective Feedback (Motion Capture)

Detailed Description

This will be a two-phase study that will take 2 years to complete.

Phase 1 will recruit staff surgeons, surgical fellows, and residents from the departments of Obstetrics and Gynecology, Urology, General Surgery, and Thoracics at The University of Ottawa and The Ottawa Hospital to complete a set of FLS tasks. Logistic regressions will be used to identify significant factors that differentiate surgical experience level (junior residents, senior residents, fellows, and staff). Classification of residents will depend on their current cohort in the CDB or traditional time-based system: A target interval will be computed for each significant factor based on the staff surgeons' average performance. This target interval will be used to assess resident performance in Phase 2.

Phase 2 will recruit Transition to Discipline residents from each department to complete 3 sets of FLS tasks. Participants will be randomized into an "objective feedback" group or a "no feedback" group. Participants randomized to the objective feedback group will receive a report that compares their performance in Set 1 to that of the staff surgeons' using the target interval as a reference. Participants receiving objective feedback will then complete Sets 2 and 3 knowing what factors to improve upon. Participants receiving no feedback will complete all sets with no intervention. Phase 2 data collection will start once Phase 1 is completed.

Trajectories of retro-reflective markers will be recorded using a 6-camera motion analysis system (40MXF-40 Vicon cameras, Oxford Metrics, Oxford, UK) sampling at 100 Hz with supporting Nexus software (v2, Oxford Metrics, Oxford, UK). Retro-reflective markers will be placed on various anatomical landmarks on the participant's body based on a cluster marker set. Laparoscopic surgical tools (grasper, scissors, or dissector) will have a marker triad fixed to the upper shaft while a laparoscopic trainer (FLS Trainer System, Limbs & Things, Savannah GA) will have markers placed at its corners. Using Vicon Nexus software, raw marker trajectories will be filtered and used to compute local coordinate systems of the upper body, joint centers, and subsequent joint angles while filtered marker trajectories on the laparoscopic tools will compute the tool tip's motion and orientation relative to the box trainer's workspace. Inertial measurement units (IMUs) will also be used to capture motion of the upper body and compare to motion capture outcomes (Secondary Objective). Wireless IMUs will be placed along the long axis of the forearm, upper arm, upper spine and lower spine.

Surface electromyography (EMG) signals of the biceps, triceps, anterior deltoid, upper trapezius, wrist extensors and flexors will also be recorded using a wireless EMG system (Trigno, Delsys Inc., Natick MA) and recorded using Nexus. Sensors will be placed over the muscle bellies following SENIAM guidelines [34] and signals will be sample data 100Hz with a 20-450Hz bandpass. All experimental EMG signals will be appropriately conditioned and normalized to maximum EMG value elicited during maximum voluntary isometric contraction (MVIC) exercises. Three repetitions of each MVIC exercise will be performed before the experimental task and will include elbow flexion and extension, arm abduction, arm flexion, and wrist flexion and extension. A minimum of 30 seconds rest will be given between each exercise to mitigate effects of fatigue.

Motion and EMG data will be exported to a custom MatLab (2019, Mathworks, Natick, MA) application to compute objective measures of laparoscopic performance: instrument tip pathlength, average (±1 stdev) excursion velocity, average (±1 stdev) excursion error, average (±1 stdev) upper body kinematics (shoulder, elbow, wrist, neck and trunk) in all three planes of motion, and integrated EMG.

Video of the surgical tasks will be recorded and synced to marker trajectory data using an external trigger. Videos will be de-identified, and the last repetition assessed by three staff surgeons. First, the raters will categorize each video as Transition to Discipline resident, Foundation of Discipline resident, Core Discipline resident, Transition to Practice resident, fellow, or staff surgeon based other their general perception of the performer's experience level. The raters will then score each video following the FLS scoring system (based on timing and penalty scores), and the OSATS evaluation grid.

• Study Type: Interventional
• Enrollment (Actual): 26
• Phase: Not Applicable

Contacts and Locations

Study Locations

• Canada
  • Ontario
    • Ottawa, Ontario, Canada, K1G 7W9
      • Ottawa Hospital Research Institute

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: Child; Adult; Older Adult
• Accepts Healthy Volunteers: Yes

Description

Inclusion Criteria:

• staff surgeons, surgical fellows and residents
• primary affiliation with the departments of Obstetrics and Gynecology, Urology, General Surgery, and Thoracics at the University of Ottawa and The Ottawa Hospital

Exclusion Criteria:

• current injury/condition that in the opinion or the participant and/or research team will affect performance

Study Plan

How is the study designed?

Design Details

• Primary Purpose: Other
• Allocation: Randomized
• Interventional Model: Sequential Assignment
• Masking: None (Open Label)

Number of Arms

2

Arms and Interventions

Participant Group / Arm	Intervention / Treatment
Other: Objective Feedback (Motion Capture); Participants in the objective feedback group will receive a report that compares their performance in Set 1 to that of the staff surgeons' using the target interval as a reference. Participants receiving objective feedback will then complete Sets 2 and 3 knowing what factors to improve upon.	Other: Objective Feedback (Motion Capture); Participants assigned to the objective feedback group will be given a report after completing Set 1 and Set 2. This report will outline their performance for each significant factor (identified in Phase 1) relative to a target interval, derived from the staff surgeon group average (± one standard deviation). Participants assigned to the objective feedback group will then be able to repeat the tasks knowing which variables to improve.
No Intervention: No Feedback; Participants receiving no feedback will complete all sets with no intervention.

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Completion time	Time to complete each task measured in seconds	1 year
Instrument pathlength	Pathlength of surgical instrument tip measured in cm	1 year
Motion smoothness	Motion smoothness of surgical instrument tip measured by the number of changes in acceleration (m/s^2)	1 year
Instrument orientation	Orientation of surgical instrument tip measured in the three planes of motion (depth, pitch, roll, yaw), quantified in degrees relative to the surgical trainer box's local coordinate system.	1 year
Fundamentals of Laparoscopy Skill (FLS) score	- A numbered score is calculated following procedures outlined in the Fundamentals of Laparoscopy Skills (FLS) modules	1 year
Objective Structured Assessment of Technical Skills (OSATS) score	- A numbered score is computed from a global rating system based evaluation grid.	1 year
Participant classification	Blinded reviews will classify participants as junior residents, senior residents, fellows/staff based on their perception of participant skill level.	1 year

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Muscle Electromyography	Muscles: erector spinae, biceps, triceps, anterior deltoid, upper trapezius, wrist extensors, wrist flexors; Amplitude quantified as %maximum measured during maximal effort trials	1 year
Muscle activation symmetry	Muscles: erector spinae, biceps, triceps, anterior deltoid, upper trapezius, wrist extensors, wrist flexors; Symmetry indices will be quantified as non-dominant side/dominant side.	1 year
Upper body kinematics	- Body orientation will be measured in degrees for the shoulder, elbow, wrist, neck, trunk	1 year

Collaborators and Investigators

• Sponsor: Ottawa Hospital Research Institute
• Investigators: Principal Investigator: Sukhbir Singh, Ottawa Hospital Research Institute

Publications and helpful links

General Publications

• Sanchez A, Rodriguez O, Sanchez R, Benitez G, Pena R, Salamo O, Baez V. Laparoscopic surgery skills evaluation: analysis based on accelerometers. JSLS. 2014 Oct-Dec;18(4):e2014.00234. doi: 10.4293/JSLS.2014.00234.
• Moorthy, Munz, Dosis, Bello, Darzi. Motion analysis in the training and assessment of minimally invasive surgery. Minim Invasive Ther Allied Technol. 2003 Jul;12(3):137-42. doi: 10.1080/13645700310011233.
• Farcas MA, Trudeau MO, Nasr A, Gerstle JT, Carrillo B, Azzie G. Analysis of motion in laparoscopy: the deconstruction of an intra-corporeal suturing task. Surg Endosc. 2017 Aug;31(8):3130-3139. doi: 10.1007/s00464-016-5337-4. Epub 2016 Dec 7.
• Gray RJ, Kahol K, Islam G, Smith M, Chapital A, Ferrara J. High-fidelity, low-cost, automated method to assess laparoscopic skills objectively. J Surg Educ. 2012 May-Jun;69(3):335-9. doi: 10.1016/j.jsurg.2011.10.014.
• Ganni S, Botden SMBI, Chmarra M, Li M, Goossens RHM, Jakimowicz JJ. Validation of Motion Tracking Software for Evaluation of Surgical Performance in Laparoscopic Cholecystectomy. J Med Syst. 2020 Jan 24;44(3):56. doi: 10.1007/s10916-020-1525-9.
• Gallagher AG, McClure N, McGuigan J, Ritchie K, Sheehy NP. An ergonomic analysis of the fulcrum effect in the acquisition of endoscopic skills. Endoscopy. 1998 Sep;30(7):617-20. doi: 10.1055/s-2007-1001366.

Study record dates

Study Major Dates

• Study Start (Actual): May 1, 2021
• Primary Completion (Actual): October 11, 2023
• Study Completion (Actual): December 31, 2024

Study Registration Dates

• First Submitted: November 3, 2020
• First Submitted That Met QC Criteria: November 22, 2020
• First Posted (Actual): November 27, 2020

Study Record Updates

• Last Update Posted (Actual): March 25, 2025
• Last Update Submitted That Met QC Criteria: March 21, 2025
• Last Verified: March 1, 2025

More Information

Terms related to this study

• Keywords: surgical trainees; surgical staff
• Other Study ID Numbers: 20200631-01H

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: NO

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No