Orsi Surgical Skills E-learning Trial (OSSET)

Prospective and Randomized Assessment of the Impact of Proficiency-based Progression E-learning Prior to Skills Training on a Robotic Surgical Task

The training of robotic surgical procedural skills has been challenged by changes in work practices and safety concerns specifically related to training. In surgery and procedural medicine simulation-based training has been demonstrated to supplant the early part of the learning curve. Training in the skills laboratory is however expensive because of equipment and supervision burdens. In this study the investigators will assess the economic impact of proficiency-based progression (PBP) e-learning training prior to training in the skills laboratory. 48 trainees will be randomly assigned to one of four groups. 1) will receive an apprenticeship type training (Group 1 ; n=12), 2) A standard or traditional trained group (Group 2; n=12) will then receive face-to-face lectures on how to perform the robotic surgical training task (i.e., ORSI chicken anastomosis task for learning robotic suturing and knot tying.) 3) The third group (Group 3; n=12) will have e-learning training prior to training in the skills laboratory and then learn the same task. 4) The fourth group (Group 4; n=12) will have the exact same pre-course e-learning curriculum as Group 3 but will be required to study it until they score at the quantitatively defined proficiency benchmark of experienced robotic surgeons, i.e., the mean performance level of experienced robotic surgeons - they can complete the task with <10 performance errors.

The research will be conducted at the laboratory skills lab of Orsi Academy, Proefhoevestraat 12 9090 Melle. It will be conducted by Maxime Lasseel and Laura Langhendries, under direct guidance of Dr. S. Puliatti, Prof. A.G. Gallagher and Prof. A. Mottrie.

Study Overview

• Status: Completed
• Conditions: Surgery; E-learning
• Intervention / Treatment: Procedure: Online course with proficiency test; Procedure: Online course; Procedure: Standard training group; Procedure: Apprenticeship group

Detailed Description

Simulation-based training has been shown to be an effective way to prepare the trainee for the operating room. Gallagher et al. have demonstrated in a number of studies that proficiency-based progression (PBP) simulation training works best when it is integrated into a curriculum. Learning is optimal when trainees receive metric-based feedback on their performance. Metrics should unambiguously characterize important aspects of procedure or skill performance. They are developed from a task analysis of the procedure or skills to be learned. The outcome of the task analysis should also shape how the simulation looks and behaves. Metric-based performance characterization can be used to establish a benchmark (i.e., a level of proficiency) which trainees must demonstrate before training progression. This approach ensures a more homogeneous skill-set in graduating trainees and can be applied to any level of training. Prospective, randomized and blinded clinical studies have shown that trainees who acquired their skills to a level of proficiency on a simulator in the skills laboratory perform significantly better in vivo in comparison to their traditionally trained colleagues. Although simulation-based training has been shown to be very effective in helping trainees acquire skills for the operating room. Acquiring skills in the skills laboratory, outside the operating is no doubt significantly safer for patients as they can acquire skills at the start of their learning curve, away from the operating room. The skills laboratory is however a very expensive resource particularly for the acquisition of skills to used advanced technologies such as surgical robots. It is recognised that inadequate training in robotic surgery results in increased complications and higher costs. Standardising modular training with defined steps and the errors to avoid will enable proficiency-based progression (PBP) training. Skills training for robotic surgery is some of the most expensive training space in medicine. It therefore should be used efficiently.

E-learning is naturally suited for the delivery of multisensory information such as material associated with learning to perform surgery. It can provide a flexible learning environment that can be used as an adjunct to face-to-face teaching, skills laboratory training and clinical surgery training for both novice and very experienced operators. E-learning offers particularly exciting opportunities for augmenting the learning process in surgery and procedural medicine. For example, in a well-delivered traditional lecture, the academic has very few ways of knowing how effectively they have imparted the information that they are trying to communicate.

Furthermore, it would be unrealistic to expect that all of the audience would be learning at the same pace, but the traditional lecture is delivered in the standard 40 - 50 minute time period to the individuals sitting in the same room. The hope is that by the time of the exam, everyone is at a sufficient standard to at least pass the course. If the material is delivered on an e-platform, the progress of each individual can be tracked with a formative assessment process. This means that individuals who learn at a slower pace can have their education supplemented automatically or they can be flagged for direct academic intervention.

E-learning also suits the PBP methodology in that only trainees who have demonstrated the requisite proficiency benchmark actually get training in the skills laboratory. The aim of this study is to quantitatively assess the economic impact and cost savings (if any) of requiring trainees to study a robotic surgical task until they demonstrate a defined performance benchmark before commencing their technical skills training in the skills laboratory.

As recently outlined during the first European multi-specialty consensus meeting on robotic surgery training, a fundamental step in creating a proficient robotic surgeon is the acquisition of basic surgical skills, such as suturing, knotting, coagulating and dissecting. Among numerous dry models, the Venezuelan chicken model seems to be ideal for suturing, anastomosis and knotting exercises, not only in the urological field but in all robotic surgical disciplines. The participants will be working with this model.

Methods:

The design of this study is shown in Figure 1. It is a prospective, randomized and blinded study where 48 trainees will be randomly assigned to one of four groups. They will all complete educational and training tasks that prepare them for the completion of the ORSI chicken anastomosis task. They will continue training on the task until they can perform the task to quantitatively defined proficiency benchmarks. The outcome of this study is the amount of resources used by each group to get to the proficiency benchmark.

Subjects:

48 university educated novice subjects will be randomly assigned to four different training conditions.

Materials:

ORSI robotic surgery chicken anastomosis task:

The chicken has all the abdominal organs removed, except 6-8 cm of the cloaca (starting measurements from the anus) and the stomach. The stomach will simulate the human bladder neck and the cloaca will simulate the human urethra. The similarity of the model is increased by the possibility of positioning a catheter inside the cloaca both during and at the end of the procedure.

Instruments and Robot positioning:

The model must be placed on the surface of a table, at the same height as a normal operating table. All models of surgical robots currently on the market can be used to perform the task. The docking of the robot must be performed following the normal rules used for robotic surgery. In particular, the camera trocar must be at a distance of 18-20 cm from the target. Robotic trocars must be positioned at a distance of 8-10 cm apart. The instruments suggested are 2 large needle drivers and 1 prograsp forceps. 2 polysorb stitches of 10 cm each, knotted at the end with 4 knots can be used to perform the anastomosis.

Metrics:

The procedure metrics for a reference anastomosis and knotting in a chicken model were developed by a procedure characterization group consisting of 4 surgeons and a behavioral scientist. The surgeons had previous large substantial hand-on training experience in the lab. Procedure phases steps, errors and critical errors were also outlined.

Suturing and knotting task:

The aim of the exercise is to perform a hermetic circular anastomosis that allows connecting two anatomical tubular structures. Always using an outside-inside direction in the stomach, and inside-outside in the cloaca, two hemi-continuous sutures must be performed, knotting the two threads at the end.

The basic steps into which the exercise is divided are:

• posterior Wall (first 4 bites on stomach and cloaca);
• right and left lateral wall (the bites between the posterior and the anterior wall;
• anterior wall (the last 4 bites on stomach and cloaca);
• knotting (one double knot and two single knots to close the anastomosis).

The errors that the trainee can commit during the execution of the task can be divided into two categories: suturing operative errors and knotting operative errors.

These technical errors result in alterations of the needle, breaking of the suture thread, lesions to the biological tissue or can lead to inadequate final result.

The breakage of the needle or suture thread and the leakage of water to the final test performed through the catheter, are considered critical errors.

The time limit for performing the exercise is 25 minutes. The lack of progress made in a specific step of the anastomosis for the time of 1 minute is considered as a "failure to progress" error.

Assessment Pre-assessment of subjects Robotic surgery requires two-handed coordination of instruments with a monitor image, in three-dimensional space and, consequently, necessitates complex ambidextrous psychomotor, visio-spatial, and perceptual skills. These abilities are not uniformly distributed in the population and therefore to ensure that subjects are of comparable abilities all participating subjects will undergo a psychometric assessment which will assess their visio-spatial, perceptual and psychomotor coordination.

Psychomotor Assessment All participating subjects will be required to perform a simple task psychomotor task on the robot prior to training proper.

Visuospatial Assessment

The visuospatial assessment includes the use of three fixed length, fixed format, self-administered tests to evaluate the ability to receive, interpret, and apply meaning to visual information as measured in constructional skills and visual perceptual tests. The cognitive tests administered during this study will be:

1. Cube Comparisons Test - designed to evaluate ability to detect differences in 3 dimensional cubes.
2. Card Rotations Test - designed to evaluate ability to detect differences in 2 dimensional figures.
3. Map Planning Test - designed to evaluate ability to determine the shortest route between two places Perceptual Assessment The perceptual assessment will include a fixed length, fixed format, self-administered, computer generated test (PicSOr) used to evaluate the ability to interpret three dimensional (3-D) structures from two dimensional (2D) displays. It is scored using an interval/ratio scale with values that range between 0.0 and 1.0. A score of 1 = perfect ability to reconstruct 3-D from 2-D cues; a sore of 0 = inability to reconstruct 3-D from 2D cues. In large-scale international studies most surgical residents perform the task well but a small group have demonstrated they have struggle with it.

Procedure Group 1: Pre-trained Group (online didactic to proficiency- PBP++) Group 1 Pre-trained group will receive information on how to optimally perform the ORSI chicken anastomosis task and the material will be delivered online via the ORSI e-learning platform. They will be given access to the material two weeks before their training. Unlike the other groups, Group 1 will be required to study the material to a pre-defined performance benchmark or proficiency level.

Group 2: Pre-trained Group (online didactic- PBP+) The Pre-trained group (Group 2) will receive the exact same information as Group 1 on how to optimally perform the ORSI chicken anastomosis task but the participants are not required to study the material to a pre-defined proficiency benchmark. The time the subjects spend on the task and effort expended will be logged.

Group 3: Standard training group The standard trained group will receive face-to-face lectures on how to perform the ORSI chicken anastomosis task. It will not differ from what they would normally receive during a traditional surgery training course when they arrive at the ORSI academy for their training. The content of the face-to face lecture is the same as in the e-learning courses.

Group 4: Apprenticeship Group The apprenticeship trained group will not receive face-to-face lectures or e-learning on how to perform the ORSI chicken anastomosis task. They will however receive hands-on practical one-to-one training during a traditional surgical training course, with deliberate practice. They will also receive published materials describing how best to perform the task and mentoring on suturing and knot tying by a task expert who will guide their performance.

Proficiency definition Proficiency will be quantitatively defined on the mean of the objectively assessed performance of experienced consultant surgeons who are also experienced in performing robotic suturing, knot tying and anastomosis.

• Study Type: Interventional
• Enrollment (Actual): 48
• Phase: Not Applicable

Contacts and Locations

Study Locations

• Belgium
  • Oost-Vlaanderen
    • Melle, Oost-Vlaanderen, Belgium, 9090
      • Orsi Academy

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: 18 years and older (ADULT, OLDER_ADULT)
• Accepts Healthy Volunteers: Yes
• Genders Eligible for Study: All

Description

Inclusion Criteria:

• Second bachelor - second master students enrolled in a University.

Exclusion Criteria:

• Previous experience on robotic surgery tasks.

Study Plan

How is the study designed?

Design Details

• Primary Purpose: OTHER
• Allocation: RANDOMIZED
• Interventional Model: PARALLEL
• Masking: SINGLE

Number of Arms

4

Arms and Interventions

Participant Group / Arm	Intervention / Treatment
ACTIVE_COMPARATOR: Group 1: Online didactic to proficiency PBP+; Group 1 Pre-trained group will receive information on how to optimally perform the ORSI chicken anastomosis task and the material will be delivered online via the ORSI e-learning platform. They will be given access to the material two weeks before their training. Unlike the other groups, Group 1 will be required to study the material to a pre-defined performance benchmark or proficiency level. Once the online course is completed, participants have to perform the orsi chicken anastomosis task.	Procedure: Online course with proficiency test; Participants will receive an online e-course with a proficiency test at the end. Afterwards, participants will have to perform a hermetic circular anastomosis that allows connecting two anatomical tubular structures using an outside-inside direction in the stomach and inside-outside in the cloaca. Two hemi-continuous sutures must be performed, knotting the two treads at the end.; Other Names: Online didactic to proficiency PBP++
ACTIVE_COMPARATOR: Group 2: Online didactic PBP+; The Pre-trained group (Group 2) will receive the exact same information as Group 1 on how to optimally perform the ORSI chicken anastomosis task but they are not required to study the material to a pre-defined proficiency benchmark. The time they spend on the task and effort expended will be logged. Once the online course is completed, participants have to perform the orsi chicken anastomosis task.	Procedure: Online course; Participants will receive an online e-course but are not required to pass the proficiency test. Afterwards, participants will have to perform a hermetic circular anastomosis that allows connecting two anatomical tubular structures using an outside-inside direction in the stomach and inside-outside in the cloaca. Two hemi-continuous sutures must be performed, knotting the two treads at the end.; Other Names: Online didactic PBP +
ACTIVE_COMPARATOR: Group 3: Standard training group; The standard trained group will receive face-to-face lectures on how to perform the ORSI chicken anastomosis task. It will not differ from what they would normally receive during a traditional surgery training course when they arrive at the ORSI academy for their training. The content of the face-to face lecture is the same as in the e-learning courses. After the face-to face lecture, the participants have to perform the orsi chicken anastomosis task.	Procedure: Standard training group; After receiving a traditional lecture, participants will have to perform a hermetic circular anastomosis that allows connecting two anatomical tubular structures using an outside-inside direction in the stomach and inside-outside in the cloaca. Two hemi-continuous sutures must be performed, knotting the two treads at the end.
ACTIVE_COMPARATOR: Group 4: Apprenticeship Group; The apprenticeship trained group will not receive face-to-face lectures or e-learning on how to perform the ORSI chicken anastomosis task. They will however receive hands-on practical one-to-one training during a traditional surgical training course, with deliberate practice. During this course, clinicians will train the participants on how to perform the orsi anastomosis task. They will also receive published materials describing how best to perform the task and mentoring on suturing and knot tying by a task expert who will guide their performance.	Procedure: Apprenticeship group; Participants will have to perform a hermetic circular anastomosis that allows connecting two anatomical tubular structures using an outside-inside direction in the stomach and inside-outside in the cloaca. Two hemi-continuous sutures must be performed, knotting the two treads at the end. Participants will only receive papers on how the task will be performed and clinicians will help them with completion of the task.

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Errors made during the task at baseline	We will measure the amount of errors made bij trainees during the procedure via a metric based system at baseline when the participants do their first chicken anastomosis task. This task takes maximum 30 minutes	30 minutes
Errors made during the task at proficiency	Participants repeat the chicken anastomosis task in order to train themselves. Every 30 minutes, a new participant starts the task. Examinators will assess the errors made by trainees via a metric based system at every attempt until they demonstrate proficiency. Every 30 minutes, errors will be assessed.	30 minutes
Cost of training (in euro)	The economic impact of proficiency-based training. We will watch and note the resources utilised (amount of sutures, amount of chickens utilised by the participants) and time needed (in minutes) to reach proficiency. By knowing these elements, we can calculate the price per hour that it costs to train trainees in the specific groups.	Up to 9 days

Collaborators and Investigators

• Sponsor: Orsi Academy
• Investigators: Principal Investigator: Stefano Puliatti, Medical Dir, ORSI Acadmy

Publications and helpful links

General Publications

• Puliatti S, Amato M, Farinha R, Paludo A, Rosiello G, De Groote R, Mari A, Bianchi L, Piazza P, Van Cleynenbreugel B, Mazzone E, Migliorini F, Forte S, Rocco B, Kiely P, Mottrie A, Gallagher AG. Does quality assured eLearning provide adequate preparation for robotic surgical skills; a prospective, randomized and multi-center study. Int J Comput Assist Radiol Surg. 2022 Mar;17(3):457-465. doi: 10.1007/s11548-021-02545-4. Epub 2022 Jan 7.

Study record dates

Study Major Dates

• Study Start (ACTUAL): August 25, 2020
• Primary Completion (ACTUAL): September 20, 2020
• Study Completion (ACTUAL): September 20, 2020

Study Registration Dates

• First Submitted: August 26, 2020
• First Submitted That Met QC Criteria: September 1, 2020
• First Posted (ACTUAL): September 9, 2020

Study Record Updates

• Last Update Posted (ACTUAL): December 22, 2020
• Last Update Submitted That Met QC Criteria: December 18, 2020
• Last Verified: December 1, 2020

More Information

Terms related to this study

• Keywords: Surgery; E-learning; Proficiency-based training
• Other Study ID Numbers: 22/07/2020

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: NO

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No