Use of an Observer Tool and Learning of Non-technical Skills

Use of an Observer Tool to Improve the Learning of Non-technical Skills of Anesthesia Residents Observing High-fidelity Simulation Training Sessions: a Randomized Controlled Trial.

The use of simulation is recommended in anesthesiology education. Given a large number of residents, all of them will not be active participants during high-fidelity simulation sessions but will only be observers. To increase the positive effects of the simulation when the learner is in the role of observer, some authors have proposed to strengthen the educational effect through the use of an observer tool that observers must complete by analyzing the progress of the task performed by their colleagues. This is a list describing the set of key points to be achieved. However, data concerning the educational value of these observer tools are limited. Studies on the use of these tools during crisis management training in the operating room assessed by high-fidelity simulation have shown improved learning outcomes for technical skills but no study has evaluated yet their effect on non-technical skills. The effect on these skills can hardly be assessed by measuring the resident's perception or knowledge and an objective assessment of performance is necessary.

The objective of this study will be to evaluate if using an observer tool (OT) dedicated to non-technical skills improves the learning of these skills in anesthesia residents during high-fidelity simulation.

Study Overview

• Status: Not yet recruiting
• Conditions: Simulation; Education
• Intervention / Treatment: Other: observer tool

Detailed Description

The use of simulation is recommended in anesthesiology education. Given a large number of residents, all of them will not be active participants during high-fidelity simulation sessions but will only be observers. To increase the positive effects of the simulation when the learner is in the role of observer, some authors have proposed to strengthen the educational effect through the use of an observer tool that observers must complete by analyzing the progress of the task performed by their colleagues. This is a list describing the set of key points to be achieved. However, data concerning the educational value of these observer tools are limited. Studies on the use of these tools during crisis management training in the operating room assessed by high-fidelity simulation have shown improved learning outcomes for technical skills but no study has evaluated yet their effect on non-technical skills. The effect on these skills can hardly be assessed by measuring the resident's perception or knowledge and an objective assessment of performance is necessary.

The objective of this study will be to evaluate if using an observer tool (OT) dedicated to non-technical skills improves the learning of these skills in anesthesia residents during high-fidelity simulation.

This prospective and randomized study will be performed in the simulation center of Paris-Saclay University (LabForSIMS) during crisis management training of anesthesia residents. The high-fidelity based-simulation session is a full-day training with 5 different crisis scenarios. For each scenario, two residents will play the role of the senior anesthesiologist and the anesthesia resident respectively. A third resident will play the role of another senior anesthesiologist and will be called as a backup person. Participants will neither be aware in advance of the role they would be assigned nor of the scenario in which they would be involved. Observers will be seated in an adjacent room in which the scenario is broadcasted by live video transmission.

After each scenario, all residents will participate to a debriefing with 2 instructors (experts in anesthesia and simulation). The debriefing will be carried out using the RAS method (reaction, analysis, and synthesis). The actors will be initially invited to give their feelings and emotions. Then the analysis phase will allow for a reflective analysis of the situation by encouraging active participants and observers to highlight the positive and negative points concerning technical and non-technical skills (defined by the educational objectives of each scenario. Decontextualization will then be promoted. The first 3 scenarios will focus on technical skills and the 2 others on technical and non-technical skills. Finally, the conclusion part will highlight the key educational objectives of each scenario.

Anesthesia residents of the Paris area will be enrolled after having given their written consent and video copyright.

Training days will be randomized according to the strategy used when residents will observe the simulation scenarios:

• OT+ group: during the whole training day, the observers will use an observer tool (based on non-technical skills) when they will be not role role-playing
• OT- group: no observer tool will be used

The observer tool used will be the Anaesthetist Non-Technical Skills (ANTS) tool previously published by Fletcher et al. and translated in French. The ANTS scoring system uses four categories assessing task management, teamworking, situation awareness, and decision-making. It will only be used to reinforce learning and will not be used by the residents for summative evaluation.

All the scenarios will be recorded and anonymously retained.

The primary outcome will be the individual performance of non-technical skills assessed by the ANTS score measured by an external evaluator analyzing each video (scenario 1 = pre-test and scenarios 2,3,4 = post-test). The ANTS scoring system uses four categories assessing task management, teamworking, situation awareness, and decision-making (1 to 4 points by category). The minimum score is 4 and the maximum 16.

During the first scenario, and whatever the randomization list, 3 residents will be active participants, and an analysis of their basic performance will be used as a pre-test. Then, depending on the randomization (use of OT or not), the observers will be given the ANTS (or not) tool for scenarios 1,2, and 3. Those residents who will be active participants in scenarios 2,3 or 4 will be assessed regarding their non-technical performance depending on whether they had used (or not) an OT when being in the role of observer in the previous scenarios. This design leads to the fact that some residents may have completed 1, 2, or 3 OTs prior to their active participation in a scenario (e.g. observers of scenarios 1 and 2 will be evaluated for their performance in Scenario 3).

At the end of the day, a questionnaire will be filled out (via a QR code) by all residents in order to collect their personal characteristics and to analyze several secondary outcomes such as satisfaction and the expected future professional impact of the training.

• Study Type: Interventional
• Enrollment (Anticipated): 46
• Phase: Not Applicable

Contacts and Locations

• Study Contact: Name: Antonia Blanié, MD PhD; Phone Number: 33 1 45 21 34 47; Email: antonia.blanie@aphp.fr
• Study Contact Backup: Name: Dan Benhamou, MD Pr; Phone Number: 33 1 45 21 34 47; Email: dan.benhamou@aphp.fr

Study Locations

• France
  • Ile De France
    • Le Kremlin-Bicêtre, Ile De France, France, 94275
      • LabForSIMS - Faculté de Médecine Paris Saclay
      • Contact: Antonia Blanié, MD PhD; Phone Number: 33 1 45 21 34 47; Email: antonia.blanie@aphp.fr
      • Contact: Dan Benhamou, MD Pr; Phone Number: 33 1 45 21 34 47; Email: dan.benhamou@aphp.fr
      • Principal Investigator: Antonia Blanié, MD PhD
      • Sub-Investigator: Dan Benhamou, MD Pr

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: 18 years and older (ADULT, OLDER_ADULT)
• Accepts Healthy Volunteers: Yes
• Genders Eligible for Study: All

Description

Inclusion Criteria:

• all anesthesiology residents of Paris participating in a full-day of crisis management training at the LabforSIMS-Faculty of Medicine Paris-Saclay

Exclusion Criteria:

• declines active participation in the session
• previously included in a CRM training session at LabforSIMS
• no video-recorded

Study Plan

How is the study designed?

Design Details

• Primary Purpose: OTHER
• Allocation: RANDOMIZED
• Interventional Model: PARALLEL
• Masking: NONE

Number of Arms

2

Arms and Interventions

Participant Group / Arm	Intervention / Treatment
EXPERIMENTAL: group (OT+); In this group (OT +), during the whole training day, the observers will use an observer tool (based on non-technical skills) when they will be not role role-playing	Other: observer tool; The observer tool used will be the Anaesthetist Non-Technical Skills (ANTS) tool previously published by Fletcher et al. and translated in French. The ANTS scoring system uses four categories assessing task management, teamworking, situation awareness and decision-making. The observers will use an observer tool it when they will be not role role-playing
NO_INTERVENTION: group (OT-); In this group (OT-), the resident will not be given the observer tool and will observe other residents without any physical support.

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
individual performance of non-technical skills	The primary outcome will be the individual performance of non-technical skills assessed by the ANTS score measured by an external evaluator analyzing each video (Kirkpatrick level 2) . The Anaesthetist Non-Technical Skills (ANTS) tool previously published by Fletcher et al. translated in French. The ANTS scoring system uses four categories assessing task management, teamworking, situation awareness and decision-making (1 to 4 points by categories). The minimum score is 4 and the maximum 16.	30 min by video after the end of the session by an external evaluator

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
each category of the ANTS score	Separate analysis of each categories of the ANTS score (task management, teamworking, situation awareness and decision-making (1 to 4 points by categories))	10 min by video after the end of the session by an external evaluator
Team performance	Team performance will be assessed by a Team performance observation tool published and translated in French by Maignan et al. (Kirkpatrick level 2) (0 to 4 points by items). Number of OT used and degree of completeness	10 min by video after the end of the session by an external evaluator
level of Satisfaction	Satisfaction will be assessed by a Likert scale (0 to 10) (Kirkpatrick level 1)	1 min and the end of the session by all residents
future professional impact	The expected future professional impact of the training will be assessed by a Likert scale (0 to 10) (Kirkpatrick level 3)	1 min and the end of the session by all residents

Collaborators and Investigators

• Sponsor: Université Paris-Sud
• Investigators: Principal Investigator: Antonia Blanié, MD PhD, LabForSIMS - Faculté de Médecine Paris Saclay

Study record dates

Study Major Dates

• Study Start (ANTICIPATED): June 13, 2022
• Primary Completion (ANTICIPATED): July 20, 2022
• Study Completion (ANTICIPATED): July 1, 2023

Study Registration Dates

• First Submitted: May 25, 2022
• First Submitted That Met QC Criteria: May 25, 2022
• First Posted (ACTUAL): May 31, 2022

Study Record Updates

• Last Update Posted (ACTUAL): May 31, 2022
• Last Update Submitted That Met QC Criteria: May 25, 2022
• Last Verified: May 1, 2022

More Information

Terms related to this study

• Keywords: simulation; education; non-technical skills; observer tool
• Other Study ID Numbers: LabForSims-006

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: NO

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No