Simulation-based training for colonoscopy: establishing criteria for competency

Abstract

The aim of this study was to create simulation-based tests with credible pass/fail standards for 2 different fidelities of colonoscopy models. Only competent practitioners should perform colonoscopy. Reliable and valid simulation-based tests could be used to establish basic competency in colonoscopy before practicing on patients. Twenty-five physicians (10 consultants with endoscopic experience and 15 fellows with very little endoscopic experience) were tested on 2 different simulator models: a virtual-reality simulator and a physical model. Tests were repeated twice on each simulator model. Metrics with discriminatory ability were identified for both modalities and reliability was determined. The contrasting-groups method was used to create pass/fail standards and the consequences of these were explored. The consultants significantly performed faster and scored higher than the fellows on both the models (P < 0.001). Reliability analysis showed Cronbach α = 0.80 and 0.87 for the virtual-reality and the physical model, respectively. The established pass/fail standards failed one of the consultants (virtual-reality simulator) and allowed one fellow to pass (physical model). The 2 tested simulations-based modalities provided reliable and valid assessments of competence in colonoscopy and credible pass/fail standards were established for both the tests. We propose to use these standards in simulation-based training programs before proceeding to supervised training on patients.

Conflict of interest statement

The authors have no funding and conflicts of interests to disclose.

Figures

FIGURE 1

Virtual-reality simulator test. (A) Establishing a pass/fail standard using the contrasting-groups method. The distributions of scores of novices (dotted line, n = 15) and experienced endoscopists (solid line, n = 10) are shown. The pass score (15.5 points/min) was set at the intersection of the score distributions of the 2 groups. (B) Box plot showing the consequences of the established pass/fail criterion. One of the experienced endoscopists failed the test and one novice passed the test.

FIGURE 2

Phantom model test. (A) Establishing a pass/fail standard using the contrasting-groups method. The distributions of scores of novice endoscopists (dotted line, n = 15) and experienced endoscopists (solid line, n = 10) are shown. The pass score (0.79 points/min) was set at the intersection of the score distributions of the 2 groups. (B) Box plot showing the consequences of the established pass/fail criterion. All the experienced endoscopists passed the test as well as a single outlier in the novice group.