Simulation Game Versus Multiple Choice Questionnaire to Assess the Clinical Competence of Medical Students: Prospective Sequential Trial

Tristan Fonteneau, Elodie Billion, Cindy Abdoul, Sebastien Le, Alice Hadchouel, David Drummond, Tristan Fonteneau, Elodie Billion, Cindy Abdoul, Sebastien Le, Alice Hadchouel, David Drummond

Abstract

Background: The use of simulation games (SG) to assess the clinical competence of medical students has been poorly studied.

Objective: The objective of this study was to assess whether an SG better reflects the clinical competence of medical students than a multiple choice questionnaire (MCQ).

Methods: Fifth-year medical students in Paris (France) were included and individually evaluated on a case of pediatric asthma exacerbation using three successive modalities: high-fidelity simulation (HFS), considered the gold standard for the evaluation of clinical competence, the SG Effic'Asthme, and an MCQ designed for the study. The primary endpoint was the median kappa coefficient evaluating the correlation of the actions performed by the students between the SG and HFS modalities and the MCQ and HFS modalities. Student satisfaction was also evaluated.

Results: Forty-two students were included. The actions performed by the students were more reproducible between the SG and HFS modalities than between the MCQ and HFS modalities (P=.04). Students reported significantly higher satisfaction with the SG (P<.01) than with the MCQ modality.

Conclusions: The SG Effic'Asthme better reflected the actions performed by medical students during an HFS session than an MCQ on the same asthma exacerbation case. Because SGs allow the assessment of more dimensions of clinical competence than MCQs, they are particularly appropriate for the assessment of medical students on situations involving symptom recognition, prioritization of decisions, and technical skills.

Trial registration: ClinicalTrials.gov NCT03884114; https://ichgcp.net/clinical-trials-registry/NCT03884114.

Keywords: assessment; asthma; pediatrics; professional competence; serious game; simulation game.

Conflict of interest statement

Conflicts of Interest: DD and AH participated to the creation of the simulation game Effic’Asthme.

©Tristan Fonteneau, Elodie Billion, Cindy Abdoul, Sebastien Le, Alice Hadchouel, David Drummond. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 16.12.2020.

Figures

Figure 1
Figure 1
Study design. HFS: high fidelity simulation; SG: simulation game; MCQ: multiple choice questionnaire; *: tutorial video.
Figure 2
Figure 2
Simulation room with the Simbaby manikin and equipment available to the student: (1) paracetamol, (2) emergency treatment (salbutamol inhaler), (3) controller treatment (fluticasone inhaler), (4) water, (5) telephone to call 911 or pediatric emergencies, (6) oral corticosteroids (prednisolone 20 mg tablets), (7) otoscope, (8) thermometer, (9) stethoscope, (10) inhalation chamber with face mask, (11) saline pipettes, and (12) paper tissue.
Figure 3
Figure 3
Effic’Asthme home screen: (A) “asthma action plan” automatically created by the app based on data (age, weight) entered by parent, (B) “asthma attack log” for monitoring, (C) “training” section, which allows the player to access different scenarios of simulated asthma exacerbation (only this section was used in this study), (D) “my performances,” providing scores on the scenarios already done, (E) “documents,” with access to detailed information on the different asthma symptoms and several questions/answers about the child's asthma, and (F) “physician” section, which helps the treating physician to know which parts of the action plan or inhalation technique should be reviewed with the family.
Figure 4
Figure 4
Effic’Asthme menu and inhalation technique page: (left) drop-down menu “emergency treatment kit” with several options (take temperature, clean nasal airway, administer paracetamol, administer emergency treatment, administer controller treatment, administer oral corticosteroids) and (right) administration of the inhaler treatment.
Figure 5
Figure 5
Concordance of actions between the modalities assessed by Cohen kappa coefficient. Each point represents the median degree of agreement (Cohen kappa coefficient) of all students for one of the 19 items on the checklist. The horizontal bar corresponds to the median kappa coefficient for all items. HFS: high-fidelity simulation; SG: simulation game; MCQ: multiple choice questionnaire; *P=.04.
Figure 6
Figure 6
Median checklist scores for the modalities. HFS: high-fidelity simulation; SG: simulation game; MCQ: multiple choice questionnaire; *P=.04.

References

    1. Epstein RM, Hundert EM. Defining and assessing professional competence. JAMA. 2002 Jan 09;287(2):226–235.
    1. Norcini JJ, McKinley DW. Assessment methods in medical education. Teaching Teacher Educ. 2007 Apr;23(3):239–250. doi: 10.1016/j.tate.2006.12.021.
    1. Wass V, Van der Vleuten C, Shatzer J, Jones R. Assessment of clinical competence. Lancet. 2001 Mar 24;357(9260):945–949. doi: 10.1016/S0140-6736(00)04221-5.
    1. Epstein RM. Assessment in medical education. N Engl J Med. 2007 Jan 25;356(4):387–396. doi: 10.1056/NEJMra054784.
    1. Doughty CB, Kessler DO, Zuckerbraun NS, Stone KP, Reid JR, Kennedy CS, Nypaver MM, Auerbach MA. Simulation in pediatric emergency medicine fellowships. Pediatrics. 2015 Jul;136(1):e152–e158. doi: 10.1542/peds.2014-4158.
    1. Alvarez J. Du Jeu Vidéo au Serious Game?: Approches Culturelle, Pragmatique et Formelle [Thesis] Toulouse: University of Toulouse; 2007.
    1. Alvarez J, Djaouti D. Une taxinomie des Serious Games dédiés au secteur de la santé?: L'application des jeux vidéo à la médecine et à la surveillance. Rev électr électron. 2008. [2020-11-28]. .
    1. Sitzmann T. A meta-analytic examination of the instructional effectiveness of computer-based simulation games. Personnel Psychol. 2011;64(2):528. doi: 10.1111/j.1744-6570.2011.01190.x.
    1. Drummond D, Hadchouel A, Tesnière A. Serious games for health: three steps forwards. Adv Simul. 2017 Feb 4;2(1):3. doi: 10.1186/s41077-017-0036-3.
    1. Charsky D. From edutainment to serious games: a change in the use of game characteristics. Games Culture. 2010 Apr;5(2):177–198. doi: 10.1177/1555412009354727.
    1. Pasin F, Giroux H. The impact of a simulation game on operations management education. Comput Educ. 2011 Aug;57(1):1240–1254. doi: 10.1016/j.compedu.2010.12.006.
    1. Salas E, Wildman JL, Piccolo RF. Using Simulation-Based Training to Enhance Management Education. AMLE. 2009 Dec;8(4):559–573. doi: 10.5465/amle.8.4.zqr559.
    1. Graafland M, Schraagen JM, Schijven MP. Systematic review of serious games for medical education and surgical skills training. Br J Surg. 2012 Oct;99(10):1322–1330. doi: 10.1002/bjs.8819.
    1. Wang R, DeMaria S, Goldberg A, Katz D. A systematic review of serious games in training health care professionals. Simul Healthc. 2016 Feb;11(1):41–51. doi: 10.1097/SIH.0000000000000118.
    1. Adjedj J, Ducrocq G, Bouleti C, Reinhart L, Fabbro E, Elbez Y, Fischer Q, Tesniere A, Feldman L, Varenne O. Medical student evaluation with a serious game compared to multiple choice questions assessment. JMIR Serious Games. 2017 May 16;5(2):e11. doi: 10.2196/games.7033.
    1. Drummond D, Fabbro E, Tesnière A, Hadchouel A. Toward virtual simulation for parents of children with asthma. J Allergy Clin Immunol Pract. 2017;5(6):1779–1781. doi: 10.1016/j.jaip.2017.04.017.
    1. Drummond D, Truchot J, Fabbro E, Ceccaldi P, Plaisance P, Tesnière A, Hadchouel A. Fixed versus variable practice for teaching medical students the management of pediatric asthma exacerbations using simulation. Eur J Pediatr. 2018 Feb;177(2):211–219. doi: 10.1007/s00431-017-3054-1.
    1. Bujang M, Baharum N. Guidelines of the minimum sample size requirements for Kappa agreement test. Epidemiol Biostatistics Public Health. 2017;14(2):122671–1226710. doi: 10.2427/12267.
    1. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977 Mar;33(1):159–174.
    1. Bick JS, Demaria S, Kennedy JD, Schwartz AD, Weiner MM, Levine AI, Shi Y, Schildcrout JS, Wagner CE. Comparison of expert and novice performance of a simulated transesophageal echocardiography examination. Simul Healthc. 2013 Oct;8(5):329–334. doi: 10.1097/SIH.0b013e31829068df.
    1. Chon S, Timmermann F, Dratsch T, Schuelper N, Plum P, Berlth F, Datta RR, Schramm C, Haneder S, Späth MR, Dübbers M, Kleinert J, Raupach T, Bruns C, Kleinert R. Serious games in surgical medical education: a virtual emergency department as a tool for teaching clinical reasoning to medical students. JMIR Serious Games. 2019 Mar 05;7(1):e13028. doi: 10.2196/13028.
    1. Drummond D, Tesniere A, Hadchouel A. Design and development of a simulation game for parents of preschool children with asthma: an evidence-based approach. Eur Respiratory Soc. 2018;52:PA5445. doi: 10.1183/13993003.congress-2018.pa5445.
    1. Shinnick MA, Woo M, Horwich TB, Steadman R. Debriefing: the most important component in simulation? Clin Simul Nurs. 2011 May;7(3):e105–e111. doi: 10.1016/j.ecns.2010.11.005.

Source: PubMed

Sponsorer og samarbeidspartnere

Medisinsk tilstand

Legemiddelintervensjoner

3
Abonnere