Script concordance tests: guidelines for construction

Jean Paul Fournier, Anne Demeester, Bernard Charlin, Jean Paul Fournier, Anne Demeester, Bernard Charlin

Abstract

Background: SCT is used to assess clinical reasoning in ambiguous or uncertain situations. It allows testing on real-life situations that are not adequately measured with current tests. It probes the multiple judgments that are made in the clinical reasoning process. Scoring reflects the degree of concordance of these judgments to those of a panel of reference experts.

Method: SCT is an item format that is gaining acceptance in education in the health professions. However, there are no detailed guidelines on item writing, test scoring or test optimization.

Results: The item format is described and the steps for preparing and administering reliable and valid SCTs are presented.

Conclusion: SCTs probe examinees on a specific clinical reasoning task: data interpretation, a crucial step in the clinical reasoning process. It is inferred that a high degree of concordance corresponds to optimal use of information in the context of these specific tasks and therefore provides an indication of clinical reasoning quality.

Figures

Figure 1
Figure 1
Example of case from the investigation section of an SCT.
Figure 2
Figure 2
Example of test item administered on line.

References

    1. Feltovich PJ, Barrows HS. Issues of generality in medical problem solving. In: Schmidt HG, De Volder ML, editor. Tutorials in Problem-based Learning: A New Direction in Teaching the Health Professions. Assen: Van Gorcum; 1984. pp. 128–142.
    1. Schmidt HG, Norman GR, Bozhuizen HA. A cognitive perspective on medical expertise: Theory and implications. Academic Medicine. 1990;65:611–621. doi: 10.1097/00001888-199010000-00001.
    1. Charlin B, Boshuizen HPA, Custers EJFM, Feltovich Paul J. Scripts and clinical reasoning. Medical Education. 2007;41:1179–1185.
    1. Charlin B, Vleuten C Van Der. Standardized assessment of reasoning in context of uncertainty. The Script Concordance Test approach. Evaluation and the Health Professions. 2004;27:304–319. doi: 10.1177/0163278704267043.
    1. Hall KH. Reviewing intuitive decision-making and uncertainty: the implications for medical education. Medical Education. 2002;36:216–224. doi: 10.1046/j.1365-2923.2002.01140.x.
    1. Gerrity MS, Earl JAL, deVellis RF. Uncertainty and professional work: Perceptions of physicians in clinical practice. Motivation and Emotion. 1992;97:1022–1051.
    1. Gagnon R, Charlin B, Coletti M, Sauvé E, Vleuten C Van der. Assessment in the context of uncertainty: How many members are needed on the panel of reference of a script concordance test? Medical Education. 2005;39:284–291. doi: 10.1111/j.1365-2929.2005.02092.x.
    1. Charlin B, Gagnon R, Pelletier J, Coletti M, Abi-Rizk G, Nasr C, Sauvé E, Vleuten C Van der. Assessment of clinical reasoning in the context of uncertainty: the effect of variability within the reference panel. Medical Education. 2006;18:22–27.
    1. Demeester A. Mémoire de maîtrise. Vol. 13. Bobigny – Université de Paris; 2004. Évaluation des processus de Raisonnement clinique des étudiants sages-femmes en fin de cursus.
    1. Meterissian S, Zabolotny B, Gagnon R, Charlin B. Is the script concordance test a valid instrument for assessment of intraoperative decision-making skills? American Journal of Surgery. 2007;193:248–51. doi: 10.1016/j.amjsurg.2006.10.012.
    1. Gagnon R, Charlin B, Roy L, Saint Martin M, Sauvé E, Boshuizen HPA, Vleuten C Van der. The cognitive validity of the script concordance test: a processing time study. Teaching and Learning in Medicine. 2006;18:22–27. doi: 10.1207/s15328015tlm1801_6.
    1. Amin Z, Chong YC, Khoo HE. Practical guide to medical student assessment. New Jersey: World Scientific Publishing Co; 2006.
    1. Crocker L, Algina J. Introduction to classical and modern test theory. Belmont: Wadsworth-Thomson learning; 1986. Process of test construction; pp. 66–86.
    1. Fournier JP, Thiercelin D, Pulcini C, Alunni-Perret V, Gilbert E, Minguet JM, Bertrand F. Évaluation du raisonnement clinique en médecine d'urgence: les tests de concordance des scripts décèlent mieux l'expérience clinique que les questions à choix multiples à contexte riche. Pédagogie Médicale. 2006;7:20–30.
    1. Page G, Bordage G. The Medical Council of Canada's key-feature project: a valid written examination of clinical decisions skills. Academic Medicine. 1995;70:104–110.
    1. Charlin B, Roy L, Braïlowsky C, Goulet F, Vleuten C Van der. The Script Concordance test: a tool to assess the reflective clinician. Teaching and Learning in Medicine. 2000;12:189–195. doi: 10.1207/S15328015TLM1204_5.
    1. Norman GR, Tugwell P, Feightner JW, Muzzin LJ. Knowledge and clinical problem-solving ability. Medical Education. 1985;19:344–56.
    1. Norman G, Bordage G, Pages G, Keane D. How specific is case specificity? Medical Education. 2006;40:618–623. doi: 10.1111/j.1365-2929.2006.02511.x.
    1. Gagnon R, Charlin B, Lambert C, et al. More cases or more questions? Advances in Health Sciences Education. 2007. accepted with revisions.
    1. Roy PM, Meyer G, Vielle B, et al. Appropriateness of diagnosis management and outcomes of suspected pulmonary embolism. Ann Intern Med. 2006;144:157–164.
    1. Petrella RJ, Davis P. Improving management of musculoskeletal disorders in primary care: the Joint Adventures Program. Clinical Rheumatology. 2007;26:1061–1066. doi: 10.1007/s10067-006-0446-4.
    1. Bland AC, Kreiter CD, Gordon JA. The psychometric properties of five scoring methods applied to the script concordance test. Academic Medicine. 2005;80:395–9. doi: 10.1097/00001888-200504000-00019.
    1. Gagnon R, Charlin B, Coletti M, Sauvé E, Vleuten C Van der. Assessment in the context of uncertainty: how many members are needed on the panel of reference of a script concordance test? Medical Education. 2005;39:284–291. doi: 10.1111/j.1365-2929.2005.02092.x.
    1. Caire F, Sol JC, Moreau JJ, Isidori P, Charlin B. Auto évaluation des internes en Neurochirurgie par test de concordance des scripts (TCS): processus d'évaluation des tests. Neurochirurgie. 2004;50:66–72. doi: 10.1016/S0028-3770(04)98309-9.
    1. Groves M, O'Rourke P, Alexander H. The relative contribution of identification, interpretation and hypothesis errors to misdiagnosis. Medical Teacher. 2003;25:621–5. doi: 10.1080/01421590310001605688.

Source: PubMed

スポンサーと協力者

医学的状態

薬物療法

3
購読する