Teaching Medical Students to Verify AI-Generated Citation Hallucinations (SARA)

Participants assigned to this arm will receive a standardized 75-minute educational intervention using the SARA citation-verification framework: Source authenticity, Accuracy, Relevance, and Alignment. Training includes facilitator modelling and think-aloud demonstration, coached practice with feedback, use of a one-page verification scaffold, articulation and reflection, and progressive fading of prompts before independent practice. The intervention is designed to teach students to distinguish source-level citation errors from deeper evidence-claim errors and to apply structured verification independently.

A standardized 75-minute educational intervention using the SARA framework: Source authenticity, Accuracy, Relevance, and Alignment. The session includes facilitator modelling and think-aloud demonstration, coached practice with feedback, a one-page verification scaffold, learner articulation and reflection, and progressive fading of prompts before independent practice. The intervention is designed to teach medical students to identify both source-level citation errors and deeper evidence-claim errors and to apply verification independently.

Participants assigned to this arm will receive an attention-matched 75-minute educational session on responsible use of generative artificial intelligence in academic work. The session covers privacy, disclosure, authorship, academic integrity, and general awareness that GenAI may produce errors. It does not teach the SARA framework or another structured citation-verification process. The session is matched to the intervention for duration, group size, facilitator interaction, and delivery format.

A standardized 75-minute attention-matched educational session on responsible use of generative artificial intelligence in academic work. The session covers privacy, disclosure, authorship, academic integrity, and general awareness that GenAI may produce errors. It does not teach the SARA framework or another structured citation-verification process. The session is matched to the SARA intervention for duration, group size, facilitator interaction, and delivery format.

Participants will review a novel approximately 650-word AI-generated literature-review passage containing eight embedded citation-claim relationships. The passage includes both valid and erroneous citations spanning the study's five-level citation-error taxonomy. Participants will not be prompted to use the SARA framework and will not be told how many or what types of errors are present. Each citation-claim relationship will be scored against a prespecified gold standard for the appropriateness of the participant's final disposition. One point will be awarded for each appropriate disposition, producing a total score from 0 to 8, with higher scores indicating better independent citation-verification performance. Responses will be scored by outcome assessors blinded to intervention allocation.

Citation-verification performance will be measured using parallel 24-item structured assessment forms. Each item requires an appropriate final disposition of a citation-claim pair against a prespecified gold standard. One point will be awarded for each correct disposition, giving a total score from 0 to 24. Higher scores indicate better structured citation-verification performance.

Accuracy will be assessed separately for valid/aligned citations and for Levels 1-5 of the prespecified citation-error taxonomy: non-existent citations, altered bibliographic details, irrelevant sources, evidence-claim misalignment, and contextual omission or overgeneralization. The outcome will be expressed as the proportion of citation-claim decisions with an appropriate final disposition within each condition.

Verification initiation will be measured as the proportion of citation-claim decisions for which the participant initiates an external verification action, such as consulting a bibliographic database, search engine, publisher source, abstract, or full text. Higher proportions indicate more frequent initiation of verification.

High-quality verification will be defined as examining the source element necessary to resolve the specific citation problem. Merely locating an article will not be considered adequate for evidence-claim errors if the participant does not assess whether the evidence supports the claim. The outcome will be expressed as the proportion of citation-claim decisions meeting the prespecified high-quality verification criterion.

Participants will encounter valid citation-claim relationships within the transfer task. The number of valid citations that are unnecessarily altered, rejected, replaced, or deleted will be recorded. The score ranges from 0 to 2, with lower values indicating more appropriate retention of valid evidence.

Participants will rate confidence in each final citation-verification judgment from 0% to 100%. Calibration error will be calculated by comparing confidence with correctness of the corresponding judgment. Lower absolute calibration error indicates closer correspondence between confidence and actual performance.

Retention will be assessed using the 24-item structured verification score at 12 weeks and compared with baseline and immediate post-intervention performance. Scores range from 0 to 24, with higher scores indicating better retained structured citation-verification performance.

A purposively selected subgroup of participants will complete think-aloud verification of novel citation-claim pairs followed by stimulated-recall interviews. Data will explore what triggers verification, how participants decide what evidence is sufficient, why verification stops, how relevance is distinguished from evidentiary support, how confidence changes, and whether SARA has become an internalized strategy or remains an explicit checklist. Interview transcripts will be analyzed using reflexive thematic analysis.