Intelligent Simulated Patient System for Gastroenterology Education

Development and Evaluation of an Intelligent Simulated Patient System Based on Large Language Models for Gastroenterology History-Taking Education: A Prospective Randomized Controlled Trial

This randomized controlled trial evaluated a large language model-based Intelligent Simulated Patient System (ISPS) for gastroenterology history-taking education. The system integrates retrieval-augmented generation (RAG), electronic health record-based knowledge retrieval, virtual patient interaction, and automated scoring. Ninety medical students were randomly assigned to conventional training or ISPS-assisted training. Primary outcomes included history-taking completeness and diagnostic accuracy. Secondary outcomes included empathy performance and technology acceptance.

Study Overview

• Status: Completed
• Conditions: Medical Education; Clinical Competence; History Taking
• Intervention / Treatment: Other: Intelligent Simulated Patient System (ISPS); Other: Conventional Clinical Teaching

Detailed Description

History-taking is a fundamental clinical skill in gastroenterology because digestive diseases often present with complex and nonspecific symptoms that require systematic information gathering for accurate diagnosis. Traditional history-taking education relies on bedside teaching, standardized patients, and faculty supervision. However, these approaches are limited by high educational costs, restricted access to standardized patients, variability in assessment, and limited opportunities for repeated practice.

This study developed an Intelligent Simulated Patient System (ISPS) powered by a large language model (LLM) and retrieval-augmented generation (RAG) technology for gastroenterology education. The system was built using de-identified electronic health records, clinical guidelines, and textbook-based knowledge to create realistic virtual patients capable of maintaining disease-specific characteristics throughout multi-turn conversations. In addition to virtual patient simulation, the platform includes an automated scoring system that evaluates history-taking completeness using semantic similarity matching and logical completeness assessment based on predefined clinical rubrics.

This prospective randomized controlled trial aims to evaluate whether ISPS-assisted training improves clinical history-taking skills compared with conventional teaching methods. Ninety medical students who had completed courses in diagnostics and internal medicine were randomly assigned in a 1:1 ratio to either an ISPS training group or a conventional teaching group. The intervention lasted four weeks.

Participants in the control group received conventional clinical education, including bedside teaching, case-based learning, and supervised clinical discussions. Participants in the intervention group independently completed repeated history-taking practice using the ISPS platform with virtual patients representing common gastroenterology diseases.

The primary outcomes are history-taking completeness scores and diagnostic accuracy assessed during a standardized Objective Structured Clinical Examination (OSCE). Secondary outcomes include empathy performance during physician-patient communication and participants' acceptance of the ISPS assessed using the Technology Acceptance Model (TAM) questionnaire.

The study also evaluates the reliability of the automated scoring system by comparing AI-generated scores with expert ratings using Pearson correlation, intraclass correlation coefficients (ICC), and Bland-Altman analysis.

The findings are expected to provide evidence regarding the effectiveness of large language model-based virtual patient systems for improving clinical communication skills, standardized assessment, and competency-based medical education in gastroenterology.

• Study Type: Interventional
• Enrollment (Actual): 90
• Phase: Not Applicable

Contacts and Locations

Study Locations

• China
  • Zhejiang
    • Hangzhou, Zhejiang, China
      • The First Affiliated Hospital of Zhejiang Chinese Medical University

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: Adult; Older Adult
• Accepts Healthy Volunteers: Yes

Description

Inclusion Criteria:

• Undergraduate medical students at Zhejiang Chinese Medical University who had completed their clinical clerkship and the required coursework in Diagnostics and Internal Medicine (Gastroenterology).
• Postgraduate students in Clinical Medicine at the First Affiliated Hospital of Zhejiang Chinese Medical University (Zhejiang Provincial Hospital of Chinese Medicine).
• Able and willing to provide written informed consent.
• Willing to complete the entire 4-week study protocol and all outcome assessments.

Exclusion Criteria:

• Individuals with extensive prior clinical experience in gastroenterology that could substantially influence baseline performance.
• Individuals with severe visual, hearing, or other impairments that would prevent effective participation in the Intelligent Simulated Patient System training.
• Individuals unable to complete the required training sessions or the final Objective Structured Clinical Examination (OSCE).
• Individuals who withdrew informed consent or had incomplete outcome assessment.

Study Plan

How is the study designed?

Design Details

• Primary Purpose: Other
• Allocation: Randomized
• Interventional Model: Parallel Assignment
• Masking: Single

Number of Arms

2

Arms and Interventions

Participant Group / Arm	Intervention / Treatment
Experimental: ISPS Group; Participants received four weeks of training using the Intelligent Simulated Patient System (ISPS), a large language model-based virtual patient platform for gastroenterology education. The system provides repeated history-taking practice, automated assessment, and individualized feedback.	Other: Intelligent Simulated Patient System (ISPS); Participants received four weeks of self-directed training using the Intelligent Simulated Patient System (ISPS), a large language model-based educational platform developed for gastroenterology history-taking training. The system integrates retrieval-augmented generation (RAG), de-identified electronic health records, disease-specific knowledge retrieval, virtual patient simulation, automated history-taking assessment, and individualized feedback. Participants independently selected clinical cases, conducted repeated history-taking interviews with virtual patients, received automated evaluations of history-taking completeness, and reviewed personalized feedback throughout the intervention period.
Active Comparator: Control Group; Participants received four weeks of conventional clinical education, including bedside teaching, case-based learning, paper-based medical record review, and faculty-guided clinical discussions according to the standard curriculum.	Other: Conventional Clinical Teaching; Participants received conventional clinical education for four weeks, including bedside teaching, case-based learning, paper-based medical record review, and faculty-guided clinical discussions according to the standard curriculum.

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Diagnostic accuracy	Diagnostic accuracy was defined as the proportion of participants who correctly identified the primary diagnosis during the standardized OSCE.	Immediately after the 4-week intervention
History-taking completeness score	History-taking completeness was evaluated during a standardized Objective Structured Clinical Examination (OSCE) using a validated rubric with a total score ranging from 0 to 100. Higher scores indicate more complete history-taking performance.	Immediately after the 4-week intervention

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Empathy performance	Empathy performance was assessed during the standardized OSCE using a quantitative empathy assessment scale developed from the "Understand the Patient's Perspective" dimension of the SEGUE framework.	Immediately after the 4-week intervention
Technology acceptance	Participants' acceptance of the Intelligent Simulated Patient System was assessed using a Technology Acceptance Model (TAM)-based questionnaire evaluating perceived usefulness, perceived ease of use, perceived enjoyment, behavioral intention, and perceived risks.	Immediately after completion of the intervention

Collaborators and Investigators

• Sponsor: The First Affiliated Hospital of Zhejiang Chinese Medical University

Study record dates

Study Major Dates

• Study Start (Actual): March 1, 2026
• Primary Completion (Actual): June 1, 2026
• Study Completion (Actual): June 1, 2026

Study Registration Dates

• First Submitted: July 15, 2026
• First Submitted That Met QC Criteria: July 15, 2026
• First Posted (Actual): July 20, 2026

Study Record Updates

• Last Update Posted (Actual): July 20, 2026
• Last Update Submitted That Met QC Criteria: July 15, 2026
• Last Verified: July 1, 2026

More Information

Terms related to this study

• Keywords: Artificial Intelligence; Medical Education; Gastroenterology; Simulation Training; Large Language Model; Virtual Patient; Clinical Skills; History Taking; Automated Assessment
• Other Study ID Numbers: ISPS-2026-001

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No