AI-Assisted Learning in Medicine

Impact of Supplementary Access to a Fine-Tuned Medical Large Language Model (MetaGP-Edu) on Learning Outcomes in Undergraduate Medicine Education

This multi-center retrospective cohort study investigates the real-world impact of integrating MetaGP-Edu, a proprietary AI tool fine-tuned for medical education, into the undergraduate Internal Medicine curriculum. Utilizing historical academic records from several major medical institutions in China across multiple academic years, the study compares the performance of student cohorts who learned via traditional methods only with subsequent cohorts who had supplementary access to MetaGP-Edu. The primary outcome measure is overall academic performance in the Internal Medicine course, assessed through final course scores. The analysis aims to determine if access to the AI tool as a supplementary resource is associated with differences in learning outcomes, while statistically accounting for baseline student characteristics and other potential confounders between the compared cohorts.

Study Overview

• Status: Completed
• Conditions: Medical Student

Detailed Description

ackground and Rationale: The effective teaching of Internal Medicine, a cornerstone of undergraduate medical education, presents significant pedagogical challenges due to the breadth and complexity of the subject matter and the large student cohorts typically enrolled in major academic medical centers. While traditional methods like lectures and textbook readings are essential, there is a recognized need for innovative approaches that can better support the development of clinical reasoning, facilitate deeper engagement with complex case material, and offer more personalized learning opportunities at scale. Artificial intelligence (AI), particularly advanced large language models (LLMs) trained on domain-specific knowledge, holds considerable potential as an educational technology to address these needs. MetaGP-Edu, a proprietary generative foundation model fine-tuned specifically for medical education using pedagogical datasets, was developed to explore this potential by serving as a supplementary learning resource. Evaluating the real-world impact of integrating such tools into established curricula is crucial for evidence-informed educational practice.

Objectives: The primary objective of this study is to retrospectively evaluate the association between the availability of the MetaGP-Edu AI tool as a supplementary learning resource and overall student academic performance in the core Internal Medicine curriculum.

Study Design: This investigation employs a multi-center, retrospective cohort study design. Routinely collected academic data from several major medical schools in China over multiple consecutive academic years will be analyzed. This approach allows for the comparison of student cohorts based on their historical exposure to different educational resource environments (with vs. without MetaGP-Edu access) within real-world academic settings.

Setting and Participants: The study encompasses data from undergraduate medical students enrolled in the mandatory Internal Medicine course at several large, academically affiliated medical institutions in China. Participants include students who completed the course across a span of academic years covering the period before and after the introduction of MetaGP-Edu (approximately Fall 2022). Inclusion is based on the availability of complete academic records for the Internal Medicine course during the specified study period.

Exposure/Intervention and Comparator: The study compares two main cohorts defined by the timing of MetaGP-Edu availability:

Comparator Cohort (Pre-MetaGP-Edu): Students who completed the Internal Medicine course in the academic years prior to the introduction of MetaGP-Edu. These students utilized the institutions' established traditional teaching methods (e.g., lectures, tutorials, standard case discussions, textbook assignments) without access to the MetaGP-Edu tool.

Exposure Cohort (Post-MetaGP-Edu): Students who completed the Internal Medicine course in the academic years following the introduction of MetaGP-Edu. These students participated in the same core traditional curriculum components but also had access to MetaGP-Edu via institutional platforms as a supplementary, optional resource for self-directed learning, case exploration, and clinical query support.

Outcome Measures: The primary outcome measure is overall academic performance in the Internal Medicine course, typically represented by the final numeric course score (e.g., on a 0-100 scale) as documented in official university records. Key covariates, including measures of baseline academic achievement (e.g., prior cumulative GPA or equivalent), demographic factors (e.g., sex), and institutional site, will be extracted from administrative data to allow for statistical adjustment.

• Study Type: Observational
• Enrollment (Actual): 1632

Contacts and Locations

Study Locations

• China
  • Beijing, China
    • Peking Union Medical College
  • Beijing, China
    • Peking University Health Science Center

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: Child; Adult; Older Adult
• Accepts Healthy Volunteers: Yes

• Sampling Method: Non-Probability Sample

Study Population

The study population comprises undergraduate medical students who completed the mandatory Internal Medicine curriculum at several major academic medical centers in China over multiple specified academic years. Inclusion in this retrospective analysis was determined by the availability of complete, official academic records for the relevant course during the defined time periods covering both before and after the implementation of the MetaGP-Edu tool.

Description

Inclusion Criteria:

Inclusion criteria required students to have completed the entire Internal Medicine course and possess a recorded final numeric score for the course within the selected timeframe

Exclusion Criteria:

Students with incomplete academic records for the course, those who transferred between institutions mid-course, or individuals identified as having repeated the course were excluded

Study Plan

How is the study designed?

Number of groups / cohorts

2

Cohorts and Interventions

Group / Cohort
Pre-MetaGP-Edu Cohorts; This group includes all students who completed the Internal Medicine course at the six participating institutions during the academic years prior to 2022 (2020-2021 and 2021-2022). These students learned exclusively through the traditional curriculum methods described above, without access to the MetaGP tool.
Post-MetaGP-Edu Cohorts; This group includes all students who completed the Internal Medicine course starting from the 2022 academic year onwards (2022-2023 and 2023-2024). These students experienced the same core traditional curriculum components but also had access to MetaGP- as a supplementary learning resource throughout their course duration.

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
academic performance	The primary outcome measure was student academic performance in the Internal Medicine course, operationalized as the final numeric course score (scaled 0-100) obtained from official university transcripts	1 year

Collaborators and Investigators

• Sponsor: Kang Zhang

Study record dates

Study Major Dates

• Study Start (Actual): June 1, 2020
• Primary Completion (Actual): December 1, 2024
• Study Completion (Actual): December 1, 2024

Study Registration Dates

• First Submitted: April 18, 2025
• First Submitted That Met QC Criteria: April 18, 2025
• First Posted (Actual): April 25, 2025

Study Record Updates

• Last Update Posted (Actual): April 25, 2025
• Last Update Submitted That Met QC Criteria: April 18, 2025
• Last Verified: April 1, 2025

More Information

Terms related to this study

• Keywords: artificial intelligence; Education; Large Language Model; MetaGP-Edu
• Other Study ID Numbers: AI and medical education

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: NO

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No