Improving the Reliability of LLMs as Medical Assistants for the General Public (LAMP-1)
1 september 2026 uppdaterad av: Ji Xunming,MD,PhD, Capital Medical University
Improving the Reliability of LLMs as Medical Assistants for the General Public: a Proof of Concept Simulation Trial
This study will evaluate whether three-minute six-dimensions education(3M-6D education) can improve the reliability of large language models as medical assistants for the general public.
Participants will be randomly assigned to receive or not receive 3M-6D education and then use ChatGPT, Gemini, or non-AI information resources.
The study will assess relevant condition identification, disposition concordance, red-flag identification, and NASA-TLX score.
Studieöversikt
Status
Avslutad
Betingelser
Intervention / Behandling
Detaljerad beskrivning
This randomized, controlled, proof-of-concept simulation trial will evaluate whether three-minute six-dimensions education (3M-6D education) can improve the reliability of large language models as medical assistants for the general public.
Eligible participants will be randomly assigned in a 1:1:1:1:1 ratio to one of five study groups: the 3M-6D education GPT group, the GPT group, the 3M-6D education Gemini group, the Gemini group, or the control group. Participants in the 3M-6D education GPT and 3M-6D education Gemini groups will receive approximately three minutes of education before using ChatGPT or Gemini.Each participant will be randomly assigned one of 10 standardized clinical scenarios and complete a simulated counseling task in unrestricted natural language within approximately 10 minutes. The study will assess relevant condition identification, disposition concordance, red-flag identification, and NASA-TLX score.
Studietyp
Interventionell
Inskrivning (Faktisk)
527
Fas
- Inte tillämpbar
Kontakter och platser
Det här avsnittet innehåller kontaktuppgifter för dem som genomför studien och information om var denna studie genomförs.
Studieorter
-
-
Beijing Municipality
-
Beijing, Beijing Municipality, Kina
- Beijing Ctiy
-
-
Deltagandekriterier
Forskare letar efter personer som passar en viss beskrivning, så kallade behörighetskriterier. Några exempel på dessa kriterier är en persons allmänna hälsotillstånd eller tidigare behandlingar.
Urvalskriterier
Åldrar som är berättigade till studier
- Vuxen
- Äldre vuxen
Tar emot friska volontärer
Ja
Beskrivning
Inclusion Criteria:
- Age 18 years or greater, male or female;
- Completed primary school or higher education;
- Able to use a smartphone or computer to complete online interaction;
- No history of acute ischemic stroke, systemic lupus erythematosus, gastric ulcer, pneumonia, acute cardiac infarction, urinary tract infection, uterine fibroids, diabetes, osteoarthritis, or migraine.
- Able to understand and comply with study procedures and to provide written informed consent.
Exclusion Criteria:
- Currently or previously employed as a healthcare worker;
- Previously received systematic medical training;
- Currently involved in concurrent research that may interfere with the results of the present trial;
- The investigator considered that the participant had other conditions that might affect compliance or preclude participation.
Studieplan
Det här avsnittet ger detaljer om studieplanen, inklusive hur studien är utformad och vad studien mäter.
Hur är studien utformad?
Designdetaljer
- Primärt syfte: Hälsovårdsforskning
- Tilldelning: Randomiserad
- Interventionsmodell: Parallellt uppdrag
- Maskning: Enda
Vapen och interventioner
Deltagargrupp / Arm |
Intervention / Behandling |
|---|---|
|
Experimentell: 3M-6D education GPT Group
Participants will first be trained in 3M-6D education, then use ChatGPT to complete a consultation task in unrestricted natural language in approximately 10 minutes.
|
Participants use ChatGPT to complete a standardized simulated clinical scenarios in unrestricted natural language.
3M-6D education is designed based on Cognitive Load Theory to reduce the cognitive burden on patients during medical interactions with AI and to improve the clarity and completeness of symptom reporting. Guided by cognitive load theory and the natural process physicians use to take medical histories, the investigators identified candidate information dimensions and developed a structured expression framework with six dimensions for public health queries through a Delphi expert consensus process. Participants were instructed to use the framework to describe their symptoms across these six dimensions; this process can typically be completed within three minutes, so the investigators call this approach three minutes six dimensions education (3M-6D education). |
|
Experimentell: 3M-6D education Gemini Group
Participants will first be trained in 3M-6D education, then use Gemini to complete a consultation task in unrestricted natural language in approximately 10 minutes.
|
Participants use Gemini to complete a standardized simulated clinical scenarios in unrestricted natural language.
3M-6D education is designed based on Cognitive Load Theory to reduce the cognitive burden on patients during medical interactions with AI and to improve the clarity and completeness of symptom reporting. Guided by cognitive load theory and the natural process physicians use to take medical histories, the investigators identified candidate information dimensions and developed a structured expression framework with six dimensions for public health queries through a Delphi expert consensus process. Participants were instructed to use the framework to describe their symptoms across these six dimensions; this process can typically be completed within three minutes, so the investigators call this approach three minutes six dimensions education (3M-6D education). |
|
Aktiv komparator: GPT Group
Participants will use ChatGPT to complete a consultation task in unrestricted natural language in approximately 10 minutes.
|
Participants use ChatGPT to complete a standardized simulated clinical scenarios in unrestricted natural language.
|
|
Aktiv komparator: Gemini Group
Participants will use Gemini to complete a consultation task in unrestricted natural language in approximately 10 minutes.
|
Participants use Gemini to complete a standardized simulated clinical scenarios in unrestricted natural language.
|
|
Inget ingripande: Control group
Participants will use non-AI tools such as internet searches and medical websites to complete a consultation task in unrestricted natural language in approximately 10 minutes.
|
Vad mäter studien?
Primära resultatmått
Resultatmått |
Åtgärdsbeskrivning |
Tidsram |
|---|---|---|
|
Relevant conditions identification of the 3M-6D education GPT group compared with the GPT group
Tidsram: 1 hour.
|
Relevant conditions identification is defined as the proportion of participants whose final response includes the expert-defined final diagnosis or a relevant differential diagnosis.
|
1 hour.
|
|
Disposition concordance of the 3M-6D education GPT group compared with the GPT group
Tidsram: 1 hour.
|
Disposition concordance is defined as the proportion of participants whose final care recommendation matches the expert-defined level.
The five levels are self-care, routine outpatient care, urgent outpatient care, emergency department visit, and emergency medical services.
|
1 hour.
|
|
Relevant conditions identification of the 3M-6D education Gemini group compared with the Gemini group
Tidsram: 1 hour.
|
1 hour.
|
|
|
Disposition concordance of the 3M-6D education Gemini group compared with the Gemini group
Tidsram: 1 hour.
|
1 hour.
|
Sekundära resultatmått
Resultatmått |
Åtgärdsbeskrivning |
Tidsram |
|---|---|---|
|
Relevant conditions identification of the 3M-6D education GPT group compared with the control group
Tidsram: 1 hour.
|
1 hour.
|
|
|
Relevant conditions identification of the 3M-6D education Gemini group compared with the control group
Tidsram: 1 hour.
|
1 hour.
|
|
|
Disposition concordance of the 3M-6D education GPT group compared with the control group
Tidsram: 1 hour.
|
1 hour.
|
|
|
Disposition concordance of the 3M-6D education Gemini group compared with the control group
Tidsram: 1 hour.
|
1 hour.
|
|
|
Red-flag identification in the 3M-6D education GPT group compared with the GPT group
Tidsram: 1 hour.
|
Red-flag identification is defined as the proportion of participants whose final response includes the key warning signs that experts defined for the assigned scenario.
|
1 hour.
|
|
Red-flag identification in the 3M-6D education GPT group compared with the control group
Tidsram: 1 hour.
|
1 hour.
|
|
|
Red-flag identification in the 3M-6D education Gemini group compared with the Gemini group
Tidsram: 1 hour.
|
1 hour.
|
|
|
Red-flag identification in the 3M-6D education Gemini group compared with the control group
Tidsram: 1 hour.
|
1 hour.
|
|
|
NASA Task Load Index score of the 3M-6D education GPT group compared with the GPT group
Tidsram: 1 hour.
|
NASA-TLX score is a self-reported task-load score measured after the simulated consultation with a physician.
It includes six domains: mental demand, physical demand, temporal demand, effort, frustration, and performance.
Each domain is scored from 0 to 100.
The total score is the mean of the six domains.
Higher scores indicate greater perceived task load.
|
1 hour.
|
|
NASA Task Load Index score of the 3M-6D education GPT group compared with the control group
Tidsram: 1 hour.
|
NASA-TLX score is a self-reported task-load score measured after the simulated consultation with a physician.
It includes six domains: mental demand, physical demand, temporal demand, effort, frustration, and performance.
Each domain is scored from 0 to 100.
The total score is the mean of the six domains.
Higher scores indicate greater perceived task load.
|
1 hour.
|
|
NASA Task Load Index score of the 3M-6D education Gemini group compared with the Gemini group
Tidsram: 1 hour.
|
NASA-TLX score is a self-reported task-load score measured after the simulated consultation with a physician.
It includes six domains: mental demand, physical demand, temporal demand, effort, frustration, and performance.
Each domain is scored from 0 to 100.
The total score is the mean of the six domains.
Higher scores indicate greater perceived task load.
|
1 hour.
|
|
NASA Task Load Index score of the 3M-6D education Gemini group compared with the control group
Tidsram: 1 hour.
|
NASA-TLX score is a self-reported task-load score measured after the simulated consultation with a physician.
It includes six domains: mental demand, physical demand, temporal demand, effort, frustration, and performance.
Each domain is scored from 0 to 100.
The total score is the mean of the six domains.
Higher scores indicate greater perceived task load.
|
1 hour.
|
|
Relevant conditions identification of the 3M-6D education GPT group compared with the 3M-6D education Gemini group
Tidsram: 1 hour.
|
1 hour.
|
|
|
Disposition concordance of the 3M-6D education GPT group compared with the 3M-6D education Gemini group
Tidsram: 1 hour.
|
1 hour.
|
|
|
Red-flag identification in the 3M-6D education GPT group compared with the 3M-6D education Gemini group
Tidsram: 1 hour.
|
1 hour.
|
|
|
NASA Task Load Index score of the 3M-6D education GPT group compared with the 3M-6D education Gemini group
Tidsram: 1 hour.
|
NASA-TLX score is a self-reported task-load score measured after the simulated consultation with a physician.
It includes six domains: mental demand, physical demand, temporal demand, effort, frustration, and performance.
Each domain is scored from 0 to 100.
The total score is the mean of the six domains.
Higher scores indicate greater perceived task load.
|
1 hour.
|
Andra resultatmått
Resultatmått |
Åtgärdsbeskrivning |
Tidsram |
|---|---|---|
|
Failure to identify red flags in the 3M-6D education GPT group compared with the GPT group
Tidsram: 1 hour.
|
Failure to identify red flags is defined as the proportion of participants whose final response does not include the expert-defined red-flag symptoms or warning signs for the assigned standardized simulated clinical scenario.
|
1 hour.
|
|
Failure to identify red flags in the 3M-6D education GPT group compared with the control group
Tidsram: 1 hour.
|
1 hour.
|
|
|
Failure to identify red flags in the 3M-6D education Gemini group compared with the Gemini group
Tidsram: 1 hour.
|
1 hour.
|
|
|
Failure to identify red flags in the 3M-6D education Gemini group compared with the control group
Tidsram: 1 hour.
|
1 hour.
|
|
|
Underestimation of disposition in the 3M-6D education GPT group compared with the GPT group
Tidsram: 1 hour.
|
Underestimation of disposition is defined as the proportion of participants whose final care recommendation is lower than the expert-defined disposition level for the assigned standardized simulated clinical scenario.
|
1 hour.
|
|
Underestimation of disposition in the 3M-6D education GPT group compared with the control group
Tidsram: 1 hour.
|
1 hour.
|
|
|
Underestimation of disposition in the 3M-6D education Gemini group compared with the Gemini group
Tidsram: 1 hour.
|
1 hour.
|
|
|
Underestimation of disposition in the 3M-6D education Gemini group compared with the control group
Tidsram: 1 hour.
|
1 hour.
|
Samarbetspartners och utredare
Det är här du hittar personer och organisationer som är involverade i denna studie.
Sponsor
Samarbetspartners
Studieavstämningsdatum
Dessa datum spårar framstegen för inlämningar av studieposter och sammanfattande resultat till ClinicalTrials.gov. Studieposter och rapporterade resultat granskas av National Library of Medicine (NLM) för att säkerställa att de uppfyller specifika kvalitetskontrollstandarder innan de publiceras på den offentliga webbplatsen.
Studera stora datum
Studiestart (Faktisk)
3 juli 2026
Primärt slutförande (Faktisk)
26 augusti 2026
Avslutad studie (Faktisk)
26 augusti 2026
Studieregistreringsdatum
Först inskickad
11 juni 2026
Först inskickad som uppfyllde QC-kriterierna
11 juni 2026
Första postat (Faktisk)
16 juni 2026
Uppdateringar av studier
Senaste uppdatering publicerad (Faktisk)
3 september 2026
Senaste inskickade uppdateringen som uppfyllde QC-kriterierna
1 september 2026
Senast verifierad
1 september 2026
Mer information
Termer relaterade till denna studie
Nyckelord
Andra studie-ID-nummer
- LAMP-1
Plan för individuella deltagardata (IPD)
Planerar du att dela individuella deltagardata (IPD)?
OBESLUTSAM
Läkemedels- och apparatinformation, studiedokument
Studerar en amerikansk FDA-reglerad läkemedelsprodukt
Nej
Studerar en amerikansk FDA-reglerad produktprodukt
Nej
Denna information hämtades direkt från webbplatsen clinicaltrials.gov utan några ändringar. Om du har några önskemål om att ändra, ta bort eller uppdatera dina studieuppgifter, vänligen kontakta register@clinicaltrials.gov. Så snart en ändring har implementerats på clinicaltrials.gov, kommer denna att uppdateras automatiskt även på vår webbplats .