Improving the Reliability of LLMs as Medical Assistants for the General Public (LAMP-1)
11 de junho de 2026 atualizado por: Ji Xunming,MD,PhD, Capital Medical University
Improving the Reliability of LLMs as Medical Assistants for the General Public: a Proof of Concept Simulation Trial
This study will evaluate whether three-minute six-dimensions education(3M-6D education) can improve the reliability of large language models as medical assistants for the general public.
Participants will be randomly assigned to receive or not receive 3M-6D education and then use ChatGPT, Gemini, or non-AI information resources.
The study will assess relevant condition identification, disposition concordance, red-flag identification, and NASA-TLX score.
Visão geral do estudo
Status
Ainda não está recrutando
Condições
Intervenção / Tratamento
Descrição detalhada
This randomized, controlled, proof-of-concept simulation trial will evaluate whether three-minute six-dimensions education (3M-6D education) can improve the reliability of large language models as medical assistants for the general public.
Eligible participants will be randomly assigned in a 1:1:1:1:1 ratio to one of five study groups: the 3M-6D education GPT group, the GPT group, the 3M-6D education Gemini group, the Gemini group, or the control group. Participants in the 3M-6D education GPT and 3M-6D education Gemini groups will receive approximately three minutes of education before using ChatGPT or Gemini.Each participant will be randomly assigned one of 10 standardized clinical scenarios and complete a simulated counseling task in unrestricted natural language within approximately 10 minutes. The study will assess relevant condition identification, disposition concordance, red-flag identification, and NASA-TLX score.
Tipo de estudo
Intervencional
Inscrição (Estimado)
525
Estágio
- Não aplicável
Contactos e Locais
Esta seção fornece os detalhes de contato para aqueles que conduzem o estudo e informações sobre onde este estudo está sendo realizado.
Contato de estudo
- Nome: Xunming Ji
- Número de telefone: 01083198962
- E-mail: jixm@ccmu.edu.cn
Estude backup de contato
- Nome: Chuanjie Wu
- Número de telefone: 01083199439
- E-mail: wuchuanjie@ccmu.edu.cn
Locais de estudo
-
-
Beijing Municipality
-
Beijing, Beijing Municipality, China, 100053
- Xuanwu Hospital, Capital Medical University
-
Contato:
- Chuanjie Wu
- Número de telefone: 010-83199439
- E-mail: wuchuanjie@ccmu.edu.cn
-
-
Critérios de participação
Os pesquisadores procuram pessoas que se encaixem em uma determinada descrição, chamada de critérios de elegibilidade. Alguns exemplos desses critérios são a condição geral de saúde de uma pessoa ou tratamentos anteriores.
Critérios de elegibilidade
Idades elegíveis para estudo
- Adulto
- Adulto mais velho
Aceita Voluntários Saudáveis
Sim
Descrição
Inclusion Criteria:
- Age 18 years or greater, male or female;
- Completed primary school or higher education;
- Able to use a smartphone or computer to complete online interaction;
- No history of acute ischemic stroke, systemic lupus erythematosus, gastric ulcer, pneumonia, acute cardiac infarction, urinary tract infection, uterine fibroids, diabetes, osteoarthritis, or migraine.
- Able to understand and comply with study procedures and to provide written informed consent.
Exclusion Criteria:
- Currently or previously employed as a healthcare worker;
- Previously received systematic medical training;
- Currently involved in concurrent research that may interfere with the results of the present trial;
- The investigator considered that the participant had other conditions that might affect compliance or preclude participation.
Plano de estudo
Esta seção fornece detalhes do plano de estudo, incluindo como o estudo é projetado e o que o estudo está medindo.
Como o estudo é projetado?
Detalhes do projeto
- Finalidade Principal: Pesquisa de serviços de saúde
- Alocação: Randomizado
- Modelo Intervencional: Atribuição Paralela
- Mascaramento: Solteiro
Armas e Intervenções
Grupo de Participantes / Braço |
Intervenção / Tratamento |
|---|---|
|
Experimental: 3M-6D education GPT Group
Participants will first be trained in 3M-6D education, then use ChatGPT to complete a consultation task in unrestricted natural language in approximately 10 minutes.
|
3M-6D education is designed based on Cognitive Load Theory to reduce the cognitive burden on patients during medical interactions with AI and to improve the clarity and completeness of symptom reporting. Guided by cognitive load theory and the natural process physicians use to take medical histories, we identified candidate information dimensions and developed a structured expression framework with six dimensions for public health queries through a Delphi expert consensus process. Participants were instructed to use the framework to describe their symptoms across these six dimensions; this process can typically be completed within three minutes, so we call this approach three minutes six dimensions education (3M-6D education).
Participants use ChatGPT to complete a standardized simulated clinical scenarios in unrestricted natural language.
|
|
Experimental: 3M-6D education Gemini Group
Participants will first be trained in 3M-6D education, then use Gemini to complete a consultation task in unrestricted natural language in approximately 10 minutes.
|
3M-6D education is designed based on Cognitive Load Theory to reduce the cognitive burden on patients during medical interactions with AI and to improve the clarity and completeness of symptom reporting. Guided by cognitive load theory and the natural process physicians use to take medical histories, we identified candidate information dimensions and developed a structured expression framework with six dimensions for public health queries through a Delphi expert consensus process. Participants were instructed to use the framework to describe their symptoms across these six dimensions; this process can typically be completed within three minutes, so we call this approach three minutes six dimensions education (3M-6D education).
Participants use Gemini to complete a standardized simulated clinical scenarios in unrestricted natural language.
|
|
Comparador Ativo: GPT Group
Participants will use ChatGPT to complete a consultation task in unrestricted natural language in approximately 10 minutes.
|
Participants use ChatGPT to complete a standardized simulated clinical scenarios in unrestricted natural language.
|
|
Comparador Ativo: Gemini Group
Participants will use Gemini to complete a consultation task in unrestricted natural language in approximately 10 minutes.
|
Participants use Gemini to complete a standardized simulated clinical scenarios in unrestricted natural language.
|
|
Sem intervenção: Control group
Participants will use non-AI tools such as internet searches and medical websites to complete a consultation task in unrestricted natural language in approximately 10 minutes.
|
O que o estudo está medindo?
Medidas de resultados primários
Medida de resultado |
Descrição da medida |
Prazo |
|---|---|---|
|
Relevant conditions identification of the 3M-6D education GPT group compared with the GPT group
Prazo: Usually within 1 hour.
|
Relevant conditions identification is defined as the proportion of participants whose final response includes the expert-defined final diagnosis or a relevant differential diagnosis.
|
Usually within 1 hour.
|
|
Disposition concordance of the 3M-6D education GPT group compared with the GPT group
Prazo: Usually within 1 hour.
|
Disposition concordance is defined as the proportion of participants whose final care recommendation matches the expert-defined level.
The five levels are self-care, routine outpatient care, urgent outpatient care, emergency department visit, and emergency medical services.
|
Usually within 1 hour.
|
|
Relevant conditions identification of the 3M-6D education Gemini group compared with the Gemini group
Prazo: Usually within 1 hour.
|
Usually within 1 hour.
|
|
|
Disposition concordance of the 3M-6D education Gemini group compared with the Gemini group
Prazo: Usually within 1 hour.
|
Usually within 1 hour.
|
Medidas de resultados secundários
Medida de resultado |
Descrição da medida |
Prazo |
|---|---|---|
|
Relevant conditions identification of the 3M-6D education GPT group compared with the control group
Prazo: Usually within 1 hour.
|
Usually within 1 hour.
|
|
|
Relevant conditions identification of the 3M-6D education Gemini group compared with the control group
Prazo: Usually within 1 hour.
|
Usually within 1 hour.
|
|
|
Disposition concordance of the 3M-6D education GPT group compared with the control group
Prazo: Usually within 1 hour.
|
Usually within 1 hour.
|
|
|
Disposition concordance of the 3M-6D education Gemini group compared with the control group
Prazo: Usually within 1 hour.
|
Usually within 1 hour.
|
|
|
Red-flag identification in the 3M-6D education GPT group compared with the GPT group
Prazo: Usually within 1 hour.
|
Red-flag identification is defined as the proportion of participants whose final response includes the key warning signs that experts defined for the assigned scenario.
|
Usually within 1 hour.
|
|
Red-flag identification in the 3M-6D education GPT group compared with the control group
Prazo: Usually within 1 hour.
|
Usually within 1 hour.
|
|
|
Red-flag identification in the 3M-6D education Gemini group compared with the Gemini group
Prazo: Usually within 1 hour.
|
Usually within 1 hour.
|
|
|
Red-flag identification in the 3M-6D education Gemini group compared with the control group
Prazo: Usually within 1 hour.
|
Usually within 1 hour.
|
|
|
NASA Task Load Index score of the 3M-6D education GPT group compared with the GPT group
Prazo: Usually within 1 hour.
|
NASA-TLX score is a self-reported task-load score measured after the simulated consultation with a physician.
It includes six domains: mental demand, physical demand, temporal demand, effort, frustration, and performance.
Each domain is scored from 0 to 100.
The total score is the mean of the six domains.
Higher scores indicate greater perceived task load.
|
Usually within 1 hour.
|
|
NASA Task Load Index score of the 3M-6D education GPT group compared with the control group
Prazo: Usually within 1 hour.
|
Usually within 1 hour.
|
|
|
NASA Task Load Index score of the 3M-6D education Gemini group compared with the Gemini group
Prazo: Usually within 1 hour.
|
Usually within 1 hour.
|
|
|
NASA Task Load Index score of the 3M-6D education Gemini group compared with the control group
Prazo: Usually within 1 hour.
|
Usually within 1 hour.
|
|
|
Relevant conditions identification of the 3M-6D education GPT group compared with the 3M-6D education Gemini group
Prazo: Usually within 1 hour.
|
Usually within 1 hour.
|
|
|
Disposition concordance of the 3M-6D education GPT group compared with the 3M-6D education Gemini group
Prazo: Usually within 1 hour.
|
Usually within 1 hour.
|
|
|
Red-flag identification in the 3M-6D education GPT group compared with the 3M-6D education Gemini group
Prazo: Usually within 1 hour.
|
Usually within 1 hour.
|
|
|
NASA Task Load Index score of the 3M-6D education GPT group compared with the 3M-6D education Gemini group
Prazo: Usually within 1 hour.
|
Usually within 1 hour.
|
Outras medidas de resultado
Medida de resultado |
Descrição da medida |
Prazo |
|---|---|---|
|
Failure to identify red flags in the 3M-6D education GPT group compared with the GPT group
Prazo: Usually within 1 hour.
|
Failure to identify red flags is defined as the proportion of participants whose final response does not include the expert-defined red-flag symptoms or warning signs for the assigned standardized simulated clinical scenario.
|
Usually within 1 hour.
|
|
Failure to identify red flags in the 3M-6D education GPT group compared with the control group
Prazo: Usually within 1 hour.
|
Usually within 1 hour.
|
|
|
Failure to identify red flags in the 3M-6D education Gemini group compared with the Gemini group
Prazo: Usually within 1 hour.
|
Usually within 1 hour.
|
|
|
Failure to identify red flags in the 3M-6D education Gemini group compared with the control group
Prazo: Usually within 1 hour.
|
Usually within 1 hour.
|
|
|
Underestimation of disposition in the 3M-6D education GPT group compared with the GPT group
Prazo: Usually within 1 hour.
|
Underestimation of disposition is defined as the proportion of participants whose final care recommendation is lower than the expert-defined disposition level for the assigned standardized simulated clinical scenario.
|
Usually within 1 hour.
|
|
Underestimation of disposition in the 3M-6D education GPT group compared with the control group
Prazo: Usually within 1 hour.
|
Usually within 1 hour.
|
|
|
Underestimation of disposition in the 3M-6D education Gemini group compared with the Gemini group
Prazo: Usually within 1 hour.
|
Usually within 1 hour.
|
|
|
Underestimation of disposition in the 3M-6D education Gemini group compared with the control group
Prazo: Usually within 1 hour.
|
Usually within 1 hour.
|
Colaboradores e Investigadores
É aqui que você encontrará pessoas e organizações envolvidas com este estudo.
Patrocinador
Colaboradores
Datas de registro do estudo
Essas datas acompanham o progresso do registro do estudo e os envios de resumo dos resultados para ClinicalTrials.gov. Os registros do estudo e os resultados relatados são revisados pela National Library of Medicine (NLM) para garantir que atendam aos padrões específicos de controle de qualidade antes de serem publicados no site público.
Datas Principais do Estudo
Início do estudo (Estimado)
20 de junho de 2026
Conclusão Primária (Estimado)
20 de julho de 2026
Conclusão do estudo (Estimado)
20 de julho de 2026
Datas de inscrição no estudo
Enviado pela primeira vez
11 de junho de 2026
Enviado pela primeira vez que atendeu aos critérios de CQ
11 de junho de 2026
Primeira postagem (Real)
16 de junho de 2026
Atualizações de registro de estudo
Última Atualização Postada (Real)
16 de junho de 2026
Última atualização enviada que atendeu aos critérios de controle de qualidade
11 de junho de 2026
Última verificação
1 de junho de 2026
Mais Informações
Termos relacionados a este estudo
Palavras-chave
Outros números de identificação do estudo
- LAMP-1
Plano para dados de participantes individuais (IPD)
Planeja compartilhar dados de participantes individuais (IPD)?
INDECISO
Informações sobre medicamentos e dispositivos, documentos de estudo
Estuda um medicamento regulamentado pela FDA dos EUA
Não
Estuda um produto de dispositivo regulamentado pela FDA dos EUA
Não
Essas informações foram obtidas diretamente do site clinicaltrials.gov sem nenhuma alteração. Se você tiver alguma solicitação para alterar, remover ou atualizar os detalhes do seu estudo, entre em contato com register@clinicaltrials.gov. Assim que uma alteração for implementada em clinicaltrials.gov, ela também será atualizada automaticamente em nosso site .