Testing an AI Tool to Help Primary Care Clinicians With Specialty Consultation Questions (SAGE)
Physician Usability and Output Quality Evaluation of a Retrieval-Augmented Language Model System for Specialty Medical Consultation: A Human-Computer Interaction Study
The goal of this study is to test an artificial intelligence (AI) tool called SAGE. SAGE helps primary care doctors with questions that often need a specialist. Primary care doctors are the doctors people usually see first. SAGE reviews a case and suggests what a specialist might advise.
The main questions it aims to answer are:
- Do doctors make sound, timely care decisions when they use SAGE?
- Do those decisions match what a specialist would advise?
Researchers will compare the decisions doctors make with and without SAGE.
Doctors in the study will:
- Review made-up patient cases (these are not real patients)
- Make a decision for each case, their usual way and with SAGE
Studieöversikt
Status
Betingelser
Intervention / Behandling
Studietyp
Inskrivning (Faktisk)
Fas
- Inte tillämpbar
Kontakter och platser
Studieorter
-
-
California
-
Stanford, California, Förenta staterna, 94305
- Stanford University
-
-
Deltagandekriterier
Urvalskriterier
Åldrar som är berättigade till studier
Tar emot friska volontärer
Kön som är behöriga för studier
Beskrivning
Inclusion Criteria:
- Physician practicing in primary care internal medicine and/or family medicine
- Attending physician, or resident physician at postgraduate year 2 (PGY-2) or higher
- Stanford-affiliated (faculty, staff, or trainee)
- Able to complete a single, approximately 60-minute remote (Zoom) session conducted in English
Exclusion Criteria:
- Not currently practicing in primary care internal medicine or family medicine
- Member of the study team or otherwise involved in the development of SAGE
- Unable to complete the remote study session
Studieplan
Hur är studien utformad?
Designdetaljer
- Primärt syfte: Hälsovårdsforskning
- Tilldelning: N/A
- Interventionsmodell: Enskild gruppuppgift
- Maskning: Ingen (Open Label)
Vapen och interventioner
Deltagargrupp / Arm |
Intervention / Behandling |
|---|---|
|
Experimentell: Primary care physicians
Primary care physicians make management decisions on synthetic patient cases. Each physician reviews four cases in random order: two with SAGE (AI clinical decision support) and two using a standard eConsult approach. |
SAGE is an artificial intelligence (AI) based clinical decision support tool. It generates specialty-informed recommendations for a primary care consultation question. Physicians review the SAGE output while making a management decision for the synthetic case. A standard electronic consultation (eConsult) format, without SAGE. Physicians review this information while making a management decision for the synthetic case. |
Vad mäter studien?
Primära resultatmått
Resultatmått |
Åtgärdsbeskrivning |
Tidsram |
|---|---|---|
|
Clinical actionability of the clinician's decision
Tidsram: Post-session independent review, through study completion (up to approximately 2 months)
|
For each case, independent reviewers judge whether the physician's decision is clinically actionable, meaning the physician can act on it now rather than wait for a specialist. The outcome is the proportion of cases judged actionable, measured separately for decisions made with SAGE and for decisions made the standard way. |
Post-session independent review, through study completion (up to approximately 2 months)
|
|
Concordance of the clinician's decision with the specialist reference answer
Tidsram: Post-session independent review, through study completion (up to approximately 2 months)
|
Agreement between the physician's management decision and a specialist reference answer, rated by independent reviewers, with inter-rater agreement summarized. Assessed for decisions made with SAGE and for decisions made the standard way. |
Post-session independent review, through study completion (up to approximately 2 months)
|
Sekundära resultatmått
Resultatmått |
Åtgärdsbeskrivning |
Tidsram |
|---|---|---|
|
System usability (System Usability Scale, SUS)
Tidsram: End of study session (Day 1, approximately 60 minutes)
|
Perceived usability of SAGE measured with the 10-item System Usability Scale (SUS), scored from 0 to 100. |
End of study session (Day 1, approximately 60 minutes)
|
|
Order acceptance in the SAGE condition
Tidsram: During study session (Day 1)
|
Proportion of SAGE-suggested orders that the clinician accepts, assessed in the SAGE condition. |
During study session (Day 1)
|
|
Time to complete case review
Tidsram: During study session (Day 1)
|
Time (in minutes) taken to review each case and reach a management decision, compared between the SAGE and standard eConsult conditions. |
During study session (Day 1)
|
Samarbetspartners och utredare
Sponsor
Utredare
- Huvudutredare: Jonathan Chen, MD, PhD, Stanford University
Studieavstämningsdatum
Studera stora datum
Studiestart (Faktisk)
Primärt slutförande (Faktisk)
Avslutad studie (Faktisk)
Studieregistreringsdatum
Först inskickad
Först inskickad som uppfyllde QC-kriterierna
Första postat (Faktisk)
Uppdateringar av studier
Senaste uppdatering publicerad (Faktisk)
Senaste inskickade uppdateringen som uppfyllde QC-kriterierna
Senast verifierad
Mer information
Termer relaterade till denna studie
Andra studie-ID-nummer
- 85430
Plan för individuella deltagardata (IPD)
Planerar du att dela individuella deltagardata (IPD)?
Läkemedels- och apparatinformation, studiedokument
Studerar en amerikansk FDA-reglerad läkemedelsprodukt
Studerar en amerikansk FDA-reglerad produktprodukt
Denna information hämtades direkt från webbplatsen clinicaltrials.gov utan några ändringar. Om du har några önskemål om att ändra, ta bort eller uppdatera dina studieuppgifter, vänligen kontakta register@clinicaltrials.gov. Så snart en ändring har implementerats på clinicaltrials.gov, kommer denna att uppdateras automatiskt även på vår webbplats .