Testing an AI Tool to Help Primary Care Clinicians With Specialty Consultation Questions (SAGE)
Physician Usability and Output Quality Evaluation of a Retrieval-Augmented Language Model System for Specialty Medical Consultation: A Human-Computer Interaction Study
The goal of this study is to test an artificial intelligence (AI) tool called SAGE. SAGE helps primary care doctors with questions that often need a specialist. Primary care doctors are the doctors people usually see first. SAGE reviews a case and suggests what a specialist might advise.
The main questions it aims to answer are:
- Do doctors make sound, timely care decisions when they use SAGE?
- Do those decisions match what a specialist would advise?
Researchers will compare the decisions doctors make with and without SAGE.
Doctors in the study will:
- Review made-up patient cases (these are not real patients)
- Make a decision for each case, their usual way and with SAGE
研究概览
研究类型
注册 (实际的)
阶段
- 不适用
联系人和位置
学习地点
-
-
California
-
Stanford、California、美国、94305
- Stanford University
-
-
参与标准
资格标准
适合学习的年龄
接受健康志愿者
有资格学习的性别
描述
Inclusion Criteria:
- Physician practicing in primary care internal medicine and/or family medicine
- Attending physician, or resident physician at postgraduate year 2 (PGY-2) or higher
- Stanford-affiliated (faculty, staff, or trainee)
- Able to complete a single, approximately 60-minute remote (Zoom) session conducted in English
Exclusion Criteria:
- Not currently practicing in primary care internal medicine or family medicine
- Member of the study team or otherwise involved in the development of SAGE
- Unable to complete the remote study session
学习计划
研究是如何设计的?
设计细节
- 主要用途:卫生服务研究
- 分配:不适用
- 介入模型:单组作业
- 屏蔽:无(打开标签)
武器和干预
参与者组/臂 |
干预/治疗 |
|---|---|
|
实验性的:Primary care physicians
Primary care physicians make management decisions on synthetic patient cases. Each physician reviews four cases in random order: two with SAGE (AI clinical decision support) and two using a standard eConsult approach. |
SAGE is an artificial intelligence (AI) based clinical decision support tool. It generates specialty-informed recommendations for a primary care consultation question. Physicians review the SAGE output while making a management decision for the synthetic case. A standard electronic consultation (eConsult) format, without SAGE. Physicians review this information while making a management decision for the synthetic case. |
研究衡量的是什么?
主要结果指标
结果测量 |
措施说明 |
大体时间 |
|---|---|---|
|
Clinical actionability of the clinician's decision
大体时间:Post-session independent review, through study completion (up to approximately 2 months)
|
For each case, independent reviewers judge whether the physician's decision is clinically actionable, meaning the physician can act on it now rather than wait for a specialist. The outcome is the proportion of cases judged actionable, measured separately for decisions made with SAGE and for decisions made the standard way. |
Post-session independent review, through study completion (up to approximately 2 months)
|
|
Concordance of the clinician's decision with the specialist reference answer
大体时间:Post-session independent review, through study completion (up to approximately 2 months)
|
Agreement between the physician's management decision and a specialist reference answer, rated by independent reviewers, with inter-rater agreement summarized. Assessed for decisions made with SAGE and for decisions made the standard way. |
Post-session independent review, through study completion (up to approximately 2 months)
|
次要结果测量
结果测量 |
措施说明 |
大体时间 |
|---|---|---|
|
System usability (System Usability Scale, SUS)
大体时间:End of study session (Day 1, approximately 60 minutes)
|
Perceived usability of SAGE measured with the 10-item System Usability Scale (SUS), scored from 0 to 100. |
End of study session (Day 1, approximately 60 minutes)
|
|
Order acceptance in the SAGE condition
大体时间:During study session (Day 1)
|
Proportion of SAGE-suggested orders that the clinician accepts, assessed in the SAGE condition. |
During study session (Day 1)
|
|
Time to complete case review
大体时间:During study session (Day 1)
|
Time (in minutes) taken to review each case and reach a management decision, compared between the SAGE and standard eConsult conditions. |
During study session (Day 1)
|
合作者和调查者
调查人员
- 首席研究员:Jonathan Chen, MD, PhD、Stanford University
研究记录日期
研究主要日期
学习开始 (实际的)
初级完成 (实际的)
研究完成 (实际的)
研究注册日期
首次提交
首先提交符合 QC 标准的
首次发布 (实际的)
研究记录更新
最后更新发布 (实际的)
上次提交的符合 QC 标准的更新
最后验证
更多信息
此信息直接从 clinicaltrials.gov 网站检索,没有任何更改。如果您有任何更改、删除或更新研究详细信息的请求,请联系 register@clinicaltrials.gov. clinicaltrials.gov 上实施更改,我们的网站上也会自动更新.