Testing an AI Tool to Help Primary Care Clinicians With Specialty Consultation Questions (SAGE)
Physician Usability and Output Quality Evaluation of a Retrieval-Augmented Language Model System for Specialty Medical Consultation: A Human-Computer Interaction Study
The goal of this study is to test an artificial intelligence (AI) tool called SAGE. SAGE helps primary care doctors with questions that often need a specialist. Primary care doctors are the doctors people usually see first. SAGE reviews a case and suggests what a specialist might advise.
The main questions it aims to answer are:
- Do doctors make sound, timely care decisions when they use SAGE?
- Do those decisions match what a specialist would advise?
Researchers will compare the decisions doctors make with and without SAGE.
Doctors in the study will:
- Review made-up patient cases (these are not real patients)
- Make a decision for each case, their usual way and with SAGE
調査の概要
研究の種類
入学 (実際)
段階
- 適用できない
連絡先と場所
研究場所
-
-
California
-
Stanford、California、アメリカ、94305
- Stanford University
-
-
参加基準
適格基準
就学可能な年齢
健康ボランティアの受け入れ
受講資格のある性別
説明
Inclusion Criteria:
- Physician practicing in primary care internal medicine and/or family medicine
- Attending physician, or resident physician at postgraduate year 2 (PGY-2) or higher
- Stanford-affiliated (faculty, staff, or trainee)
- Able to complete a single, approximately 60-minute remote (Zoom) session conducted in English
Exclusion Criteria:
- Not currently practicing in primary care internal medicine or family medicine
- Member of the study team or otherwise involved in the development of SAGE
- Unable to complete the remote study session
研究計画
研究はどのように設計されていますか？
デザインの詳細
- 主な目的：ヘルスサービス研究
- 割り当て：なし
- 介入モデル：単一グループの割り当て
- マスキング：なし（オープンラベル）
武器と介入
参加者グループ / アーム |
介入・治療 |
|---|---|
|
実験的：Primary care physicians
Primary care physicians make management decisions on synthetic patient cases. Each physician reviews four cases in random order: two with SAGE (AI clinical decision support) and two using a standard eConsult approach. |
SAGE is an artificial intelligence (AI) based clinical decision support tool. It generates specialty-informed recommendations for a primary care consultation question. Physicians review the SAGE output while making a management decision for the synthetic case. A standard electronic consultation (eConsult) format, without SAGE. Physicians review this information while making a management decision for the synthetic case. |
この研究は何を測定していますか？
主要な結果の測定
結果測定 |
メジャーの説明 |
時間枠 |
|---|---|---|
|
Clinical actionability of the clinician's decision
時間枠：Post-session independent review, through study completion (up to approximately 2 months)
|
For each case, independent reviewers judge whether the physician's decision is clinically actionable, meaning the physician can act on it now rather than wait for a specialist. The outcome is the proportion of cases judged actionable, measured separately for decisions made with SAGE and for decisions made the standard way. |
Post-session independent review, through study completion (up to approximately 2 months)
|
|
Concordance of the clinician's decision with the specialist reference answer
時間枠：Post-session independent review, through study completion (up to approximately 2 months)
|
Agreement between the physician's management decision and a specialist reference answer, rated by independent reviewers, with inter-rater agreement summarized. Assessed for decisions made with SAGE and for decisions made the standard way. |
Post-session independent review, through study completion (up to approximately 2 months)
|
二次結果の測定
結果測定 |
メジャーの説明 |
時間枠 |
|---|---|---|
|
System usability (System Usability Scale, SUS)
時間枠：End of study session (Day 1, approximately 60 minutes)
|
Perceived usability of SAGE measured with the 10-item System Usability Scale (SUS), scored from 0 to 100. |
End of study session (Day 1, approximately 60 minutes)
|
|
Order acceptance in the SAGE condition
時間枠：During study session (Day 1)
|
Proportion of SAGE-suggested orders that the clinician accepts, assessed in the SAGE condition. |
During study session (Day 1)
|
|
Time to complete case review
時間枠：During study session (Day 1)
|
Time (in minutes) taken to review each case and reach a management decision, compared between the SAGE and standard eConsult conditions. |
During study session (Day 1)
|
協力者と研究者
スポンサー
捜査官
- 主任研究者：Jonathan Chen, MD, PhD、Stanford University
研究記録日
主要日程の研究
研究開始 (実際)
一次修了 (実際)
研究の完了 (実際)
試験登録日
最初に提出
QC基準を満たした最初の提出物
最初の投稿 (実際)
学習記録の更新
投稿された最後の更新 (実際)
QC基準を満たした最後の更新が送信されました
最終確認日
詳しくは
この情報は、Web サイト clinicaltrials.gov から変更なしで直接取得したものです。研究の詳細を変更、削除、または更新するリクエストがある場合は、register@clinicaltrials.gov。 までご連絡ください。 clinicaltrials.gov に変更が加えられるとすぐに、ウェブサイトでも自動的に更新されます。