Preliminary Evaluation of a Large Language Model-Based Tool for Complex Surgical Decision Support in Lung Cancer
2026年6月13日 更新者：XiuYuan Chen、Peking University People's Hospital
This study is an exploratory effect-size estimation study, with the following specific objectives: ① to estimate the point estimate and 95% confidence interval of the Win Ratio for the experimental group (GAPS-Agent) versus the control group (large language model) in blinded pairwise preference judgments by thoracic surgery expert adjudicators, to serve as a sample size planning parameter for subsequent multicenter confirmatory clinical trials; ② to preliminarily evaluate the value of GAPS-Agent within clinical workflows.The hypothesis of this study is as follows: compared with a general-purpose large language model without medical enhancement (control group), a structured agentic workflow optimized on the basis of the GAPS evaluation framework (GAPS-Agent, experimental group) can help junior resident physicians generate clinical decision plans for complex lung cancer cases that are more strongly preferred by senior thoracic surgery expert adjudicators.
調査の概要
研究の種類
介入
入学 (推定)
12
段階
- 適用できない
連絡先と場所
このセクションには、調査を実施する担当者の連絡先の詳細と、この調査が実施されている場所に関する情報が記載されています。
研究場所
-
-
Beijing Municipality
-
Beijing、Beijing Municipality、中国、100044
- Peking University People's Hospital
-
-
参加基準
研究者は、適格基準と呼ばれる特定の説明に適合する人を探します。これらの基準のいくつかの例は、人の一般的な健康状態または以前の治療です。
適格基準
就学可能な年齢
- 大人
- 高齢者
健康ボランティアの受け入れ
いいえ
説明
Inclusion Criteria:
Resident Physician Subjects:
- Holds a valid and legally effective Physician Practice License of the People's Republic of China;
- Currently holds the rank of resident physician in a thoracic surgery department at a tertiary Class A (3A) hospital;
- Agrees to complete all assessment tasks of the main study phase in accordance with the study protocol;
- Can guarantee the time and effort required to complete all assessment tasks of the main study.
Study Cases:
- The case was discussed at the Thoracic Oncology Multidisciplinary Team (MDT) conference of Peking University People's Hospital between January 2025 and May 2026;
- The current version of the NCCN guidelines does not provide an explicit recommendation covering the management of the case;
- Does not overlap with the GAPS evaluation set;
- The case is presented in pure text in a structured format, with all direct and indirect identifiers removed and complete de-identification performed prior to inclusion;
- From the pool of eligible cases, 12 cases will be randomly drawn using Python (numpy.random, with a fixed and archived seed) to serve as the main study cases. The cases will cover 6 themes (chest mass of undetermined diagnosis, early-stage lung cancer, locally advanced lung cancer, oligometastatic/oligoprogressive disease, special intraoperative situations, and tumor recurrence), with 2 cases per theme.
Adjudication Expert Panel:
- Holds a valid and legally effective Physician Practice License of the People's Republic of China;
- Currently holds the rank of attending physician or above in a thoracic surgery department at a tertiary Class A hospital;
- Chairs or regularly participates in lung cancer multidisciplinary team (MDT) work in their department.
Exclusion Criteria:
Resident Physician Subjects:
- Has previously participated in the construction of the GAPS evaluation set or the development of GAPS-Agent;
- Unable to complete the tasks of the study phase.
Study Cases:
- Key case information is missing, such as text-form data on pathology (including IHC/NGS), imaging, laboratory tests, prior medical history, comorbidities, or PS score;
- Decision-making for the case is strictly dependent on non-text information.
Adjudication Expert Panel:
- Participated in the construction of the GAPS evaluation set, the content validity verification, or the development of GAPS-Agent for this study;
- Has a direct conflict of interest with any specific product among the two-arm tools of this study.
研究計画
このセクションでは、研究がどのように設計され、研究が何を測定しているかなど、研究計画の詳細を提供します。
研究はどのように設計されていますか？
デザインの詳細
- 主な目的：他の
- 割り当て：ランダム化
- 介入モデル：並列代入
- マスキング：独身
武器と介入
参加者グループ / アーム |
介入・治療 |
|---|---|
|
実験的：test arm
GAPS-Agent
|
The research group has previously developed the GAPS evaluation framework for complex clinical decision-making in lung cancer.
In this framework, G (Grounding) characterizes the cognitive depth of decision-making (ranging from knowledge retrieval to decisions that go beyond clinical guidelines), A (Authority) corresponds to the grading of evidence strength, P (Perturbation) describes the identification and management of real-world clinical confounding factors, and S (Strength) corresponds to the calibration of recommendation strength.
Within this framework, the research group has completed the construction of a 100-item complex lung cancer decision-making evaluation set along with its corresponding rubrics, and has invited multiple thoracic oncology experts to complete content validity validation.
Based on this, the research group developed GAPS-Agent, which uses an open-source large language model as its foundation and integrates functional modules such as guideline and evidence retri
|
|
アクティブコンパレータ：control arm
LLM
|
Open source large language model that is not specifically enhanced in medical field.
|
この研究は何を測定していますか？
主要な結果の測定
結果測定 |
メジャーの説明 |
時間枠 |
|---|---|---|
|
Overall plan Win Ratio
時間枠：Measured at the time when experts completed their preference judgements. Calculated up to 3 weeks after the preference judgements.
|
A total of 10 blinded expert judges made Win/Tie/Loss ternary preference judgments on 192 paired scheme comparisons in terms of overall scheme quality.
The win ratio was calculated as Wins ÷ Losses, and the 95% confidence interval was estimated using a two-level (physician × case) cluster bootstrap resampling method (B = 10,000, quantile method on the log scale).
|
Measured at the time when experts completed their preference judgements. Calculated up to 3 weeks after the preference judgements.
|
二次結果の測定
結果測定 |
メジャーの説明 |
時間枠 |
|---|---|---|
|
Inter-rater agreement
時間枠：Measured at the time when experts completed their preference judgements. Calculated up to 3 weeks after the preference judgements.
|
For the ternary preference judgment results of 10 expert judges across 192 paired comparisons and 6 evaluation domains, Fleiss' kappa was used to assess inter-rater agreement.
The kappa value and its 95% confidence interval are reported for each evaluation domain.
|
Measured at the time when experts completed their preference judgements. Calculated up to 3 weeks after the preference judgements.
|
|
Redundancy Win Ratio
時間枠：Measured at the time when experts completed their preference judgements. Calculated up to 3 weeks after the preference judgements.
|
A total of 10 blinded expert judges made Win/Tie/Loss ternary preference judgments on 192 paired scheme comparisons in terms of overall scheme quality.
The win ratio was calculated as Wins ÷ Losses, and the 95% confidence interval was estimated using a two-level (physician × case) cluster bootstrap resampling method (B = 10,000, quantile method on the log scale).
|
Measured at the time when experts completed their preference judgements. Calculated up to 3 weeks after the preference judgements.
|
|
Evidence-based medicine adherence Win Ratio
時間枠：Measured at the time when experts completed their preference judgements. Calculated up to 3 weeks after the preference judgements.
|
A total of 10 blinded expert judges made Win/Tie/Loss ternary preference judgments on 192 paired scheme comparisons in terms of overall scheme quality.
The win ratio was calculated as Wins ÷ Losses, and the 95% confidence interval was estimated using a two-level (physician × case) cluster bootstrap resampling method (B = 10,000, quantile method on the log scale).
|
Measured at the time when experts completed their preference judgements. Calculated up to 3 weeks after the preference judgements.
|
|
Actionability Win Ratio
時間枠：Measured at the time when experts completed their preference judgements. Calculated up to 3 weeks after the preference judgements.
|
A total of 10 blinded expert judges made Win/Tie/Loss ternary preference judgments on 192 paired scheme comparisons in terms of overall scheme quality.
The win ratio was calculated as Wins ÷ Losses, and the 95% confidence interval was estimated using a two-level (physician × case) cluster bootstrap resampling method (B = 10,000, quantile method on the log scale).
|
Measured at the time when experts completed their preference judgements. Calculated up to 3 weeks after the preference judgements.
|
|
Completeness Win Ratio
時間枠：Measured at the time when experts completed their preference judgements. Calculated up to 3 weeks after the preference judgements.
|
A total of 10 blinded expert judges made Win/Tie/Loss ternary preference judgments on 192 paired scheme comparisons in terms of overall scheme quality.
The win ratio was calculated as Wins ÷ Losses, and the 95% confidence interval was estimated using a two-level (physician × case) cluster bootstrap resampling method (B = 10,000, quantile method on the log scale).
|
Measured at the time when experts completed their preference judgements. Calculated up to 3 weeks after the preference judgements.
|
|
Safety Win Ratio
時間枠：Measured at the time when experts completed their preference judgements. Calculated up to 3 weeks after the preference judgements.
|
A total of 10 blinded expert judges made Win/Tie/Loss ternary preference judgments on 192 paired scheme comparisons in terms of overall scheme quality.
The win ratio was calculated as Wins ÷ Losses, and the 95% confidence interval was estimated using a two-level (physician × case) cluster bootstrap resampling method (B = 10,000, quantile method on the log scale).
|
Measured at the time when experts completed their preference judgements. Calculated up to 3 weeks after the preference judgements.
|
|
GAPS automated rubric score
時間枠：Generated up to 3 weeks after residents finished their plan generation.
|
A third-party large language model, independent of the two study arms' base models, served as the judge model and automatically scored all 96 plans according to the GAPS rubric.
|
Generated up to 3 weeks after residents finished their plan generation.
|
|
Subject physician's self-confidence score
時間枠：Completed at the time when residents submitted their plans. Calculated up to 3 weeks after the submission.
|
After submitting each case plan, the participating physicians self-rated their confidence in their own plan using a 1-5 point Likert scale.
|
Completed at the time when residents submitted their plans. Calculated up to 3 weeks after the submission.
|
|
Tool satisfaction score
時間枠：Completed at the time when residents submitted their plans. Calculated up to 3 weeks after the submission.
|
After submitting each case plan, the participating physicians rated their satisfaction with the tool using a 1-5 point Likert scale.
|
Completed at the time when residents submitted their plans. Calculated up to 3 weeks after the submission.
|
|
Tool trustworthiness score
時間枠：Completed at the time when residents submitted their plans. Calculated up to 3 weeks after the submission.
|
After submitting each case plan, the participating physicians rated the tool's credibility using a 1-5 point Likert scale.
|
Completed at the time when residents submitted their plans. Calculated up to 3 weeks after the submission.
|
|
Decision-making time
時間枠：Completed at the time when residents submitted their plans. Calculated up to 3 weeks after the submission.
|
The time taken (in minutes) by each participating physician to complete the production of each case plan was automatically recorded by the evaluation platform.
Differences between groups were analyzed using a linear mixed-effects model.
|
Completed at the time when residents submitted their plans. Calculated up to 3 weeks after the submission.
|
協力者と研究者
ここでは、この調査に関係する人々や組織を見つけることができます。
研究記録日
これらの日付は、ClinicalTrials.gov への研究記録と要約結果の提出の進捗状況を追跡します。研究記録と報告された結果は、国立医学図書館 (NLM) によって審査され、公開 Web サイトに掲載される前に、特定の品質管理基準を満たしていることが確認されます。
主要日程の研究
研究開始 (実際)
2026年6月10日
一次修了 (推定)
2026年6月21日
研究の完了 (推定)
2026年6月21日
試験登録日
最初に提出
2026年6月10日
QC基準を満たした最初の提出物
2026年6月13日
最初の投稿 (実際)
2026年6月17日
学習記録の更新
投稿された最後の更新 (実際)
2026年6月17日
QC基準を満たした最後の更新が送信されました
2026年6月13日
最終確認日
2026年6月1日
詳しくは
この情報は、Web サイト clinicaltrials.gov から変更なしで直接取得したものです。研究の詳細を変更、削除、または更新するリクエストがある場合は、register@clinicaltrials.gov。 までご連絡ください。 clinicaltrials.gov に変更が加えられるとすぐに、ウェブサイトでも自動的に更新されます。
肺がん（NSCLC）の臨床試験
-
Novartis Pharmaceuticals終了しましたメラノーマ | 高度なEGFR変異体非小さな細胞肺cancer（NSCLC） | KRAS G12変異NSCLC | 食道扁平上皮がん（SCC） | ヘッド/ネックSCC | 進行した胃腸間質腫瘍（GIST） | 進行したNRAS/BRAFT WT皮膚黒色腫アメリカ, 台湾, オランダ, カナダ, スペイン, シンガポール, イタリア, 日本, 韓国
-
Xiaorong Dongわからない
-
CSPC Megalith Biopharmaceutical Co.,Ltd.まだ募集していません
-
Tianjin Medical University Cancer Institute and...募集
-
Radboud University Medical CenterPfizer; ImaginAb, Inc.; University Hospital Tuebingenまだ募集していません
-
Guangdong Provincial People's Hospital積極的、募集していない
GAPS-Agentの臨床試験
-
ImmunityBio, Inc.引きこもった敗血症 | リンパ球減少症 | 急性呼吸窮迫症候群（ARDS） | 市中肺炎 (CAP) | 免疫麻痺
-
ImmunityBio, Inc.引きこもった敗血症 | 急性呼吸促拍症候群 | 重度市中肺炎 | 重症成人患者におけるリンパ球減少症／免疫麻痺