Concordance Between Large Language Model and Multidisciplinary Team Recommendations in Rectal Cancer
A Prospective Single-Center Observational Study Evaluating Concordance Between Large Language Model-Generated Recommendations and Multidisciplinary Team Recommendations in Rectal Cancer
This prospective single-center observational study will evaluate the concordance between recommendations generated by a locally deployed large language model and standardized multidisciplinary team recommendations for patients with rectal cancer.
Consecutive adult patients with pathologically confirmed rectal adenocarcinoma who are scheduled for routine rectal cancer multidisciplinary team discussion will be enrolled. For each case, investigators will prepare a standardized de-identified clinical summary before the multidisciplinary team meeting. The same summary will be used for large language model generation and routine multidisciplinary team discussion.
The large language model recommendation will not be disclosed to the clinical team and will not influence actual patient management. Concordance between the large language model recommendation and the multidisciplinary team reference recommendation will be assessed using predefined structured rules and blinded expert review.
Studienübersicht
Status
Bedingungen
Intervention / Behandlung
Detaillierte Beschreibung
Management of rectal cancer often requires multidisciplinary decision-making based on tumor location, pelvic magnetic resonance imaging findings, clinical stage, mesorectal fascia or circumferential resection margin status, extramural vascular invasion, lateral lymph node status, metastatic status, previous treatment, surgical feasibility, organ preservation considerations, and patient preferences. Large language models have shown potential in medical information processing and clinical decision support, but their performance in complex oncologic decision-making has not been fully validated.
This study is designed as a prospective, single-center, observational concordance study. Consecutive patients with rectal adenocarcinoma who are scheduled for routine rectal cancer multidisciplinary team discussion at the study center will be screened. Before the multidisciplinary team meeting, investigators will prepare a standardized de-identified case summary using a predefined template. The summary will include relevant demographic information, clinical status, endoscopic findings, pathological and molecular information, key imaging findings, previous treatments, and patient preferences or practical constraints when available.
The same standardized case summary will be used as the input for a locally deployed large language model. A fixed prompt, fixed model version, and fixed inference parameters will be used throughout the study. Each case will be processed in an independent session, without additional interactive prompting or manual correction. The model will not use internet access, external knowledge retrieval, or retrieval-augmented generation during the study.
Routine multidisciplinary team discussion will proceed independently according to standard clinical workflow. The large language model output will not be provided to the multidisciplinary team and will not be used to guide patient treatment. Actual treatment decisions will be made by the treating physicians and multidisciplinary team according to routine clinical practice.
After both recommendations have been generated, the large language model recommendation and the multidisciplinary team recommendation will be transformed into a structured format. The structured recommendations will include the preferred treatment pathway, specific treatment plan, acceptable alternative options, key rationale, and need for additional examinations or information. De-identified and randomly ordered recommendations will then be evaluated using predefined concordance rules and blinded expert review.
The primary objective is to estimate the complete concordance rate between the large language model recommendation and the multidisciplinary team reference recommendation for the preferred treatment pathway. Secondary objectives include evaluation of concordance in specific treatment implementation, acceptable alternative options, identification of additional examinations or information needs, and the rate of major discordance. Exploratory analyses will assess patterns of major discordance and clinical features associated with concordant or discordant recommendations.
Studientyp
Einschreibung (Geschätzt)
Kontakte und Standorte
Studienkontakt
- Name: Jinbo Yue, MD, PhD
- Telefonnummer: 053167626929
- E-Mail: jbyue@sdfmu.edu.cn
Studienorte
-
-
Shandong
-
Jinan, Shandong, China, 0531
- Department of Radiation Oncology, Shandong Cancer Hospital and Institute
-
-
Teilnahmekriterien
Zulassungskriterien
Studienberechtigtes Alter
- Erwachsene
- Älterer Erwachsener
Akzeptiert gesunde Freiwillige
Probenahmeverfahren
Studienpopulation
Beschreibung
Inclusion Criteria:
- Age 18 years or older.
- Pathologically confirmed rectal adenocarcinoma.
- Scheduled for routine rectal cancer multidisciplinary team discussion at the study center.
- Availability of complete or substantially complete standardized decision-making information before multidisciplinary team discussion, including clinical, pathological, and key imaging information.
- Availability of a structured pelvic magnetic resonance imaging report meeting the requirements of the institutional rectal cancer multidisciplinary team, including at least tumor location, clinical T stage, clinical N stage, circumferential resection margin or mesorectal fascia status, extramural vascular invasion status, and lateral lymph node status.
- Presence of a defined clinical treatment decision question.
- Clinical data can be sufficiently de-identified for research use.
Exclusion Criteria:
- Severely incomplete clinical information preventing preparation of a standardized case summary.
- Non-rectal primary tumor.
- Routine follow-up cases without a defined treatment decision question.
- Absence of a structured pelvic magnetic resonance imaging report meeting the requirements of the institutional rectal cancer multidisciplinary team, or missing pelvic magnetic resonance imaging elements that preclude key rectal cancer decision-making.
- Cases containing sensitive information considered unsuitable for large language model input by the study team.
- Cases in which a definitive treatment decision has already been made before the multidisciplinary team discussion and the meeting serves only as a formal review.
Studienplan
Wie ist die Studie aufgebaut?
Designdetails
Kohorten und Interventionen
Gruppe / Kohorte |
Intervention / Behandlung |
|---|---|
|
Rectal Cancer MDT Cases
Adult patients with pathologically confirmed rectal adenocarcinoma who are scheduled for routine rectal cancer multidisciplinary team discussion at the study center.
For each enrolled case, a standardized de-identified clinical summary will be prepared and used for both large language model recommendation generation and routine multidisciplinary team discussion.
The large language model output will not be disclosed to the clinical team and will not influence actual patient management.
|
For each enrolled case, a standardized de-identified clinical summary will be entered into a locally deployed large language model using a fixed prompt and fixed inference parameters.
The model will generate a structured treatment recommendation for concordance assessment.
The large language model output will not be disclosed to the multidisciplinary team and will not influence actual patient management.
|
Was misst die Studie?
Primäre Ergebnismessungen
Ergebnis Maßnahme |
Maßnahmenbeschreibung |
Zeitfenster |
|---|---|---|
|
Percentage of Cases With Complete Concordance in Preferred Treatment Pathway Assessed Using a Structured Concordance Adjudication Form
Zeitfenster: From enrollment to completion of recommendation adjudication for each case, up to 12 months.
|
Percentage of enrolled cases in which the preferred treatment pathway recommended by the large language model is completely concordant with the multidisciplinary team reference recommendation, as assessed using a predefined structured concordance adjudication form.
The unit of measure is percentage of cases.
|
From enrollment to completion of recommendation adjudication for each case, up to 12 months.
|
Sekundäre Ergebnismessungen
Ergebnis Maßnahme |
Maßnahmenbeschreibung |
Zeitfenster |
|---|---|---|
|
Percentage of Cases With Concordant Specific Treatment Implementation Assessed Using a Structured Concordance Adjudication Form
Zeitfenster: From enrollment to completion of recommendation adjudication for each case, up to 12 months.
|
Percentage of cases in which the large language model recommendation and the multidisciplinary team reference recommendation are rated as concordant for the specific treatment implementation plan using a predefined structured concordance adjudication form and blinded expert review.
The unit of measure is percentage of cases.
|
From enrollment to completion of recommendation adjudication for each case, up to 12 months.
|
|
Percentage of Applicable Cases With Concordant Alternative Treatment Options Assessed Using a Structured Concordance Adjudication Form
Zeitfenster: From enrollment to completion of recommendation adjudication for each case, up to 12 months.
|
Percentage of applicable cases in which the large language model recommendation and the multidisciplinary team reference recommendation are rated as concordant regarding acceptable alternative treatment options using a predefined structured concordance adjudication form and blinded expert review.
The unit of measure is percentage of applicable cases.
|
From enrollment to completion of recommendation adjudication for each case, up to 12 months.
|
|
Percentage of Cases With Concordant Identification of Additional Examination or Information Needs Assessed Using a Structured Concordance Adjudication Form
Zeitfenster: From enrollment to completion of recommendation adjudication for each case, up to 12 months.
|
Percentage of cases in which the large language model recommendation and the multidisciplinary team reference recommendation agree on whether additional examinations, restaging, or key information are needed before treatment decision-making, as assessed using a predefined structured concordance adjudication form.
The unit of measure is percentage of cases.
|
From enrollment to completion of recommendation adjudication for each case, up to 12 months.
|
|
Percentage of Cases With Major Discordance Assessed by Blinded Expert Review
Zeitfenster: From enrollment to completion of recommendation adjudication for each case, up to 12 months.
|
Percentage of enrolled cases in which differences between the large language model recommendation and the multidisciplinary team reference recommendation are classified as major discordance using a predefined structured adjudication form and blinded expert review.
Major discordance is defined as a clinically substantial difference potentially associated with undertreatment, overtreatment, incorrect treatment sequencing, inappropriate organ preservation or local control judgment, or omission of necessary additional evaluation.
The unit of measure is percentage of cases.
|
From enrollment to completion of recommendation adjudication for each case, up to 12 months.
|
|
Cohen's Kappa Coefficient for Inter-Rater Agreement in Blinded Expert Assessment
Zeitfenster: At completion of blinded expert review, up to 12 months.
|
Inter-rater agreement between independent blinded reviewers for judgment-based endpoints will be assessed using Cohen's kappa coefficient.
Judgment-based endpoints include specific treatment implementation concordance, alternative treatment option concordance, major discordance classification, and failure mode classification when applicable.
The unit of measure is the kappa coefficient.
|
At completion of blinded expert review, up to 12 months.
|
Mitarbeiter und Ermittler
Ermittler
- Hauptermittler: Jinbo Yue, MD, PhD, Shandong Cancer Hospital and Institute
Studienaufzeichnungsdaten
Haupttermine studieren
Studienbeginn (Geschätzt)
Primärer Abschluss (Geschätzt)
Studienabschluss (Geschätzt)
Studienanmeldedaten
Zuerst eingereicht
Zuerst eingereicht, das die QC-Kriterien erfüllt hat
Zuerst gepostet (Tatsächlich)
Studienaufzeichnungsaktualisierungen
Letztes Update gepostet (Tatsächlich)
Letztes eingereichtes Update, das die QC-Kriterien erfüllt
Zuletzt verifiziert
Mehr Informationen
Begriffe im Zusammenhang mit dieser Studie
Schlüsselwörter
Zusätzliche relevante MeSH-Bedingungen
Andere Studien-ID-Nummern
- LLM-MDT-RECTAL-001
Plan für individuelle Teilnehmerdaten (IPD)
Planen Sie, individuelle Teilnehmerdaten (IPD) zu teilen?
Arzneimittel- und Geräteinformationen, Studienunterlagen
Studiert ein von der US-amerikanischen FDA reguliertes Arzneimittelprodukt
Studiert ein von der US-amerikanischen FDA reguliertes Geräteprodukt
Diese Informationen wurden ohne Änderungen direkt von der Website clinicaltrials.gov abgerufen. Wenn Sie Ihre Studiendaten ändern, entfernen oder aktualisieren möchten, wenden Sie sich bitte an register@clinicaltrials.gov. Sobald eine Änderung auf clinicaltrials.gov implementiert wird, wird diese automatisch auch auf unserer Website aktualisiert .