Tämä sivu käännettiin automaattisesti, eikä käännösten tarkkuutta voida taata. Katso englanninkielinen versio lähdetekstiä varten.

Shadow-Mode Evaluation of an AI Assistant for Multidisciplinary Tumor Boards (OnkoBot)

maanantai 9. marraskuuta 2026 päivittänyt: Lisa Adams, Technical University of Munich

OnkoBot: Prospective Shadow-Mode Evaluation of a Locally Operated, Source-Grounded Large Language Model System for Drafting Multidisciplinary Tumor Board Recommendations

This study tests whether an artificial intelligence (AI) system that drafts tumor board recommendations performs as well on new cases as it did on archived cases. The system runs in shadow mode at three multidisciplinary tumor boards of TUM University Hospital rechts der Isar. For each case it drafts a recommendation before the board meets. The board does not see the draft, and the draft has no influence on patient care. After the meeting, each draft is compared with the board decision. The agreement rate is then compared with the rate measured on archived decisions of the same boards.

Tutkimuksen yleiskatsaus

Yksityiskohtainen kuvaus

The system is a locally operated large language model (LLM) pipeline with retrieval-augmented generation. Its input is the case as presented to the board, with diagnosis, course, question and all patient documents dated before the board date. It retrieves guideline recommendations (Onkopedia and AWMF S3 guidelines in the version in force on the board date), laboratory values, precedent cases from the board archive and PubMed abstracts published before the board date. It then drafts a recommendation with justification. Each statement of the draft is linked to a verbatim quote from its source. All models run on local GPU servers of the hospital with no data retention beyond the immediate analysis, and no patient data are sent to external providers.

The study includes the hematology board, the orthopedics and sarcoma board and the head and neck board. For each case registered for a session, the system drafts a recommendation after the pre-board documents are pulled and before the session starts. Board members do not see the drafts. The decision pipeline (generator Gemma-4-31B at temperature 0) is frozen and versioned for the study period. The study uses routine clinical data only. Patients are not contacted, and no study-specific procedures are performed.

The draft is compared with the consensus decision in the board report. An LLM judge (GLM-5.3-Flash, majority of three votes) assigns both texts to one of seven decision categories: systemic therapy, surgery, radiotherapy, watch and wait, further diagnostics, best supportive care and referral to the molecular tumor board. It also scores agreement in substance from 0 to 3 and records the direction of any disagreement. The judge was validated against clinician ratings on a sealed test set. The same prompt version is used to score the prospective cases and the retrospective reference sample, separately from any output that clinicians see.

The retrospective reference is the evaluation sample of archived decisions of the same three boards (about 2,000 cases, one per patient, drawn with a fixed seed and stratified by time). Patients who also appear in the prospective cohort are removed from the reference. The primary analysis tests non-inferiority of the prospective category match rate against the retrospective rate, pooled over the three boards, with a margin of 7.5 percentage points and a one-sided alpha of 0.025. The variance is cluster-robust for board sessions. A case without a draft counts as not concordant. Per-board results are reported descriptively with 95% confidence intervals. An exploratory component evaluates a trial-matching engine that proposes recruiting clinical trials per case.

In a subset of 30 to 50 cases per board, physicians rate the system draft against the board decision. This subset tests whether the agreement between judge and clinicians, measured retrospectively, also holds prospectively.

Opintotyyppi

Havainnollistava

Ilmoittautuminen (Arvioitu)

450

Yhteystiedot ja paikat

Tässä osiossa on tutkimuksen suorittajien yhteystiedot ja tiedot siitä, missä tämä tutkimus suoritetaan.

Opiskeluyhteys

  • Nimi: Lisa Adams, MD
  • Puhelinnumero: +49 89 4150 1084
  • Sähköposti: lisa.adams@tum.de

Tutki yhteystietojen varmuuskopiointi

Opiskelupaikat

    • Bavaria
      • Munich, Bavaria, Saksa, 81675
        • Ei vielä rekrytointia
        • TUM University Hospital rechts der Isar
      • Munich, Bavaria, Saksa
        • Rekrytointi
        • TUM Klinikum rechts der Isar
        • Ottaa yhteyttä:

Osallistumiskriteerit

Tutkijat etsivät ihmisiä, jotka sopivat tiettyyn kuvaukseen, jota kutsutaan kelpoisuuskriteereiksi. Joitakin esimerkkejä näistä kriteereistä ovat henkilön yleinen terveydentila tai aiemmat hoidot.

Kelpoisuusvaatimukset

Opintokelpoiset iät

  • Lapsi
  • Aikuinen
  • Vanhempi Aikuinen

Hyväksyy terveitä vapaaehtoisia

Ei

Sukupuolet, jotka voivat opiskella

Kaikki

Näytteenottomenetelmä

Ei-todennäköisyysnäyte

Tutkimusväestö

All cases registered for the hematology, orthopedics and sarcoma, and head and neck tumor boards of TUM University Hospital rechts der Isar during the study period. One case is one patient presentation on one board date.

Kuvaus

Inclusion Criteria:

  • Case registered for a session of a participating tumor board (hematology, orthopedics and sarcoma, head and neck) of TUM University Hospital rechts der Isar during the study period
  • Board registration and pre-board patient documents available before the session starts

Exclusion Criteria:

  • Case not discussed at the session, defined as no board report with a consensus decision for that date (deferred or withdrawn registrations)
  • Session without a shadow run of the system

Excluded cases are counted and reported by reason.

Opintosuunnitelma

Tässä osiossa on tietoja tutkimussuunnitelmasta, mukaan lukien kuinka tutkimus on suunniteltu ja mitä tutkimuksella mitataan.

Miten tutkimus on suunniteltu?

Suunnittelun yksityiskohdat

  • Havaintomallit: COHORT
  • Aikanäkymät: PROSPECTIVE

Kohortit ja interventiot

Ryhmä/Kohortti
Interventio / Hoito
Hematology board

Consecutive cases presented at the hematology tumor board of TUM University Hospital rechts der Isar.

A locally operated, source-grounded large language model system drafts a tumor board recommendation with justification and verified citations for each case before the board meeting. The draft is not shown to the board and does not influence patient care.

Orthopedics and sarcoma board

Consecutive cases presented at the orthopedics and sarcoma tumor board of TUM University Hospital rechts der Isar.

A locally operated, source-grounded large language model system drafts a tumor board recommendation with justification and verified citations for each case before the board meeting. The draft is not shown to the board and does not influence patient care.

Head and neck board

Consecutive cases presented at the head and neck tumor board of TUM University Hospital rechts der Isar.

A locally operated, source-grounded large language model system drafts a tumor board recommendation with justification and verified citations for each case before the board meeting. The draft is not shown to the board and does not influence patient care.

Mitä tutkimuksessa mitataan?

Ensisijaiset tulostoimenpiteet

Tulosmittaus
Toimenpiteen kuvaus
Aikaikkuna
Proportion of Cases With Category Match Between System Draft and Board Decision
Aikaikkuna: Each case is assessed once, after the board report of its session is available; analysis after accrual is complete (anticipated January 2027)

Proportion of discussed cases in which the system draft and the board consensus decision fall into the same of seven decision categories (systemic therapy, surgery, radiotherapy, watch and wait, further diagnostics, best supportive care, referral to the molecular tumor board). Overlap counts where either side carries a second category. Assessed by a pre-specified LLM judge (GLM-5.3-Flash, prompt V3, majority of three votes), pooled over the three boards. Tested for non-inferiority against the rate of the same system and judge on the retrospective reference sample of the same boards, margin 7.5 percentage points, one-sided alpha 0.025.

Each case is assessed once, after the board report of its session is available; analysis after accrual is complete (anticipated January 2027)

Toissijaiset tulostoimenpiteet

Tulosmittaus
Toimenpiteen kuvaus
Aikaikkuna
Proportion of Cases Rated Concordant or Partially Concordant
Aikaikkuna: After the board report of each session is available; analysis after accrual is complete (anticipated January 2027)

Category match with a substance score of at least 1 of 3 (concordant requires at least 2 of 3 on a primary-category match, 3 of 3 where only second categories overlap). Non-inferiority versus retrospective reference, margin 7.5 percentage points, tested after the primary outcome (hierarchical testing).

After the board report of each session is available; analysis after accrual is complete (anticipated January 2027)
Proportion of Cases Rated Concordant, Partially Concordant and Not Concordant
Aikaikkuna: After the board report of each session is available; analysis after accrual is complete (anticipated January 2027)

Proportion of cases rated concordant, partially concordant and not concordant by the LLM judge.

After the board report of each session is available; analysis after accrual is complete (anticipated January 2027)
Proportion of Cases With Category Match per Board
Aikaikkuna: After the board report of each session is available; analysis after accrual is complete (anticipated January 2027)

Category match and concordant or partially concordant per board, with 95% confidence intervals, descriptive.

After the board report of each session is available; analysis after accrual is complete (anticipated January 2027)
Proportion of Discordant Cases by Direction of Disagreement
Aikaikkuna: After the board report of each session is available; analysis after accrual is complete (anticipated January 2027)

Among cases without category match, proportion in which the draft is more conservative, more aggressive or a different modality than the board decision.

After the board report of each session is available; analysis after accrual is complete (anticipated January 2027)
Proportion of Cases With Category Match as Rated by Clinicians
Aikaikkuna: Within 3 months after the board session

In 30 to 50 cases per board, agreement between draft and board decision as rated by physicians who did not attend the session, and agreement between judge and clinicians on category match (percentage agreement, Cohen's kappa).

Within 3 months after the board session
Proportion of Citations Verified Word for Word Against Their Source
Aikaikkuna: At draft generation, before the board session

Proportion of citations per case that verify word for word against their source, and proportion of cases in which every citation verifies.

At draft generation, before the board session
Proportion of Cases With a Draft Completed Before the Board Session
Aikaikkuna: At draft generation, before the board session

Proportion of cases with a draft completed before the session starts, and generation time per case in minutes.

At draft generation, before the board session

Muut tulostoimenpiteet

Tulosmittaus
Toimenpiteen kuvaus
Aikaikkuna
Number of Clinical Trials Shortlisted per Case by the Trial-Matching Engine
Aikaikkuna: At draft generation, before the board session

Number of trials shortlisted per case, proportion recruiting on the board date, rule-based error rate (wrong disease, line of therapy, setting or status) and clinician screening verdicts where given.

At draft generation, before the board session

Yhteistyökumppanit ja tutkijat

Täältä löydät tähän tutkimukseen osallistuvat ihmiset ja organisaatiot.

Tutkijat

  • Päätutkija: Lisa Adams, MD, Technical University of Munich
  • Päätutkija: Krischan Braitsch, MD, Technical University of Munich

Opintojen ennätyspäivät

Nämä päivämäärät seuraavat ClinicalTrials.gov-sivustolle lähetettyjen tutkimustietueiden ja yhteenvetojen edistymistä. National Library of Medicine (NLM) tarkistaa tutkimustiedot ja raportoidut tulokset varmistaakseen, että ne täyttävät tietyt laadunvalvontastandardit, ennen kuin ne julkaistaan ​​julkisella verkkosivustolla.

Opi tärkeimmät päivämäärät

Opiskelun aloitus (Arvioitu)

Maanantai 9. marraskuuta 2026

Ensisijainen valmistuminen (Arvioitu)

Sunnuntai 31. tammikuuta 2027

Opintojen valmistuminen (Arvioitu)

Perjantai 30. huhtikuuta 2027

Opintoihin ilmoittautumispäivät

Ensimmäinen lähetetty

Maanantai 9. marraskuuta 2026

Ensimmäinen toimitettu, joka täytti QC-kriteerit

Maanantai 9. marraskuuta 2026

Ensimmäinen Lähetetty (Todellinen)

Maanantai 16. marraskuuta 2026

Tutkimustietojen päivitykset

Viimeisin päivitys julkaistu (Todellinen)

Maanantai 16. marraskuuta 2026

Viimeisin lähetetty päivitys, joka täytti QC-kriteerit

Maanantai 9. marraskuuta 2026

Viimeksi vahvistettu

Sunnuntai 1. marraskuuta 2026

Lisää tietoa

Tähän tutkimukseen liittyvät termit

Yksittäisten osallistujien tietojen suunnitelma (IPD)

Aiotko jakaa yksittäisten osallistujien tietoja (IPD)?

EI

IPD-suunnitelman kuvaus

Individual participant data are routine clinical records that may not leave the hospital under the ethics approval and data protection concept. Aggregate results and the analysis code will be published.

Lääke- ja laitetiedot, tutkimusasiakirjat

Tutkii yhdysvaltalaista FDA sääntelemää lääkevalmistetta

Ei

Tutkii yhdysvaltalaista FDA sääntelemää laitetuotetta

Ei

Nämä tiedot haettiin suoraan verkkosivustolta clinicaltrials.gov ilman muutoksia. Jos sinulla on pyyntöjä muuttaa, poistaa tai päivittää tutkimustietojasi, ota yhteyttä register@clinicaltrials.gov. Heti kun muutos on otettu käyttöön osoitteessa clinicaltrials.gov, se päivitetään automaattisesti myös verkkosivustollemme .

Tilaa