Deze pagina is automatisch vertaald en de nauwkeurigheid van de vertaling kan niet worden gegarandeerd. Raadpleeg de Engelse versie voor een brontekst.

Benchmarking Large Language Models Against Tumour Boards for Oncology Treatment Recommendations (BEACON)

28 juli 2026 bijgewerkt door: Assistance Publique - Hôpitaux de Paris

Benchmarking AI for Clinical Oncology decisioNmaking (BEACON): A Prospective, Multicentre, Blinded Evaluation of Frontier Large Language Models Against Multidisciplinary Tumour Board Recommendations in Oncology Treatment Planning

BEACON (Benchmarking AI for Clinical Oncology decisioNmaking) is a prospective, multicentre, comparative, blinded, non-interventional benchmark evaluating the treatment recommendations of five frontier large language models (LLMs) against the recommendations of multidisciplinary tumour boards (RCP) in oncology treatment planning. One hundred standardised synthetic cases (20 per localisation, across breast, lung, urological, digestive and gynaecological cancers) are submitted as identical structured input to two independent tumour boards per localisation and to five frontier LLMs. Each recommendation - human or model - is decomposed into five predefined decision domains (intent, surgery, radiotherapy, systemic therapy, work-up and biomarkers) and scored 0/1/2 for concordance against a two-tier reference: the consensus of the two tumour boards, complemented by an a priori locked guideline matrix (ESMO, NCCN). The primary endpoint is domain-level concordance between LLM and RCP consensus, expressed as a linearly weighted Cohen's kappa. A co-primary safety endpoint captures the proportion of recommendations carrying serious harm potential, because concordance alone can conceal dangerous errors. Because expert boards may disagree with one another on identical cases, model performance is always interpreted against the human consensus. BEACON is designed as reusable, openly licensed, pre-registered infrastructure: all synthetic cases, evaluation rubrics, the locked guideline matrix, scoring algorithms and verbatim prompts are released for full reproducibility.

Studie Overzicht

Toestand

Werving

Conditie

Interventie / Behandeling

Gedetailleerde beschrijving

BEACON is a prospective, multicentre, blinded benchmark using automated, criteria-based scoring. It is built on three design decisions that distinguish it from the existing literature: (i) synthetic, standardised cases remove the record-completeness variability that confounds retrospective comparisons and allow the identical input to be given to every board and every model; (ii) two independent tumour boards per localisation let human-human agreement be measured rather than assumed; and (iii) a guideline matrix, locked a priori, provides an objective anchor applied identically to human and model recommendations.

Reference standard. For each case-domain, a guideline matrix (guideline-recommended / acceptable / unsupported options per case-domain; ESMO, NCCN), locked and time-stamped before data collection, is applied identically to boards and models.

Five decision domains. Every recommendation is decomposed into D1 Intent, D2 Surgery, D3 Radiotherapy, D4 Systemic therapy (class + line), and D5 Work-up & biomarkers before any comparison.

Studietype

Observationeel

Inschrijving (Geschat)

100

Contacten en locaties

In dit gedeelte vindt u de contactgegevens van degenen die het onderzoek uitvoeren en informatie over waar dit onderzoek wordt uitgevoerd.

Studiecontact

Studie Contact Back-up

Studie Locaties

Deelname Criteria

Onderzoekers zoeken naar mensen die aan een bepaalde beschrijving voldoen, de zogenaamde geschiktheidscriteria. Enkele voorbeelden van deze criteria zijn iemands algemene gezondheidstoestand of eerdere behandelingen.

Geschiktheidscriteria

Leeftijden die in aanmerking komen voor studie

  • Volwassen
  • Oudere volwassene

Accepteert gezonde vrijwilligers

Nee

Bemonsteringsmethode

Niet-waarschijnlijkheidssteekproef

Studie Bevolking

100 synthetic oncology treatment-planning cases (20 per localisation) across five localisations: breast, lung, urological (prostate, bladder / upper-tract urothelial, kidney), digestive and gynaecological. Each case is a structured JSON input specifying UICC 8th-edition stage, biomarkers, ECOG performance status, comorbidities and a standardised clinical question. No human participants, no patient data and no identifiable individuals. Recommendations are produced by two independent tumour boards per localisation and by five frontier LLMs (queried May 2026).

Beschrijving

Inclusion Criteria:

  • Synthetic oncology case within one of the five predefined localisations (breast, lung, urological, digestive, gynaecological).
  • Complete structured schema: UICC 8th-edition stage, biomarkers, ECOG performance status, comorbidities and a standardised clinical question.
  • A clinically answerable treatment-planning question that is mappable to the locked guideline matrix.

Exclusion Criteria:

  • Case outside the five predefined localisations.
  • Incomplete, internally inconsistent or ambiguous schema.
  • Duplicate or near-duplicate of an existing case in the set.
  • Question not resolvable by current guidelines.

Studie plan

Dit gedeelte bevat details van het studieplan, inclusief hoe de studie is opgezet en wat de studie meet.

Hoe is de studie opgezet?

Ontwerpdetails

Cohorten en interventies

Groep / Cohort
Interventie / Behandeling
Breast cancers
Ander: Multidisciplinary tumour boards
Two independent tumour boards per localisation (10 boards in total) issue a categorical recommendation for every synthetic case. Where both boards agree, their consensus defines the reference standard; where they differ, the case-domain is classified as EQUIPOISE and analysed separately.
Ander: Frontier large language models
Five frontier LLMs (GPT-5.6, Claude Fable 5, Gemini 3.1 Pro, DeepSeek V4 Pro, Llama 4 Maverick) each receive the identical structured input for every case, three times in independent sessions, under locked prompts, versions and settings.
Lung cancers
Ander: Multidisciplinary tumour boards
Two independent tumour boards per localisation (10 boards in total) issue a categorical recommendation for every synthetic case. Where both boards agree, their consensus defines the reference standard; where they differ, the case-domain is classified as EQUIPOISE and analysed separately.
Ander: Frontier large language models
Five frontier LLMs (GPT-5.6, Claude Fable 5, Gemini 3.1 Pro, DeepSeek V4 Pro, Llama 4 Maverick) each receive the identical structured input for every case, three times in independent sessions, under locked prompts, versions and settings.
Urological cancers
Ander: Multidisciplinary tumour boards
Two independent tumour boards per localisation (10 boards in total) issue a categorical recommendation for every synthetic case. Where both boards agree, their consensus defines the reference standard; where they differ, the case-domain is classified as EQUIPOISE and analysed separately.
Ander: Frontier large language models
Five frontier LLMs (GPT-5.6, Claude Fable 5, Gemini 3.1 Pro, DeepSeek V4 Pro, Llama 4 Maverick) each receive the identical structured input for every case, three times in independent sessions, under locked prompts, versions and settings.
Digestive cancers
Ander: Multidisciplinary tumour boards
Two independent tumour boards per localisation (10 boards in total) issue a categorical recommendation for every synthetic case. Where both boards agree, their consensus defines the reference standard; where they differ, the case-domain is classified as EQUIPOISE and analysed separately.
Ander: Frontier large language models
Five frontier LLMs (GPT-5.6, Claude Fable 5, Gemini 3.1 Pro, DeepSeek V4 Pro, Llama 4 Maverick) each receive the identical structured input for every case, three times in independent sessions, under locked prompts, versions and settings.
Gynaecological cancers
Ander: Multidisciplinary tumour boards
Two independent tumour boards per localisation (10 boards in total) issue a categorical recommendation for every synthetic case. Where both boards agree, their consensus defines the reference standard; where they differ, the case-domain is classified as EQUIPOISE and analysed separately.
Ander: Frontier large language models
Five frontier LLMs (GPT-5.6, Claude Fable 5, Gemini 3.1 Pro, DeepSeek V4 Pro, Llama 4 Maverick) each receive the identical structured input for every case, three times in independent sessions, under locked prompts, versions and settings.

Wat meet het onderzoek?

Primaire uitkomstmaten

Uitkomstmaat
Maatregel Beschrijving
Tijdsspanne
Domain-level performance between LLM recommendations and the locked guidelines.
Tijdsspanne: Assessed once at central scoring, after data collection (~October 2026)
For each recommendation domain and each LLM, proportion of LLM recommendation concordant with locked guidelines
Assessed once at central scoring, after data collection (~October 2026)

Secundaire uitkomstmaten

Uitkomstmaat
Maatregel Beschrijving
Tijdsspanne
Proportion of recommendations carrying serious harm potential ( LLM and tumour boards)
Tijdsspanne: Up to October 2026
Up to October 2026
Domain-level recommendation concordance between LLM and tumour-boards
Tijdsspanne: Up to October 2026
Each recommendation domain, decomposed into the five decision domains and scored per domain on an ordinal scale (2 = complete concordance; 1 = partial concordance; 0 = discordance).
Up to October 2026
Inter-tumour board domain-level recommendation concordance
Tijdsspanne: Up to October 2026
Agreement between the two independent tumour boards scored per recommendation domain
Up to October 2026
Equipoise rate
Tijdsspanne: Up to October 2026
Proportion of case-domains where the two tumour boards give different categorical recommendations
Up to October 2026
Completeness
Tijdsspanne: Up to October 2026
Proportion of required domains addressed (LLM and tumour boards)
Up to October 2026
Missingness
Tijdsspanne: Up to October 2026
Proportion of critical omissions (LLM and tumour boards)
Up to October 2026
Intensity bias
Tijdsspanne: Up to October 2026
Proportion of recommendation corresponding to over- or under-treatment
Up to October 2026

Medewerkers en onderzoekers

Hier vindt u mensen en organisaties die betrokken zijn bij dit onderzoek.

Sponsor

Assistance Publique - Hôpitaux de Paris

Studie record data

Deze datums volgen de voortgang van het onderzoeksdossier en de samenvatting van de ingediende resultaten bij ClinicalTrials.gov. Studieverslagen en gerapporteerde resultaten worden beoordeeld door de National Library of Medicine (NLM) om er zeker van te zijn dat ze voldoen aan specifieke kwaliteitscontrolenormen voordat ze op de openbare website worden geplaatst.

Bestudeer belangrijke data

Studie start (Werkelijk)

1 mei 2026

Primaire voltooiing (Geschat)

1 oktober 2026

Studie voltooiing (Geschat)

1 oktober 2026

Studieregistratiedata

Eerst ingediend

28 juli 2026

Eerst ingediend dat voldeed aan de QC-criteria

28 juli 2026

Eerst geplaatst (Werkelijk)

31 juli 2026

Updates van studierecords

Laatste update geplaatst (Werkelijk)

31 juli 2026

Laatste update ingediend die voldeed aan QC-criteria

28 juli 2026

Laatst geverifieerd

1 juli 2026

Meer informatie

Termen gerelateerd aan deze studie

Trefwoorden

Aanvullende relevante MeSH-voorwaarden

Andere studie-ID-nummers

  • APHP261032

Plan Individuele Deelnemersgegevens (IPD)

Bent u van plan om gegevens van individuele deelnemers (IPD) te delen?

ONBESLIST

Informatie over medicijnen en apparaten, studiedocumenten

Bestudeert een door de Amerikaanse FDA gereguleerd geneesmiddel

Nee

Bestudeert een door de Amerikaanse FDA gereguleerd apparaatproduct

Nee

Deze informatie is zonder wijzigingen rechtstreeks van de website clinicaltrials.gov gehaald. Als u verzoeken heeft om uw onderzoeksgegevens te wijzigen, te verwijderen of bij te werken, neem dan contact op met register@clinicaltrials.gov. Zodra er een wijziging wordt doorgevoerd op clinicaltrials.gov, wordt deze ook automatisch bijgewerkt op onze website .

Abonneren