Better Leukemia Diagnostics Through AI (BELUGA) (BELUGA)

A Case-Control Study To Determine The Suitability Of Artificial Intelligence For Leukemia Diagnostics

To the best of our knowledge, BELUGA will be the first prospective trial investigating the usefulness of deep learning-based hematologic diagnostic algorithms. Taking advantage of an unprecedented collection of diagnostic samples consisting of flow cytometry datapoints and digitalized blood-smears, categorization of yet undiagnosed patient samples will prospectively be compared to current state-of-the-art diagnosis at the Munich Leukemia Laboratory (hereafter MLL). In total, a collection of 25,000 digitalized blood smears and 25,000 flow cytometry datapoints will be prospectively used to train an AI-based deep neuronal network for correct categorization. Subsequently, the superiority will be challenged for the primary endpoints: sensitivity and specificity of diagnosis, most probable diagnosis, and time to diagnose. The secondary endpoints will compare the consequences regarding further diagnostic work-up and, thus, clinical decision making between routine diagnosis and AI guided diagnostics. BELUGA will set the stage for the introduction of AI-based hematologic diagnostics in a real-world setting.

Study Overview

• Status: Recruiting
• Conditions: Lymphoma; Leukemia; Hematologic Malignancy; Minimal Residual Disease; Blood Cancer
• Intervention / Treatment: Diagnostic test: Automated AI-Guided Diagnosis of Hematological Malignancies

Detailed Description

In numerous recent studies, deep neuronal networks (DNN) have been leveraged to examine the usefulness of artificial intelligence (AI)-based DNN for diagnostic purposes. In essence, they have successfully proved to recapitulate state-of-the-art diagnoses currently performed by humans.

Specifically, the use of artificial intelligence for pattern recognition showed that DNN could categorize complex and composite data points, chiefly images, with high fidelity to a specific pathogenic condition or disease. The majority of these studies are primarily based on extensive training sample collections that were categorized a priori. Subsequently, this "training" provided the necessary input to classify newly delivered specimens into the correct subgroups, frequently even outperforming independent human investigators. So far, these studies have thus provided the rationale for the use of DNN in real-world diagnostics. However, the prerequisite for using DNN in a real-world setting, where specimen sampling and analysis would need to outperform human diagnosis prospectively, would be a blinded and prospective trial. Currently, there is a lack of prospective data, therefore still challenging the notion that DNN can outperform state-of-the-art human-based diagnostic algorithms. Here we want to investigate the validity and usefulness of AI-based diagnostic capabilities prospectively in a real-world setting.

Hematologic diagnostics heavily rely on multiple methodically distinct approaches, of which phenotyping aberrant blood or bone marrow cells from affected patients represents a cornerstone for all subsequent methods, such as chromosomal or molecular genetic analyses. At the MLL, five different diagnostic pillars are required to provide diagnostic evidence for a specific malignant blood disorder faithfully: cytomorphology and immunophenotyping first, guiding more specific methods such as cytogenetics, FISH, and a diversity of molecular genetic assays.

+++ Objectives +++

Phenotyping of blood cells is primarily based on two distinct challenges; (1) the morphological appearance and abundance of specific cell types and (2) the presence of particular lineage markers detected by flow cytometry. These two methods are critical for each subsequent decision-making process and, thus ultimately, the final diagnosis. Simultaneously, these two methods are ideally suited for automated analysis by DNN due to their inherent image-based nature. This has been recently illustrated by a publication by Marr and colleagues (Matek et al., 2019; https://doi.org/10.1038/s42256-019-0101-9)

In BELUGA, we want to investigate whether the automated analysis of blood (from peripheral blood and bone marrow aspirates) smears and flow-cytometry-based analyses can provide a benefit for diagnostic quality and, ultimately, patient care. Moreover, BELUGA will provide evidence for the cooperative nature of image-based diagnostic tools for other pillars of hematologic diagnostic decision making such as genetic and molecular genetic characterization.

BELUGA, therefore, consists of three parts (A-C) (See Figure in the attached File). In A, we want to train a DNN with an unprecedented collection of blood smears and flow-cytometry-based data points collected during the course of 15 years. These samples consist of all hematological malignancies currently identified and recognized by the current WHO classification for hematologic malignancies. Due to the varying incidences of these entities, the total number of training items varies from 1,000 to 20,000 for 15 years. However, we deem this discrepancy a benefit to this trial's overall aims, because this diverse spectrum will inform us on the number of training items needed for outperforming the state-of-the-art diagnostics in cytomorphology or flow cytometry.

In part B, we will compare the overall performance of our trained DNN prospectively to new yet undiagnosed samples arriving at our laboratory (see the main section for details). The superiority of DNN based categorization will be challenged based on the pre-defined outcome parameters accuracy with respect to state-of-the-art diagnostics, mismatch-rate, and time needed to provide a diagnostic probability.

Lastly, in C, we will investigate the effects on faster and more accurate diagnostic power by leveraging our trained DNN to aid downstream diagnostic methodologies such as chromosomal analysis or panel sequencing of patient samples.

• Study Type: Observational
• Enrollment (Estimated): 25000

Contacts and Locations

• Study Contact: Name: Adam Wahida, MD; Phone Number: +49 (0)89 99017 338; Email: adam.wahida@mll.com
• Study Contact Backup: Name: Torsten Haferlach, Prof. Dr.Dr.; Phone Number: +49 (0)89 99017 100; Email: torsten.haferlach@mll.com

Study Locations

• Germany
  • Bavaria
    • Munich, Bavaria, Germany, 81377
      • Recruiting
      • MLL Munich Leukemia Laboratory
      • Contact: Adam Wahida, MD; Email: adam.wahida@mll.com
      • Contact: Torsten Haferlach, Prof.Dr.Dr.; Email: torsten.haferlach@mll.com

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: 18 years and older (Adult, Older Adult)
• Accepts Healthy Volunteers: Yes

• Sampling Method: Non-Probability Sample

Study Population

The training cohort of BELUGA consists of 50,000 annotated samples for which cytomorphological smears (25,000 samples) and immunophenotyping (25,000 samples) data points have been collected. This cohort serves as a foundation for the DNN to perform training. Our test cohort will consist of all samples for which cytomorphology and immunophenotyping will be performed for one year.

Description

Inclusion Criteria:

• Patients having been diagnosed with a suspected hematological disorder
• The suspected diagnoses constitute a primary diagnosis
• Only samples of patients min.18 years of age will be used
• Samples must suffice quality attributes which are denoted in "Exclusion Criteria"

Exclusion Criteria:

• The sample is not fit for state-of-the-art diagnosis or fails initial quality control. For quality insurance, we will exclude samples in heparin- instead of EDTA. Samples with damage due to atmospheric reasons (freeze-thaw damage or elevated temperature) will be excluded.
• Samples with too scarce material jeopardizing routine gold-standard diagnosis will be excluded.
• Bone marrow aspirates without sufficient material to assess malignant or healthy hematopoiesis.

Study Plan

How is the study designed?

Design Details

• Observational Models: Case-Control
• Time Perspectives: Prospective

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Sensitivity and Specificity of AI Guided diagnostics in Hematology	As a primary endpoint, we will examine the ability of DNN to classify disorders according to (after initial assessment disease/healthy) to the gold-standard diagnosis. The gold-standard diagnosis is defined as an integrated diagnosis, including cytomorphology, flow cytometry, cytogenetics, FISH, and molecular genetics. DNN will independently provide a bi-directional (probabilistic) diagnosis, with the most probable diagnosis. The primary analysis will include a direct comparison between the human cytomorphological examination and the pattern recognition software. Secondly, this result will be provided to downstream diagnostic departments to assess phenotypic diagnosis's usefulness for genetic characterization. We hypothesize that the turn-around time will be significantly enhanced, further providing quality at sooner timepoint.	08-01-2020 until 07-31-2021

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
comparison of clinical consequences	We will compare the clinical recommendation obtained after routine gold-standard diagnostics and after AI-guided categorization of all samples enrolled in this study	08-01-2020 until 07-31-2021
predictive diagnostic value	We will assess the predictive value of unsupervised categorization and diagnosis in comparison to gold-standard routine testing.	08-01-2020 until 07-31-2021
turn-around-time	We will measure the turn-around-time of gold-standard diagnostics in comparison to AI-guided diagnosis.	08-01-2020 until 07-31-2021
enumerate entity-specific benchmarks (e.g., blast count in leukemia) count)	We will assess secondary disease specific values determined by AI/DNN based unsupervised diagnosis versus routine testing.	08-01-2020 until 07-31-2021

Collaborators and Investigators

• Sponsor: Munich Leukemia Laboratory
• Investigators: Principal Investigator: Wolfgang Kern, Prof. Dr., MLL Munich Leukemia Laboratory

Publications and helpful links

General Publications

• Zhao M, Mallesh N, Hollein A, Schabath R, Haferlach C, Haferlach T, Elsner F, Luling H, Krawitz P, Kern W. Hematologist-Level Classification of Mature B-Cell Neoplasm Using Deep Learning on Multiparameter Flow Cytometry Data. Cytometry A. 2020 Oct;97(10):1073-1080. doi: 10.1002/cyto.a.24159. Epub 2020 Jun 9.

Helpful Links

• Related Info
• WHO Classification of hematological malignancies according to Swerdlow et al.
• Related Info
• Related Info

Study record dates

Study Major Dates

• Study Start (Actual): January 5, 2020
• Primary Completion (Estimated): July 31, 2024
• Study Completion (Estimated): July 31, 2025

Study Registration Dates

• First Submitted: July 6, 2020
• First Submitted That Met QC Criteria: July 9, 2020
• First Posted (Actual): July 10, 2020

Study Record Updates

• Last Update Posted (Actual): October 19, 2023
• Last Update Submitted That Met QC Criteria: October 17, 2023
• Last Verified: October 1, 2023

More Information

Terms related to this study

• Keywords: artificial intelligence; hematology; laboratory medicine; AI-based diagnostics; deep neuronal networks
• Additional Relevant MeSH Terms: Pathologic Processes; Neoplasms by Histologic Type; Neoplasms; Neoplasms by Site; Hematologic Diseases; Neoplastic Processes; Hematologic Neoplasms; Leukemia; Neoplasm, Residual
• Other Study ID Numbers: MLL_001

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: UNDECIDED

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No