MIRA Clinical Learning Environment (MIRACLE): Lung (MIRACLE)

The goal of this quality improvement (QI) study is to develop automated clinical pipelines to implement machine learning models in the care pathway of lung cancer patients. The main questions it aims to answer are:

• Can model-prompted risk classifications be incorporated into clinician workflows to enable informed clinical decision-making?
• What are clinicians' perceptions of the information from model outputs, and do they change their decision about data already available to them as a result of the model-prompted risk classification (i.e., to re-review or further assess patients identified by the models as being higher risk)?

Participating radiation oncologists will receive the risk prediction from the model and be asked to complete a survey to give feedback on how they used the prediction in their decision-making.

Study Overview

• Status: Recruiting
• Conditions: Lung Cancer
• Intervention / Treatment: Other: Application of ILD prediction machine learning model to planning imaging; Other: Routine, automatic presentation of ILD risk level for evaluation by the clinician.; Other: Application of SGR machine learning model to diagnostic and planning imaging; Other: Routine estimation of tumor specific growth rate (SGR) for lesions being considered for radiation therapy presented to clinician.; Other: Application of CBCT machine learning model to on-treatment imaging; Other: Routine monitoring of lung density changes during the course of treatment presented to clinician.

Detailed Description

Novel data science and imaging-based methods to personalize care are being identified retrospectively and explored at many centers. Unfortunately, most of these methods require significant manual intervention to apply to any given patient situation and are difficult to deploy in a timely fashion to affect patient treatment decisions. Clinical implementation of data science research will require automated pipelines that are tied into the entire treatment pathway in ways that facilitate real-time data analysis and enable translational research.

The current process for clinical/translational researchers within Princess Margaret Hospital (PM)/University Health Network (UHN) to analyze imaging data involves extensive manual curation consisting of interactions with electronic databases and analysis tools to: identify patients with imaging data; collect that data; delineate targets of interest manually (minutes-to-hours per patient); analyze targets based on manually-selected images; and then correlate the analyzed images with clinical information sources (e.g. outcomes or correlative data). Thus, projects with large patient numbers often encounter insurmountable obstacles that limit research productivity.

MIRA (an in-house developed programming toolkit) solves a common problem for all researchers at PM/UHN studying diagnostic, radiotherapy treatment planning, and/or on-treatment imaging by providing a consistent automated analysis environment for these data. MIRA also enhances ethics approved studies with direct linkage to real-time clinical data including diagnostic imaging via collaboration with the Joint Department of Medical Imaging, radiation oncology treatment planning information, and daily radiation oncology on-treatment imaging. The MIRA Clinical Learning Environment (MIRACLE) quality improvement project intends to use the MIRA platform to develop automated clinical pipelines to address three specific study aims:

To identify lung cancer patients with undiagnosed underlying inflammatory lung disease (ILD) from pre-treatment diagnostic images

To estimate individual patients' tumor growth-rate between diagnostic and treatment planning images (specific growth-rate, SGR)

To provide each patient with an estimate of dynamic radiation treatment toxicity risk using radiation treatment planning information, while continuously updating risk estimates using daily cone-beam computed tomography (CBCT) images routinely obtained before each radiation treatment.

MIRACLE is linked safely to active clinical data repositories and has the potential to directly impact daily cancer treatment decisions by making existing imaging data findable, rapidly accessible, interoperable, and reusable for both clinical and research analysis by end users including the physicians caring for lung cancer patients, and cancer researchers. This facilitates evaluation of novel imaging research findings in large patient numbers for clinical and research use. The MIRACLE project's goal is to specifically demonstrate the clinical implementation feasibility of automatically linking and analyzing clinical imaging data alongside clinical outcome; ultimately, helping to deliver value-based healthcare via better patient selection (ILD/SGR) and monitoring/adjusting treatment to decrease toxicity (CBCT).

Feedback from the participating radiation oncologists will be gathered to assess the feasibility and effectiveness of showing patient-specific insights for inflammatory lung disease (ILD), a specific tumour growth rate greater than 0.04 (SGR) and cone-beam computed tomography system (CBCT) changes to clinicians at the point of care. The analysis will help to understand clinicians' perceptions of information provided to them from the model regarding ILD prediction, SGR and lung density changes over the QI period and whether clinicians changed their decision about data already available to them as a result of the model-prompted risk classification (i.e., to re-review or further assess patients for ILD, SGR and CBCT changes based on those patients highlighted by the model as being higher risk).

• Study Type: Observational
• Enrollment (Anticipated): 1000

Contacts and Locations

• Study Contact: Name: Andrew Hope, MD, FRCPC; Phone Number: 416-946-2124; Email: Andrew.Hope@rmp.uhn.ca

Study Locations

• Canada
  • Ontario
    • Toronto, Ontario, Canada
      • Recruiting
      • Princess Margaret Hospital
      • Contact: Andrew Hope, MD, FRCPC

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: 18 years and older (Adult, Older Adult)
• Accepts Healthy Volunteers: No
• Genders Eligible for Study: All

• Sampling Method: Non-Probability Sample

Study Population

All lung cancer patients 18 years or older receiving radiation therapy (RT) will be included in the study between 2000 until the end of the study. The type of RT received by the patient will further determine which of the three aims their data is suitable for, and therefore which clinical trial methodology will be used.

Description

Inclusion Criteria:

• Diagnosed with lung cancer stage I-IV and planned for treatment with radiotherapy at Princess Margaret hospital. The three aims of this project have specific inclusion criteria as follows.
• Aim 1 ILD: All lung cancer patients receiving RT.
• Aim 2 SGR: Node negative lung cancer patients receiving stereotactic body RT.
• Aim 3 CBCT: Node positive lung cancer patients receiving standard RT.

Exclusion Criteria:

• No exclusion criteria

Study Plan

How is the study designed?

Design Details

• Observational Models: Cohort
• Time Perspectives: Prospective

Number of groups / cohorts

6

Cohorts and Interventions

Group / Cohort	Intervention / Treatment
ILD Silent Mode; The ILD model will be run on patients undergoing routine treatment planning imaging where the notification is sent to the study team for a period of one month to ensure the pipeline is operating as intended.	Other: Application of ILD prediction machine learning model to planning imaging; The ILD prediction machine learning model will be applied to the treatment planning imaging of lung cancer patients receiving radiation therapy (RT). The model will score the risk as high risk or low risk for having underlying ILD.
ILD Prospective Mode; Following successful silent mode, the ILD model will be run on patients undergoing routine treatment planning imaging and the notifications will be sent to the treating physician to incorporate into their workflow.	Other: Application of ILD prediction machine learning model to planning imaging; The ILD prediction machine learning model will be applied to the treatment planning imaging of lung cancer patients receiving radiation therapy (RT). The model will score the risk as high risk or low risk for having underlying ILD.; Other: Routine, automatic presentation of ILD risk level for evaluation by the clinician.; Participating clinicians will be provided with an ILD risk estimate for all lung cancer patients receiving RT who are deemed potentially high-risk based on the model. In these cases, the clinician will receive an email identifying the patient medical record number (MRN) and 'potential high-risk for ILD' flag. Clinicians will then be able to decide whether, based on the information, they want to reassess the patient for ILD prior to starting treatment. Clinicians will also be presented with a short survey each time they are sent an email for a potential high-risk for ILD case so the study team can better understand how that information was used, if at all.
SGR Silent Mode; The SGR model will be run on patients undergoing routine treatment planning imaging where the notification is sent to the study team for a period of one month to ensure the pipeline is operating as intended.	Other: Application of SGR machine learning model to diagnostic and planning imaging; The SGR machine learning model will be applied to the imaging of lung cancer patients with node negative lung cancer receiving stereotactic RT. The automatic calculation will compare target lesions on the patient's diagnostic images with those same lesions on treatment planning images.
SGR Prospective Mode; Following successful silent mode, the SGR model will be run on patients undergoing routine treatment planning imaging and the notifications will be sent to the treating physician to incorporate into their workflow.	Other: Application of SGR machine learning model to diagnostic and planning imaging; The SGR machine learning model will be applied to the imaging of lung cancer patients with node negative lung cancer receiving stereotactic RT. The automatic calculation will compare target lesions on the patient's diagnostic images with those same lesions on treatment planning images.; Other: Routine estimation of tumor specific growth rate (SGR) for lesions being considered for radiation therapy presented to clinician.; Participating clinicians will be provided with an SGR calculation for each lung cancer patient with node negative lung cancer receiving stereotactic RT. This SGR calculation will be presented to clinicians, who will then be able to decide, based on the information, how they want to address and track a patient's overall survival and failure free survival. Clinicians will also be presented with a short survey each time they are provided with a patient's SGR calculation so the study team can better understand how that information was used, if at all.
CBCT Silent Mode; The CBCT model will be run on patients receiving routine on-treatment imaging where the notification is sent to the study team for a period of one month to ensure the pipeline is operating as intended.	Other: Application of CBCT machine learning model to on-treatment imaging; The CBCT machine learning model will be applied to on-treatment imaging as part of routine care for patients with node positive lung cancer receiving standard RT. An indicator of lung density changes will be calculated automatically by comparing cone beam CTs (CBCTs) completed prior to each treatment.
CBCT Prospective Mode; Following successful silent mode, The CBCT model will be run on patients receiving routine on-treatment imaging and the notifications will be sent to the treating physician to incorporate into their workflow.	Other: Application of CBCT machine learning model to on-treatment imaging; The CBCT machine learning model will be applied to on-treatment imaging as part of routine care for patients with node positive lung cancer receiving standard RT. An indicator of lung density changes will be calculated automatically by comparing cone beam CTs (CBCTs) completed prior to each treatment.; Other: Routine monitoring of lung density changes during the course of treatment presented to clinician.; Participating clinicians will be provided with a daily indicator of lung density changes for each patient with node positive lung cancer receiving standard RT. This measurement will be presented to the clinical team, who will then be able to decide, based on the information, how they want to address and track relevant outcomes such as pneumonitis. Additionally, this information may provide the clinical team with feedback about the lung reaction occurring as a result of treatment. Density changes will be documented and monitored for future validation studies, which are outside of the scope of this application.

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Rates of true positive diagnosis of ILD increase with high/low patient risk predictions being made available to clinicians.	An expert review of the cases and chart review will be correlated with survey responses to determine whether the rate of true positive cases were impacted by the implementation of the MIRACLE pathways.	January 2022 - December 2023
Previously difficult-to-assess information are made available during the clinical workflow as an easily accessible information source available to clinicians	Clinicians will provide feedback on the communication of the predictions, the integration into their clinical workflow and timeliness of receiving the predictions in order to incorporate into their decision-making.	January 2022 - December 2023
Radiation oncologists use predictions provided from the model to support their clinical decision-making.	Clinicians will indicate in the survey their perceptions of accuracy and usefulness of the predictions and whether they have incorporated the predictions into their decision-making.	January 2022 - December 2023

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Additional expertise is focused on patients identified as being higher risk for ILD, SGR > 0.04, or possible pneumonitis.	Clinicians will indicate in the survey whether they have gone back and reassessed or flagged patients in cases where the model identifies a possible high-risk for ILD, SGR > 0.04, or pneumonitis.	January 2022 - December 2023

Collaborators and Investigators

• Sponsor: University Health Network, Toronto
• Collaborators: University of Toronto
• Investigators: Principal Investigator: Hope, University Health Network, Toronto

Publications and helpful links

General Publications

• Mori Y, Kudo SE, Misawa M, Saito Y, Ikematsu H, Hotta K, Ohtsuka K, Urushibara F, Kataoka S, Ogawa Y, Maeda Y, Takeda K, Nakamura H, Ichimasa K, Kudo T, Hayashi T, Wakamura K, Ishida F, Inoue H, Itoh H, Oda M, Mori K. Real-Time Use of Artificial Intelligence in Identification of Diminutive Polyps During Colonoscopy: A Prospective Study. Ann Intern Med. 2018 Sep 18;169(6):357-366. doi: 10.7326/M18-0249. Epub 2018 Aug 14.
• Topol EJ. High-performance medicine: the convergence of human and artificial intelligence. Nat Med. 2019 Jan;25(1):44-56. doi: 10.1038/s41591-018-0300-7. Epub 2019 Jan 7.
• Steiner DF, MacDonald R, Liu Y, Truszkowski P, Hipp JD, Gammage C, Thng F, Peng L, Stumpe MC. Impact of Deep Learning Assistance on the Histopathologic Review of Lymph Nodes for Metastatic Breast Cancer. Am J Surg Pathol. 2018 Dec;42(12):1636-1646. doi: 10.1097/PAS.0000000000001151.
• Li C, Jing B, Ke L, Li B, Xia W, He C, Qian C, Zhao C, Mai H, Chen M, Cao K, Mo H, Guo L, Chen Q, Tang L, Qiu W, Yu Y, Liang H, Huang X, Liu G, Li W, Wang L, Sun R, Zou X, Guo S, Huang P, Luo D, Qiu F, Wu Y, Hua Y, Liu K, Lv S, Miao J, Xiang Y, Sun Y, Guo X, Lv X. Development and validation of an endoscopic images-based deep learning model for detection with nasopharyngeal malignancies. Cancer Commun (Lond). 2018 Sep 25;38(1):59. doi: 10.1186/s40880-018-0325-9.
• Dascalu A, David EO. Skin cancer detection by deep learning and sound analysis algorithms: A prospective clinical study of an elementary dermoscope. EBioMedicine. 2019 May;43:107-113. doi: 10.1016/j.ebiom.2019.04.055. Epub 2019 May 14.
• Phillips M, Marsden H, Jaffe W, Matin RN, Wali GN, Greenhalgh J, McGrath E, James R, Ladoyanni E, Bewley A, Argenziano G, Palamaras I. Assessment of Accuracy of an Artificial Intelligence Algorithm to Detect Melanoma in Images of Skin Lesions. JAMA Netw Open. 2019 Oct 2;2(10):e1913436. doi: 10.1001/jamanetworkopen.2019.13436. Erratum In: JAMA Netw Open. 2019 Nov 1;2(11):e1916430.
• Abramoff MD, Lavin PT, Birch M, Shah N, Folk JC. Pivotal trial of an autonomous AI-based diagnostic system for detection of diabetic retinopathy in primary care offices. NPJ Digit Med. 2018 Aug 28;1:39. doi: 10.1038/s41746-018-0040-6. eCollection 2018.
• Gulshan V, Rajan RP, Widner K, Wu D, Wubbels P, Rhodes T, Whitehouse K, Coram M, Corrado G, Ramasamy K, Raman R, Peng L, Webster DR. Performance of a Deep-Learning Algorithm vs Manual Grading for Detecting Diabetic Retinopathy in India. JAMA Ophthalmol. 2019 Sep 1;137(9):987-993. doi: 10.1001/jamaophthalmol.2019.2004.
• Kanagasingam Y, Xiao D, Vignarajan J, Preetham A, Tay-Kearney ML, Mehrotra A. Evaluation of Artificial Intelligence-Based Grading of Diabetic Retinopathy in Primary Care. JAMA Netw Open. 2018 Sep 7;1(5):e182665. doi: 10.1001/jamanetworkopen.2018.2665.
• Lee H, Yune S, Mansouri M, Kim M, Tajmir SH, Guerrier CE, Ebert SA, Pomerantz SR, Romero JM, Kamalian S, Gonzalez RG, Lev MH, Do S. An explainable deep-learning algorithm for the detection of acute intracranial haemorrhage from small datasets. Nat Biomed Eng. 2019 Mar;3(3):173-182. doi: 10.1038/s41551-018-0324-9. Epub 2018 Dec 17.
• Glick D, Lyen S, Kandel S, Shapera S, Le LW, Lindsay P, Wong O, Bezjak A, Brade A, Cho BCJ, Hope A, Sun A, Giuliani M. Impact of Pretreatment Interstitial Lung Disease on Radiation Pneumonitis and Survival in Patients Treated With Lung Stereotactic Body Radiation Therapy (SBRT). Clin Lung Cancer. 2018 Mar;19(2):e219-e226. doi: 10.1016/j.cllc.2017.06.021. Epub 2017 Jul 10.
• Atallah S, Cho BC, Allibhai Z, Taremi M, Giuliani M, Le LW, Brade A, Sun A, Bezjak A, Hope AJ. Impact of pretreatment tumor growth rate on outcome of early-stage lung cancer treated with stereotactic body radiation therapy. Int J Radiat Oncol Biol Phys. 2014 Jul 1;89(3):532-8. doi: 10.1016/j.ijrobp.2014.03.003.

Helpful Links

• U.S. Department of Health and Human Services. Common Terminology Criteria for Adverse Events (CTCAE) v5.0. Published November 27, 2017. Accessed February 4, 2021.

Study record dates

Study Major Dates

• Study Start (Actual): January 1, 2022
• Primary Completion (Anticipated): December 31, 2023
• Study Completion (Anticipated): December 31, 2023

Study Registration Dates

• First Submitted: January 9, 2023
• First Submitted That Met QC Criteria: January 9, 2023
• First Posted (Estimate): January 19, 2023

Study Record Updates

• Last Update Posted (Estimate): January 19, 2023
• Last Update Submitted That Met QC Criteria: January 9, 2023
• Last Verified: January 1, 2023

More Information

Terms related to this study

• Keywords: Artificial Intelligence; Quality Improvement; Machine Learning; Clinical Implementation
• Other Study ID Numbers: QI ID#: 21-0193

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: Yes
• IPD Plan Description: The individual patient data collected during the trial after de-identification will be made available 1-2 years after trial publication and ending 4 years after trial publication. The study protocol will be available. Researchers who provide a methodologically sound proposal will be granted access to the data made available to achieve the approved aims. To gain access to the data location, data requestors will need to sign a data access agreement.
• IPD Sharing Time Frame: 1-2 years following publication.
• IPD Sharing Access Criteria: Researchers who provide a methodologically sound proposal will be granted access to the data made available to achieve the approved aims. To gain access to the data location, data requestors will need to sign a data access agreement.
• IPD Sharing Supporting Information Type: Study Protocol; Analytic Code

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No