Usability and Clinical Effectiveness of an Interpretable Deep Learning Framework for Post-Hepatectomy Liver Failure Prediction

Usability and Clinical Effectiveness of an Interpretable Deep Learning Framework (VAE-MILP) Using Counterfactual Explanations and Layerwise Relevance Propagation Framework for Post-Hepatectomy Liver Failure Prediction

The goal of this in-silico clinical trial is to learn about the usability and clinical effectiveness of an interpretable deep learning framework (VAE-MLP) using counterfactual explanations and layerwise relevance propagation for prediction of post-hepatectomy liver failure (PHLF) in patients with hepatocellular carcinoma (HCC). The main questions it aims to answer are:

• To investigate the usability of the VAE-MLP framework for explanation of the deep learning model.
• To investigate the clinical effectiveness of VAE-MLP framework for prediction of post-hepatectomy liver failure in patients with hepatocellular carcinoma.

In the usability trial the clinicians and radiologists will be shown the counterfactual explanations and layerwise relevance propagation (LRP) plots to evaluate the usability of the framework.

In the clinical trial the clinicians and radiologists will make the prediction under two different conditions: with model explanation and without model explanation with a washout period of at least 14 days to evaluate the clinical effectiveness of the explanation framework.

Study Overview

• Status: Recruiting
• Conditions: Hepatocellular Carcinoma; Artificial Intelligence; Post-hepatectomy Liver Failure
• Intervention / Treatment: Other: The explanation of deep learning framework (VAE-MLP) , including counterfactual explanations and layerwise relevance propagation; Other: The model prediction; Other: The model prediction and the explanation of deep learning framework (VAE-MLP) , including counterfactual explanations and layerwise relevance propagation

Detailed Description

Post-hepatectomy liver failure (PHLF) is a severe complication after liver resection. It is important to develop an interpretable model for predicting PHLF in order to facilitate effective collaboration with clinicians for decision-making. Two-dimensional shear wave elastography (2D-SWE) is a liver stiffness measurement (LSM) technology that was proven to be useful in liver fibrosis staging. Therefore 2D-SWE shows the potential value for liver function assessment and PHLF prediction. 2D-SWE images display color-coded tissue stiffness map of liver parenchyma, with red representing a solid tissue (higher stiffness) and blue representing a soft tissue (lower stiffness). Routine analysis of 2D-SWE fails to fully utilize all information available in the images and also suffers from inter-observer variance in choosing the optimal quantification region.

Deep learning (DL) has demonstrated state-of-the-art performance on many medical imaging tasks such as classification or segmentation. However, despite significant progress in DL, the clinical translation of DL tools has so far been limited, partially due to a lack of interpretability of models, the so-called "black box" problem. Interpretability of DL systems is important for fostering clinical trust as well as timely correcting any faulty processes in the algorithms.

Here, the investigators present a novel interpretable DL framework (VAE-MLP) which incorporates counterfactual analysis for the explanation of 2D medical images and LRP for the explanation of feature attributions of both medical images and clinical variables.

The goal of this in-silico clinical trial is to learn about the usability and clinical effectiveness of an interpretable deep learning framework (VAE-MLP) using counterfactual explanations and layerwise relevance propagation for prediction of post-hepatectomy liver failure (PHLF) in patients with hepatocellular carcinoma. The main questions it aims to answer are:

• To investigate the usability of the the interpretable deep learning framework (VAE-MLP) for explanation of the deep learning model.
• To investigate the clinical effectiveness of the interpretable deep learning framework (VAE-MLP) for prediction of post-hepatectomy liver failure in patients with hepatocellular carcinoma.

In the usability trial the clinicians and radiologists will be shown the counterfactual explanations and layerwise relevance propagation plots of 6 examples. The score of the Likert scale of a designed questionnaire is used to evaluate the usability of the framework.

In the clinical trial the clinicians and radiologists will make the prediction under two different conditions: with model explanation and without model explanation with a washout period of at least 14 days. The accuracy, sensitivity and specificity is used to compare the clinical effectiveness of the explanation framework.

• Study Type: Observational
• Enrollment (Estimated): 80

Contacts and Locations

• Study Contact: Name: Xian Zhong; Phone Number: 86-13632460144; Email: x.zhong@maastrichtuniversity.nl

Study Locations

• China
  • Guangdong
    • Guangzhou, Guangdong, China, 510000
      • Recruiting
      • The First Affiliated Hospital of Sun Yat-sen University
      • Contact: Xian Zhong; Phone Number: 86-13632460144; Email: x.zhong@maastrichtuniversity.nl

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: Child; Adult; Older Adult
• Accepts Healthy Volunteers: No

• Sampling Method: Non-Probability Sample

Study Population

Patients who underwent curative liver resection for HCC.

Description

Inclusion Criteria:

1. patients with treatment-naive and resectable HCC;
2. performance status Eastern Cooperative Oncology Group (PS) score 0-1.

Exclusion Criteria:

1. liver resection was not performed;
2. pathological diagnosis of non-HCC;
3. failure in liver stiffness measurement defined as the elastography color map was less than 75% filled or interquartile range (IQR)/median > 30%;
4. immune-active chronic hepatitis indicated by an elevation of alanine aminotransferase (ALT) levels ≥ 2×upper limit of normal (ULN);
5. obstructive jaundice or dilated intrahepatic bile ducts with a diameter of >3 mm;
6. hypoalbuminemia, hyperbilirubinemia, or coagulopathy not related to the liver.

Study Plan

How is the study designed?

Number of groups / cohorts

1

Cohorts and Interventions

Group / Cohort

Intervention / Treatment

Patients with HCC; Patients who underwent curative liver resection for HCC in the First Affiliated Hospital of Sun Yat-Sen University in China.

Other: The explanation of deep learning framework (VAE-MLP) , including counterfactual explanations and layerwise relevance propagation; The radiologist and clinicians will be provided the model prediction results with the explanation of the model and they will fill in a questionnaire to evaluate the usability of the interpretable framework.; Other: The model prediction; The radiologist and clinicians will be provided the model prediction results without the explanation of the model and they will be asked to give their own prediction.; Other: The model prediction and the explanation of deep learning framework (VAE-MLP) , including counterfactual explanations and layerwise relevance propagation; The radiologist and clinicians will be provided the model prediction results with the explanation of the model and they will be asked to give their own prediction.

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Clinical effectiveness of the explanation framework	The accuracy, sensitivity and specificity will be compared between the prediction made with and without the explanation of the DL model to determine the clinical effectiveness of the explanation framework.	From enrollment to the end of trial at 8 weeks

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Usability of the explanation framework	The score of the Likert scale of a designed questionnaire is used to evaluate the usability of the framework. Each item is given a score from 1 to 5. Higher scores mean a better outcome.	From enrollment to the end of trial at 8 weeks

Collaborators and Investigators

• Sponsor: Maastricht University
• Collaborators: First Affiliated Hospital, Sun Yat-Sen University
• Investigators: Principal Investigator: Philippe Lambin, Maastricht University

Study record dates

Study Major Dates

• Study Start (Actual): December 10, 2023
• Primary Completion (Estimated): February 28, 2024
• Study Completion (Estimated): March 15, 2024

Study Registration Dates

• First Submitted: September 4, 2023
• First Submitted That Met QC Criteria: September 4, 2023
• First Posted (Actual): September 11, 2023

Study Record Updates

• Last Update Posted (Estimated): February 6, 2024
• Last Update Submitted That Met QC Criteria: February 5, 2024
• Last Verified: February 1, 2024

More Information

Terms related to this study

• Keywords: hepatocellular carcinoma; shear wave elastography; deep learning; post-hepatectomy liver failure; interpretability
• Additional Relevant MeSH Terms: Digestive System Diseases; Neoplasms by Histologic Type; Neoplasms; Neoplasms by Site; Adenocarcinoma; Carcinoma; Neoplasms, Glandular and Epithelial; Digestive System Neoplasms; Liver Diseases; Liver Neoplasms; Liver Failure; Hepatic Insufficiency; Carcinoma, Hepatocellular
• Other Study ID Numbers: Interpretable DL; 92059201 (Other Grant/Funding Number: National Natural Science Foundation of China)

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: UNDECIDED
• IPD Plan Description: Individual participant data (IPD) can be shared if asked

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No