Prospective Validation of Liver Cancer Risk Computation (LIRIC) Models

Prospective Validation of Liver Cancer Risk Computation (LIRIC) Models on Multicenter EHR Data

The goal of this prospective observational cohort study is to validate previously developed Hepatocellular Carcinoma (HCC) risk prediction algorithms, the Liver Risk Computation (LIRIC) models, which are based on electronic health records.

The main questions it aims to answer are:

• Will our retrospectively developed general population LIRIC models, developed on routine EHR data, perform similarly when prospectively validated, and reliably and accurately predict HCC in real-time?
• What is the average time from model deployment and risk prediction, to the date of HCC development and what is the stage of HCC at diagnosis?

The risk model will be deployed on data from individuals eligible for the study. Each individual will be assigned a risk score and tracked over time to assess the model's discriminatory performance and calibration.

Study Overview

• Status: Active, not recruiting
• Conditions: Predictive Cancer Model
• Intervention / Treatment: Other: Liver Risk Computation Model (LIRIC); Other: Liver Risk Computation Model (LIRIC)_cirrhosis; Other: Liver Risk Computation Model (LIRIC)_no_cirrhosis

Detailed Description

We will conduct a prospective observational cohort study, separately deploying three separate LIRIC models (the general population, cirrhosis, and no_cirrhosis models) on retrospective de-identified EHR data of 44 HCOs in the USA, using the TriNetX federated network platform. LIRIC will generate a risk score for each individual. All risk-stratified individuals will be prospectively, electronically followed for up to 3-years to assess the primary end-point of HCC development. At the end of this period, model discrimination will be assessed, using the following metrics: AUROC, sensitivity, specificity, PPV/NPV. Risk scores generated by the model will be divided into quantiles. For each quantile, we will evaluate the following: number of individuals in each quantile, number of HCC cases, PPV, NNS, SIR. Model calibration will be used for assessing the accuracy of estimates, based on the estimated to observed number of events. The model will dynamically re-evaluate all individual data every 6 months, re-classifying individuals (as needed).

• Study Type: Observational
• Enrollment (Actual): 6000000

Contacts and Locations

Study Locations

• United States
  • Massachusetts
    • Boston, Massachusetts, United States, 02115
      • Beth Israel Deaconess Medical Center

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: Adult; Older Adult
• Accepts Healthy Volunteers: Yes

• Sampling Method: Non-Probability Sample

Study Population

The cohort will be selected from 44 eligible HCOs comprised of community hospitals, outpatient clinics and academic medical centers from across the US.

Description

We will utilize the following criteria for all 3 models:

Inclusion criteria:

• Male and females age ≥40 years from all US HCOs available on the platform
• at least at least 2 clinical encounters to the HCO, within the last year, before the study start date

Exclusion Criteria:

• Personal history of HCC or current HCC (ICD-9: 155.0; ICD-10: C22.0)
• Age below 40. The same dataset will be utilized for the non-cirrhosis validation, with exclusion of all cases with cirrhosis (ICD-9: 571.2, 571.5; ICD-10: K70, K70.3, K70.30, K70.31, K74, K74.0, K74.6, K74.60, K74.69). For the cirrhosis validation, we will include only patients with the above cirrhosis codes.

Study Plan

How is the study designed?

Number of groups / cohorts

3

Cohorts and Interventions

Group / Cohort	Intervention / Treatment
prospective general population cohort; Males and females age >= 40 years, without a personal history of HCC or current HCC and at least two clinical visits to their HCO, within the last year, before the study start date.	Other: Liver Risk Computation Model (LIRIC); A neural network model (LIRIC-NN) and a logistic regression model (LIRIC-LR) that use routinely collected EHR data to stratify individuals into HCC risk groups for the general population
Prospective cirrhosis population cohort; Males and females age >= 40 years, with liver cirrhosis and without a personal history of HCC or current HCC, that have at least two clinical visits to their HCO, within the last year, before the study start date.	Other: Liver Risk Computation Model (LIRIC)_cirrhosis; A neural network model (LIRIC-NN) and a logistic regression model (LIRIC-LR) that use routinely collected EHR data to stratify individuals into HCC risk groups for the population with liver cirrhosis
Prospective no_cirrhosis population cohort; Males and females age >= 40 years, without a personal history of HCC or current HCC and without a diagnosis of liver cirrhosis, that have at least two clinical visits to their HCO, within the last year, before the study start date.	Other: Liver Risk Computation Model (LIRIC)_no_cirrhosis; neural network model (LIRIC-NN) and a logistic regression model (LIRIC-LR) that use routinely collected EHR data to stratify individuals into HCC risk groups for the population without liver cirrhosis

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Area under the receiver operating characteristic curve (AUROC) of LIRIC for all groups stratified	To assess the discriminatory performance of LIRIC for prospective identification of high-risk individuals for HCC development. ROCs and AUROC numbers will be calculated for the whole population and groups stratified by age, sex, race, and geographical location.	6 months from index date, at 1 year, 2 years and 3 years
Calibration of LIRIC for all groups stratified	To assess how well the risk prediction by LIRIC aligns with observed risk without recalibration. Calibration plots will be created for the whole population and groups stratified by age, sex, race, and geographical location.	6 months from index date, at 1 year, 2 years and 3 years
Performance metrics for LIRIC model risk quantiles	To evaluate the sensitivity, specificity, number of individuals/number of HCC cases, PPV, NNS in each predicted risk quantile for multiple risk thresholds	6 months from index date, at 1 year, 2 years and 3 years

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Timing of incident HCC occurrence	To evaluate how long in advance before HCC occurrence should be expected for LIRIC models to make high-risk predictions based on different thresholds for high-risk. Distribution plots of the date of HCC incidence for multiple risk thresholds will be created.	6 months from index date, at 1 year, 2 years and 3 years
Tumor stage at HCC diagnosis	TNM staging at HCC diagnosis	6 months from index date, at 1 year, 2 years and 3 years

Collaborators and Investigators

• Sponsor: Beth Israel Deaconess Medical Center
• Collaborators: Massachusetts Institute of Technology; TriNetX, LLC

Study record dates

Study Major Dates

• Study Start (Actual): April 1, 2023
• Primary Completion (Estimated): March 31, 2026
• Study Completion (Estimated): March 31, 2027

Study Registration Dates

• First Submitted: November 15, 2023
• First Submitted That Met QC Criteria: November 15, 2023
• First Posted (Actual): November 20, 2023

Study Record Updates

• Last Update Posted (Actual): November 20, 2023
• Last Update Submitted That Met QC Criteria: November 15, 2023
• Last Verified: November 1, 2023

More Information

Terms related to this study

• Additional Relevant MeSH Terms: Digestive System Diseases; Neoplasms; Neoplasms by Site; Digestive System Neoplasms; Liver Diseases; Liver Neoplasms; Hematinics; Liver Extracts
• Other Study ID Numbers: Nov2023Trial

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No