Deep Learning-Based Analysis of Colorectal Cancer Pathology Images: An Innovative Approach for Predicting Colorectal Cancer Subtypes

AI-Powered Copilots for Precision Diagnosis and Surgical Assessment of Histological Growth Patterns in Resectable Colorectal Liver Metastases: A Prospective Study

Colorectal cancer (CRC) is a leading cause of mortality in China, with metastasis significantly contributing to poor outcomes. Histopathological growth patterns (HGPs) in colorectal liver metastasis (CRLM) provide vital prognostic insights, yet the limited number of pathologists highlights the need for auxiliary diagnostic tools. Recent advancements in artificial intelligence (AI) have demonstrated potential in enhancing diagnostic precision, prompting the development of specialized AI models like COFFEE to improve the classification and management of HGPs in CRLM patients. This study aims to develop and validate a Transformer-based deep learning model, COFFEE, for the classification of colorectal cancer subtypes using whole slide images (WSIs) from patients diagnosed with colorectal cancer liver metastasis. The model is pre-trained using self-supervised learning (DINO) on WSIs from the TCGA-COAD cohort, utilizing a Vision Transformer (ViT) architecture to extract 384-dimensional feature vectors from 256×256 pixel patches. The COFFEE model integrates a Transformer-based Multiple Instance Learning (TransMIL) framework, incorporating multi-head self-attention and Pyramid Position Encoding Generator (PPEG) modules to aggregate spatial and morphological information. The study includes training, testing, and prospective validation cohorts and evaluates the performance of the model in both binary and multi-class classification settings, as well as its potential to assist pathologists in clinical workflows.

Study Overview

• Status: Completed
• Conditions: Colorectal Liver Metastasis (CRLM); Histopathological Growth Patterns (HGPs); Artificial Intelligence (AI) in Diagnosis; Vision Transformer (ViT); Desmoplastic Classification
• Intervention / Treatment: Procedure: CRLM surgery

• Study Type: Observational
• Enrollment (Actual): 431

Contacts and Locations

Study Locations

• China
  • Guangdong
    • Guangzhou, Guangdong, China, 510655
      • Ethics Committee of the Sixth Affiliated Hospital of Sun Yat-sen University

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: Adult; Older Adult
• Accepts Healthy Volunteers: No

• Sampling Method: Non-Probability Sample

Study Population

The study involved 431 patients with colorectal cancer liver metastasis, all undergoing surgery at the Sixth Affiliated Hospital of Sun Yat-sen University. The cohort consisted of 297 patients in the training set and 104 patients in the testing set.

Description

Inclusion Criteria:

1. Patients diagnosed with colorectal cancer liver metastasis (CRLM) undergoing surgical treatment;
2. The maximum diameter of resected metastatic lesions should be ≥ 2 cm;
3. Availability of pathology slides along with baseline clinical, biological, and pathological features.

Exclusion Criteria:

1. Tissue sections obtained from biopsy specimens;
2. Absence of viable tumor tissue in metastatic lesions;
3. Lesions previously treated with ablation followed by surgical resection, resulting in inadequate tissue slide quality.

Study Plan

How is the study designed?

Number of groups / cohorts

3

Cohorts and Interventions

Group / Cohort	Intervention / Treatment
Surgical pathology slides from the SAHSYSU, 1,994 WSIs from 297 slides dated July 3, 2013.; This group includes 297 patients with colorectal cancer liver metastasis (CRLM), from which 1,994 whole slide images (WSIs) were collected. These slides were used for developing and testing the COFFEE AI model for histopathological growth pattern (HGP) classification, providing valuable insights for tumor characterization and prognosis.	Procedure: CRLM surgery; Surgical resection of colorectal cancer liver metastasis (CRLM) involves the removal of metastatic lesions from the liver. This procedure is aimed at improving survival rates and reducing tumor burden in patients diagnosed with CRLM. The resection is performed to treat liver metastasis, and clinical outcomes, such as progression-free survival (PFS) and overall survival (OS), are assessed post-surgery to determine treatment efficacy.
Surgical pathology slides from the SAHSYSU , 972 WSIs from 104 patients dated April 21, 2023.; This cohort contains 104 patients diagnosed with CRLM. 972 WSIs were collected to validate the COFFEE model on a more recent dataset, evaluating the model's performance in both binary and four-class HGP classifications.	Procedure: CRLM surgery; Surgical resection of colorectal cancer liver metastasis (CRLM) involves the removal of metastatic lesions from the liver. This procedure is aimed at improving survival rates and reducing tumor burden in patients diagnosed with CRLM. The resection is performed to treat liver metastasis, and clinical outcomes, such as progression-free survival (PFS) and overall survival (OS), are assessed post-surgery to determine treatment efficacy.
Surgical pathology slides from the SAHSYSU, 114 WSIs from 30 patients dated 2024.; This prospective cohort consists of 30 patients with CRLM, from which 114 WSIs were obtained in 2024. The cohort was used to assess the clinical applicability of the COFFEE AI model through a prospective trial, comparing the diagnostic performance of pathologists with and without AI assistance.	Procedure: CRLM surgery; Surgical resection of colorectal cancer liver metastasis (CRLM) involves the removal of metastatic lesions from the liver. This procedure is aimed at improving survival rates and reducing tumor burden in patients diagnosed with CRLM. The resection is performed to treat liver metastasis, and clinical outcomes, such as progression-free survival (PFS) and overall survival (OS), are assessed post-surgery to determine treatment efficacy.

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Classification Accuracy (%) of the COFFEE AI Model in Binary Identification of Histopathological Growth Patterns (HGPs) in CRLM Using Whole Slide Images	This outcome measures the diagnostic classification accuracy of the COFFEE AI model in detecting histopathological growth patterns (HGPs) in patients with colorectal cancer liver metastasis (CRLM). Accuracy is defined as the proportion of correctly predicted HGP labels compared to the ground truth labels determined by consensus of expert pathologists. The analysis includes binary classification (desmoplastic vs. non-desmoplastic). Accuracy will be calculated as: Accuracy = Total number of predictions / Number of correct predictions×100%. The outcome will be assessed using digital whole slide images obtained from liver metastasis specimens collected during surgery. Model performance will be evaluated 6 months post-surgery in a prospective validation cohort.	6 months post-surgery (for prospective cohort)

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Classification Accuracy (%) of the COFFEE AI Model in Multi-Class Identification of Histopathological Growth Patterns (HGPs) in CRLM Using Whole Slide Images	This outcome measures the diagnostic classification accuracy of the COFFEE AI model in detecting histopathological growth patterns (HGPs) in patients with colorectal cancer liver metastasis (CRLM). Accuracy is defined as the proportion of correctly predicted HGP labels compared to the ground truth labels determined by consensus of expert pathologists. The analysis includes four-class classification (desmoplastic, replacement, pushing, and mixed). Accuracy will be calculated as: Accuracy = Total number of predictions / Number of correct predictions×100%. The outcome will be assessed using digital whole slide images obtained from liver metastasis specimens collected during surgery. Model performance will be evaluated 6 months post-surgery in a prospective validation cohort.	6 months post-surgery (for prospective cohort)

Other Outcome Measures

Outcome Measure	Measure Description	Time Frame
Progression-Free Survival (PFS, in months) in Colorectal Cancer Liver Metastasis (CRLM) Patients Stratified by AI-based Histopathological Growth Pattern (HGP) Classification	This outcome evaluates the association between AI-based HGP classification (desmoplastic and non-desmoplastic) and progression-free survival (PFS) in patients with colorectal cancer liver metastasis (CRLM) following curative-intent resection. PFS is defined as the time from surgery to disease progression or death from any cause. Kaplan-Meier analysis will be used to estimate PFS for each HGP group, with comparisons by log-rank test. Multivariate Cox regression models will assess the prognostic value of HGPs, adjusting for clinical covariates (e.g., age, sex, metastasis number/size, chemotherapy, margin status, tumor burden score). Hazard ratios with 95% confidence intervals will be reported. Model assumptions will be tested and adjusted if necessary.	Up to 3 years post-surgery
Overall Survival (OS, in months) in Colorectal Cancer Liver Metastasis (CRLM) Patients Stratified by AI-based Histopathological Growth Pattern (HGP) Classification	This outcome evaluates the association between AI-based HGP classification (desmoplastic and non-desmoplastic) and overall survival (OS) in patients with colorectal cancer liver metastasis (CRLM) following curative-intent resection. OS is defined as the time from surgery to death from any cause. Kaplan-Meier analysis will be used to estimate OS for each HGP group, with comparisons by log-rank test. Multivariate Cox regression models will assess the prognostic value of HGPs, adjusting for clinical covariates (e.g., age, sex, metastasis number/size, chemotherapy, margin status, tumor burden score). Hazard ratios with 95% confidence intervals will be reported. Model assumptions will be tested and adjusted if necessary.	Up to 3 years post-surgery
Time to Diagnosis (in minutes) by Pathologists With and Without AI-Assisted COFFEE Model in CRLM HGP Classification	This outcome assesses the impact of the AI-assisted COFFEE model on diagnostic efficiency by comparing the time required by pathologists to classify histopathological growth patterns (HGPs) of colorectal cancer liver metastasis (CRLM), with and without COFFEE assistance. The metric is the time (minutes) from slide review start to final diagnosis, measured for each pathologist using a standardized digital whole slide image platform. The comparison includes two arms: the AI-assisted diagnosis arm, where junior pathologists use COFFEE as a decision-support tool, and the conventional diagnosis arm, where pathologists perform manual classification based on visual histopathological assessment. All participants review the same set of slides in randomized order, and diagnostic time is logged by the viewing software. Descriptive statistics (median, IQR) will be reported.	During the prospective trial period (6 months)
Diagnostic Accuracy (percentage of correct classifications) of Pathologists With and Without AI-Assisted COFFEE Model in CRLM HGP Classification	This outcome evaluates the diagnostic accuracy of pathologists in classifying histopathological growth patterns (HGPs) of colorectal cancer liver metastasis (CRLM), comparing AI-assisted versus conventional diagnostic workflows. Accuracy is defined as the proportion of correctly classified whole slide images (WSIs) relative to a gold-standard consensus diagnosis by expert gastrointestinal pathologists. Each pathologist will independently classify the same set of CRLM WSIs under two conditions: with AI assistance (COFFEE model) and without AI assistance (manual assessment). Classification will be evaluated for both binary HGP categories (desmoplastic vs. non-desmoplastic) and four-class HGP categories (desmoplastic, replacement, pushing, mixed). Accuracy will be calculated as: Accuracy = Total number of predictions / Number of correct predictions×100%.	During the prospective trial period (6 months)

Collaborators and Investigators

• Sponsor: Sun Yat-Sen Memorial Hospital of Sun Yat-Sen University

Study record dates

Study Major Dates

• Study Start (Actual): May 22, 2023
• Primary Completion (Actual): March 6, 2024
• Study Completion (Actual): March 6, 2024

Study Registration Dates

• First Submitted: April 3, 2025
• First Submitted That Met QC Criteria: April 13, 2025
• First Posted (Actual): April 20, 2025

Study Record Updates

• Last Update Posted (Actual): April 20, 2025
• Last Update Submitted That Met QC Criteria: April 13, 2025
• Last Verified: April 1, 2025

More Information

Terms related to this study

• Additional Relevant MeSH Terms: Pathologic Processes; Neoplasms; Neoplastic Processes; Disease; Neoplasm Metastasis
• Other Study ID Numbers: 2023ZSLYEC-256

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: UNDECIDED

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No