COLORS-Validate Study of Decision-Support Software for Surgical Sequencing in Colorectal Liver Metastases
Clinical Utility Evaluation of a Surgical Sequencing Decision-Support Software in Simulated Cases of Colorectal Liver Metastases: A Multicenter Prospective Study (COLORS-Validate Study)
This study evaluates a surgical decision-support software designed to assist physicians in selecting the optimal surgical sequence for patients with colorectal cancer liver metastases.
In this multicenter prospective simulation study, participating surgeons from multiple centers will review standardized clinical case scenarios. For each case, physicians will first make a surgical sequencing decision based on their own clinical judgment. They will then review the recommendation provided by the decision-support software and make a second decision if they choose to revise their initial plan.
The study will assess whether the software influences clinical decision-making, including changes in surgical strategy, decision confidence, decision time, and agreement with software recommendations. Physician user experience will also be evaluated using standardized questionnaires, including system usability and cognitive workload scales.
The goal of this study is to determine the clinical utility and usability of the decision-support software in improving surgical decision-making consistency and supporting clinical reasoning in colorectal liver metastases cases.
Study Overview
Status
Conditions
Intervention / Treatment
Detailed Description
The selection of surgical sequencing-whether to resect the primary tumor first or the liver metastases first-in patients with colorectal cancer liver metastases (CRLM) undergoing synchronous resection represents a critical and highly complex decision in current clinical practice. With the advancement of systemic therapies and surgical techniques, an increasing number of patients with initially unresectable CRLM have gained the opportunity for surgical treatment. However, due to substantial heterogeneity among patients in terms of tumor burden, biological behavior of the disease, and overall physical condition, no unified standard currently exists for determining the optimal surgical sequence. Existing studies comparing different surgical strategies have yielded inconsistent prognostic results, and most are based on retrospective analyses lacking consistent conclusions and clear stratification criteria. Consequently, in real-world clinical practice, such decisions rely heavily on physician experience and institutional preference, resulting in considerable inter-physician variability and uncertainty. This scenario, characterized by the absence of a clearly optimal strategy, renders the decision-making process inherently a form of decision-making under uncertainty.
Based on this context, we previously developed, using multicenter retrospective data, two multi-outcome predictive models corresponding to different surgical strategies. These models integrate postoperative major complications, Comprehensive Complication Index (CCI), progression-free survival (PFS), and overall survival (OS), and have been externally validated. However, in real-world clinical decision-making, the central challenge faced by surgeons is not merely the prediction of a single outcome, but rather the need to balance multiple competing outcomes. For example, one surgical strategy may be associated with a lower risk of postoperative complications but limited long-term survival benefit, whereas another may involve higher perioperative risk in exchange for improved oncological outcomes. This inherent tension among multidimensional outcomes transforms clinical decision-making into a multi-objective decision-making problem, rather than a simple comparison of a single endpoint.
In current practice, such trade-offs across outcomes are typically based on physicians' experiential judgment, whereby the relative importance of different outcomes is implicitly weighted. However, this weighting process lacks explicit representation and standardized criteria, and is prone to influence by individual experience, risk preference, and institutional norms, thereby leading to substantial variability and inconsistency in decision-making. To address this key limitation, the present study further constructs a structured multi-outcome weighting framework using the Delphi method combined with the analytic hierarchy process (AHP). This approach establishes the relative importance of different clinical outcomes through expert consensus and translates it into a quantifiable weighting system, thereby enabling the integration of multidimensional predictive results into a comparable composite score. Based on this framework, we developed a decision-support software system capable of simultaneously providing individualized multi-outcome predictions and composite reference scores, transforming the previously experience-dependent implicit trade-off process into a transparent, interpretable, and standardized decision-support process.
Nevertheless, the establishment of predictive models and weighting frameworks does not necessarily translate into practical clinical value. For decision-support systems, focusing solely on predictive performance is insufficient to reflect their real-world clinical utility. More importantly, it is necessary to evaluate whether such systems can influence physician decision-making behavior under uncertainty, reduce unnecessary variability, and provide a structured reference framework for decision-making. In addition, because clinical decision-making involves counterfactual comparisons (i.e., potential outcomes under alternative strategies for the same patient cannot be simultaneously observed), traditional outcome-based evaluation methods have inherent limitations. Therefore, it is necessary to adopt a model-informed standardized reference framework to evaluate the impact of AI assistance at the level of decision behavior.
Based on the above considerations, this study adopts a multicenter, prospective crossover simulation design to systematically evaluate the clinical utility of the decision-support system in a standardized setting. The evaluation focuses on the following two dimensions:
- From a scientific decision perspective: whether the system alters physician decision behavior, improves decision consistency, and reduces variability;
- From a human factors perspective: whether the system affects cognitive load, decision confidence, and user experience.
This study extends AI evaluation beyond predictive performance toward its influence on decision-making behavior, with a particular focus on the human-AI collaborative decision-making process in uncertain clinical scenarios.
Study Type
Enrollment (Estimated)
Contacts and Locations
Study Contact
- Name: Hong Zhao, MD
- Phone Number: +86 01087787100
- Email: tongjinliang0202@126.com
Study Contact Backup
- Name: QICHEN CHEN, MD
- Phone Number: +86101881055
- Email: chenqichen0822@126.com
Participation Criteria
Eligibility Criteria
Ages Eligible for Study
- Adult
- Older Adult
Accepts Healthy Volunteers
Sampling Method
Study Population
Description
Inclusion Criteria:
- Licensed physicians specializing in hepatobiliary surgery or colorectal surgery
- At least 1 year of clinical experience after graduation
- Willing to participate in the simulation-based decision-making study
- Practicing at one of the participating tertiary academic hospitals
Exclusion Criteria:
- Not actively involved in clinical surgical decision-making
- Unable to complete all required simulation sessions
- Prior involvement in the development of the decision-support software
Study Plan
How is the study designed?
Design Details
Cohorts and Interventions
Group / Cohort |
Intervention / Treatment |
|---|---|
|
Unaided Clinical Decision-Making
Physicians make surgical sequencing decisions based on routine clinical judgment without decision-support software.
|
Usual Clinical Decision-Making (Without Decision Support Software)
|
|
Decision-Support Software-Assisted Decision-Making
Physicians make surgical sequencing decisions after reviewing recommendations from the decision-support software.
|
Physicians make surgical sequencing decisions after reviewing recommendations generated by the decision-support software, including weighted scores and predicted outcomes.
|
What is the study measuring?
Primary Outcome Measures
Outcome Measure |
Measure Description |
Time Frame |
|---|---|---|
|
Change in Surgical Decision-Making Between Software-Assisted and Non-Assisted Conditions
Time Frame: During each simulated case evaluation session
|
The primary outcome is the proportion of cases in which physicians change their surgical sequencing decision after reviewing decision-support software recommendations compared with their initial unaided decision.
|
During each simulated case evaluation session
|
Secondary Outcome Measures
Outcome Measure |
Measure Description |
Time Frame |
|---|---|---|
|
Software utilization outcomes
Time Frame: During each simulated case evaluation session
|
Software adoption rate (agreement between final decision and software recommendation); Post-change adoption rate among changed decisions
|
During each simulated case evaluation session
|
|
Decision performance outcomes
Time Frame: During each simulated case evaluation session
|
Decision-making time (seconds per case); Inter-group comparison between unaided and software-assisted conditions
|
During each simulated case evaluation session
|
|
Human factors outcomes
Time Frame: During each simulated case evaluation session
|
Decision confidence score (5-point Likert scale)
|
During each simulated case evaluation session
|
Collaborators and Investigators
Investigators
- Study Chair: Hong Zhao, MD, Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College
Publications and helpful links
General Publications
- Chen Q, Chen J, Deng Y, Bi X, Zhao J, Zhou J, Huang Z, Cai J, Xing B, Li Y, Li K, Zhao H. Personalized prediction of postoperative complication and survival among Colorectal Liver Metastases Patients Receiving Simultaneous Resection using machine learning approaches: A multi-center study. Cancer Lett. 2024 Jul 1;593:216967. doi: 10.1016/j.canlet.2024.216967. Epub 2024 May 18.
- Chen Q, Tong J, Deng Y, Bi X, Li Y, Li K, Zhao H. Impact of an AI prognostic tool on clinician performance in colorectal liver metastases. NPJ Digit Med. 2026 Apr 8;9(1):432. doi: 10.1038/s41746-026-02606-5.
- Chen Q, Deng Y, Wang K, Li Y, Bi X, Li K, Zhao H. Dynamic Prognostic Models for Colorectal Cancer With Liver Metastases. JAMA Netw Open. 2025 Aug 1;8(8):e2529093. doi: 10.1001/jamanetworkopen.2025.29093.
Study record dates
Study Major Dates
Study Start (Estimated)
Primary Completion (Estimated)
Study Completion (Estimated)
Study Registration Dates
First Submitted
First Submitted That Met QC Criteria
First Posted (Actual)
Study Record Updates
Last Update Posted (Actual)
Last Update Submitted That Met QC Criteria
Last Verified
More Information
Terms related to this study
Other Study ID Numbers
- NCC-023039
Plan for Individual participant data (IPD)
Plan to Share Individual Participant Data (IPD)?
IPD Plan Description
Drug and device information, study documents
Studies a U.S. FDA-regulated drug product
Studies a U.S. FDA-regulated device product
This information was retrieved directly from the website clinicaltrials.gov without any changes. If you have any requests to change, remove or update your study details, please contact register@clinicaltrials.gov. As soon as a change is implemented on clinicaltrials.gov, this will be updated automatically on our website as well.