Evaluation of an Artificial Intelligence-enabled Clinical Assistant to Support Thyroid Cancer Management

A Randomized Controlled Trial to Evaluate an Artificial Intelligence-enabled Clinical Assistant for Thyroid Cancer Staging and Risk Stratification Among Medical Students and Clinicians

This study aims to evaluate the clinical feasibility of adopting artificial intelligence (AI)-based models to improve clinical management of thyroid cancer.

Study Overview

• Status: Enrolling by invitation
• Conditions: Thyroid Cancer; Large Language Models
• Intervention / Treatment: Other: AI-enabled clinical assistant

Detailed Description

With recent advancements in technology, AI has become widely applicable to visual text recognition in clinical settings. AI-powered text recognition is emerging as a highly efficient, sustainable, and cost-effective tool for decision making and personalised medicine. Numerous studies have employed natural language processing (NLP) algorithms, particularly large language models (LLMs), to convert unstructured free-text from clinical consultation notes within electronic health records (EHR) into structured data, thus enriching individual clinical profiles in the EHR databases. Over time, these AI models have continuously improved their predictive accuracy and performance through self-learning (or unsupervised learning). While AI models had made a significant impact in oncology practices overseas, their utility for text recognition in oncology remains limited in Hong Kong. This proposed study aims to evaluate the clinical feasibility of adopting AI-based models to improve the end-user confidence in diagnostic accuracy and risk prediction using AI-assisted workflows in thyroid cancer management.

• Study Type: Interventional
• Enrollment (Estimated): 70
• Phase: Not Applicable

Contacts and Locations

Study Locations

• Hong Kong
  • Hong Kong, Hong Kong
    • School of Public Health, The University of Hong Kong
  • Hong Kong, Hong Kong
    • Department of Surgery, School of Clinical Medicine, The University of Hong Kong

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: Adult; Older Adult
• Accepts Healthy Volunteers: Yes

Description

Inclusion Criteria:

• medical students
• clinicians (including but not limited to surgeons, oncologists, pathologists)

Exclusion Criteria:

• medical students and clinicians who had reviewed the clinical notes or were involved in the processing of the clinical notes prior to the commencement of clinical experimental studies

Study Plan

How is the study designed?

Design Details

• Primary Purpose: Health Services Research
• Allocation: Randomized
• Interventional Model: Crossover Assignment
• Masking: Single

Number of Arms

2

Arms and Interventions

Participant Group / Arm	Intervention / Treatment
Experimental: AI-enabled clinical assistant; Participants will provide the caner staging and risk category of each thyroid cancer patient as well as the participants' confidence for the above diagnostic assessments with AI-enabled clinical assistant as the intervention. The AI assistant is powered by LLMs and comprises a clinical dashboard. The clinical dashboard displays the original clinical notes and summarizes cancer staging and risk category of each thyroid cancer patient generated from the backend processing of the clinical assistant. Supporting evidence from original clinical notes is also highlighted for participants' verification.	Other: AI-enabled clinical assistant; Participants will provide the caner staging and risk category of each thyroid cancer patient as well as the participants' confidence for the above diagnostic assessments with AI-enabled clinical assistant as the intervention. The AI assistant is powered by LLMs and comprises a clinical dashboard. The clinical dashboard displays the original clinical notes and summarizes cancer staging and risk category of each thyroid cancer patient generated from the backend processing of the clinical assistant. Supporting evidence from original clinical notes is also highlighted for participants' verification.
No Intervention: Manural chart review; Participants will provide the caner staging and risk category of each thyroid cancer patient as well as the participants' confidence for the above diagnostic assessments with manual chart review.

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Accuracy of Cancer Staging and Risk Stratification by Participants Compared with Ground Truth across Intervention and Non-intervention Groups	The study will compare the accuracy of cancer staging and risk category assessed by the participants across the intervention group with AI assitance and non-intervention group without AI asssitance. The participants will review the clinical notes and assess the cancer staging and risk category for each thyroid cancer patient with or without the AI assistant. Participant provided assessments will be compared against the ground truth established by the clinical investigators of the study to guage the accuracy which is quantified as the percentage of correctly graded cancer staging and risk stratification. The accuracy will be compared between the intervention group and non-intervention groups using t-tests to evaluate the clinical impact of the AI assistant.	Between intervention group and non-intervention group. Cross-over in 3-4 weeks
Participants' Confidence in Cancer Staging and Risk Stratification as Assessed by a 0-10 Scale Questionnaire	The study will compare the participants' confidence in grading cancer staging and risk category between the intervention group with AI assistance and non-intervention group without AI-assistance. After evaluating each thyroid cancer case for providing cancer staging and risk category, participants will complete a short questionnaire rating their confidence in providing their assessments on a scale from 0 (lowest) to 10 (hightest). Meanw confidence score will be compared between the intervention group and non-intervention group to evaluate the clinical impact of the AI assitant.	Between intervention group and non-intervention group. Cross-over in 3-4 weeks
Efficiency	The time required to complete reviewing one set of clinical notes is compared between intervention and non-intervention groups	Between intervention group and non-intervention group. Cross-over in 3-4 weeks

Collaborators and Investigators

• Sponsor: The University of Hong Kong
• Collaborators: Innovation and Technology Commission, Hong Kong
• Investigators: Principal Investigator: King Ho Carlos Wong, School of Public Health The University of Hong Kong; Principal Investigator: Man Him Matrix Fung, Department of Surgery, School of Clinical Medicine, The University of Hong Kong

Publications and helpful links

General Publications

• Murphy GS, Szokol JW, Marymont JH, Avram MJ, Vender JS, Rosengart TK. Impact of shorter-acting neuromuscular blocking agents on fast-track recovery of the cardiac surgical patient. Anesthesiology. 2002 Mar;96(3):600-6. doi: 10.1097/00000542-200203000-00015.

Study record dates

Study Major Dates

• Study Start (Actual): October 2, 2025
• Primary Completion (Estimated): March 31, 2026
• Study Completion (Estimated): April 30, 2026

Study Registration Dates

• First Submitted: September 29, 2025
• First Submitted That Met QC Criteria: November 16, 2025
• First Posted (Estimated): November 18, 2025

Study Record Updates

• Last Update Posted (Actual): December 17, 2025
• Last Update Submitted That Met QC Criteria: December 16, 2025
• Last Verified: November 1, 2025

More Information

Terms related to this study

• Keywords: Thyroid Cancer; NLP; natural language processing; LLMs; large language models
• Additional Relevant MeSH Terms: Endocrine System Diseases; Neoplasms by Site; Neoplasms; Endocrine Gland Neoplasms; Head and Neck Neoplasms; Thyroid Diseases; Thyroid Neoplasms
• Other Study ID Numbers: UW24-319-RCT

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: YES
• IPD Plan Description: Only anonymized IPD used in results publications will be shared so that re-identification of individuals is not possible.
• IPD Sharing Time Frame: The IPD and supporting information will be available upon the completion of study (anticipated date as 31 December 2025) with results dissemination or publication, and will remain unending until required of removal.
• IPD Sharing Access Criteria: The IPD and supporting information will be available with results dissemination and publication as documents uploads or attachment. Anyone who has access to the articles will be able to access all the documents.
• IPD Sharing Supporting Information Type: STUDY_PROTOCOL; SAP; ICF; ANALYTIC_CODE

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No