Improving the Accuracy of Artificial Intelligence Triage in Primary Care

November 14, 2025 updated by: Benjamin Brown, University of Manchester

WHY ARE WE DOING THIS? When patients contact their GP practice, the first step is to work out what kind of help they need and how quickly it's needed. This is called 'triage' and is important for patient safety.

Artificial Intelligence (AI) can help make triage faster. While AI is already being used in the NHS, we don't know how accurate it is or if it treats all patients fairly.

WHAT WILL WE DO?

We will collect anonymised data from patients that use an AI triage system called Patchs in GP practices in England. The project will last four years. We will analyse the data in four steps:

  1. Look at data from GP practices using Patchs without AI triage to see how they currently triage patients and what problems they face.
  2. Use data from GP practices using Patchs (both with AI on and off) to make the AI triage more accurate.
  3. Check data from GP practices using Patchs with AI triage off to measure how well the updated AI system works.
  4. Give the improved AI triage system to GP practices already using AI.

At each step, we will check whether patients from different backgrounds are treated fairly.

HOW WILL WE ANALYSE THE DATA? We will use statistical methods to compare the triage decisions made by the AI with those made by clinical staff. This analysis will also be used to check that the AI works fairly for patients from different backgrounds.

WHAT DIFFERENCE WILL WE MAKE? Our research will show the problems with triage and explain how an improved AI system could help patients get the care they need more quickly.

Study Overview

Status

Recruiting

Conditions

Intervention / Treatment

Detailed Description

Background GP practice staff triage patients contacting them to make the best use of resources and maintain patient safety. Online consultation systems are used by most GP practices and allow patients to contact their GP practice using an online form. They can be submitted without talking to a member of staff, thereby circumventing the usual triage process. Online consultation systems can triage patients using 'Artificial Intelligence' (AI), though there is a lack of research on their performance. We (The University of Manchester; UoM) propose to fill this gap by collaborating with an online consultation system provider with optional AI triage functionality (Patchs).

Research questions Overall research question: is it possible to develop AI models that can replicate clinicians' triage decisions?

  1. What challenges do patients and GP practices face when triaging patients in primary care, and what are their drivers?
  2. What is the best performing AI model for triaging patients in primary care?
  3. Is AI triage performance maintained across different geographical regions?
  4. Is AI triage performance maintained over time?
  5. How does AI triage performance compare to current clinical practice?
  6. Does AI triage performance change when deployed into clinical practice?
  7. Does AI triage work fairly for all patients? Methods Workstream 1: Triage problem quantification. We will analyse anonymised historic data from GP practices using Patchs with AI triage disabled. Where publicly available, we will compare this to practice-level data from GP practices not using Patchs (control practices). We will undertake descriptive and inferential analyses to understand potential triage problems and factors that influence them, such as delays in providing patient care.

Workstream 2: AI development. We will use anonymised historic data from GP practices using Patchs to build new versions of the AI triage models currently in use with four different approaches: logistic regression, XGBoost, long short-term memory (LSTM), and large language model (LLM). We will use internal-external cross-validation by geographical region and compare their performance using random-effects meta-analysis and sub-group analyses to assess fairness (e.g. across ethnicities). We will compare their performance to the current AI triage models in use. The final version of the best-performing AI models will be developed using the entire dataset.

Workstream 3: Prospective background evaluation. We will obtain predictions from the best-performing AI models on prospectively collected data from GP practices using Patchs without AI triage by running the models in the 'background'. We will undertake sub-group analyses to assess fairness as described above.

Workstream 4: Prospective implementation evaluation. In accordance with the normal Patchs software updates, we will update the AI models in GP practices already using AI triage with the best-performing versions. We will prospectively measure how often GP practice staff and patients agree with the new versions' triage predictions to test whether its performance translates to real patient care. We will undertake sub-group analyses to assess fairness as described above.

Anticipated benefits We will help understand the problems currently faced by GP practices during online consultation triage. If we developed improved AI models, there may be improved patient safety (e.g. by helping patients receive help sooner) and reduced GP practice workload (e.g. by automating the triage process). GP practices and their patients in Workstream 4 would benefit immediately. We will provide evidence for GP practices not currently using AI triage whether to adopt it.

Study Type

Interventional

Enrollment (Estimated)

226821

Phase

  • Not Applicable

Contacts and Locations

This section provides the contact details for those conducting the study, and information on where this study is being conducted.

Study Contact

Study Locations

Participation Criteria

Researchers look for people who fit a certain description, called eligibility criteria. Some examples of these criteria are a person's general health condition or prior treatments.

Eligibility Criteria

Ages Eligible for Study

  • Child
  • Adult
  • Older Adult

Accepts Healthy Volunteers

Yes

Description

Inclusion Criteria:

  • GP practices using the Patchs system

Exclusion Criteria:

  • N/A

Study Plan

This section provides details of the study plan, including how the study is designed and what the study is measuring.

How is the study designed?

Design Details

  • Primary Purpose: Health Services Research
  • Allocation: Non-Randomized
  • Interventional Model: Parallel Assignment
  • Masking: Triple

Arms and Interventions

Participant Group / Arm
Intervention / Treatment
Experimental: AI triage
GP practices using AI triage
Other: AI triage
Patchs AI automates parts of the triage and workflow process for GP practices when any patient uses the Patchs online consultation system. It is intended to assist, not replace human decision-making. Patchs AI aims to reduce GP practice workload by minimising manual tasks and improve patient safety by helping patients receive appropriate care sooner. Patchs AI uses information about the patient and their online consultation to suggest an urgency, clinical topic, staff role, and mode to conduct the consultation. Based on these suggestions it can provide patients with relevant health information, ask questions to elicit further information, and/or advise them to contact alternative care providers. Suggestions from Patchs AI can be accepted or rejected by GP practice staff and patients, which is used to monitor its safety and re-train the system.
Active Comparator: No AI triage
GP practices not using AI triage
Other: No AI triage
Use of the Patchs system with no AI triage i.e. manual triage only

What is the study measuring?

Primary Outcome Measures

Outcome Measure
Measure Description
Time Frame
F1 score
Time Frame: From enrollment to the end of the study, anticipated to be 4 years
F1 score of each AI triage module (harmonic mean of its precision and recall)
From enrollment to the end of the study, anticipated to be 4 years

Collaborators and Investigators

This is where you will find people and organizations involved with this study.

Sponsor

University of Manchester

Collaborators

University of Cambridge

Publications and helpful links

The person responsible for entering information about the study voluntarily provides these publications. These may be about anything related to the study.

Helpful Links

Study record dates

These dates track the progress of study record and summary results submissions to ClinicalTrials.gov. Study records and reported results are reviewed by the National Library of Medicine (NLM) to make sure they meet specific quality control standards before being posted on the public website.

Study Major Dates

Study Start (Actual)

April 1, 2025

Primary Completion (Estimated)

April 1, 2029

Study Completion (Estimated)

July 1, 2029

Study Registration Dates

First Submitted

September 19, 2025

First Submitted That Met QC Criteria

November 14, 2025

First Posted (Actual)

November 20, 2025

Study Record Updates

Last Update Posted (Actual)

November 20, 2025

Last Update Submitted That Met QC Criteria

November 14, 2025

Last Verified

November 1, 2025

More Information

Terms related to this study

Other Study ID Numbers

  • 340776
  • 7hzfm (Other Identifier: Open Science Framework)

Plan for Individual participant data (IPD)

Plan to Share Individual Participant Data (IPD)?

NO

IPD Plan Description

Patient consent does not cover sharing data with other research teams.

Drug and device information, study documents

Studies a U.S. FDA-regulated drug product

No

Studies a U.S. FDA-regulated device product

No

This information was retrieved directly from the website clinicaltrials.gov without any changes. If you have any requests to change, remove or update your study details, please contact register@clinicaltrials.gov. As soon as a change is implemented on clinicaltrials.gov, this will be updated automatically on our website as well.

Clinical Trials on Primary Care

Clinical Trials on AI triage

Search Similar Trials

Sponsors and Collaborators

Medical Conditions

Drug Interventions

CROs by country

CROs in Sri Lanka

Conditions

Rare Diseases

Drug Interventions

Dietary Supplements

Sponsor/Collaborators

Locations

Subscribe