Spirometry Interpretation Performance of Primary Care Clinicians With/Without AI Software (SPIRO-AID)

A Randomized Controlled Trial Comparing Performance of Primary Care Clinicians in the Interpretation of SPIROmetry With or Without Artificial Intelligence Decision Support Software

To evaluate whether an artificial intelligence decision support software (ArtiQ.Spiro) improves the diagnostic accuracy of spirometry interpreted by primary care clinicians, as measured by Clinician Diagnostic Accuracy (vs Reference Standard).

Study Overview

• Status: Recruiting
• Conditions: Lung Disease
• Intervention / Treatment: Other: Artificial Intelligence-powered Spirometry Interpretation Report

Detailed Description

This is a randomised controlled study to evaluate the effects of AI support software on the performance of primary care clinicians in the interpretation of spirometry. Clinicians will be provided with a clinical dataset of 50 entirely anonymous, previously recorded real-world spirometry records to interpret and will be asked to complete specific questions about diagnosis and quality assessment. The records will be randomly selected from a database comprising spirometry records from 1122 patients undergoing spirometry in primary care and community -based respiratory clinics in Hillingdon borough between 2015-2018.

Participating clinicians will be allocated at random to receive either spirometry records alone or spirometry records with the addition of an AI spirometry interpretation eport. The clinical spirometry records will be de-identified (name, date of birth, address, postcode, occupation, GP, medications data removed), by a member of the clinical care team.

Study participants (participating clinicians) will independently examine the same 50 spirometry records through an online platform. For each spirometry record, the primary care clinician participant will answer questions about technical quality, pattern interpretation, preferred diagnosis, differential diagnosis and self-rated confidence with these answers.

The study statistician will be blinded to treatment allocation up to completion of analysis and interpretation.

The reference standards for spirometry technical quality and pattern interpretation will be made by a senior experienced respiratory physiologist but without access to AI report.

The reference standard for diagnosis will be made by a panel of three respiratory specialists from the clinical care team with access to medical notes and results of relevant investigations but without access to AI report.

• Study Type: Interventional
• Enrollment (Estimated): 228
• Phase: Not Applicable

Contacts and Locations

• Study Contact: Name: Ethaar El-Emir, PhD; Phone Number: 85952 01895 823737; Email: e.el-emir@rbht.nhs.uk
• Study Contact Backup: Name: George Edwards, MSc; Phone Number: 01895 823737; Email: G.Edwards2@rbht.nhs.uk

Study Locations

• United Kingdom
  • Uxbridge, United Kingdom, UB9 6JH
    • Recruiting
    • Royal Brompton & Harefield Hospitals
    • Contact: Ethaar El-Emir, PhD; Phone Number: 85952 01895 823737; Email: e.el-emir@rbht.nhs.uk
    • Contact: George Edwards, MSc; Phone Number: 01895 823737; Email: G.Edwards2@rbht.nhs.uk
    • Principal Investigator: William Man

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: Adult; Older Adult
• Accepts Healthy Volunteers: Yes

Description

Inclusion Criteria:

1. Clinicians working in primary care (for at least 50% of their job plan) in the UK, who refer for or perform spirometry (typically GP, practice nurse)
2. Able to access spirometry traces on study platform
3. Provide written informed consent via study platform

Exclusion Criteria:

1. Clinicians who have completed specialist training in respiratory medicine and recognised by the General Medical Council with a right to practise as a NHS consultant in respiratory medicine

Study Plan

How is the study designed?

Design Details

• Primary Purpose: Health Services Research
• Allocation: Randomized
• Interventional Model: Parallel Assignment
• Masking: Double

Number of Arms

2

Arms and Interventions

Participant Group / Arm	Intervention / Treatment
No Intervention: Control; Participants to report 50 spirometry records alone
Experimental: Intervention; Participants report the same 50 spirometry records provided in the control arm with an artificial intelligence-powered spirometry interpretation report	Other: Artificial Intelligence-powered Spirometry Interpretation Report; A report generated by artificial intelligence powered software that assessed technical quality of spirometry and estimates the diagnostic probability of six categories: COPD/Asthma/ILD/ Normal/Other obstructive/Other Unidentified; Other Names: ArtiQ.Spiro

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Preferred Diagnostic Performance	A correct case is where the preferred diagnosis matches the reference final diagnosis. Units will be percentage of total cases that are correct.	Six months

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Pattern interpretation	A correct case is where the participants' selected pattern matches the reference pattern. Options are: Normal, Airflow obstruction, Possible restriction or non-specific pattern, Possible Mixed Disorder. Units will be percentage of total cases that are correct.	Six months
Differential diagnostic performance	A correct case is where the preferred or differential diagnosis matches the reference final diagnosis. Units will be percentage of total cases that are correct.	Six months
Quality assessment performance	A correct case is where the participant's quality grade matches the reference quality grade. Options are: Acceptable (Grade A/B) or Not Acceptable (Grades C/D/E/F/U). Units will be percentage of total cases that are correct.	Six months
Pattern interpretation self-rated confidence	Pattern interpretation self-rated confidence will be measured on a visual analogue scale (0-10) where 0 = not confident at all; 10= very confident)	Six months
Diagnostic self-rated confidence	Diagnostic self-rated confidence will be measured on a visual analogue scale (0-10) where 0 = not confident at all; 10= very confident)	Six months
Quality Assessment self-rated confidence	Quality Assessment self-rated confidence will be measured on a visual analogue scale (0-10) where is 0 = not confident at all; 10= very confident)	Six months

Collaborators and Investigators

• Sponsor: Royal Brompton & Harefield NHS Foundation Trust
• Collaborators: National Institute for Health Research, United Kingdom
• Investigators: Principal Investigator: William Man, Royal Brompton & Harefield Hospitals

Study record dates

Study Major Dates

• Study Start (Actual): June 27, 2023
• Primary Completion (Estimated): June 27, 2024
• Study Completion (Estimated): September 30, 2024

Study Registration Dates

• First Submitted: June 27, 2023
• First Submitted That Met QC Criteria: June 27, 2023
• First Posted (Actual): July 6, 2023

Study Record Updates

• Last Update Posted (Actual): February 16, 2024
• Last Update Submitted That Met QC Criteria: February 15, 2024
• Last Verified: February 1, 2024

More Information

Terms related to this study

• Additional Relevant MeSH Terms: Respiratory Tract Diseases; Lung Diseases
• Other Study ID Numbers: 323361

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: NO

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No