Real World Evaluation of Lunit INSIGHT MMG Technology

Real World Evaluation of Lunit INSIGHT MMG Technology in a Population of Women With Breast Symptoms Referred to a "One-stop" Breast Clinic

This study is a single-centre prospective observational cohort study designed to assess and compare the sensitivity and specificity of a Lunit INSIGHT MMG assisted human reading to the standard care double human reading process within mammography review at a "one-stop" breast clinic (non-inferiority study).

The current imaging reporting process is a sequential double read of mammography and ultrasound (if available) images, by consultant radiologists or radiographers. The first reader produces a report which is then sent to the second reader who reviews it. If the second reader agrees with the first, this is reflected in the second reader's report which translates into a decision for further action; in the event of disagreement, a third reader arbitrates and produces the final report.

In the past, breast clinics have had to resort to single reader reporting due to staff shortages and high demand. This results in delays to any further assessments that may be required. It is worth noting however that despite difficulties in meeting the target, the current clinical pathway has proven to be cost effective.

The Lunit INSIGHT MMG tool could generate benefits and potential efficiencies if it were introduced to the clinical service as an assistant reader within the mammography reporting process, by replacing one of the two human readers in the current standard of care. Before this can be assessed however, its non-inferiority in combination with a human reader in comparison to standard of care (double human reading) must first be established. This study will aim to address this issue in the first instance, maintaining standard of care for all patients seen within the 2 week wait pathway, by introducing the use of Lunit INSIGHT MMG into one of two arms within this prospective, observational parallel cohort study.

Study Overview

• Status: Not yet recruiting
• Conditions: Breast Cancer; Breast Diseases
• Intervention / Treatment: Device: Lunit INSIGHT MMG

Detailed Description

The study has three main objectives, which will be met through separate sub-studies and analyses.

First, it will investigate and compare the performance of the Lunit INSIGHT MMG tool in combination with a human reader, against the current standard of care (two human readers). Quantitative data will be collected from a prospective, observational cohort study which will take place during routine clinical care at the participating site in parallel to standard care. The main aims will be:

1. To assess and compare the sensitivity and specificity of Lunit INSIGHT MMG combined with a single human reader for non-inferiority compared to double human reading
2. To assess and compare the rate of escalation to arbitration (following non-agreement) for non-inferiority of Lunit INSIGHT MMG combined with a single human reader against double human reading
3. To assess clinician reported agreement with the Lunit INSIGHT MMG report
4. To address any differences in performance characteristics as a function of patient age 6 To assess how Lunit score relates to tumour grade on biopsy

Second, the acceptability and perceptions surrounding the AI will be investigated using qualitative data obtained from patient and workforce interviews and questionnaires from the latter. The main aims will be to assess the acceptability of the Lunit INSIGHT MMG tool to patients and the workforce and explore patient and workforce perceptions surrounding it and its implementation in clinical pathways.

Third, a health economic evaluation will be carried out in a cost effectiveness analysis to assess the value of introducing the Lunit INSIGHT MMG tool at a "one-stop" breast clinic. The main aims will be two. The implementation costs of the Lunit Insight MMG tool and costs of a scan read by it, in combination with a human reader, will be compared against the cost of double human readings based on standard practices. In addition, a valuation of outcomes will also be undertaken following the lifetime of the cohort with the use of the Lunit INSIGHT MMG tool compared with standard of care.

• Study Type: Observational
• Enrollment (Estimated): 5500

Contacts and Locations

• Study Contact: Name: Mohammad MS Sayeem; Phone Number: +44 (0) 20 784 89527; Email: kitec@kcl.ac.uk
• Study Contact Backup: Name: Sarah Hammoudi; Phone Number: +44 (0) 20 784 89527; Email: kitec@kcl.ac.uk

Study Locations

• United Kingdom
  • Leeds, United Kingdom
    • The Leeds Teaching Hospitals NHS Trust
    • Principal Investigator: Nisha Sharma
    • Contact: Dr Nisha Sharma; Email: leedsth-tr.breastscreeningresearch@nhs.net
    • Contact: Aneesha Qadeer; Email: leedsth-tr.breastscreeningresearch@nhs.net

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: Adult; Older Adult
• Accepts Healthy Volunteers: No

• Sampling Method: Non-Probability Sample

Study Population

The symptomatic breast population will be recruited from the breast service 2 week pathway referred to a 'one-stop' breast clinic

Description

Inclusion Criteria:

Non-Inferiority Study

• Women 18 years of age or older.
• Referred to a "one-stop" breast clinic following GP appointment for breast symptoms.

Patient and workforce qualitative studies Patients: as above. Workforce: Clinical staff and other key stakeholders involved in the implementation of Lunit INSIGHT MMG both male and female

• Able to undergo mammography.

Exclusion Criteria:

Non-Inferiority Study

• Men will not be included in this study.
• Women who are not able to tolerate a mammogram or are unable to have one due to physical difficulties in engagement with the technology caused by immobility or being wheelchair bound.

Patient and workforce qualitative studies Patients: as above. Workforce: Has no connection to Lunit INSIGHT MMG implementation irrespective of being male or female

Study Plan

How is the study designed?

Number of groups / cohorts

2

Cohorts and Interventions

Group / Cohort	Intervention / Treatment
Standard Pathway Arm; All participating patients will have their mammograms reported using the standard of care. The mammograms randomised to the Standard Pathway arm will be reported using the current standard of care, which involves two human readers; The readers will be blind to Lunit INSIGHT MMG report
Lunit Assisted Arm; All participating patients will have their mammograms reported using the standard of care. Those randomised to the Lunit assisted arm will also undergo double reader process , ensuring the current standard of clinical care is provided. In addition to that, the Lunit INSIGHT MMG report will also be available to the readers. Within this arm, each reader will be presented with a Likert scale to rank their level of agreement with the Lunit INSIGHT MMG report	Device: Lunit INSIGHT MMG; In the Lunit Assisted arm, the radiographers will also be seeing the Lunit INSIGHT MMG reports while reading the mammograms. The standard of care will be followed in all the cases

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Sensitivity, specificity and cost effectiveness of Lunit INSIGHT MMG	To assess and compare the sensitivity and specificity (further follow up or not) of Lunit INSIGHT MMG combined with a single human reader for non-inferiority against double human reading	Jan-Nov 2024
Acceptability of Lunit INSIGHT MMG to patients and NHS workforce	To assess the acceptability of Lunit INSIGHT MMG to patients and NHS workforce. This will be done through a patient and workforce qualitative study involving in-depth interviews and questionnaires	Jan-Nov 2024
Cost Effectiveness of introducing Lunit INSIGHT MMG	To assess the cost effectiveness of introducing Lunit INSIGHT MMG at the 'one-stop' breast clinic	Jan-Nov 2024

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Sensitivity and specificity of the Lunit INSIGHT MMG assisted single Human reading compared to double human reading	The sensitivity and specificity of the Lunit INSIGHT MMG assisted single human reading will be compared to double human reading through ROC curve comparison	Jan-Nov 2024
Number and proportion of non-agreement/arbitrations in the Lunit INSIGHT MMG assisted arm vs the Standard of care pathway arm	Chi squared and Wilson tests on the number and proportion of non-agreement/arbitration in the Lunit INSIGHT MMG assisted arm vs the Standard of Care pathway arm	Jan-Nov 2024
Correlation between the Lunit INSIGHT MMG abnormality score and the level of confidence of human reader 1 in the Lunit result	Correlation between Lunit INSIGHT MMG abnormality score (percentage continuous) and the level of confidence of human reader 1 in the Lunit result as measured by Likert scale	Jan-Nov 2024
Correlation of reader level of concern score	Correlation of reader level of concern score (categorical ordinal) with Lunit INSIGHT MMG abnormality score (percentage continuous). This can be done using tests Kendall's tau or Somers' D if the abnormality percentage are converted to categorical scores	Jan-Nov 2024
Investigation of performance of the Lunit assisted arm in sub-categories of patients	Investigation of performance of the Lunit assisted arm in sub-categories of patients for different ages, breast density, BMI, smoking status and family history using ANOVA or Kruskal-Wallis tests	Jan-Nov 2024
Comparison of time to final report between the two arms	Comparison of time to final report between the two arms with T-test and Mann-Whitney test.	Jan-Nov 2024

Collaborators and Investigators

• Sponsor: Lunit Inc.
• Collaborators: King's College London; The Leeds Teaching Hospitals NHS Trust
• Investigators: Principal Investigator: Nisha Sharma, The Leeds Teaching Hospital NHS Trust

Study record dates

Study Major Dates

• Study Start (Estimated): February 1, 2024
• Primary Completion (Estimated): November 1, 2024
• Study Completion (Estimated): November 1, 2024

Study Registration Dates

• First Submitted: January 22, 2024
• First Submitted That Met QC Criteria: January 22, 2024
• First Posted (Actual): January 30, 2024

Study Record Updates

• Last Update Posted (Actual): January 30, 2024
• Last Update Submitted That Met QC Criteria: January 22, 2024
• Last Verified: January 1, 2024

More Information

Terms related to this study

• Additional Relevant MeSH Terms: Skin Diseases; Breast Diseases
• Other Study ID Numbers: Lunit INSIGHT protocol

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: NO
• IPD Plan Description: No patient identifiable data will be shared outside of the direct care team for the main study. The qualitative interview data will only be available to the King's College Research team which also be anonymized after transcription.

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: Yes