DERM NMSC Validation Study

Effectiveness of an Image Analysing Algorithm (DERM) to Diagnose Non-melanoma Skin Cancer (NMSC) and Benign Skin Lesions Compared to Gold Standard Clinical and Histological Diagnosis

This study aims to establish the effectiveness of an Artificial Intelligence (AI) algorithm (DERM) to determine the presence of Basal Cell Carcinoma (BCC) and Squamous Cell Carcinoma (SCC) and frequently observed benign conditions, when used to analyse images of skin lesions taken by commonly available smart phone cameras.

Study Overview

• Status: Completed
• Conditions: Non-melanoma Skin Cancer
• Intervention / Treatment: Device: Deep Ensemble for the Recognition of Malignancy (DERM)

Detailed Description

DERM, an Artificial Intelligence (AI)-based diagnosis support tool, has been shown to be able to accurately identify Non-melanoma skin cancers (NMSC) and other conditions from historical images of suspicious skin lesions (moles). This study aims to establish how well DERM determines the presence of these conditions in images of skin lesions collected in a clinical setting.

Suspicious skin lesions that are due to be assessed by a dermatologist and a patch of healthy skin will be photographed using three commonly available smart phone cameras with a specific lens attachment. The images will be analysed by DERM, and the results compared to the clinician's diagnosis (all lesions) and histologically-conformed diagnosis (any lesion that is biopsied).

• Study Type: Observational
• Enrollment (Actual): 572

Contacts and Locations

Study Locations

• United Kingdom
  • London, United Kingdom, NW3 2QG
    • Royal Free London NHS Foundation Trust
  • Newcastle Upon Tyne, United Kingdom, NE7 7DN
    • Royal Victoria Infirmary
  • Poole, United Kingdom, BH15 2JB
    • Poole General Hospital

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: 14 years and older (Adult, Older Adult)
• Accepts Healthy Volunteers: No
• Genders Eligible for Study: All

• Sampling Method: Non-Probability Sample

Study Population

Patients attending a dermatology clinic with at least 1 suspicious skin lesion

Description

Inclusion Criteria:

• Participant is willing and able to give informed consent for participation in the study,
• Male or Female, aged 18 years or above,
• Have at least suspicious skin lesion which is suitable for photographing (<15mm, not located on an anatomical site inappropriate to photograph (genitalia, hair-bearing areas, under nails), not previously biopsied, not located in an area of visible scarring or tattooing),
• In the Investigator's opinion, able and willing to comply with all study requirements.

Exclusion Criteria:

• Any other significant disease or disorder which, in the opinion of the Investigator, may either put the participant at risk because of participation in the study, or may influence the result of the study, or the participant's ability to participate in the study

Study Plan

How is the study designed?

Design Details

• Observational Models: Cohort
• Time Perspectives: Prospective

Number of groups / cohorts

1

Cohorts and Interventions

Group / Cohort

Intervention / Treatment

All patients; Recruited participants will be attending a dermatology clinic with at least one skin lesion where there is a suspicion of skin cancer. All suspicious lesions suitable for photographing will be photographed six times in a single visit. A macro and dermoscopic image of each lesion will be captured by three different mobile phones: an iPhone, a Samsung and a Nokia smart phone, without (macro image) or with (dermoscopic image) a Dermlite DL1 lens attached. Dermoscopic images of healthy skin will also be captured by each camera. Images of the lesions will be analysed by DERM. The DERM results for lesions biopsied will be compared to the biopsy result; the DERM results for lesions not biopsied will be compared to the clinical assessment.

Device: Deep Ensemble for the Recognition of Malignancy (DERM); An AI-based diagnosis support tool

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
AUROC of DERM performance when analysing images of biopsied lesions	Area Under the Receiver Operating Characteristic Curve (AUROC) of the DERM result of biopsied lesions, using histopathological-confirmed diagnosis as gold standard	Study completion

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
AUROC of DERM performance when analysing images of non-biopsied lesions	Area Under the Receiver Operating Characteristic Curve (AUROC) of the DERM result of biopsied lesions, using clinical diagnosis as gold standard	Study completion: on average 2 days
The sensitivity of DERM when used to assess biopsied lesions	The sensitivity of DERM when used to assess biopsied lesions	Study completion: on average 2 days
The specificity of DERM when used to assess biopsied lesions	The specificity of DERM when used to assess biopsied lesions	Study completion: on average 2 days
The false positive rate of DERM when used to assess biopsied lesions	The false positive rate of DERM when used to assess biopsied lesions	Study completion: on average 2 days
The false negative rate of DERM when used to assess biopsied lesions	The false negative rate of DERM when used to assess biopsied lesions	Study completion: on average 2 days
The positive predictive value of DERM when used to assess biopsied lesions	The positive predictive value of DERM when used to assess biopsied lesions	Study completion: on average 2 days
The negative predictive value of DERM when used to assess biopsied lesions	The negative predictive value of DERM when used to assess biopsied lesions	Study completion: on average 2 days
The sensitivity of DERM when used to assess non-biopsied lesions	The sensitivity of DERM when used to assess non-biopsied lesions	Study completion: on average 2 days
The specificity of DERM when used to assess non-biopsied lesions	The specificity of DERM when used to assess non-biopsied lesions	Study completion: on average 2 days
The false positive rate of DERM when used to assess non-biopsied lesions	The false positive rate of DERM when used to assess non-biopsied lesions	Study completion: on average 2 days
The false negative rate of DERM when used to assess non-biopsied lesions	The false negative rate of DERM when used to assess non-biopsied lesions	Study completion: on average 2 days
The positive predictive value of DERM when used to assess non-biopsied lesions	The positive predictive value of DERM when used to assess non-biopsied lesions	Study completion: on average 2 days
The negative predictive value of DERM when used to assess non-biopsied lesions	The negative predictive value of DERM when used to assess non-biopsied lesions	Study completion: on average 2 days
Concordance of clinician assessment with histologically confirmed diagnosis	Concordance of clinician assessment with histologically confirmed diagnosis	Study completion: on average 2 days
The concordance of DERM result generated using images from each camera	The concordance of DERM result generated using images from each camera	Study completion: on average 2 days
The proportion of skin lesions with 3 images that can be analysed by DERM;	The proportion of skin lesions with 3 images that can be analysed by DERM;	Study completion: on average 2 days
The proportion of skin lesions with at least 1 readable image that can be analysed by DERM	The proportion of skin lesions with at least 1 readable image that can be analysed by DERM	Study completion: on average 2 days

Other Outcome Measures

Outcome Measure	Measure Description	Time Frame
Impact of patient characteristics on the DERM and clinician assessment	The impact of patient characteristics (such as sex, age, location of lesion, total body lesion count, Fitzpatrick skin type, past medical history of skin cancer) on the diagnostic accuracy of DERM and clinician assessment;	Study completion: on average 2 days
Impact of lesion characteristics on the DERM and clinician assessment	The impact of lesions characteristic (such as growth over last 6 months, stage and sub-type) on the diagnostic accuracy of DERM and clinician assessment	Study completion: on average 2 days
The impact of image variables on the diagnostic accuracy of DERM assessment	The impact of image variables (such as macro and dermoscopic images) on the diagnostic accuracy of DERM assessment	Study completion: on average 2 days
DERM performance (AUROC) when macro images are used both to train the algorithm and as test images	Exploration of whether macro images can be used as part of DERM's assessment	Study completion: on average 2 days

Collaborators and Investigators

• Sponsor: Skin Analytics Limited
• Collaborators: Innovate UK

Study record dates

Study Major Dates

• Study Start (Actual): June 26, 2020
• Primary Completion (Actual): February 28, 2022
• Study Completion (Actual): March 16, 2022

Study Registration Dates

• First Submitted: September 30, 2019
• First Submitted That Met QC Criteria: October 2, 2019
• First Posted (Actual): October 7, 2019

Study Record Updates

• Last Update Posted (Actual): May 18, 2022
• Last Update Submitted That Met QC Criteria: May 17, 2022
• Last Verified: May 1, 2022

More Information

Terms related to this study

• Additional Relevant MeSH Terms: Skin Diseases; Neoplasms; Neoplasms by Site; Skin Neoplasms
• Other Study ID Numbers: DERM-003

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: Undecided
• IPD Plan Description: Research to improve or test the performance of DERM only allowed in consent

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No