AI-based Medical Device Validation for Early Melanoma Detection (LEGIT_MC_EVCDA)

Clinical Validation Study of a CAD System With Artificial Intelligence Algorithms for Early Noninvasive in Vivo Cutaneous Melanoma Detection

The goal of this observational study is to learn if a computer-aided diagnosis (CAD) system can help identify skin cancer (cutaneous melanoma). The research focuses on adults who have skin spots that a doctor thinks might be cancerous. The main questions the study aims to answer are:

Can the artificial intelligence (AI) tool accurately identify melanoma in skin images?

How does the tool's accuracy compare to the clinical judgment of expert skin doctors (dermatologists)?

Researchers will compare the results from the AI tool to the final diagnosis made by doctors or through a skin biopsy. A biopsy is a medical test where a small piece of skin is removed and checked in a lab.

Participants will:

Have their skin spots photographed using a special camera attached to a smartphone.

Allow researchers to use their clinical data and biopsy results for the study.

The study does not change the medical care participants receive. Doctors will continue to treat participants as they normally would. By testing this tool, researchers hope to find a way to detect skin cancer earlier and more accurately

Study Overview

• Status: Completed
• Conditions: Melanoma; Melanoma, Skin
• Intervention / Treatment: Device: AI-based Computer-Aided Diagnosis (CAD) Software for Skin Lesion Analysis.

Detailed Description

This study is designed to clinically validate a computer-aided diagnosis (CAD) system that utilizes artificial intelligence (AI) and machine vision to assist in the detection of cutaneous melanoma in its early stages. Cutaneous melanoma is a form of skin cancer that is treatable when identified early; however, differentiating early melanoma from benign skin lesions during visual examination presents a challenge for healthcare professionals.

Study Design and Methodology The research is a prospective, observational, and cross-sectional study conducted at Hospital Universitario Cruces and Hospital Universitario Basurto in Spain. The protocol evaluates the diagnostic performance of an AI device using clinical images without interfering with routine patient care.

• Participant Selection: The study focuses on adults with skin lesions suspected of malignancy during regular clinical visits.
• Image Acquisition: Researchers capture photographs of skin lesions using a smartphone equipped with a specialized dermoscopic camera.
• Data Collection: Clinical and demographic data, such as age and sex, are collected alongside the digital images.
• Preprocessing: All images undergo a standardized preprocessing step where the lesion is cropped to minimize background noise for the algorithm.
• AI Analysis: The system processes the cropped images to generate a malignancy probability and a list of possible disease categories.
• Reference Standard: AI predictions are compared against a composite "Gold Standard." This standard is based on pathological anatomy results from a biopsy or, where a biopsy is not clinically indicated, the consensus diagnosis of expert dermatologists with extensive clinical experience.

Study Phases and Sample Size Plan

The investigation was planned in two phases to ensure a representative dataset:

• Initial Phase: An initial pilot phase to assess the preliminary workflow.
• Extension Phase: A second phase intended to broaden the sample to include a wider range of benign lesions, such as nevi, to better reflect the distribution of cases seen in daily clinical practice.
• Sample Size Target: The study was designed to recruit participants until a sufficient number of confirmed melanoma cases were achieved to provide the statistical power required to validate the device's performance metrics.

Performance Evaluation Measures

The device's effectiveness is evaluated through the following pre-specified statistical metrics:

• Primary Metrics: Area Under the ROC Curve (AUC), sensitivity, and specificity for the detection of melanoma.
• Secondary Metrics: Top-K accuracy (evaluating if the correct diagnosis is within the top 1, 3, or 5 predictions) and malignancy prediction values, including Positive Predictive Value (PPV) and Negative Predictive Value (NPV).
• Safety Assessment: The protocol includes the systematic recording of any adverse events or product deficiencies encountered during the use of the device to evaluate its safety profile as a clinical decision-support tool.

• Study Type: Observational
• Enrollment (Actual): 105

Contacts and Locations

Study Locations

• Spain
  • Biscay
    • Barakaldo, Biscay, Spain, 48903
      • University Hospital of Cruces

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: Adult; Older Adult
• Accepts Healthy Volunteers: No

• Sampling Method: Non-Probability Sample

Study Population

Patients with skin lesions with suspected malignancy seen at the Dermatology Department of the Hospital Universitario Cruces and Hospital Universitario Basurto.

Description

Inclusion Criteria:

• Patients with skin lesions with suspected malignancy
• Age over 18 years old
• Patients who consent to participate in the study by signing the Informed Consent form

Exclusion Criteria:

• Patients under 18 years of age

Study Plan

How is the study designed?

Number of groups / cohorts

1

Cohorts and Interventions

Group / Cohort

Intervention / Treatment

Patients with suspected cutaneous malignancy; Group/Cohort Description The study group consists of adult patients (over 18 years old) who presented at the Dermatology Departments of Hospital Universitario Cruces and Hospital Universitario Basurto with skin lesions suspected of being malignant. As this is an observational study, participants were not assigned to any new medical interventions, drugs, or treatments as part of the research protocol.

Device: AI-based Computer-Aided Diagnosis (CAD) Software for Skin Lesion Analysis.; The intervention is a software-only medical device that utilizes artificial intelligence and machine vision algorithms to analyze digital images of the skin. Unlike traditional diagnostic tools, this system is designed to provide quantitative data on visible clinical signs and an interpretative distribution of possible disease categories (ICD codes). Key Distinguishing Features Non-Invasive Diagnostic Support: It acts as a clinical decision-support tool to help practitioners prioritize patients based on malignancy risk, rather than providing a standalone or confirmatory diagnosis. Broad ICD Recognition: While many tools focus only on melanoma, this system is capable of recognizing a variety of ICD categories, including basal cell carcinoma, nevi, and dermatofibroma Advanced Image Preprocessing: The system includes a Dermatology Image Quality Assessment (DIQA) algorithm to ensure images have sufficient visual quality before analysis.

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Area Under the ROC Curve (AUC) for Melanoma Detection	Measures the device's ability to distinguish between melanoma and non-melanoma cases using predicted probabilities.	At the time of the single clinical visit (Baseline).
Accuracy for Melanoma Detection	Accuracy represents the percentage of all cases where the AI software's primary (top-ranked) prediction correctly matched the confirmed medical diagnosis. The "confirmed diagnosis" was determined by either a laboratory biopsy (the gold standard) or a consensus of expert dermatologists. To calculate this, the AI analyzed high-resolution dermoscopic images of skin lesions. The software succeeded if its highest-probability diagnosis category matched the actual disease category of the lesion. Only images meeting a minimum visual quality score (DIQA ≥ 5) were included in this analysis to ensure the results reflect performance in a professional clinical setting.	At the time of the single clinical visit (Baseline)
Sensitivity for Melanoma Detection	The percentage of true positive melanoma cases correctly identified by the device.	At the time of the single clinical visit (Baseline).
Specificity for Melanoma Detection	The percentage of true negative (benign) cases correctly identified by the device.	At the time of the single clinical visit (Baseline).

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Top-1 Accuracy for Multiple ICD Categories	Evaluates if the correct diagnosis is within the Top-1 predictions across various skin disease categories (International Classification of Diseases).	At the time of the single clinical visit (Baseline).
Top-3 Accuracy for Multiple ICD Categories	Evaluates if the correct diagnosis is within the Top-3 predictions across various skin disease categories (International Classification of Diseases).	At the time of the single clinical visit (Baseline).
Top-5 Accuracy for Multiple ICD Categories	Evaluates if the correct diagnosis is within the Top-5 predictions across various skin disease categories (International Classification of Diseases).	At the time of the single clinical visit (Baseline).
Area Under the ROC Curve (AUC) for Malignancy Detection	Includes AUC, Sensitivity, and Specificity for detecting any malignant lesion (not limited to melanoma).	At the time of the single clinical visit (Baseline).
Sensitivity for Multiple Malignant Conditions Detection	The percentage of true positive malignant cases correctly identified by the device.	At the time of the single clinical visit (Baseline).
Specificity for Multiple Malignant Conditions Detection	The percentage of true negative (benign) cases correctly identified by the device.	At the time of the single clinical visit (Baseline).
Predictive Values (PPV and NPV) for Malignancy	Measures the Positive Predictive Value (PPV) and Negative Predictive Value (NPV) to determine the probability that a "malignant" or "benign" result from the device is correct.	At the time of the single clinical visit (Baseline).

Collaborators and Investigators

• Sponsor: AI Labs Group S.L
• Collaborators: Hospital de Basurto; Osakidetza; Hospital de Cruces; Servicio Vasco de Salud Osakidetza, Spain
• Investigators: Principal Investigator: Rosa María Ize, PhD, Hospital Universitario Basurto; Principal Investigator: Jesús Gardeazabal, PhD, Hospital de Cruces

Study record dates

Study Major Dates

• Study Start (Actual): September 17, 2020
• Primary Completion (Actual): November 13, 2023
• Study Completion (Actual): November 13, 2023

Study Registration Dates

• First Submitted: January 15, 2024
• First Submitted That Met QC Criteria: January 23, 2024
• First Posted (Actual): January 24, 2024

Study Record Updates

• Last Update Posted (Actual): March 16, 2026
• Last Update Submitted That Met QC Criteria: February 23, 2026
• Last Verified: February 1, 2026

More Information

Terms related to this study

• Keywords: dermatology; primary care; Artificial Intelligence; diagnosis; melanoma; Severity
• Additional Relevant MeSH Terms: Pathologic Processes; Neoplasms by Site; Neoplasms; Neoplasms by Histologic Type; Skin Diseases; Neuroectodermal Tumors; Neoplasms, Germ Cell and Embryonal; Neoplasms, Nerve Tissue; Neuroendocrine Tumors; Nevi and Melanomas; Skin Neoplasms; Pathological Conditions, Signs and Symptoms; Skin and Connective Tissue Diseases; Disease; Melanoma; Diagnosis; Diagnosis, Computer-Assisted
• Other Study ID Numbers: LEGIT_MC_EVCDAO_2019

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: UNDECIDED

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No