Using Artificial Intelligence to Help Doctors Identify Different Skin Conditions and Improve Patient Care (LegitHealthSAN)
A Multi-Reader Multi-Case Study for Evaluating the Impact of Legit.Health Plus Device on the Healthcare Practitioners' Assessment of Skin Lesions
This study aims to determine if an artificial intelligence (AI) medical device can help doctors more accurately identify a wide variety of skin conditions and improve the efficiency of patient consultations. While many patients visit primary care for skin issues, general doctors may sometimes have different opinions from specialists, which can lead to delays in getting the right treatment.
The researchers hypothesized that using the AI tool would increase the true diagnostic accuracy of healthcare professionals for multiple skin conditions. To test this, 16 doctors (including 10 general practitioners and 6 dermatologists) evaluated 29 different medical images.
For each case, the doctors followed a structured process:
- Initial Assessment: Doctors first gave a diagnosis based only on the patient's image and medical history.
- AI Support: Doctors were then shown the AI's top five suggested diagnoses and confidence levels to see if they wished to adjust their final decision.
- Clinical Utility: Doctors also indicated if the patient required a specialist referral and if the case could be handled through a remote (online) consultation.
The primary question the study tried to answer was whether AI support could significantly improve correct diagnoses across 13 different types of skin pathologies-ranging from common rashes to skin cancer-while also making the consultation process faster and more effective for both doctors and patients.
Study Overview
Status
Status
Conditions
Conditions
Intervention / Treatment
Intervention / Treatment
Detailed Description
This detailed description outlines the clinical methodology, technical framework, and data integrity protocols utilized in the investigation of the Legit Health Plus medical device for skin pathologies in primary care and dermatology.
Study Design and Technical Methodology The research was conducted as a prospective observational and cross-sectional self-controlled study. It utilized a Multi-Reader Multi-Case (MRMC) framework to evaluate the impact of Computer-Aided Diagnosis (CAD) on clinician performance.
- Self-Controlled Framework: Each healthcare professional (HCP) served as their own comparator, providing diagnoses first without the use of the device and subsequently with the support of the device on the same set of images.
- Evaluation Workflow: Participants accessed a secure web-based platform to review 29 clinical cases. For each case, doctors provided an initial diagnosis based on an image and medical history, followed by a final diagnosis after reviewing the AI's top 5 suggested International Classification of Diseases (ICD) categories and confidence levels.
- Clinical Utility Assessment: The study included a specific questionnaire to evaluate the utility of the data, consultation time reductions, and confidence in making remote clinical decisions.
- Pathology Diversity: The dataset included 13 distinct skin conditions, ranging from common ailments like Acne and Dermatitis to malignant conditions such as Melanoma and Basal Cell Carcinoma.
Quality Assurance and Data Management
To ensure the scientific integrity of the investigation, the following quality and monitoring protocols were implemented:
- Data Validation and Checks: A validation process was carried out by running computer filters based on validation rules to automatically identify missing values or inconsistencies. Manual editing and exploratory statistical techniques were also used to complement error detection.
- Monitoring Plan: The investigation was monitored by a designated clinical monitor, independent of the investigational site, to ensure the protection of subject rights and data accuracy. Monitoring included remote video or telephone meetings every 3 months.
- Bias Minimization: HCPs were randomly selected to ensure that outcomes were not influenced by pre-existing participant characteristics. Standardized procedures for conducting the study and measuring outcomes were strictly followed to reduce variability.
Ethical and Confidentiality Framework The study adhered to ISO 14155:2021, the Declaration of Helsinki, and the General Data Protection Regulation (GDPR).
- Anonymization: All clinical images were completely anonymized and sourced from public dermatological atlases, containing no information that would allow the identification of patients.
- Data Security: All data entries were timestamped, encrypted using industry-standard security protocols, and stored in a secure central database.
- Data Retention: Upon completion of the study and drafting of the final report, all information stored in the device platform is scheduled to be permanently deleted.
Study Type
Study Type
Enrollment (Actual)
Enrollment
Contacts and Locations
Study Locations
-
-
Basque Country
-
Bilbao, Basque Country, Spain
- AI Labs Group S.L.
-
-
Participation Criteria
Eligibility Criteria
Eligibility Criteria
Ages Eligible for Study
- Adult
- Older Adult
Accepts Healthy Volunteers
Sampling Method
Study Population
The study population consists of board-certified healthcare professionals recruited from the clinical fields of general medicine and dermatology. The participant group includes:
- Primary Care Practitioners: General practitioners who often serve as the first point of contact for patients with dermatological symptoms.
- Specialist Dermatologists: Physicians with advanced expertise in skin pathologies and rare conditions.
- Experience Level: The cohort includes practitioners with at least 5 years of clinical experience in their respective specialities.
Participants were recruited to engage in a remote, web-based evaluation environment rather than being selected from a single physical hospital or town. The clinical images evaluated as part of the study "cases" were sourced from international public dermatology atlases and existing research databases from the sponsor, representing a diverse global patient population.
Description
Inclusion Criteria:
- Board-certified primary care practitioners and dermatologists, regardless of their professional experience.
- High-quality images of patients with different skin conditions.
Exclusion Criteria:
- Low-quality images of patients which can not be properly analyzed.
Study Plan
How is the study designed?
Design Details
Number of groups / cohorts
Cohorts and Interventions
Group / CohortGroup / Cohort |
Intervention / TreatmentIntervention / Treatment |
|---|---|
|
Healthcare Professionals (Primary Care Physicians and Dermatologists)
This group is composed of board-certified healthcare professionals (HCPs) who serve as the "readers" in this multi-reader multi-case (MRMC) study.
The cohort is uniquely characterized by its internal comparison: each participant acts as their own control.
- Dual Professional Roles: The group includes 10 primary care physicians (PCPs) and 6 dermatologists, allowing for a comparison between generalist and specialist diagnostic baseline performance.
- Interventional Exposure: All participants are evaluated under two distinct conditions: first, providing a diagnosis based solely on clinical images and patient history; second, providing a diagnosis assisted by the AI-based medical device's top 5 suggestions and confidence levels.
- Clinical Expertise: Every member of the cohort has a minimum of 5 years of clinical experience in their respective field.
|
The intervention consists of a Computer-Aided Diagnosis (CAD) software-only medical device that utilizes computer vision algorithms to analyze digital images of skin structures.
During the study, healthcare professionals use the tool as a diagnostic support system to assist in the evaluation of complex dermatological conditions.
|
What is the study measuring?
Primary Outcome Measures
Primary Outcome Measures
Outcome Measure |
Measure Description |
Time Frame |
|---|---|---|
|
Diagnostic Accuracy for Multiple Dermatological Conditions with and without Artificial Intelligence Support
Time Frame: Day 1
|
This measure evaluates the "Top-1" diagnostic accuracy of healthcare professionals (HCPs).
Accuracy is determined by comparing the clinician's identified diagnosis-both before and after receiving the AI device's top 5 suggestions and confidence levels-against a confirmed reference standard (confirmed by dermatologists or anatomical pathology)
|
Day 1
|
Secondary Outcome Measures
Secondary Outcome Measures
Outcome Measure |
Measure Description |
Time Frame |
|---|---|---|
|
Change in Dermatology Referral Rate Assisted by Artificial Intelligence.
Time Frame: Day 1
|
This outcome validates the percentage of cases that practitioners determine should be referred to a dermatology specialist after reviewing the AI-provided information, which includes malignancy indices and referral recommendations.
|
Day 1
|
|
Percentage of Cases Deemed Manageable via Remote Consultation.
Time Frame: Day 1
|
This measure assesses the practitioners' evaluation of whether a case can be confirmed and treated remotely through teledermatology based on the AI analysis.
|
Day 1
|
|
Clinical Utility and Usability Scores for Diagnostic Support.
Time Frame: Day 1
|
This outcome assesses the perceived value of the device using a Clinical Utility Questionnaire.
It measures average utility of data (on a scale of 0-10, where 10 is most useful), system usability scores, and the impact on consultation time reduction.
|
Day 1
|
Collaborators and Investigators
Sponsor
Sponsor
Investigators
Investigators
- Principal Investigator: Antonio Martorell, PhD, Hospital Universitari de Manises
Publications and helpful links
Helpful Links
Study record dates
Study Major Dates
Study Start (Actual)
Study Start
Primary Completion (Actual)
Primary Completion
Study Completion (Actual)
Study Completion
Study Registration Dates
First Submitted
First Submitted
First Submitted That Met QC Criteria
First Submitted That Met QC Criteria
First Posted (Actual)
First Posted
Study Record Updates
Last Update Posted (Actual)
Last Update Posted
Last Update Submitted That Met QC Criteria
Last Update Submitted That Met QC Criteria
Last Verified
Last Verified
More Information
Terms related to this study
Additional Relevant MeSH Terms
- Pathologic Processes
- Neoplasms by Site
- Neoplasms
- Pathological Conditions, Anatomical
- Connective Tissue Diseases
- Immune System Diseases
- Infections
- Virus Diseases
- Neoplasms by Histologic Type
- Hypersensitivity, Immediate
- Hypersensitivity
- DNA Virus Infections
- Skin Diseases, Papulosquamous
- Skin Ulcer
- Neuroectodermal Tumors
- Neoplasms, Germ Cell and Embryonal
- Neoplasms, Nerve Tissue
- Hypotrichosis
- Hair Diseases
- Bacterial Infections and Mycoses
- Skin Diseases, Vascular
- Skin Diseases, Infectious
- Neuroendocrine Tumors
- Acneiform Eruptions
- Sebaceous Gland Diseases
- Herpesviridae Infections
- Nevi and Melanomas
- Skin Neoplasms
- Keratosis
- Skin Diseases, Viral
- Granuloma
- Collagen Diseases
- Mycoses
- Dermatomycoses
- Necrobiotic Disorders
- Pathological Conditions, Signs and Symptoms
- Skin and Connective Tissue Diseases
- Alopecia
- Psoriasis
- Dermatitis
- Acne Vulgaris
- Melanoma
- Skin Diseases
- Herpes Simplex
- Pressure Ulcer
- Urticaria
- Granuloma Annulare
- Nevus
- Tinea
- Keratosis, Seborrheic
Other Study ID Numbers
Other Study ID Numbers
- Legit.Health_SAN_2024
- EUPAS1000000911 (Other Identifier: European Medicines Agency)
Plan for Individual participant data (IPD)
Plan to Share Individual Participant Data (IPD)?
Drug and device information, study documents
Studies a U.S. FDA-regulated drug product
Studies a U.S. FDA-regulated device product
This information was retrieved directly from the website clinicaltrials.gov without any changes. If you have any requests to change, remove or update your study details, please contact register@clinicaltrials.gov. As soon as a change is implemented on clinicaltrials.gov, this will be updated automatically on our website as well.
Clinical Trials on Psoriasis
-
NCT04099979WithdrawnPsoriasis | Psoriasis Vulgaris | Psoriasis of Scalp | Psoriatic Plaque | Psoriasis Universalis | Psoriasis Face | Psoriasis Nail | Psoriasis Diffusa | Psoriasis Punctata | Psoriasis Palmaris
-
NCT05938361Active, not recruitingPsoriasis of Scalp | Psoriasis Nail | Psoriasis Palmaris | Psoriasis Genital | Psoriasis Plantaris
-
NCT06846541Active, not recruitingPsoriasis | Plaque Psoriasis | Psoriasis (PsO) | Moderate Psoriasis | Severe Psoriasis
-
NCT02078297CompletedScalp Psoriasis | Pustular Palmo-plantar Psoriasis | Non-pustular Palmo-plantar Psoriasis | Elbow Psoriasis | Lower Leg Psoriasis
-
NCT07448337Not yet recruitingPsoriasis | Psoriasis (PsO) | Psoriasis Arthritis | Psoriasis Patients
-
NCT05144165RecruitingPsoriasis | Psoriasis Vulgaris | Psoriasis of Scalp | Psoriatic Plaque | Psoriasis Universalis | Psoriasis Palmaris | Psoriatic Erythroderma | Psoriatic Nail | Psoriasis Guttate | Psoriasis Inverse
-
NCT00521339CompletedPsoriasis-Type Psoriasis | Plaque-Type Psoriasis
-
NCT03051217CompletedModerate to Severe Psoriasis | Generalized Pustular Psoriasis and Erythrodermic Psoriasis
-
NCT06295692Active, not recruitingGeneralized Pustular Psoriasis | Erythrodermic Psoriasis
-
NCT03942042CompletedGeneralized Pustular Psoriasis | Erythrodermic Psoriasis
Clinical Trials on AI-based medical device for aided diagnosis in Dermatology
-
NCT07428915CompletedPlaque Psoriasis | Seborrheic Dermatitis | Palmoplantar Pustulosis | Acne | Impetigo | Hidradenitis Suppurativa (HS) | Generalized Pustular Psoriasis (GPP) | Tinea Corporis | Acute Generalized Exanthematous Pustulosis | Pustular Psoriasis (PP)
-
NCT07428941CompletedPlaque Psoriasis | Pustular Psoriasis | Actinic Keratoses | Urticaria | Melanoma (Skin Cancer) | Hidradenitis Suppurativa (HS) | Skin Lesions | Melanocytic Nevi | Basal Cell Carcinoma of Skin
-
NCT06221397CompletedMelanoma | Melanoma, Skin
-
NCT04325815CompletedPolyps | Colon Polyp | Adenoma Colon | Colorectal Polyp
-
NCT07170345RecruitingHepatocellular Carcinoma (HCC) | Microvascular Invasion (MVI)
-
NCT07127939RecruitingDeep Learning | Artificial Intelligence (AI) | Ophthalmic Imaging | Traditional Chinese Medicine Constitution
-
NCT02652143TerminatedInfertility | Reproductive Sterility
-
NCT06973915Not yet recruiting
-
NCT05733481Not yet recruitingChronic Total Occlusion of Coronary Artery
-
NCT04824885CompletedHemodialysis | End Stage Renal Disease