AI-Augmented Skin Cancer Diagnosis in Teledermatoscopy (AIDMel)

AI-Augmented Skin Cancer Diagnosis in Teledermatoscopy: A Prospective Randomized Study

In this study an artificial intelligence (AI) tool for skin cancer diagnosis is implemented in a teleldermatoscopy platform. The aim is to study the effects on clinician diagnostic accuracy, management decisions, and confidence. Furthermore, this prospective randomized study investigates the role of human factors in determining clinician reliance on AI tools and the consequent accuracy in a real-world setting.

Study Overview

• Status: Enrolling by invitation
• Conditions: Melanoma; Skin Cancer
• Intervention / Treatment: Other: AI assistance

Detailed Description

Deep-learning algorithms can potentially benefit many areas in healthcare, including the diagnosis of skin cancer using teledermatoscopy. However, there is a dearth of clinical, prospective research on human-AI interaction in diagnostic tasks that take human factors into account.

In this study we will examine the impact of such factors in a real-world setting where we integrate an algorithm in an existing teledermatoscopy platform that is used clinically at a tertiary hospital in Sweden. We will investigate what impact various implementations of AI tool output in relation to human factors have on diagnostic accuracy and management decisions.

Study subjects are recruited at the Department of Dermatology at Karolinska University Hospital and will be asked to rate prospective teledermatoscopic consults with and without AI-support. Each consult will be randomized into one of three workflows with or without one pre-defined implementation of the AI tool. Study subjects are also asked to complete two surveys with demographic information and questions relating to various human factors. Patients participating in the study will be diagnosed outside the study prior to inclusion without any involvement of an AI tool, notably by two experienced dermatologists who do not participate as study subjects.

• Study Type: Interventional
• Enrollment (Estimated): 30
• Phase: Not Applicable

Contacts and Locations

Study Locations

• Sweden
  • Stockholm, Sweden
    • Karolinska University Hospital

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: Adult; Older Adult
• Accepts Healthy Volunteers: Yes

Description

Inclusion Criteria:

• Licensed physician
• Working at a dermatology clinic
• Sufficient knowledge in Swedish
• Written consent to participate

Exclusion Criteria:

• No experience of using dermatoscopy
• Does not wish to participate
• Incomplete answers
• Physicians that are involved in the patients' clinical care relating to the teledermoscopical consult

Study Plan

How is the study designed?

Design Details

• Primary Purpose: Diagnostic
• Allocation: Randomized
• Interventional Model: Crossover Assignment
• Masking: None (Open Label)

Number of Arms

3

Arms and Interventions

Participant Group / Arm	Intervention / Treatment
No Intervention: Workflow 1; Standard of care
Experimental: Workflow 2; Consult with AI assistance	Other: AI assistance; Participants will be informed of the diagnostic probabilities for each of ten differential diagnoses according to the AI tool
Experimental: Workflow 3; First workflow 1, then workflow 2	Other: AI assistance; Participants will be informed of the diagnostic probabilities for each of ten differential diagnoses according to the AI tool

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Diagnostic accuracy	Determine sensitivity, specificity, accuracy and AUROC in terms of diagnostic accuracy for dermatologists with vs without AI advice. Further, to investigate the role of the different workflows (diagnosis with or without AI with varying sequencing) and the influence of demographics and human factors (e.g. level of experience) on diagnostic accuracy	1 year
Accuracy of management decisions	Determine sensitivity, specificity, accuracy and AUROC in terms of accuracy for management decisions for dermatologists with vs without AI and investigate the role of the different workflows (with or without AI with varying sequencing) and the influence of demographics and human factors (e.g. level of experience) on management decisions (biopsy/surgery, no intervention, or follow-up)	1 year
Tendency to change initial diagnosis or management decision	Evaluate which factors affect the likelihood of a physician changing their evaluation after receiving algorithmic input	1 year
Self-reported confidence in diagnosis and management decisions	Investigate whether AI or other factors affect the physician's confidence in their diagnosis and management decisions	1 year

Collaborators and Investigators

• Sponsor: Karolinska University Hospital
• Collaborators: Karolinska Institutet; Medical University of Vienna; Stockholm School of Economics

Study record dates

Study Major Dates

• Study Start (Actual): February 15, 2023
• Primary Completion (Estimated): June 30, 2024
• Study Completion (Estimated): October 30, 2024

Study Registration Dates

• First Submitted: August 30, 2023
• First Submitted That Met QC Criteria: October 6, 2023
• First Posted (Actual): October 12, 2023

Study Record Updates

• Last Update Posted (Actual): October 12, 2023
• Last Update Submitted That Met QC Criteria: October 6, 2023
• Last Verified: October 1, 2023

More Information

Terms related to this study

• Keywords: artificial intelligence; melanoma; skin cancer; deep learning; dermatoscopy; AI augmentation; teledermatoscopy
• Additional Relevant MeSH Terms: Skin Diseases; Neoplasms by Histologic Type; Neoplasms; Neoplasms by Site; Neuroectodermal Tumors; Neoplasms, Germ Cell and Embryonal; Neoplasms, Nerve Tissue; Neuroendocrine Tumors; Nevi and Melanomas; Melanoma; Skin Neoplasms
• Other Study ID Numbers: 960024

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: NO

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No