Application of the Belle.AI Dermatological Image Reference System for Patient Diagnosis in an Active Clinical Setting

Application of the Belle.AI Dermatological Image Reference System on Patient Diagnosis Within an Active Clinical Setting

The goal of this research study is to test a new, investigational tool that uses artificial intelligence (AI) to help primary care providers assess skin conditions. This tool is an AI-powered dermatology image reference app that works with a smartphone. For clarity, the AI makes no diagnoses; it provides reference images. Primary care providers then use their own medical judgement and training to make the diagnosis.

The sponsor aims to compare the diagnoses made by primary care providers (such as doctors, nurse practitioners, and physician assistants) with the support of the AI tool compared to a panel of dermatologists, who are setting the gold standard. By doing so, the sponsor can determine the value of the AI tool for primary care providers and understand how it might be used alongside traditional clinical care.

This AI capability complies with FDA regulatory guidelines and is not considered a medical device, similar to a Google image search, which returns similar looking images for reference purposes. For intervention, they healthcare providers use their own training and clinical judgement to make the diagnosis, and not the AI.

Study Overview

• Status: Recruiting
• Conditions: Skin Condition; Dermatology/Skin - Other; Dermatology
• Intervention / Treatment: Diagnostic test: Dermatology/Skin - Other

Detailed Description

Access to dermatologists is often limited, leading to around 60% of skin, hair, and nail issues being treated by non-specialists. This study will evaluate the effectiveness of an AI dermatology decision support tool in assisting primary care providers (PCPs) with the diagnosis of skin conditions. AI-based image analysis has been shown to enhance diagnostic accuracy, particularly for non-dermatologists. Previous studies have primarily focused on dermatologists, but AI could be more beneficial for PCPs, as it has been shown to improve their diagnostic accuracy and agreement with dermatologists.

Globally, about 1.9 billion people suffer from skin diseases annually, with 1 in 3 Americans seeking dermatological care from non-specialists. Skin-related issues make up a significant proportion of visits to general practitioners and emergency departments. AI has proven effective in diagnosing skin conditions such as melanoma and other inflammatory diseases, and studies indicate that AI tools can enhance diagnostic accuracy, particularly for non-dermatologists.

The Belle AI tool, which will be used in this study, employs a convolutional neural network trained on over 500,000 images to identify over 2,000 skin conditions. It provides image match scores to help physicians identify conditions and offers a protocol for second opinions from board-certified dermatologists. The study aims to assess the tool's utility in real-time clinical settings, with potential improvements in triage accuracy, referral quality, and cost savings.

This study is supported by the Advanced Research Projects Agency for Health (ARPA-H) and will be one of the first to prospectively examine AI's impact on dermatology decision support in primary care.

The study aims to evaluate the accuracy and utility of the Belle AI dermatological reference system in a real-world clinical environment, in partnership with Urban Health Plan (UHP). Key endpoints include assessing diagnostic utility and accuracy compared to a final diagnosis from a dermatological review committee, as well as gathering feedback from primary care providers and physician extenders on their experiences with the AI. The sponsor will also analyze the cost implications of the system's use to demonstrate its value in frontline medicine.

Participants will use a smartphone app to capture images of their skin conditions, which will then be analyzed by the AI. Participants will receive financial incentives for submitting images after their initial visit. A follow-up appointment will be scheduled two weeks after the initial consultation, though some visits may be canceled based on the AI analysis.

Participants will be included if the participants present with a primary dermatological complaint and can provide informed consent. Exclusions apply to those unable to comply with procedures or pediatric participants with genital conditions for privacy reasons.

Upon entering UHP, participants register and are triaged. Those with qualifying dermatological conditions will be approached for recruitment by a study coordinator, who will explain the study and obtain consent. Participants will download the Belle Image Capture App to their smartphones, where a Study ID code linked to their EMR will be created, ensuring privacy.

During the initial appointment, providers will examine the patient, document their history, and diagnose the condition. The BellePro Physician App will be used to capture images and generate a differential diagnosis, which the provider will review before making a final diagnosis. Participants will be scheduled for a follow-up visit, and the study coordinator will notify providers of any received images captured through the app.

Beginning seven days after enrollment, push notifications will prompt participants to submit images using the app. The coordinator will follow up with participants who do not respond, aiming for a clear image within a specified timeframe. Upon receipt, the provider will reassess the diagnosis using updated AI analysis. Decisions regarding follow-up appointments will be communicated by the coordinator.

If a follow-up appointment is deemed unnecessary, participants will still be asked to submit images on Day 14. The coordinator will follow up similarly to ensure compliance.

The study spans from Day 1 (initial clinic visit) to Day 14-18, when final images or follow-up appointments will occur. Case notes will be updated continuously in eClinicalWorks, determining whether cases are resolved or require ongoing care.

Primary care providers at UHP will undergo onboarding, including an electronic intake survey and training on the BellePro Physician App via group video chat. Providers will be trained on the app's use, and their feedback will be collected in an exit survey to evaluate their experiences and willingness to continue using the AI system.

Given the complexity of dermatological diagnoses, a review committee of senior board-certified dermatologists will confirm diagnoses from the study. The review process involves three phases: initial image assessment, review of redacted medical records, and consideration of AI analysis results. The committee's consensus will determine the final diagnosis, which will be documented for analysis. Cases lacking a unanimous decision will be excluded from the study's final evaluation. Reviews will occur once enrollment is complete.

• Study Type: Interventional
• Enrollment (Estimated): 263
• Phase: Not Applicable

Contacts and Locations

• Study Contact: Name: John Romano, MD; Phone Number: 617-520-4730; Email: jromano@belle.ai
• Study Contact Backup: Name: Franco Barsanti, PharmD; Phone Number: 347-446-0227; Email: franco.barsanti@urbanhealthplan.org

Study Locations

• United States
  • New York
    • Bronx, New York, United States, 07059
      • Recruiting
      • Urban Health Plan Inc.
      • Contact: Franco Barsanti, PharmD; Phone Number: 347-446-0227; Email: franco.barsanti@urbanhealthplan.org
      • Contact: Carina Puello, BSN; Phone Number: 917-251-0223; Email: carina.puello@urbanhealthplan.org
      • Principal Investigator: Samuel DeLeon, MD
      • Principal Investigator: Acklema Mohammad, MD

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: Child; Adult; Older Adult
• Accepts Healthy Volunteers: No

Description

Inclusion Criteria:

• Patient must present to walk-in clinic with a primary dermatological complaint.
• Patient must have the ability and willingness to provide informed consent and comply with study procedures and visits.
• Participants must have access to the required technology (e.g., smartphone with internet access) and be capable of using it for the required image capture.

Exclusion Criteria:

• Patients who are unable to comply with study procedures due to physical or mental health limitations (as assessed by study coordinator).
• Pediatric, adolescent, and teen patients who present with dermatological conditions on their genitalia will not be included in the study (in support of patient privacy concerns).

Study Plan

How is the study designed?

Design Details

• Primary Purpose: Diagnostic
• Allocation: N/A
• Interventional Model: Single Group Assignment
• Masking: None (Open Label)

Number of Arms

1

Arms and Interventions

Participant Group / Arm	Intervention / Treatment
Other: Standard of care	Diagnostic test: Dermatology/Skin - Other; Belle AI for skin image assessment.

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Accuracy of Belle AI Image References	Comparison of Belle AI's image reference system primary probability diagnosis to the reference diagnosis established by the dermatological review committee; additional evaluation of the accuracy of the second and third highest probability diagnoses determined by the Bell Image Match Score	Through study completion, an average of 14 days
Impact on Economic Burden	Measure economic impact by enabling remote assessment for patients	Through study completion, an average of 14 days

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Agreement between diagnoses	Assessment of agreement between Belle AI's dermatological image reference system primary probability diagnosis and diagnoses made by primary care providers	Through study completion, an average of 14 days
Provider-Rated Usefulness	Primary care providers rating of the usefulness of the Belle AI in supporting dermatological diagnoses, including subgroup analyses by dermatological training and experience; separate into cases of agreement and disagreement between AI and provider diagnoses.	Through study completion, an average of 14 days
Cost-Impact Analysis	Cost-impact assessment of Belle AI's diagnostic effectiveness using theoretical cost savings in cases of accurate AI diagnosis.	Through study completion, an average of 14 days

Other Outcome Measures

Outcome Measure	Measure Description	Time Frame
Protocol Adherence	Separate analyses for per-protocol and intent-to-treat populations to understand adherence effects.	Through study completion, an average of 14 days

Collaborators and Investigators

• Sponsor: BelleTorus Corporation
• Collaborators: Urban Health Plan Inc.
• Investigators: Principal Investigator: John Romano, MD, BelleTorus Corporation; Study Director: Franco Barsanti, PharmD, Urban Health Plan Inc.

Publications and helpful links

Helpful Links

• AI for skin image assessment
• Bronx, Queens and Harlem-based clinics serving 90,000 patients

Study record dates

Study Major Dates

• Study Start (Actual): September 16, 2024
• Primary Completion (Estimated): March 30, 2025
• Study Completion (Estimated): June 30, 2025

Study Registration Dates

• First Submitted: October 10, 2024
• First Submitted That Met QC Criteria: December 5, 2024
• First Posted (Estimated): December 9, 2024

Study Record Updates

• Last Update Posted (Estimated): December 9, 2024
• Last Update Submitted That Met QC Criteria: December 5, 2024
• Last Verified: December 1, 2024

More Information

Terms related to this study

• Keywords: Artificial Intelligence; Digital Health; Dermatology; AI; Health Equity; Skin Health
• Additional Relevant MeSH Terms: Skin Diseases
• Other Study ID Numbers: 1376629; 75N91023C00045 (Other Grant/Funding Number: National Institute of Health | National Cancer Institute)

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: NO

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No