- ICH GCP
- US Clinical Trials Registry
- Clinical Trial NCT06334796
Artificial Intelligence-powered Virtual Assistant for Emergency Triage in Neurology (AIDEN)
Phase 1 Trial of the Implementation of an Artificial Intelligence-powered Virtual Assistant for Emergency Triage in Neurology
Study Overview
Status
Conditions
Intervention / Treatment
Detailed Description
Background and Objectives: Neurological emergencies pose significant challenges in medical care, especially in resource-limited countries. Artificial Intelligence (AI), particularly health chatbots, offers a promising solution. However, rigorous validation is required to ensure safety and accuracy. The objective of our work is to evaluate the diagnostic accuracy and resolution effectiveness of an AI-powered virtual assistant designed for the triage of emergency neurological pathologies, to ensure the minimum standard of safety that allows for the progression to successive validation tests.
Methods: This Phase 1 trial evaluates the performance of an AI-powered virtual assistant for emergency neurological triage. Ten patients over 18 years old with urgent neurological pathologies were selected. In the first stage, nine neurologists assessed the safety of the virtual assistant using their clinical records. In the second part, the assistant's accuracy when used by patients was evaluated. Finally, its performance was compared with Chat GPT 3.5 and 4.
Study Type
Enrollment (Actual)
Phase
- Early Phase 1
Contacts and Locations
Study Locations
-
-
-
Buenos Aires, Argentina, 1428
- FLENI
-
-
Participation Criteria
Eligibility Criteria
Ages Eligible for Study
- Adult
- Older Adult
Accepts Healthy Volunteers
Description
Inclusion Criteria:
- Patients over 18 years old consulting in the ER due to a neurological emergency
Exclusion Criteria:
- Pregnancy
Study Plan
How is the study designed?
Design Details
- Primary Purpose: Diagnostic
- Allocation: Non-Randomized
- Interventional Model: Single Group Assignment
- Masking: None (Open Label)
Arms and Interventions
Participant Group / Arm |
Intervention / Treatment |
---|---|
Experimental: Virtual Assistant
Patients answer question with a virtual assistant about their recent visit to the ER.
|
Stage 1 focused on safety, using only medical information from clinical records for the virtual assistant.
In Stage 2, which evaluated accuracy, participants interacted with the virtual assistant post-medical stabilization.
Additionally, participants also provided initial symptom details for Chat-GPT input.
Nine neurologists specializing in emergency participated in the study.
In Stage 1, they assessed the virtual assistant's performance using clinical history information.
In Stage 2, they analyzed the results from participant interactions with the assistant and performed a comparative evaluation of Chat-GPT.
The virtual assistant functioned as a chatbot on WhatsApp and Telegram, using Spanish and incorporating advanced algorithms, decision trees, and large language models for interaction.
For comparison, we utilized Chat-GPT versions 3.5 and 4, employing two prompt types in natural Spanish: one incorporating clinical record data and the other based on participant narratives.
|
Active Comparator: ChatGPT 3.5
Patients answer question with ChatGPT about their recent visit to the ER.
|
Stage 1 focused on safety, using only medical information from clinical records for the virtual assistant.
In Stage 2, which evaluated accuracy, participants interacted with the virtual assistant post-medical stabilization.
Additionally, participants also provided initial symptom details for Chat-GPT input.
Nine neurologists specializing in emergency participated in the study.
In Stage 1, they assessed the virtual assistant's performance using clinical history information.
In Stage 2, they analyzed the results from participant interactions with the assistant and performed a comparative evaluation of Chat-GPT.
The virtual assistant functioned as a chatbot on WhatsApp and Telegram, using Spanish and incorporating advanced algorithms, decision trees, and large language models for interaction.
For comparison, we utilized Chat-GPT versions 3.5 and 4, employing two prompt types in natural Spanish: one incorporating clinical record data and the other based on participant narratives.
|
Active Comparator: ChatGPT 4
Patients answer question with ChatGPT about their recent visit to the ER.
|
Stage 1 focused on safety, using only medical information from clinical records for the virtual assistant.
In Stage 2, which evaluated accuracy, participants interacted with the virtual assistant post-medical stabilization.
Additionally, participants also provided initial symptom details for Chat-GPT input.
Nine neurologists specializing in emergency participated in the study.
In Stage 1, they assessed the virtual assistant's performance using clinical history information.
In Stage 2, they analyzed the results from participant interactions with the assistant and performed a comparative evaluation of Chat-GPT.
The virtual assistant functioned as a chatbot on WhatsApp and Telegram, using Spanish and incorporating advanced algorithms, decision trees, and large language models for interaction.
For comparison, we utilized Chat-GPT versions 3.5 and 4, employing two prompt types in natural Spanish: one incorporating clinical record data and the other based on participant narratives.
|
What is the study measuring?
Primary Outcome Measures
Outcome Measure |
Measure Description |
Time Frame |
---|---|---|
Diagnostic performance
Time Frame: The first interaction between participants and the virtual assistant occurred within less than a year after the event. Outcome measures were evaluated immediately after the interaction between patients and the virtual assistant.
|
Refers to the accuracy and effectiveness of medical tests or diagnostic tools in correctly identifying a disease or condition in patients. Syndromic diagnosis agreement: evaluating neurologists considered a syndromic diagnosis accurate when AI tools could identify a condition based on a set of commonly coexisting signs and symptoms, rather than identifying a specific disease. This method is applied when the precise disease causing the symptoms is not immediately identifiable, allowing healthcare providers to effectively monitor and treat the patient's presenting symptoms. Differential diagnosis agreement: a differential diagnosis was considered accurate when the differentials provided by each AI tool matched those presented by the participants. The gold standard for diagnosis was considered to be the one given in the emergency department, unchanged over a one-month period. |
The first interaction between participants and the virtual assistant occurred within less than a year after the event. Outcome measures were evaluated immediately after the interaction between patients and the virtual assistant.
|
Secondary Outcome Measures
Outcome Measure |
Measure Description |
Time Frame |
---|---|---|
Appropriate medical conduct or recommendation
Time Frame: The first interaction between participants and the virtual assistant occurred within less than a year after the event. Outcome measures were evaluated immediately after the interaction between patients and the virtual assistant.
|
Case resolution was evaluated based on appropriate medical conduct or recommendation, categorizing 1) urgency as immediate, 2) short-term (within 48 hours), 3) or non-urgent. The recommendations provided by each AI tool were assessed based on information gathered from clinical histories and input from participants. The gold standard of appropriate medical conduct or recommendation was considered to be that given in the emergency department, with no changes over a period of one month. |
The first interaction between participants and the virtual assistant occurred within less than a year after the event. Outcome measures were evaluated immediately after the interaction between patients and the virtual assistant.
|
Assessment of Usability and Satisfaction
Time Frame: The first interaction between participants and the virtual assistant occurred within less than a year after the event. Outcome measures were evaluated immediately after the interaction between patients and the virtual assistant.
|
Usability was measured by the time and number of questions needed for final diagnosis and resolution, both by neurologists and participants. For Chat GPT, we evaluated the time taken to draft the consultation reason. A satisfaction scale from 1 to 5 was implemented, with 1 indicating a negative experience ("poor", potentially risky for the patient) and 5 highly positive ("excellent", potentially surpassing non-specialized human triage). A simple yes/no survey was also applied to participants, asking about the comprehensibility of the assistant's questions, the adequacy of referral according to urgency, and whether they considered the assistant could replace non-specialized triage or reduce emergency arrival time. |
The first interaction between participants and the virtual assistant occurred within less than a year after the event. Outcome measures were evaluated immediately after the interaction between patients and the virtual assistant.
|
Collaborators and Investigators
Collaborators
Investigators
- Principal Investigator: Mauricio F Farez, MD MPH, Fundación para la Lucha contra las Enfermedades Neurológicas de la Infancia
Publications and helpful links
General Publications
- Haug CJ, Drazen JM. Artificial Intelligence and Machine Learning in Clinical Medicine, 2023. N Engl J Med. 2023 Mar 30;388(13):1201-1208. doi: 10.1056/NEJMra2302038. No abstract available.
- Au Yeung J, Wang YY, Kraljevic Z, Teo JTH. Artificial intelligence (AI) for neurologists: do digital neurones dream of electric sheep? Pract Neurol. 2023 Nov 23;23(6):476-488. doi: 10.1136/pn-2023-003757.
- Patel UK, Anwar A, Saleem S, Malik P, Rasul B, Patel K, Yao R, Seshadri A, Yousufuddin M, Arumaithurai K. Artificial intelligence as an emerging technology in the current care of neurological disorders. J Neurol. 2021 May;268(5):1623-1642. doi: 10.1007/s00415-019-09518-3. Epub 2019 Aug 26.
Study record dates
Study Major Dates
Study Start (Actual)
Primary Completion (Actual)
Study Completion (Actual)
Study Registration Dates
First Submitted
First Submitted That Met QC Criteria
First Posted (Actual)
Study Record Updates
Last Update Posted (Actual)
Last Update Submitted That Met QC Criteria
Last Verified
More Information
Terms related to this study
Additional Relevant MeSH Terms
- Pathologic Processes
- Cardiovascular Diseases
- Vascular Diseases
- Cerebrovascular Disorders
- Brain Diseases
- Central Nervous System Diseases
- Nervous System Diseases
- Immune System Diseases
- Autoimmune Diseases of the Nervous System
- Demyelinating Diseases
- Autoimmune Diseases
- Pain
- Neurologic Manifestations
- Disease Attributes
- Neuromuscular Diseases
- Stomatognathic Diseases
- Otorhinolaryngologic Diseases
- Mouth Diseases
- Peripheral Nervous System Diseases
- Labyrinth Diseases
- Ear Diseases
- Cranial Nerve Diseases
- Vestibular Diseases
- Intracranial Hemorrhages
- Paralysis
- Facial Nerve Diseases
- Polyradiculoneuropathy
- Polyneuropathies
- Trigeminal Nerve Diseases
- Facial Neuralgia
- Seizures
- Chronic Disease
- Neuroinflammatory Diseases
- Post-Infectious Disorders
- Neuralgia
- Hemorrhage
- Vertigo
- Benign Paroxysmal Positional Vertigo
- Meningitis
- Subarachnoid Hemorrhage
- Guillain-Barre Syndrome
- Trigeminal Neuralgia
- Status Epilepticus
- Facial Paralysis
Other Study ID Numbers
- 10476 (CTEP)
Plan for Individual participant data (IPD)
Plan to Share Individual Participant Data (IPD)?
IPD Plan Description
Drug and device information, study documents
Studies a U.S. FDA-regulated drug product
Studies a U.S. FDA-regulated device product
This information was retrieved directly from the website clinicaltrials.gov without any changes. If you have any requests to change, remove or update your study details, please contact register@clinicaltrials.gov. As soon as a change is implemented on clinicaltrials.gov, this will be updated automatically on our website as well.
Clinical Trials on Stroke
-
University Hospital, GhentRecruitingStroke | Stroke, Ischemic | Stroke, Acute | Stroke Sequelae | Stroke HemorrhagicBelgium
-
Moleac Pte Ltd.RecruitingStroke | Stroke, Ischemic | Stroke Sequelae | Stroke, Cardiovascular | Strokes Thrombotic | Stroke, Embolic | Stroke, CryptogenicSingapore, Philippines
-
Moleac Pte Ltd.Not yet recruitingStroke | Stroke, Ischemic | Stroke Sequelae | Stroke, Cardiovascular | Strokes Thrombotic | Stroke, Embolic | Stroke, Cryptogenic
-
IRCCS San Camillo, Venezia, ItalyRecruitingStroke | Stroke, Ischemic | Stroke Sequelae | Stroke HemorrhagicItaly
-
Vanderbilt University Medical CenterPatient-Centered Outcomes Research Institute; University of Alabama at BirminghamEnrolling by invitationStroke | Stroke, Ischemic | Stroke, Acute | Stroke Sequelae | Engagement, Patient | Stroke HemorrhagicUnited States
-
University of MinnesotaAmerican Occupational Therapy FoundationRecruitingStroke | Stroke Sequelae | Stroke Hemorrhagic | Stroke IschemicUnited States
-
University of British ColumbiaCanadian Institutes of Health Research (CIHR); Michael Smith Foundation for...RecruitingStroke | Stroke, Ischemic | Stroke Hemorrhagic | Chronic StrokeCanada
-
University of CincinnatiMedical University of South Carolina; University of California, Los Angeles; University...RecruitingStroke | Stroke, Ischemic | Stroke, Acute | Stroke HemorrhagicUnited States
-
University of LiegeCompletedStroke, Acute | Stroke Hemorrhagic | Stroke, ComplicationBelgium
-
Turkish Stroke Research and Clinical Trials NetworkElectroCore INC; Turkish Neurological SocietyCompletedStroke | Stroke, Ischemic | Stroke, Acute | Stroke, HemorrhagicTurkey
Clinical Trials on Virtual Assistant
-
University of ValenciaUniversitat Politècnica de València; Iniciativa Social Integral; Las NavesNot yet recruitingDepression | Quality of Life | Anxiety | Loneliness
-
McGill UniversityCompletedSurgical EducationCanada
-
Seoul National University HospitalCompletedDental Contact During Direct LaryngoscopyKorea, Republic of
-
Centro Hospitalar do PortoUniversity of Trás-os-Montes and Alto Douro; Foundation for Science and Technology... and other collaboratorsRecruitingIntermittent Claudication | Peripheral Arterial Disease (PAD)Portugal
-
Convergent Engineering, Inc.National Heart, Lung, and Blood Institute (NHLBI); National Institutes of Health... and other collaboratorsNot yet recruitingRespiratory Therapy
-
University of NebraskaRecruitingLoneliness | Pain, ChronicUnited States
-
Convergent Engineering, Inc.National Heart, Lung, and Blood Institute (NHLBI); University of Miami; National...Not yet recruitingRespiratory TherapyUnited States
-
University of PittsburghWithdrawnMedication Adherence | Adverse Reaction to Drug | Medication NonadherenceUnited States