Clinical Validation of a Video-based Epilepsy Examination Service (Nelli)

The main purpose of this study is to validate the ability of the Nelli system (video-based epilepsy examination system) to detect epileptic motor seizure behaviors in patients at rest.

Study Overview

• Status: Completed
• Conditions: Epilepsy; Seizure

Detailed Description

Proper seizure documentation is an important tool to evaluate treatment outcomes and risks associated with epileptic seizures. Apart from a costly in-hospital video electro-encephalogram (VEEG) monitoring, most patients with epilepsy log and track their seizures with the help of the seizure diaries only. However, the reliability of the diaries is heavily dependent on an accurate recognition and recording of the seizures. There is an obvious need for a more objective and reliable seizure detection method. The investigators propose a low-cost video-based examination system, designed to detect motor seizure behaviors in patients at rest.

The main aim of this study is to clinically validate the ability of the Nelli system (video-based epilepsy examination system) to detect epileptic motor seizure behaviors in patients at rest, using simultaneous VEEG as the ground truth with the reference to all motor seizures and individual motor seizure types.

Study design established as follows:

1. Study population is represented by 90 epilepsy patients with suspected history of motor seizures undergoing a standard VEEG examination for clinical evaluation of epilepsy.
2. Co-registration of standard VEEG with the Nelli video- and audio-based system is performed, duration of the registration is 1-4 days.

3. The inclusion criteria for seizures:

   1. Epileptic motor seizures appearing from rest while patients are in supine position in their monitoring bed.
   2. The seizures are determined to be epileptic based on electroclinical characteristics by the responsible neurophysiologist / epileptologist.
   3. The seizures can be classified into motor seizure types (both focal and generalized) using the 2017 International League Against Epilepsy (ILAE) seizure classification.

4. The registrations are analyzed by the Nelli system blinded to any clinical information. Nelli system consists of two evaluation components:

   1. Algorithmic component (pre-trained machine learning model based on video and audio signals)
   2. Supervisory component (performed by experienced VEEG technicians)

5. Reports containing the seizure type classification (time-stamped) are provided by the VEEG laboratory, the performance of the Nelli system is assessed using these reports as the ground truth.

   a. Specificity and sensitivity of the Nelli system are evaluated with reference to both all motor seizures and individual motor seizure types.

6. The secondary outcome measures are established as follows:

   1. To assess the latency from the onset of the seizure detection by the Nelli system in comparison to VEEG registration.
   2. To determine an inter-rater agreement for the supervisory component of the Nelli system.
   3. To investigate the performance of the algorithmic component of the Nelli system alone.

• Study Type: Observational
• Enrollment (Actual): 182

Contacts and Locations

Study Locations

• Denmark
  • Dianalund, Denmark
    • Danish Epilepsy Center, Filadelfia

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: Child; Adult; Older Adult
• Accepts Healthy Volunteers: No
• Genders Eligible for Study: All

• Sampling Method: Non-Probability Sample

Study Population

90 epilepsy patients of various ages undergoing a standard VEEG examination for clinical evaluation in either one of the following medical facilities: Danish Epilepsy Center (Filadelfia) or Aarhus University Hospital.

Description

Inclusion Criteria:

Patients with epilepsy undergoing a standard VEEG examination for clinical evaluation consenting to participate in the present study

Exclusion Criteria:

Patients not consenting to participate in the study

Study Plan

How is the study designed?

Design Details

• Observational Models: Cohort
• Time Perspectives: Prospective

Number of groups / cohorts

2

Cohorts and Interventions

Group / Cohort
Adults; Subjects 22+ years of age
Pediatric; Subjects less than 22 years of age

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
FDR (per hour) and TPR	False detection rate (FDR/hour) as well as true positive rate (TPR) of the Nelli service is to be evaluated. Here FDR stands for detections which do not intersect with the ground truth events, and true positive events match ground truth with the Nelli detections.	Examination period specific to the patient (1-4 days)

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Latency (seconds)	Latency from the seizure onset (according to the ground truth) and the seizure detection by the Nelli system.	Examination period specific to the patient (1-4 days)
Inter-rater agreement (Cohen's kappa coefficient)	Inter-rater agreement for the supervisory component of the Nelli system	Examination period specific to the patient (1-4 days)
Performance of the algorithmic component	Performance of the algorithmic component evaluated using the ROC (receiver operating characteristic) curve at different operating points (thresholds).	Examination period specific to the patient (1-4 days)

Collaborators and Investigators

• Sponsor: Tampere University
• Collaborators: Aarhus University Hospital; Filadelfia Epilepsy Hospital; Neuro Event Labs Oy

Study record dates

Study Major Dates

• Study Start (Actual): June 17, 2019
• Primary Completion (Actual): July 2, 2021
• Study Completion (Actual): December 15, 2021

Study Registration Dates

• First Submitted: November 12, 2019
• First Submitted That Met QC Criteria: November 18, 2019
• First Posted (Actual): November 21, 2019

Study Record Updates

• Last Update Posted (Actual): October 13, 2022
• Last Update Submitted That Met QC Criteria: October 8, 2022
• Last Verified: October 1, 2022

More Information

Terms related to this study

• Additional Relevant MeSH Terms: Brain Diseases; Central Nervous System Diseases; Nervous System Diseases; Neurologic Manifestations; Epilepsy; Seizures
• Other Study ID Numbers: CS001

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No