Validation Study of Sleep Tracking Devices

Validation Study of an Artificial Intelligence-based Sleep Stage Classification for a Home Sleep Tracking Device

In this study, a two-part recursive convolutional neural networks model was developed, extracting features for each epoch window independently from before and after sleep onset (epoch encoder), and then trained in the context of long-term relationships in the sleep process (sequence encoder), using an approach similar to human expert classification based on information from single-channel forehead EEG and PPG (IR, Green, Red). The classification is based on guidelines from the American Academy of Sleep Medicine and calculated six parameters: total sleep duration (TST), wake (W), N1, N2, N3, and REM.

The validation study of the developed model and the device was conducted at the Sleep Disorders Centre of the Istanbul Medical Faculty using concurrent polysomnographic data from 305 male and female patients aged 18 to 65 years.

Study Overview

• Status: Completed
• Conditions: Sleep Disorder; Sleep
• Intervention / Treatment: Device: Sleep tracking device

• Study Type: Observational
• Enrollment (Actual): 305

Contacts and Locations

Study Locations

• Turkey
  • Başıbüyük, Maltepe
    • Istanbul, Başıbüyük, Maltepe, Turkey, 34854
      • Pnaps Health Informatics and Space Technologies Inc

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: Adult; Older Adult
• Accepts Healthy Volunteers: Yes

• Sampling Method: Non-Probability Sample

Study Population

The study population includes individuals aged between 18 and 65 who have voluntarily agreed to undergo sleep measurements specifically individuals suffering from sleep disorders such as sleep apnea. Participants in the study have been referred for polysomnography (PSG) testing by neurologists, pulmonologists, psychiatrists, and otolaryngologists.

Description

Inclusion Criteria:

• Participants suffering from sleep disorders
• Participants sent to PSG test by neurologists, pulmonologists, psychiatrists, and otolaryngologists

Exclusion Criteria:

• Anyone who has been diagnosed as having a contagious skin disease
• Participants who do not have consent to have an additional device in their forehead area
• Incomplete of sleep measurement

Study Plan

How is the study designed?

Number of groups / cohorts

2

Cohorts and Interventions

Group / Cohort	Intervention / Treatment
Female; Participants aged 18-65 who identify as female for biological	Device: Sleep tracking device; In addition to polysomnography, a device containing EEG+PPG sensors for sleep classification was placed on the forehead, and another device containing PPG and accelerometer sensors was placed on the wrist. The wrist to which the device is attached is randomly assigned.
Male; Participants aged 18-65 who identify as male for biological	Device: Sleep tracking device; In addition to polysomnography, a device containing EEG+PPG sensors for sleep classification was placed on the forehead, and another device containing PPG and accelerometer sensors was placed on the wrist. The wrist to which the device is attached is randomly assigned.

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Sleep Stages Classification Accuracy	The collected EEG data were classified according to Cohen's kappa (>85), which is considered successful in the literature. Initially the open source codes YASA, tinysleepnet and attentionsleep have been implemented. These codes yielded kappa 0.64, accuracy 0.80, kappa 0.69, accuracy 0.79 and kappa 0.65, accuracy 0.78 respectively. The values obtained do not correspond to those reported in the classification articles. Subsequently, 29 participants from our own dataset were tested in these classifications as a preliminary test, with poor results. On an individual basis, the highest cappa score was 0.51. Development of our own classification system is in progress.	4-5 months
Interoception analysis from PPG data collected from facial skin	According to our preliminary analyses, we found that the intermediary rhythm (0.12-0.18 Hz) associated with interoception is also present in sleep patients. In one participant, for example, a value of 0.19 was obtained as a ratio of total sleep time. In addition, an intermediary rhythm is observed in all stages of sleep, including wakefulness, light sleep, deep sleep and REM.	4-5 months

Collaborators and Investigators

• Sponsor: PNAPS Health Informatics and Space Technologies Inc.
• Collaborators: Analog Devices
• Investigators: Study Chair: Asuman Çevik, Master's, PNAPS Health Informatics & Space Technologies Inc.; Study Director: Hasan Birol Çotuk, PhD, Marmara University

Study record dates

Study Major Dates

• Study Start (Actual): March 21, 2023
• Primary Completion (Actual): May 31, 2023
• Study Completion (Actual): July 17, 2023

Study Registration Dates

• First Submitted: February 22, 2024
• First Submitted That Met QC Criteria: April 4, 2024
• First Posted (Actual): April 10, 2024

Study Record Updates

• Last Update Posted (Actual): April 25, 2024
• Last Update Submitted That Met QC Criteria: April 23, 2024
• Last Verified: April 1, 2024

More Information

Terms related to this study

• Keywords: Sleep; Polysomnography; EEG; Wearable Device; PPG
• Additional Relevant MeSH Terms: Mental Disorders; Nervous System Diseases; Neurologic Manifestations; Sleep Wake Disorders
• Other Study ID Numbers: 09.2023.422; 2220124 (Other Grant/Funding Number: Scientific and Technological Research Council of Türkiye (TÜBİTAK)); 1065371 (Other Grant/Funding Number: Small and Medium Enterprises Development Organization (KOSGEB))

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: UNDECIDED
• IPD Plan Description: The PSG data will be shared, but the main structure is not yet planned.

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No