Development and Validation of Deep Neural Networks for Blinking Identification and Classification

Primary objective of this study is the development and validation of a system of deep neural networks which automatically detects and classifies blinks as "complete" or "incomplete" in image sequences.

Study Overview

• Status: Completed
• Conditions: Deep Learning; Blinking
• Intervention / Treatment: Diagnostic test: Comparison of the proposed artificial network with the ground truth

Detailed Description

This method is based on iris and sclera segmentation in both eyes from the acquired images, using state of the art deep learning encoder-decoder neural architectures (DLED). The sequence of the segmented frames is post-processed to calculate the distance between the eyelids of each eye (palpebral fissure) and the corresponding iris diameter. Theses quantities are temporally filtered and their fraction is subject to adaptive thresholding to identify blinks and determine their type, independently for each eye. The two DLEDs were trained with manually segmented images and the post-process was parameterized using a 4-minute video. After DLED training, the proposed system was tested on 8 different subjects, each one with a 4-10-minute video. Several metrics of blink detection and classification accuracy were calculated against the ground truth, which was generated by 3 independent experts, whose conflicts were resolved by a senior expert. Two independent blink identifications are assumed to be in agreement, if and only if there is sufficient temporal overlapping and the type of blink is the same between the DLED system and the ground truth.

• Study Type: Observational
• Enrollment (Actual): 8

Contacts and Locations

Study Locations

• Greece
  • Evros
    • Alexandroupolis, Evros, Greece, 68100
      • Department of Ophthalmology, University Hospital of Alexandroupolis
  • Thessaly
    • Lamia, Thessaly, Greece, 35100
      • Department of Computer Science and Biomedical Informatics, University of Thessaly

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: 18 years to 75 years (Adult, Older Adult)
• Accepts Healthy Volunteers: No
• Genders Eligible for Study: All

• Sampling Method: Probability Sample

Study Population

Patients aged from 18 to 75 years with Uncorrected Distance Visual Acuity ≥ 5/10

Description

Inclusion Criteria

• Uncorrected Distance Visual Acuity above 6/12

Exclusion Criteria:

• corneal opacities
• age-related macular degeneration
• diagnosis of psychiatric diseases
• former eyelid surgery

Study Plan

How is the study designed?

Design Details

• Observational Models: Cohort
• Time Perspectives: Prospective

Number of groups / cohorts

1

Cohorts and Interventions

Group / Cohort

Intervention / Treatment

Study group; 8 patients aged between 18 to 75 years with Uncorrected Distance Visual Acuity ≥ 5/10

Diagnostic test: Comparison of the proposed artificial network with the ground truth; Both eyes will be included for each study participant. Participants watched a 4-10-minute video in standard mesopic environmental lighting conditions at 3.5m viewing distance. Simultaneously, all blinking moves will be recorded through a web infrared camera. The proposed system was tested on the 8 different subjects. Several metrics of blink detection and classification accuracy were calculated against the ground truth, which was generated by 3 independent experts, whose conflicts were resolved by a senior expert. Two independent blink identifications are assumed to be in agreement, if and only if there is sufficient temporal overlapping and the type of blink is the same between the DLED system and the ground truth.

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Identification of complete and incomplete blinks	Complete and incomplete blinks are defined by the "length of palpebral fissure-to-iris diameter" ratio	up to 1 week
First frame of each blink	The frame in which the upper eyelid starts to move down and cover the cornea	up to 1 week
Last frame of each blink	The frame in which eyelids open fully after a blink	up to 1 week

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Length of palpebral fissure of both eyes	The distance between the upper eyelid margin and the lower eyelid margin (ie. the vertical dimension of the palpebral fissure),	up to 1 week
Iris diameter of both eyes	The horizontal diameter of the iris (ie. the horizontal white-to white distance)	up to 1 week

Collaborators and Investigators

• Sponsor: Democritus University of Thrace
• Collaborators: University of Thessaly

Publications and helpful links

General Publications

• Nousias G, Panagiotopoulou EK, Delibasis K, Chaliasou AM, Tzounakou AM, Labiris G. Video-Based Eye Blink Identification and Classification. IEEE J Biomed Health Inform. 2022 Jul;26(7):3284-3293. doi: 10.1109/JBHI.2022.3153407. Epub 2022 Jul 1.

Study record dates

Study Major Dates

• Study Start (Actual): October 1, 2020
• Primary Completion (Actual): March 10, 2021
• Study Completion (Actual): March 25, 2021

Study Registration Dates

• First Submitted: March 29, 2021
• First Submitted That Met QC Criteria: March 31, 2021
• First Posted (Actual): April 1, 2021

Study Record Updates

• Last Update Posted (Actual): January 4, 2023
• Last Update Submitted That Met QC Criteria: January 2, 2023
• Last Verified: January 1, 2023

More Information

Terms related to this study

• Keywords: deep learning; DeepLabv3+; iris and sclera segmentation; eye blink detection; complete and incomplete blink classification
• Other Study ID Numbers: ES2/Th15/25-2-2021

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: Undecided

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No