SeeMe: An Automated Tool to Detect Early Recovery After Brain Injury

SeeMe: A Pilot Study Developing Predictive, Real-Time Consciousness Assessment Metrics Based on Facial Expression Changes

Early prediction of outcomes after acute brain injury (ABI) remains a major unsolved problem. Presently, physicians make predictions using clinical examination, traditional scoring systems, and statistical models. In this study, we will use a novel technique, "SeeMe," to objectively assess the level of consciousness in patients suffering from comas following ABI. SeeMe is a program that quantifies total facial motion over time and compares the response after a spoken command (i.e. "open your eyes") to a pre-stimulus baseline.

Study Overview

• Status: Recruiting
• Conditions: Traumatic Brain Injury; Disorder of Consciousness; Acute Brain Injury; Trauma, Brain; Consciousness, Loss of
• Intervention / Treatment: Diagnostic test: SeeMe; Diagnostic test: Coma Recovery Scale-Revised (CRS-R)

Detailed Description

Acute brain injury (ABI) recovers at a variable rate. While some progress has been made in predicting long-term outcomes in traumatic brain injury (TBI) and intracranial hemorrhage, there is a critical need for short-term prediction of outcomes, in the first days and weeks after injury. With advances in machine learning and artificial intelligence, there is a growing interest in facial analysis and its application in neurological and psychiatric disorders. Here we describe "SeeMe," a novel automated objective measure of consciousness based on microexpression analyses in response to auditory commands. In measuring the smallest muscular movements undetectable by clinical observation, this technique has the high spatial resolution needed to detect hidden signs of recovery and the high temporal resolution needed to study neural circuits.

• Study Type: Observational
• Enrollment (Estimated): 200

Contacts and Locations

• Study Contact: Name: Sima Mofakham, PhD; Phone Number: 631-444-1278; Email: sima.mofakham@stonybrookmedicine.edu
• Study Contact Backup: Name: Charles Mikell, MD; Phone Number: 631-444-7328; Email: charles.mikell@stonybrookmedicine.edu

Study Locations

• United States
  • New York
    • Stony Brook, New York, United States, 11794
      • Recruiting
      • Stony Brook University Hospital
      • Contact: Charles Mikell, MD; Phone Number: 631-444-7328; Email: charles.mikell@stonybrookmedicine.edu
      • Contact: Sima Mofakham, PhD; Phone Number: 631-444-7328; Email: sima.mofakham@stonybrookmedicine.edu

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: Adult; Older Adult
• Accepts Healthy Volunteers: Yes

• Sampling Method: Non-Probability Sample

Study Population

Healthy volunteers: Adults 18-85 years old with no history of a neurologically debilitating disease (i.e., dementia, glioblastoma, Alzheimer's, multiple sclerosis, major vessel stroke, previous severe TBI, etc.) that are recruited via flyer and word of mouth

Patients in a coma: Adults 18-85 years old with no history of a neurologically debilitating disease that are comatose (GCS < 9) due to an acute brain injury. Patients in a comatose state will be identified during neurosurgical consultation in the hospital by the study physicians

Description

Inclusion Criteria:

• 18 years old or older
• Healthy Volunteers
• Comatose patients (patients with a GCS < 9) due to an acute brain injury (traumatic brain injury, spontaneous subarachnoid hemorrhage, severe meningoencephalitis, etc.)

Exclusion Criteria:

• A history of a neurologically debilitating disease (i.e., dementia, glioblastoma, Alzheimer's, multiple sclerosis, major vessel stroke, previous severe TBI, etc.)
• Any other medical condition that, in the judgment of the investigator, makes participation in the study unsafe.
• Pregnant subjects
• Comatose patients without a legal authorized representative (LAR)
• Prisoners or wards of the state
• Persons who have not attained the legal age for consent to treatments or procedures

Study Plan

How is the study designed?

Number of groups / cohorts

2

Cohorts and Interventions

Group / Cohort	Intervention / Treatment
Acute Brain Injury (ABI); Patients that have suffered an ABI resulting in Coma (Glasgow Coma Scale (GCS) < 9) will undergo SeeMe and CRS-R assessment once a day until hospital discharge	Diagnostic test: SeeMe; A video-recorded SeeMe command following assessment by a trained professional once a day Each session involves three command sets that are played from an audio recording 10 times over the course of 8 minutes. These commands are "Stick out your tongue", "Open your eyes", and "Show me a smile" Each command set is recorded separately for a total of 3 videos per session. These videos are then analyzed by SeeMe to detect if subjects are responding to commands.; Other Names: SeeMe session; Diagnostic test: Coma Recovery Scale-Revised (CRS-R); A video-recorded CRS-R score assessment by a trained professional once a day. A score of 10 or greater, an auditory score >2, or an arousal score > 0 means that a subject is responding to commands
Control; Healthy subjects will undergo SeeMe and CRS-R assessment once.	Diagnostic test: SeeMe; A video-recorded SeeMe command following assessment by a trained professional once a day Each session involves three command sets that are played from an audio recording 10 times over the course of 8 minutes. These commands are "Stick out your tongue", "Open your eyes", and "Show me a smile" Each command set is recorded separately for a total of 3 videos per session. These videos are then analyzed by SeeMe to detect if subjects are responding to commands.; Other Names: SeeMe session; Diagnostic test: Coma Recovery Scale-Revised (CRS-R); A video-recorded CRS-R score assessment by a trained professional once a day. A score of 10 or greater, an auditory score >2, or an arousal score > 0 means that a subject is responding to commands

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Detection of Command Following	Length of time until an intervention is able to detect that a subject is following commands	Measured Daily from enrollment until subject is following commands or date of death of any cause, whichever comes first, up to 60 days.

Collaborators and Investigators

• Sponsor: Stony Brook University
• Investigators: Principal Investigator: Sima Mofakham, PhD, Stony Brook Medicine

Publications and helpful links

General Publications

• Kalmar K, Giacino JT. The JFK Coma Recovery Scale--Revised. Neuropsychol Rehabil. 2005 Jul-Sep;15(3-4):454-60. doi: 10.1080/09602010443000425.
• Cheng F, Yu J, Xiong H. Facial expression recognition in JAFFE dataset based on Gaussian process classification. IEEE Trans Neural Netw. 2010 Oct;21(10):1685-90. doi: 10.1109/TNN.2010.2064176. Epub 2010 Aug 19.
• Zhao Y, Xu J. A Convolutional Neural Network for Compound Micro-Expression Recognition. Sensors (Basel). 2019 Dec 16;19(24):5553. doi: 10.3390/s19245553.
• Valstar MF, Pantic M. Fully automatic recognition of the temporal phases of facial actions. IEEE Trans Syst Man Cybern B Cybern. 2012 Feb;42(1):28-43. doi: 10.1109/TSMCB.2011.2163710. Epub 2011 Sep 15.
• Chouinard B, Scott K, Cusack R. Using automatic face analysis to score infant behaviour from video collected online. Infant Behav Dev. 2019 Feb;54:1-12. doi: 10.1016/j.infbeh.2018.11.004. Epub 2018 Nov 30.
• Saadon JR, Yang F, Burgert R, Mohammad S, Gammel T, Sepe M, Rafailovich M, Mikell CB, Polak P, Mofakham S. Real-time emotion detection by quantitative facial motion analysis. PLoS One. 2023 Mar 10;18(3):e0282730. doi: 10.1371/journal.pone.0282730. eCollection 2023.

Study record dates

Study Major Dates

• Study Start (Actual): June 16, 2019
• Primary Completion (Estimated): June 1, 2025
• Study Completion (Estimated): August 1, 2025

Study Registration Dates

• First Submitted: October 4, 2023
• First Submitted That Met QC Criteria: October 10, 2023
• First Posted (Actual): October 16, 2023

Study Record Updates

• Last Update Posted (Actual): October 16, 2023
• Last Update Submitted That Met QC Criteria: October 10, 2023
• Last Verified: October 1, 2023

More Information

Terms related to this study

• Keywords: Traumatic Brain Injury; Consciousness; Acute brain injury; Consciousness Recovery; Consciousness Measurement; Facial Motion Analysis; Microscopic movements
• Additional Relevant MeSH Terms: Mental Disorders; Brain Diseases; Central Nervous System Diseases; Nervous System Diseases; Neurologic Manifestations; Neurobehavioral Manifestations; Neurocognitive Disorders; Craniocerebral Trauma; Trauma, Nervous System; Brain Injuries; Wounds and Injuries; Brain Injuries, Traumatic; Consciousness Disorders; Unconsciousness
• Other Study ID Numbers: IRB2019-00199

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: NO

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No
• product manufactured in and exported from the U.S.: No