Novel Multimodal Neural, Physiological, and Behavioral Sensing and Machine Learning for Mental States

In this program, the investigators will develop novel multimodal neural-behavioral-physiological monitoring tools (software and hardware), and machine learning models for mental states within social processes and beyond. The tools consist of a multimodal skin-like wearable sensor for physiological and biochemical sensing; a conversational virtual human platform to evoke naturalistic social processes; audiovisual affect recognition software; synchronization tools; and machine learning methods to model the multimodal data. The investigators will demonstrate the tools in healthy subjects without neural recordings and in patients with drug-resistant epilepsy who already have intracranial EEG (iEEG) electrodes implanted based on clinical criteria for standard monitoring to localize seizures, which is unrelated to our study.

Study Overview

• Status: Recruiting
• Conditions: No Condition; Study Mental State in Healthy Populations and Drug-resistant Epilepsy Patients With Existing iEEG
• Intervention / Treatment: Behavioral: Virtual human social interaction; Behavioral: Cold pressor test

• Study Type: Interventional
• Enrollment (Estimated): 90
• Phase: Not Applicable

Contacts and Locations

Study Locations

• United States
  • California
    • Downey, California, United States, 90242
      • Recruiting
      • Rancho Los Amigos National Rehabilitation Center
      • Contact: Charles Liu, MD, PhD; Phone Number: 323-226-7421; Email: cliu@usc.edu
    • Los Angeles, California, United States, 90007
      • Recruiting
      • University of Southern California
      • Contact: Maryam M Shanechi, PhD; Phone Number: 213-740-1377; Email: shanechi@usc.edu
      • Principal Investigator: Maryam M Shanechi, PhD
      • Contact: HyeongChan Jo, PhD; Email: hj_380@usc.edu

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: Adult; Older Adult
• Accepts Healthy Volunteers: Yes

Description

For the healthy population, all subjects over the age of 18 who are able and willing to give informed consent will be eligible.

For the epilepsy population, inclusion criteria are:

• Patients who suffer from drug-resistant epilepsy and already have intracranial EEG (iEEG) electrodes implanted based on clinical criteria for their standard seizure localization (unrelated to our study) will be eligible. Most patients are healthy adults outside of their epilepsy.
• Subjects >= 18 are only included in this study.
• All patients with the above conditions and with already-implanted electrodes who are willing to participate and able to cooperate and follow research instructions will be recruited.

Study Plan

How is the study designed?

Design Details

• Primary Purpose: Basic Science
• Allocation: N/A
• Interventional Model: Single Group Assignment
• Masking: None (Open Label)

Number of Arms

1

Arms and Interventions

Participant Group / Arm	Intervention / Treatment
Experimental: Behavioral tasks to assess mental states	Behavioral: Virtual human social interaction; The participants will perform a social interaction with the virtual human agent platform to assess mental states.; Behavioral: Cold pressor test; Healthy participants will immerse one hand in cold water for less than 3 minutes, while physiological data will be collected by the new wearable sensor from the other hand.

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Positive and Negative Affect Schedule (PANAS)	Momentary self-reports of the Positive and Negative Affect Schedule (PANAS) will be administered in a tablet-based application, which measures momentary positive and negative affect. These self-reports return scores indicative of momentary mental state. Self-reports are obtained intermittently from the participants.	7-10 days
Visual Analog Scale (VAS)	Momentary self-reports of the Visual Analog Scale (VAS) of depression, anxiety, and energy will be administered in a tablet-based application. These are fast self-reports to indicate the affect level along a line from none to prominent. These self-reports return scores indicative of momentary mental state. Self-reports are obtained intermittently from the participants.	7-10 days
Hamilton Depression Rating Scales (HAMD-6)	Momentary self-reports of Hamilton Depression Rating Scales (HAMD-6) will be administered in a tablet-based application, which is widely used to measure the severity of the depressive state. These self-reports return scores indicative of momentary mental state. Self-reports are obtained intermittently from the participants.	7-10 days
Hamilton Anxiety Rating Scales (HAM-A)	Momentary self-reports of Hamilton Anxiety Rating scales (HAM-A) will be administered in a tablet-based application, which is widely used to measure the severity of the anxiety state. These self-reports return scores indicative of momentary mental state. Self-reports are obtained intermittently from the participants.	7-10 days
Immediate Mood Scalar (IMS)	Momentary self-reports of the Immediate Mood Scalar (IMS) will be administered in a tablet-based application, which measures depression and anxiety severity. These self-reports return scores indicative of momentary mental state. Self-reports are obtained intermittently from the participants.	7-10 days
Fear of Negative Evaluation Brief	Momentary self-reports of Fear of Negative Evaluation Brief will be administered in a tablet-based application. These self-reports return scores indicative of momentary mental state. Self-reports are obtained intermittently from the participants.	7-10 days

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Physiological measurements - heart rate variability (HRV)	Heart rate variability will be collected from each participant by using either commercial systems or the new wearable sensor.	7-10 days
Physiological measurements - skin conductance (SC)	Skin conductance will be collected from each participant by using either commercial systems or the new wearable sensor.	7-10 days
Physiological measurements - cortisol level	Cortisol level will be collected from each participant by using either commercial systems or the new wearable sensor.	7-10 days
Neural measurements	In epilepsy patients who already have intracranial EEG recordings for their standard clinical monitoring (unrelated to this study), we will also obtain iEEG activity measurements. EEG power will be the main focus of the analysis, but the raw signal will also be analyzed.	7-10 days

Collaborators and Investigators

• Sponsor: University of Southern California
• Collaborators: National Institute of Mental Health (NIMH)
• Investigators: Principal Investigator: Maryam Shanechi, PhD, University of Southern California

Study record dates

Study Major Dates

• Study Start (Actual): August 1, 2024
• Primary Completion (Estimated): November 30, 2028
• Study Completion (Estimated): November 30, 2028

Study Registration Dates

• First Submitted: July 18, 2025
• First Submitted That Met QC Criteria: July 31, 2025
• First Posted (Actual): August 7, 2025

Study Record Updates

• Last Update Posted (Estimated): October 1, 2025
• Last Update Submitted That Met QC Criteria: September 26, 2025
• Last Verified: July 1, 2025

More Information

Terms related to this study

• Keywords: neurotechnology; neuropsychiatric disorders; brain-machine interface
• Additional Relevant MeSH Terms: Brain Diseases; Central Nervous System Diseases; Nervous System Diseases; Epilepsy; Drug Resistant Epilepsy
• Other Study ID Numbers: R61MH135407 (U.S. NIH Grant/Contract)

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: YES
• IPD Plan Description: Upon completion of the study and after publication, fully de-identified IPD may be made available for research purposes.
• IPD Sharing Time Frame: Any sharing will happen upon completion of the study and after publication.

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No