Effects of Peripheral Haptic Feedback on Intracortical Brain-Computer Interface Control and Associated Sensory Responses in Motor Cortex

Abstract

Intracortical brain-computer interfaces (iBCIs) provide people with paralysis a means to control devices with signals decoded from brain activity. Despite recent impressive advances, these devices still cannot approach able-bodied levels of control. To achieve naturalistic control and improved performance of neural prostheses, iBCIs will likely need to include proprioceptive feedback. With the goal of providing proprioceptive feedback via mechanical haptic stimulation, we aim to understand how haptic stimulation affects motor cortical neurons and ultimately, iBCI control. We provided skin shear haptic stimulation as a substitute for proprioception to the back of the neck of a person with tetraplegia. The neck location was determined via assessment of touch sensitivity using a monofilament test kit. The participant was able to correctly report skin shear at the back of the neck in 8 unique directions with 65% accuracy. We found motor cortical units that exhibited sensory responses to shear stimuli, some of which were strongly tuned to the stimuli and well modeled by cosine-shaped functions. In this article, we also demonstrated online iBCI cursor control with continuous skin-shear feedback driven by decoded command signals. Cursor control performance increased slightly but significantly when the participant was given haptic feedback, compared to the purely visual feedback condition.

Figures

Fig. 1:

Skin-shear haptic stimulation on back of neck. (A) Target stimulus location for participant T5 is at the center of the C4 dermatome. The inlay illustrates the 8 radial directions along which shear force was provided during the perception study. (B) Actual photograph of the haptic device configuration. Dermatome images adapted from Janet Fong [33].

Fig. 2:

Artificial proprioception via skin-shear haptic feedback. Neural firing rates are measured and translated to a two-dimensional velocity command vector VKF by a Kalman filter. The VKF command vector simultaneously drives the velocity of a virtual cursor (VC) on a computer monitor, and the shear force (Fshear) produced by a haptic device on the back of the participant’s neck. Dermatome image adapted from Janet Fong [33].

Fig. 3:

Velocity-shear mapping function and force tracking. (A) Distribution of X- and Y-direction velocity commands from a typical closed-loop iBCI cursor control task. Units are in workspace width per second (WW/s). Black lines bound the 95th percentile (±0.35 WW/s). (B) Velocity-shear saturation function (black line) with average measured force plotted as a function of velocity command for the X (red) and Y (blue) directions. The 95% CIs are plotted but are smaller than the width of the plotted points. Data is from a 10-minute iBCI cursor control block with haptic feedback. (C) Force tracking example from a 10 second snippet of data from the task used in panel B. Velocity commands (gold) are mapped to shear force commands (purple) using the function in panel B. The black trace is measured force sensor data.

Fig. 4:

Sensitivity to touch. (A) Able-bodied group average and T5’s sensitivity with corresponding classification colors. (B) Means and 95% CIs for sensitivity (gram-force) at each probing location for the able-bodied group (□) and T5’s responses (x). Dermatome images adapted from Janet Fong [33].

Fig. 5:

Validation of shear force stimuli. The Polar plot depicts radial directions in degrees where each ring represents a force magnitude in units of Newtons. Values of the average measured directions and magnitudes are provided in Table 1.

Fig. 6:

Cognitive perception of sensory stimulus versus decoding sensory stimulus from neural firing rates. (A) Confusion matrix of participant T5’s shear direction perception. (B) A Gaussian Naïve Bayes classifier was used to classify each trial’s stimulus using firing rates within the 500 to 900 ms window after stimulus onset. Only channels which significantly responded to shear stimulation were used (21 channels). For each matrix, the entry (i,j) in the matrix is colored according to the percentage of trials where stimulus j was decoded/predicted (out of all trials where stimulus i was cued).

Fig. 7:

Peristimulus time histograms, shown as red traces for four example electrode channels (columns) across skin-shear direction (rows). Firing rates (mean ± 95% CIs) and the window during which shear stimuli were provided (gray) as a step input are shown.

Fig. 8:

Shear-related tuning strength across electrode arrays. The strength of each electrode’s tuning to shear stimulus is indicated with a shaded gold color (darker indicates more tuning). Tuning strength was quantified by computing the fraction of total firing rate variance accounted for (FVAF) by changes in firing rate due to the stimulus directions. Crosses represent “non-functioning” electrodes. Small gray circles indicate channels with no significant tuning to shear stimulation. Larger colored circles indicate significantly tuned channels.

Fig. 9:

Tuning curves for shear stimulation. Example tuning curves show the range of shapes observed across significantly tuned electrode channels. Each panel corresponds to a single exemplary electrode channel. Blue curves indicated mean threshold crossing firing rates within a 500 to 900 ms window after stimulus onset with linear interpolation; error bars show standard error of mean. Red curves are fits to a cosine-tuning model with corresponding adjusted R2 values shown in the upper-right corner of each panel. Bottom right: adjusted R2 histogram for all 21 significantly tuned electrode channels. Values less than 0 indicate that a linear fit is better than the cosine-tuning model.

Fig. 10:

iBCI cursor control performance with and without haptic feedback. (A) Timeline of iBCI comparison blocks across 7 sessions (day numbers are days after implantation). Red clusters indicate Haptics blocks where the participant received skin-shear haptic feedback, and gray clusters indicate No Haptics blocks where haptic feedback was disabled. Each dot shows one trial’s time to target. Horizontal bars show the median time to target of each block. Arrows on the right show the median across all trials of each condition. (B) Median time to target, path efficiency, and dial-in time within each session is plotted for each condition. Asterisks indicate statistical significance level: * p≤ 0.05, ** p≤ 0.01.