The Neural Representation of Force across Grasp Types in Motor Cortex of Humans with Tetraplegia

Abstract

Intracortical brain-computer interfaces (iBCIs) have the potential to restore hand grasping and object interaction to individuals with tetraplegia. Optimal grasping and object interaction require simultaneous production of both force and grasp outputs. However, since overlapping neural populations are modulated by both parameters, grasp type could affect how well forces are decoded from motor cortex in a closed-loop force iBCI. Therefore, this work quantified the neural representation and offline decoding performance of discrete hand grasps and force levels in two human participants with tetraplegia. Participants attempted to produce three discrete forces (light, medium, hard) using up to five hand grasp configurations. A two-way Welch ANOVA was implemented on multiunit neural features to assess their modulation to force and grasp Demixed principal component analysis (dPCA) was used to assess for population-level tuning to force and grasp and to predict these parameters from neural activity. Three major findings emerged from this work: (1) force information was neurally represented and could be decoded across multiple hand grasps (and, in one participant, across attempted elbow extension as well); (2) grasp type affected force representation within multiunit neural features and offline force classification accuracy; and (3) grasp was classified more accurately and had greater population-level representation than force. These findings suggest that force and grasp have both independent and interacting representations within cortex, and that incorporating force control into real-time iBCI systems is feasible across multiple hand grasps if the decoder also accounts for grasp type.

Keywords: brain-computer interface; force; grasp; kinetic; motor cortex.

Copyright © 2021 Rastogi et al.

Figures

Figure 1.

Data collection scheme for research sessions. A, Experimental setup (adapted from Rastogi et al., 2020). Participants had two 96-channel microelectrode arrays placed chronically in motor cortex, which recorded neural activity while participants completed a force task. TC and SBP features were extracted from these recordings. Figure 1A is reprinted by permission from Springer Nature as indicated in the Terms and Conditions of a Creative Commons Attribution 4.0 International license (https://www.nature.com/srep/). B, Research session architecture. Each session consisted of 12–21 blocks, each of which contained ∼20 trials (see Table 1). In each trial, participants attempted to generate one of three visually-cued forces with one of four grasps: power, closed pinch, open pinch, ring pinch. During session 5, participant T8 also attempted force production using elbow extension. Each trial contained a preparatory (prep) phase, a go phase where forces were actively embodied, and a stop phase where neural activity was allowed to return to baseline. Participants were prompted with both audio and visual cues, in which a researcher squeezed or lifted an object associated with each force level. During pinch blocks, the researcher squeezed the pinchable objects (cotton ball, eraser, nasal aspirator tip) using the particular pinch grip dictated by the block (ring pinch, open pinch, closed pinch). Here, only closed pinches of objects are shown.

Figure 2.

Exemplary TC and SBP features tuned to task parameters of interest in participant T8 (TC and SBP features in participant T5 are illustrated in Extended Data Fig. 2-1). Rows indicate average per-condition activity (PSTH) of four exemplary features tuned to force, grasp, both factors, and an interaction between force and grasp, recorded during session 5 from participant T8 (two-way Welch-ANOVA, corrected p < 0.05, Benjamini–Hochberg method). Bolded, starred p values indicate significant tuning to force (Rows 1 and 3), grasp (Rows 2 and 3), or a force-grasp interaction (Row 4). Neural activity was normalized by subtracting block-specific mean feature activity within each recording block, and then smoothed with a 100-ms Gaussian kernel to aid in visualization. Column 1 contains PSTHs averaged within individual force levels (across multiple grasps), such that observable differences between data traces are because of force alone. Similarly, column 2 shows PSTHs averaged within individual grasps (across multiple forces). Column 3 shows a graphical representation of the simple main effects as normalized mean neural deviations from baseline activity during force trials within each of the five grasps. (cp, c-pinch = closed pinch; op, o-pinch = open pinch; rp, r-pinch = ring pinch, pow = power, elb = elbow extension). Mean neural deviations were computed over the go phase of each trial and subsequently averaged within each force-grasp pair. Error bars indicate 95% confidence intervals.

Figure 3.

Summary of neural feature population tuning to force and grasp. Row 1, Fraction of neural features significantly tuned to force, grasp, both force, and grasp and an interaction between force and grasp in participants T8 and T5 (two-way Welch-ANOVA, corrected p < 0.05). Row 2, Fraction of neural features significantly tuned to an interaction between force and grasp, subdivided into force-tuned features within each individual grasp (c-pinch = closed pinch, o-pinch = open pinch, r-pinch = ring pinch). Note that the number of grasp types differed between sessions (see Table 1).

Figure 4.

Simulated models of independent and interacting (grasp-dependent) neural representations of force. Row 1, Equations corresponding to the independent and interacting models of force representation. Here, xij represents neural feature activity generated during a particular grasp i and force j, gi represents baseline feature activity during grasp i, f represents force-related neural feature activity, and sj is a discrete force level. Row 2, Simulated population neural activity projected into a two-dimensional PCA space. Estimated force axes within the low-dimensional spaces are shown as blue lines. Row 3, Summary of variances accounted for by the top 20 dPCs extracted from the simulated neural data from each model. Here, the variance of each individual component is separated by marginalization (force, grasp, and interaction between force and grasp). Pie charts indicate the percentage of total signal variance due to these marginalizations.

Figure 5.

Neural population-level activity patterns. A, Demixed principal components (dPCs) isolated from all force-grasp conditions from T8 session 5, all force-grasp conditions from T5 session 7, and power versus elbow conditions from T8 session 5 neural data. dPCs were tuned to four marginalizations of interest: Condition-Independent (CI) tuning (i.e., time), Force, Grasp, and an interaction between force and grasp (FxGrasp). dPCs that account for the highest amount of variance in the per-marginalization neural activity are shown here. These variances are included in brackets next to each component number. Vertical bars indicate the start and end of the go phase. Horizontal bars indicate time points at which the decoder axes of the pictured components classified forces (row 2), grasps (row 3), or force-grasp pairs (row 4) significantly above chance. B, Summary of variances accounted for by the top 20 dPCs and PCs from each session. Here, the variance accounted for by the dPCs approaches the variance accounted for by traditional PCs. Horizontal dashed lines indicate total signal variance, excluding noise. Row 2 shows the variance of each individual component, separated by marginalization. C, Go-phase activity within a two-dimensional PCA space. Estimated force axes within the low-dimensional PCA spaces are shown as blue lines. Here, c-pinch = closed pinch, o-pinch = open pinch, r-pinch = ring pinch. D, Encoding model performances. The task-dependent components of neural feature activity were fit to the additive, scalar, and combined encoding models via cross-validated ordinary least squares regression. Tables contain the fit model coefficients for each session. Bar graphs indicate mean R2 values for each model over 100 iterations of Monte Carlo leave-group-out cross-validation. Error bars indicate SDs across iterations. Stars indicate statistically significant differences between model R2 values; **p < 0.01 and ***p < 0.001.

Figure 6.

Time-dependent classification accuracies for force (rows 1–2) and grasp (row 3). Data traces were smoothed with a 100-ms boxcar filter to aid in visualization. Shaded areas surrounding each data trace indicate the SD across 240 session-runs for most trials in participant T8, 40 session-runs for elbow extension trials in participant T8, and 40 session-runs in participant T5. Gray shaded areas indicate the upper and lower bounds of chance performance over S × 100 shuffles of trial data, where S is the number of sessions per participant. Time points at which force or grasp is decoded above the upper bound of chance are deemed to contain significant force-related or grasp-related information. Blue shaded regions indicate the time points used to compute go-phase confusion matrices in Figure 7. Here, c-pinch = closed pinch, o-pinch = open pinch, r-pinch = ring pinch. Time-dependent classification accuracies for individual force levels and grasp types are shown in Extended Data Figure 6-1. Grasp classification accuracies, separated by number of attempted grasp types, are presented in Extended Data Figure 6-2. Force classification accuracies, separated by individual session, are presented in Extended Data Figure 6-3.

Figure 7.

Go-phase confusion matrices. A, Time-dependent classification accuracies (shown in Fig. 6) were averaged over go-phase time windows that commenced when performance exceeded 90% of maximum and ended with the end of the go phase. These yielded mean trial accuracies, which were then averaged over all session-runs in each participant. Overall force and grasp classification accuracies are indicated above each confusion matrix. SDs across multiple session-runs are indicated next to mean accuracies (cp = closed pinch, op = open pinch, rp = ring pinch, pow = power, elb = elbow extension). Statistical comparisons between the achieved classification accuracies are shown in Extended Data Figure 7-1. B, Confusion matrices, now separated by the grasps (c-pinch = closed pinch, o-pinch = open pinch, r-pinch = ring pinch, power, elbow) that participants T8 (row 1) and T5 (row 2) used to attempt producing forces. Statistical comparisons between the achieved force accuracies are shown in Extended Data Figures 7-2, 7-3.

Figure 8.

Go-phase force classification accuracy for novel (test) grasps. Within each session (rows), dPCA force decoders were trained on neural data generated during all grasps, excluding a single leave-out grasp type (columns). The force decoder was then evaluated over the set of training grasps (gray bars), as well as the novel leave-out grasp type (white bars). Stars indicate statistically significant differences in performance between training and novel grasps; **p < 0.01, ***p < 0.001. Error bars indicate the 95% confidence intervals. The horizontal dotted line in each panel indicates upper bound of the empirical chance distribution for force classification. Here, c-pinch = closed pinch, o-pinch = open pinch, r-pinch = ring pinch.