Supplemental vibrotactile feedback control of stabilization and reaching actions of the arm using limb state and position error encodings

Abstract

Background: Deficits of kinesthesia (limb position and movement sensation) commonly limit sensorimotor function and its recovery after neuromotor injury. Sensory substitution technologies providing synthetic kinesthetic feedback might re-establish or enhance closed-loop control of goal-directed behaviors in people with impaired kinesthesia.

Methods: As a first step toward this goal, we evaluated the ability of unimpaired people to use vibrotactile sensory substitution to enhance stabilization and reaching tasks. Through two experiments, we compared the objective and subjective utility of two forms of supplemental feedback - limb state information or hand position error - to eliminate hand position drift, which develops naturally during stabilization tasks after removing visual feedback.

Results: Experiment 1 optimized the encoding of limb state feedback; the best form included hand position and velocity information, but was weighted much more heavily toward position feedback. Upon comparing optimal limb state feedback vs. hand position error feedback in Experiment 2, we found both encoding schemes capable of enhancing stabilization and reach performance in the absence of vision. However, error encoding yielded superior outcomes - objective and subjective - due to the additional task-relevant information it contains.

Conclusions: The results of this study have established the immediate utility and relative merits of two forms of vibrotactile kinesthetic feedback in enhancing stabilization and reaching actions performed with the arm and hand in neurotypical people. These findings can guide future development of vibrotactile sensory substitution technologies for improving sensorimotor function after neuromotor injury in survivors who retain motor capacity, but lack proprioceptive integrity in their more affected arm.

Keywords: Kinesthesia; Proprioception; Sensory augmentation; Sensory substitution; Stroke.

Figures

Fig. 1

Simplified model of closed-loop feedback control for goal-directed reaching. a Simplified model demonstrating how feedback delay (∆) and information content (Sensor Function) impacts performance of a proportional controller regulating the position θa of a damped inertial “limb” modeled as a second order differential equation relating changes in limb kinematics (position, velocity and acceleration) to changes in the control input u. Controller gain φ was varied to test the capabilities of the model system. b Simulation results of limb displacement (vertical axis) plotted as a function of time (horizontal axis) when the feedback path emulates proprioception (i.e., Delay ∆ = 0.06 s and Sensor Function θf = θa + 0.15 dθa/dt). Arrow indicates the time of change in desired position (depicted in arbitrary units of displacement). Dotted line: t = 1 s. Grey band: goal target zone. The limb obtains the goal within the time constraint over a broad range of controller gains with position + velocity feedback (Thick blue trace: φ = 20; Thin trace: φ = 130). c Simulated reaching under visual guidance (Red: Delay ∆ = 0.12 s and Sensor Function θf = θa; Thick red trace: φ = 5; Thin trace: φ = 10; dashed trace: φ = 20). With position feedback, no value of φ enables success when ∆ = 0.12 s. Also shown (Purple; φ = 20) is an acceptable solution obtained when simulated visual feedback also includes velocity information: θa + 0.15 dθa/dt. For panels b and c, the horizontal scale bar depicts 1 s whereas the vertical scale bar represents 5 cm displacement

Fig. 2

Experimental setup and protocol. a Participant at robot holding the end effector of a planar manipulandum, with integrated forearm support. An opaque screen occluded direct visual feedback of task performance; the left arm shows the standard placement of the four tactors (red dots). b Tasks. Left: stabilizing the hand at a fixed point in space against robotic perturbations. Right: example of a center-out reaching movement. c Sequence of events in each experiment. E1: Experiment 1. E2: Experiment 2; baseline 2 and test 2 were counter balanced in order across participants. Visual feedback (V) and vibrotactile feedback (T) was either continuous (+), absent (−), or only used for providing the results at the end of each task (KR). This sequence was used during 2 sessions, in which the only difference was that the vibration feedback encoded either error or state

Fig. 3

Tactor activation characteristics. a Exemplar activation mapping for state feedback of hand position wherein displacements of the hand were mapped onto tactor excitation voltages, as a percentage of Full Scale Range (FSR = 5.0 V). b Realized mapping between tactor activation and vibration amplitude (data points), and a solid line representing the best-fit polynomial reveal a nonlinear, monotonically increasing relationship over the entire half-workspace encoded by each tactor. A dashed line fit between the lowest and highest sample points provides an “ideal” linear reference for comparison. Grey shading indicates the half-workspace spanned by the home and far targets. c Pulse-step scheme employed to decrease the response time of the tactors. See text for details

Fig. 4

Vibrotactile information encoding schemes: spectrograms and time-aligned time series waveforms. a Error feedback encoding scheme. Time series (bottom) and spectrogram (top) of vibrotactile feedback during an exemplar reach (left) and a portion of a stabilization trial (right). Frequency axes and time scale as in panel (c). For the time series, the scale bars in the bottom panel (left) represent 0.1 G (vertical axis) and 1 s (horizontal axis). b Optimal state encoding scheme (λ = 0.2). c Sham feedback encoding scheme. See text for details. Colorbar: signal power relative to total signal power, in units of dB

Fig. 5

Experiment 1: Selected subject performance in the stabilization task (λ = 1.0). a Hand trajectory showing drift over time (line shading). Drift was modeled from t = 5 s to the end of the trial at t = 60 s. b Time course of the x (black) and y (blue) components of the endpoint trajectory from t = 5 s to t = 60 s. Scale bars: as in panel c. c Time course of the x (black) and y (blue) components of the endpoint trajectory residuals after removal of the drift, from t = 5 s to t = 60 s

Fig. 6

Experiment 1: Population performance in the stabilization task as a function of state mixture parameter lambda, with 3rd order polynomial population fit and 95% function bounds. a RMSE of the end-effector trajectory. b RMSE of the drift component of the end-effector trajectory. c RMSE of the residuals after removal of the drift

Fig. 7

Experiment 1: “Birds-eye view” of selected subject performance in the reaching task for each λ value in Vkr visual condition. Light grey circles: the 16 targets; small black dots: final reach endpoints for movements to the far targets; small, dark grey dots: final reach endpoints for movements to the near targets; white dots: final reach endpoints for movements returning to the central home target. Yellow ellipses represent the two-dimensional 95% confidence intervals of the return-to-home reach endpoints

Fig. 8

Experiment 1: Population statistics for reaching task, as a function of state mixture parameter lambda. Error bars represent ± 1 SEM. a Variability of raw reach endpoints about the home target (area of an ellipse fit to the reach endpoints). b Variability of reach endpoints at the central target location after collapsing across movement directions. c Mean absolute error |Error| at the central target. Red lines: significant Dunnett comparisons at p < 0.05

Fig. 9

Experiment 2: “Birds-eye view” of selected subject performance in the stabilization task. Cursor trajectory showing drift over time (line shading) varies with the presence and type of vibration feedback. Light grey dot: hand position at time t0; black dot: hand position at time t0 = 5 s; white dot: hand position at time t0 = 60s. Drift was modeled from t = 5 s to the end of the trial at t = 60 s. Values in red are the RMSEDrift for that trial

Fig. 10

Experiment 2: Population statistics in the stabilization task for error and state feedback. Red lines: p < 0.05. Vertical dashed lines mark the occurrence of training. The black horizontal dashed line provides a reference to assist visual comparison across training groups. No significant difference was observed across groups in this condition

Fig. 11

Experiment 2: Selected subject performance in the reaching task. Figure elements as described in the legend for Fig. 8. Compare performance in the test phases (red dashed box) to the baseline 2 and sham phases

Fig. 12

Experiment 2: Population statistics for reaching to the (unrotated) center target. Error bars represent ± 1 SEM. Red lines: p < 0.05. Vertical dashed lines mark the occurrence of training. Red horizontal dashed line: significant across-group comparison at p < 0.05

Fig. 13

Experiment 2: Population results for reaching task. Error bars represent ± 1 SEM. a-c Variability of reach endpoints for the three target sets after collapsing across movement directions. d-f Mean absolute error |Error| relative to the center of the target. Vertical dashed lines mark the occurrence of training. Red solid lines: significant within-group comparisons at p < 0.05. Red horizontal dashed lines: significant across-group comparisons at p < 0.05. Blue lines: secondary analysis with p < 0.05

Fig. 14

Experiment 2: Assessment of usefulness on a 1–7 scale for state and error feedback for three tasks. Error bars represent ± 1 SEM. Red line: p < 0.05