Reinforcement Learning Is Impaired in the Sub-acute Post-stroke Period

Abstract

Background: Neurorehabilitation approaches are frequently predicated on motor learning principles. However, much is left to be understood of how different kinds of motor learning are affected by stroke causing hemiparesis. Here we asked if two kinds of motor learning often employed in rehabilitation, (1) reinforcement learning and (2) error-based adaptation, are altered at different times after stroke.

Methods: In a cross-sectional design, we compared learning in two groups of patients with stroke, matched for their baseline motor execution deficit on the paretic side. The early group was tested within 3 months following stroke (N = 35) and the late group was tested more than 6 months after stroke (N = 30). Two types of task were studied: one based on reinforcement learning and the other on error-based learning.

Results: We found that reinforcement learning was impaired in the early but not the late group, whereas error-based learning was unaffected compared to controls. These findings could not be attributed to differences in baseline execution, cognitive impairment, gender, age, or lesion volume and location.

Conclusions: The presence of a specific impairment in reinforcement learning in the first 3 months after stroke has important implications for rehabilitation. It might be necessary to either increase the amount of reinforcement feedback given early or even delay onset of certain forms of rehabilitation training, e.g., like constraint-induced movement therapy, and instead emphasize others forms of motor learning in this early time period. A deeper understanding of stroke-related changes in motor learning capacity has the potential to facilitate the development of new, more precise treatment interventions.

Conflict of interest statement

Potential conflict of interest None of the authors have a conflict of interest to declare.

Figures

Figure 1. Study design and task overview with feedback conditions.

(A) We recruited two separate groups of patients with stroke; one at the subacute stage (≤2 months, early group) and another one in the chronic period (≥6 months, late group). At the first time point (T1) participants were tested in two different motor learning tasks, a reinforcement-based motor task that relies predominantly on corticomotor-basal ganglia loops and a visuomotor error-based task that relies mostly on error-based learning processes driven by cerebellar plasticity. (B & D) In the reinforcement task no cursor feedback was provided, instead participants received only binary feedback about task success or failure if their reaches fell between the mean of the participant’s previous 10 reaches and the outer bound of the reward zone to −15°. (C & E) In the error-based learning task participants received online feedback on the cursor trajectory. After the baseline 40- trial period, a visuomotor rotation of 1 degree was imposed, and kept increasing by 2 degrees every 20 trials. Reward zone in both tasks is marked in light orange. To assess learning we compared Baseline and End perturbation trials (first and last 40 trials of task). After a baseline period to familiarize participants with the task, cursor rotation was gradually introduced until −15° in the error-based task. Within both groups clockwise or counter-clockwise rotation was counterbalanced, but later flipped for analysis. Image adapted after Therrien et al., 2016.

Figure 2. Reinforcement versus error-based learning at different stages after stroke.

Results for the reinforcement task are on the left, error-based task on the right. (A) and (B) Changes in reaching angle over trials. (Familiarization = 40 trials, Baseline = 40 trials before introducing rotation, End perturbation = last 40 trials of rotation, Wash out = 100 trials without any feedback). Early group in blue, late group in green. (C) and (D) Comparison for Baseline versus End perturbation in both groups.

Figure 3. Subgroup analysis of reinforcement versus error-based learning in the early (blue, n=15) and late (green, n=16) groups.

(A & B) Changes in reaching angle over trials. (Familiarization = 40 trials, Baseline = 40 trials before introducing rotation, End perturbation = last 40 trials of rotation, Wash Out = 100 trials without any feedback). Please note the reduced learning in the early vs. late group in the reinforcement task only. Shading indicates SEM.

Figure 4. Recovery trajectory for impairment, measured by Fugl-Meyer Score for the upper limb at T1 and T2, in the early and the late groups.

Note that early group improves over time and has overall lower impairment.

Figure 5. Stroke lesion overlay.

Upper row lesion location for the early group. Lower row lesion location for the late group. Note that the late group had overall larger lesion volume than the early group.