Multiple Motor Learning Processes in Humans: Defining Their Neurophysiological Bases

Abstract

Learning new motor behaviors or adjusting previously learned actions to account for dynamic changes in our environment requires the operation of multiple distinct motor learning processes, which rely on different neuronal substrates. For instance, humans are capable of acquiring new motor patterns via the formation of internal model representations of the movement dynamics and through positive reinforcement. In this review, we will discuss how changes in human physiological markers, assessed with noninvasive brain stimulation techniques from distinct brain regions, can be utilized to provide insights toward the distinct learning processes underlying motor learning. We will summarize the findings from several behavioral and neurophysiological studies that have made efforts to understand how distinct processes contribute to and interact when learning new motor behaviors. In particular, we will extensively review two types of behavioral processes described in human sensorimotor learning: (1) a recalibration process of a previously learned movement and (2) acquiring an entirely new motor control policy, such as learning to play an instrument. The selected studies will demonstrate in-detail how distinct physiological mechanisms contributions change depending on the time course of learning and the type of behaviors being learned.

Keywords: TMS; brain stimulation; motor learning; physiology; skill learning.

Conflict of interest statement

Declaration of Conflicting Interests: The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Figures

Figure 1.

Processes underlying motor learning. (A) Motor learning is constituted by several different processes all involved in acquiring novel behaviors or calibrating already known ones. This includes error-based, reinforcement, use-dependent plasticity, and strategy-based forms of learning. Error-based is a type of learning based on sensory-predictions errors where the intended movement outcome is compared with the actual executed movement. In other words, a type of learning driven by a mismatch between what you think you are doing and what you perceive you are doing. Reinforcement learning refers to a success-based process in which actions leading to a successful outcome are reinforced, whereas those leading to unsuccessful outcome are avoided. Use-dependent learning is used to describe a phenomenon where behavioral changes are induced through the simple repetition of movements, regardless of whether errors are present or not. Strategy-based learning simply refers to utilizing cognition or explicit knowledge to solve motor problem. Each of these forms of learning are continuously involved in guiding the performance of our movements toward the correct solution. (B) Although these mechanisms work to achieve a common goal (i.e., learning a skill), it is important to consider that the relative contributions of these forms of learning maybe weighed differently throughout the time course of the same motor skill training (i.e., initially picking up a new task vs. after several attempts at the same task; heat map panel: red = more, yellow = less). (C) Similarly, the contributions of these forms of learning may also shift depending on the specific component of the motor task that one is asked to learn. For example, to successfully hit a tennis ball, our brain must develop an understanding of how to interact with a racket/environment/ball (e.g., weight of the racket and ball, type of court), as well as to coordinate an appropriate sequence of movements (i.e., fluid serve).

Figure 2.

Behavioral studies over the past several years have used several forms of motor adaptation tasks (e.g., rotations, prisms, force-field, split-belt walking) to elucidate the learning processes responsible for calibrating the mapping between desired outcomes and motor commands. This process is critical for our ability to adjust our daily movements to environment demands, such as walking over different surfaces (i.e., concrete, sand, ice). In adaptation tasks, a perturbation is suddenly introduced, altering the relationship between a movement and its resulting sensory feedback in the task workspace. (A) This figure depicts a commonly used visuomotor reaching adaptation task. In this setup, participants are asked to make “shooting” reaches toward a visual target by moving an on-screen cursor (yellow dot). The cursor represents the position of their hand (i.e., participants vision of the arm is blocked), therefore perturbations can be applied to the cursor (i.e., visuomotor transformations). The “shooting” action does not allow participants to correct movements within a trial, but rather only for learning via through endpoint feedback error to adjust movements on the next trial (i.e., feed-forward sensory prediction errors). Participants are capable of performing accurate reaches to targets within the workspace when there is no visuomotor manipulation (baseline). When a rotation is applied to the cursor unknowingly to the participants, they initially execute and observe large errors (early adaptation). The goal of the task for the participant then becomes to minimize the movement error between the cursor and target (i.e., minimize sensory prediction errors). This can be accomplished over multiple reaches. If the cursor perturbation is then removed, participants execute errors in the opposite direction, depicting the retention of the previously acquired sensory-motor map. (B) By providing only binary feedback about task performance (i.e., success or failure), participants can also learn a cursor rotation using reinforcement learning. In this example, if the participants land in the correct zone (highlighted in yellow) a “success” visual feedback is provided (green). However, if the participant does not land in this zone, they are given a “failure” feedback (red). Unlike learning from sensory prediction errors, learning via reinforcement does not lead to sensorimotor recalibration. Explicit processes (i.e., aiming strategies; not shown) also influences visuomotor learning, in particular when large rotations are introduced.

Figure 3.

(A) Paired-pulse transcranial magnetic stimulation (TMS) configuration for assessing cerebellar-M1 connectivity (cerebellar-inhibition or CBI). A figure-of-eight coil is placed over the left M1 (test stimulus; TS), whereas a double-cone coil is placed over the right cerebellum, 3 cm inferior and lateral to the inion (conditioning stimulus; CS). CBI refers to the ratio between the motor-evoked potential (MEP) amplitudes after applying the conditioned test stimulus (CS + TS; magenta) and the MEP amplitudes produced following the unconditioned test stimulus (TS alone; dark blue). (B) Schematic representation of the interpretation illustrating how cerebellar-M1 connectivity changes following learning. Although the physiology of stimulating this region are not completely understood, double cone coil stimulation is thought to result in the parallel-fiber-mediated activation of Purkinje cells, which in turn inhibit the deep cerebellar nuclei (Celnik 2015) that have a disynaptic excitatory connection to M1 via the thalamus. Prior to learning, a test pulse over M1 combined with a cerebellar conditioning pulse on highly active Purkinje cells (marked in red) results in a decreased activation of M1. This is depicting by smaller MEP amplitudes as a result of combined stimulation (magenta trace) when compared to the MEPs elicited by just stimulating over M1 (blue trace). CBI has been found to reduce following a variety of motor learning tasks (Jayaram and others 2011; Schlerf and others 2012a; Schlerf and others 2015; Spampinato and Celnik 2017; Spampinato and others 2017), which has been interpreted as resulting from long-term depression like changes in the activity of parallel fiber–Purkinje cell synapses, as shown in animal studies (Medina and Lisberger 2008). Thus, following learning, stimulation of less active Purkinje cells (marked in yellow) is not as likely to inhibit M1. Further evidence of this interpretation has been provided by studies utilizing transcranial direct current stimulation (tDCS) to modulate cerebellar activity. One study showed that anodal cerebellar tDCS effects were consistent with the concept of increased excitability of Purkinje cells, since the authors were able to elicit a stronger CBI effect at low conditioned stimulation intensity following tDCS stimulation (Galea and others 2009). Following this logic, the authors also showed that cathodal tDCS resulted in a reduced CBI effect. Together, this collection of results indicates that cerebellar stimulation, to a degree, reflects the state of Purkinje cells and how they respond to both behavioral and brain stimulation interventions.

Figure 4.

Schematic representation of the interpretation of occlusion of M1 long-term potentiation (LTP)-like plasticity after motor learning. The concept of occlusion relates to the idea that induction of LTP and long-term depression (LTD) within M1 reach a saturation point. This is consistent with a homeostatic plasticity mechanism that acts to control neuronal activity and excitability levels within a physiologically useful modification range. Thus, there is a limited capacity or a particular synaptic modification range for how much potentiation M1 can undergo. At rest, M1 excitability is at the center of the synaptic modification range and has a certain capacity for LTP-like plasticity (red arrow). In this scenario, when anodal transcranial direct current stimulation (tDCS; i.e., LTP-like inducing protocol) is applied to M1, that facilitates excitability via LTP-like plasticity. This is evident by comparing motor-evoked potential (MEP) amplitudes via transcranial amgnetic stimulation (TMS) over M1 before (light gray) and after (dark gray) anodal tDCS application, which can modulate excitability for ~30 minutes. However, since motor learning induces LTP-like plasticity in M1, as shown in animal studies, then there is a reduced range for further LTP-like plasticity. Thus, the effect of anodal tDCS on M1 excitability facilitation following learning (blue) can be smaller than that in baseline condition. This phenomenon is referred to as occlusion of LTP-like plasticity and can be seen as a signature indicating the presence of homeostatic M1 LTP-like plasticity.

Figure 5.

Learning a visuomotor cursor rotation via different forms of learning (i.e., via sensory prediction errors vs. reinforcement) relies on different physiological mechanisms. (A) Visuomotor task version relying on error-based learning. Participants performed a center-out reaching task in which they controlled the movement of a yellow computer-cursor from a central starting position to a target (eight possible locations) while receiving online and endpoint cursor feedback (light blue dot). (B) Behavioral reach angle data. Positive values indicate counterclockwise deviation. Solid lines and shaded areas show the mean and standard errors of the mean (SEM) for each 8-trial epoch for the constant perturbation (blue; i.e., a constant 30° rotation was administered in “Perturb” section) and the random perturbation sessions (gray). Only the Constant group shifted the reaching direction when exposed to the constant perturbation, but not when exposed to the random perturbation. (C) Cerebellar inhibition (CBI) results. Bar graphs and vertical error bars indicate the mean CBI ratio (the ratio of the conditioned/unconditioned test stimulus [TS] motor-evoked potential [MEP] amplitude) for the constant and the random sessions at each time point. The authors found that CBI selectively reduced after participants of the Constant group accounted for the perturbation (i.e., reduced CBI at “Post” time-point). (D) The AtDCS effects on M1 excitability for the Constant and Random groups. MEP amplitudes normalized to that of preAtDCS are presented for each time point (Pre, P0, . . ., P15). No significant occlusion was found for either group after training and adapting via sensory-motor prediction errors. (E) Motor task version relying on reinforcement learning. Here, participants reached from a central starting position to one target. Only binary feedback (success or failure) was presented (in the form of target colors) instead of vector cursor feedback. (F) Reach angle. Solid lines and shaded areas indicate the mean and SEM of each 8-trial epoch for the Learner (light blue) and the Non-Learner groups (gray). (G) The AtDCS effects on M1 excitability and (H) CBI results for the Learner and the Non-Learner groups. Here, the authors showed learning via binary feedback elicited changes in M1 LTP-like plasticity (i.e., occlusion), but did not modulate cerebellar excitability changes. This study elegantly showed that we can learn the same motor behavior via different forms of motor learning engaging distinct neurophysiological mechanisms (Uehara and others 2018). Figure used with permission from Oxford University Press https://doi.org/10.1093/cercor/bhx214).

Figure 6.

The role of positive-feedback reward signals to learning via use-dependent plasticity (UDP). (A) Schematic of an experimental task that assess UDP. Transcranial magnetic stimulation (TMS) is delivered over the left M1 to elicit thumb movements before and after training. The direction of TMS-evoked thumb movements was recorded and the proportion of TMS-evoked thumb movements falling in the training direction zone (TDZ; magenta) are measured. (B) Representative subject data displaying TMS-evoked thumb movements before (gray, left) and after (blue, middle) training. Mawase and others calculated the group average depicting the probability distribution of the thumb directions before and after task performance. Individuals who trained on this paradigm showed a significant increase in the proportion of TMS-evoked thumb movements within the TDZ. (C) The study design and setup of experiment 2 used in Mawase and others (2018) in which the authors showed that explicit rewards modulate UDP. In short, participants were randomly assigned to either a reward-group or random-reward group. Importantly, only the reward-group (blue) received explicit reward coinciding with task success, whereas the random reward group (red) received an explicit reward randomly throughout performance, independent of task success. TMS-evoked thumb movements were assessed before and after training for both groups. (D) A 2-dimensional histogram showing the total number of TMS-evoked movements for all participants, separated by group. Here, only the reward-group significantly benefited from receiving meaningful reward as they showed a dramatic change in TMS-evoked thumb movement direction, whereas the random-reward group did not show a significant change from baseline. Figures adapted from Mawase and others 2017; https://dx.doi.org/10.1523/jneurosci.3303-16.2017.

Figure 7.

Results from Spampinato and Celnik (2017), which investigated the temporal dynamics of two physiological mechanisms during motor skill learning. (A) The experimental design where individuals performed a sequence of movements by squeezing a pinch-force transducer to control the movement of an on-screen computer cursor. Individuals were split into three separate groups: Long (blue), Short (light blue), and Random (red). The Long and Random groups completed a total of 150 trials (5 blocks), whereas the Short group completed 1 block of 30 trials. Importantly, only the Long and Short groups had a consistent sensorimotor mapping between cursor movements and pinch-force production. Cerebellar inhibition (CBI) was assessed throughout training (Black arrows; Pre, P1, P2, P3), while occlusion was measured at the end of each day. (B) The skill performances for the Long (blue), Short (light blue), and Random (red) groups are presented for each block and day of training. Importantly, no learning is present in the random group (C) Cerebellar-M1 connectivity (CBI) changes throughout learning a de novo skill. The bar graphs show the mean CBI ratio on the y-axis for groups where learning occurred (blue and light blue) and for a random movement scenario (red). The x-axis represents different stimulation time points: before training (Pre), during (P1, P2) after the training session (P3). The authors show that only the training group (individuals capable of learning a new sensorimotor relationship) had a reduction in CBI (i.e., less inhibition). Specifically, the cerebellar physiological changes were prominent at the beginning of learning, which progressively returned toward baseline levels despite further skill improvement. (D) The AtDCS effects on M1 excitability for the long (blue), short (light blue) and random groups (red). The average MEP amplitude (standardized to the pretDCS MEP amplitude) is depicted on the y-axis and the x-axis represents successive TMS measurements taken prior to application of AtDCS (Pre), immediately after AtDCS (Post 1, P1) and repeated every 5 minutes up to 25 minutes post-AtDCS (P2, . . ., P6). Here, the authors show that in comparison to baseline (dark gray) occlusion only occurs after significant amount of learning occurs following a training session (i.e., Long group, blue and light gray). Overall, these results indicate that learning a new skill involves cerebellar-dependent processes early in learning; and as learning proceeds, M1-LTP-like plasticity can be observed, suggesting that other forms of learning are incorporated (e.g., reinforcement or use-dependent). Figures adapted from Spampinato and Celnik (2017); https://doi.org/10.1038/srep40715.

Figure 8.

Results from a study in which a motor skill task was broken down into distinct motor components (a sensorimotor mapping vs. sequence) while physiological changes were assessed. (A) The experimental design where individuals learned a logarithmic mapping between the movement of an onscreen cursor and pinch-force production. Participants were trained on this mapping for three days prior to introducing a skill version (i.e., integration of the logarithmic map and a sequence of movements). Cerebellar inhibition (CBI) was assessed throughout the map and skill training, while occlusion was measured at the end of each day. (B) Behavioral map and skill data. Overall, participants were able to learn the sensorimotor map as shown by a reduction in the time to reach successfully a target and improve their skill score with training. (C) CBI results. The y-axis depicts the CBI ratio and the x-axis represents different time points of map and skill learning. First, the authors show that CBI significantly reduces when learning the sensorimotor map. Interestingly, despite having knowledge of the mapping, CBI still reduces early on during skill learning. (D) The AtDCS effects on M1 excitability recorded at rest (Baseline Day) and following map and skill training. Here, when compared to baseline responses, significant occlusion was found only after skill training and not following sensorimotor map learning. (E) In another experiment, participants were placed in either Training or Random groups. The Training group was exposed to the same nine-element sequence throughout training, whereas the Random group received a randomized nine-element sequence order on each trial. (F) Sequence behavioral data. Only the participants from the Training group were able to learn as shown by a reduction of both sequence movement-time and response time to execute the sequence. (G) CBI results and (H) the AtDCS effects on M1 excitability for the Training and Random groups. The authors found that only learning a consistent sequence elicited changes in cerebellar excitability changes and resulted in occlusion of M1 LTP-like plasticity. Altogether these results indicate that a complex skill is learned by different forms of learning engaging distinct and specific neurophysiological mechanisms (Spampinato and Celnik 2018). Figure used with permission from Elsevier; https://doi.org/10.1016/j.cortex.2018.03.017.