How to be patient. The ability to wait for a reward depends on menstrual cycle phase and feedback-related activity

Abstract

Dopamine (DA) plays a major role in reinforcement learning with increases promoting reward sensitivity (Go learning) while decreases facilitate the avoidance of negative outcomes (NoGo learning). This is also reflected in adaptations of response time: higher levels of DA enhance speeding up to get a reward, whereas lower levels favor slowing down. The steroid hormones estradiol and progesterone have been shown to modulate dopaminergic tone. Here, we tested 14 women twice during their menstrual cycle, during the follicular (FP) and the luteal phase (LP), applying functional magnetic resonance imaging while they performed a feedback learning task. Subsequent behavioral testing assessed response time preferences with a clock task, in which subjects had to explore the optimal response time (RT) to maximize reward. In the FP subjects displayed a greater learning-related change of their RT than during the LP, when they were required to slow down. Final RTs in the slow condition were also predicted by feedback-related brain activation, but only in the FP. Increased activation of the inferior frontal junction and rostral cingulate zone was thereby predictive of slower and thus better adapted final RTs. Conversely, final RT was faster and less optimal for reward maximization if activation in the ventromedial prefrontal cortex was enhanced. These findings show that hormonal shifts across the menstrual cycle affect adaptation of response speed during reward acquisition with higher RT adjustment in the FP in the condition that requires slowing down. Since high estradiol levels during the FP increase synaptic DA levels, this conforms well to our hypothesis that estradiol supports Go learning at the expense of NoGo learning. Brain-behavior correlations further indicated that the compensatory capacity to counteract the follicular Go bias may be linked to the ability to more effectively monitor action outcomes and suppress bottom-up reward desiring during feedback processing.

Keywords: RCZ; VMPFC; dopamine; estradiol; fMRI; menstrual cycle; reinforcement learning; time perception.

Figures

Figure 1

Clock task. Reward values varied as a cosine function of response time. Subjects were instructed to figure out the optimal time point during a whole clock-arm turn of 5 s to stop the ticking clock and achieve maximum reward. In the fast clock condition, fast responses yielded the highest reward value (FAST clock for “Go learning”), whereas in the slow clock subjects had to wait longer to win maximum points (SLOW clock for “NoGo learning”). The RANDOM clock acted as a control condition, in which there existed no relation between response time and reward value. In all clock conditions random noise ranging between −5 and 4 points was added to the reward value in order to prevent subjects from memorizing an exact reward value at a specific response time. The three different clock types were presented in three separate runs for each clock consisting of 50 trials, which were counterbalanced across subjects and cycle phases.

Figure 2

Learning outcome from the probabilistic learning task predicts response speed in the first block of the fast clock condition during the FP. (A) Subjects who showed a higher percentage of choosing the best option in the second test session of the probabilistic learning task (i.e., “Go learners”) were also better able to speed up during the fast condition at the very beginning. (B) In contrast, subjects who showed a higher tendency to avoid the most punished option (i.e., “NoGo learners”) took longer to respond.

Figure 3

Functional opponency of slowing and speeding. Relative slowing and speeding were negatively correlated in both cycle phases (FP: R = −0.918, p < 0.001; LP: R = −0.637, p = 0.014).

Figure 4

RT change from first to last block of the clock task in both cycle phases. In both clock conditions subjects showed greater RT change in the course of the experimental run during the FP (*p < 0.05). Error bars represent standard error of the mean (sem).

Figure 5

Feedback-related activity during probabilistic learning predicted performance in the last block of the SLOW clock during the FP. (A) Correlations between neural responses based on the contrast positive feedback > baseline in the probabilistic learning task and optimized RTs of the last block during the SLOW clock condition. (B) Correlations between neural responses based on the contrast negative feedback > baseline in the probabilistic learning task and optimized RTs of the last block during the SLOW clock condition. Displayed positive correlations survived FWE correction for multiple comparisons at cluster-level (p < 0.05). The negative correlations are reported at p < 0.05, SVC (based on the coordinates reported in McClure et al., 2004).

Figure 6

Correlations between RTs of the last block in the SLOW clock and the parameter estimates of peak voxels in feedback-related clusters found to be associated with response time adjustment in the SLOW clock during FP. (A) Increased activation in RCZ and IFJ predicted effective response time adaption in the SLOW clock. (B) Increased activation in the VMPFC was associated with compromised RT adaptation in the SLOW clock.