Striatal structure and function predict individual biases in learning to avoid pain

Abstract

Pain is an elemental inducer of avoidance. Here, we demonstrate that people differ in how they learn to avoid pain, with some individuals refraining from actions that resulted in painful outcomes, whereas others favor actions that helped prevent pain. These individual biases were best explained by differences in learning from outcome prediction errors and were associated with distinct forms of striatal responses to painful outcomes. Specifically, striatal responses to pain were modulated in a manner consistent with an aversive prediction error in individuals who learned predominantly from pain, whereas in individuals who learned predominantly from success in preventing pain, modulation was consistent with an appetitive prediction error. In contrast, striatal responses to success in preventing pain were consistent with an appetitive prediction error in both groups. Furthermore, variation in striatal structure, encompassing the region where pain prediction errors were expressed, predicted participants' predominant mode of learning, suggesting the observed learning biases may reflect stable individual traits. These results reveal functional and structural neural components underlying individual differences in avoidance learning, which may be important contributors to psychiatric disorders involving pathological harm avoidance behavior.

Keywords: avoidance learning; individual differences; pain; prediction errors; striatum.

Conflict of interest statement

The authors declare no conflict of interest.

Figures

Fig. 1.

Experimental design and learning performance. n = 41 participants. (A) Experimental design. On each trial, participants were presented with one of three possible decks and a number between 1 and 9 drawn by the computer. If participants decided to gamble, a shock was delivered only if the number that they drew was lower than the computer’s number. Participants were only informed whether they won or lost the gamble, not which number they drew. Participants had to learn by trial and error how likely gambles were to succeed with each of the three decks. One deck contained a uniform distribution of numbers between 1 and 9 (even deck), one deck contained more 1’s (low deck), making gambles 30% less likely to succeed, and one deck contained more 9’s (high deck), making gambles 30% more likely to succeed. Opting to decline the gamble resulted in a 50% probability of shock regardless of which numbers were drawn by the computer. (B) Gambles taken with each deck as a function of time. Percentages were computed separately for each set of 15 contiguous trials (4 sets/60 trials per block). (C) Participants’ propensity to gamble in first (Left) and last (Right) blocks of trials as a function of learning bias. Propensity to gamble was computed by regressing out the effects of deck and computer’s number on participant’s choices using logistic regression. The numbers 1 and –1 correspond to always and never gamble, respectively. Learning bias was inferred from a participant’s choices using the learning model. (D) Proportion of bad gambles that were declined and good gambles that were taken in last block of trials as a function of learning bias. Participants with a positive learning bias (positive learners) declined fewer bad gambles and took more good gambles than participants with a negative learning bias (negative learners). Gambles were defined as good or bad based on probability of winning (good: >50%; bad <50%). Error bars: 95% bootstrap CI. (E) Average number drawn by computer as a function of time and learning bias. To maintain participants at a 50% gambling rate, numbers increased for positive learners and decreased for negative learners. In B and E, dotted line indicates simulated task performance of learning model. Shaded areas: 95% bootstrap CI.

Fig. 2.

PE signaling in the striatum. n = 41 participants. (A) Striatal area where BOLD response to no-shock outcomes was modulated by expectations. Map is masked and FWE corrected for volume of striatum (P < 0.05, GLM). Extent: 22 voxels. Peak MNI coordinates: [–6 10 2] t40 = 4.5, corrected P = 0.01; [10 8 4] t40 = 4.7, corrected P = 0.007. x and z denote Montreal Neurological Institute (MNI) coordinates. (B) The two components of the prediction error. Effects of deck-based expectation and the number drawn by the computer on BOLD response to shock (Bottom) and no shock (Top) outcomes in the striatal ROI identified in A. Results are shown separately for positive (Left) and negative (Right) learners. A higher deck expectation and a lower number indicate lower expectation of a shock, and thus, an appetitive PE is consistent with a rise in the effect of number and a dip in the effect of deck expectation on the response to the outcomes. The converse pattern is consistent with an aversive PE. Time 0 indicates outcome onset. Shaded area: SEM. (C) Effect of expectations on BOLD response to no-shock and shock outcomes as a function learning bias. Positive values indicate an effect that is consistent with an appetitive PE. *P < 0.05, **P < 0.005, error bars: 95% bootstrap CI. (D) Propensity to gamble in the last task block as a function of a participant’s PE index, computed as the average effect of expectation on striatal response across both types of outcomes. An appetitive PE index predicted subsequent propensity to gamble (computed as in Fig. 1C). GLM coefficients in C and D were taken from the striatal area identified in A.

Fig. 3.

Behavioral and neural responses to the even deck. (A) Propensity to gamble and BOLD response to even deck as a function of learning bias. Propensity to gamble was computed as in Fig. 1C but exclusively for the even deck. By contrast, all participants avoided gambling with the low deck (propensity to gamble –0.42, CI –0.54 to –0.30) and favored gambling with the high deck (propensity to gamble 0.72, CI 0.60–0.80). BOLD response of 1 indicates that the response to the even deck was identical to the response to the high deck, whereas a value of –1 indicates that it was identical to the response to the low deck. Similarity of BOLD responses was computed as the Euclidian distance between the two responses’ GLM coefficients across all gray matter. Error bars: 95% bootstrap confidence intervals, *P = 0.05, **P = 0.01, permutation test. (B) BOLD response to even deck, compared with high and low decks, as a function of striatal PE index. PE index taken from Fig. 2D, and BOLD response was computed as in A.

Fig. 4.

Striatal gray matter density predicts learning bias. n = 41 participants. (A) Learning bias predicted by gray matter density in the 6,315 voxels of the striatum as a function of the true learning bias inferred from participants’ choices. Learning biases were predicted from gray matter density by using cross-validated regularized linear regression. (B) Gray matter density coefficients used to predict learning bias. To create the map, predictive coefficients were averaged across participants and generalized across the striatum by assigning a fraction of each coefficient to each voxel proportionally to the gray matter-density variance shared between that voxel and the coefficient’s designated voxel. (C) Representation of expectations in the response to shocks. t values were computed by using a group-level GLM that included both negative learners, whose BOLD response was regressed against aversively signed prediction errors, and positive learners, whose BOLD response was regressed against appetitively signed prediction errors. There were no significant differences between positive and negative learners within the striatum (P > 0.5, FWE small-volume corrected). The map is masked for the volume of the striatum and not thresholded. z value in B and C denotes MNI coordinate.

Fig. 5.

Individual learning biases outside of the striatum. (A) Effect of expectation on BOLD response to shocks as a function of learning bias in the amygdala region where this effect was significant (MNI coordinates [28 −6 −18]). Responses were most consistent with an aversive prediction error in participants who mostly learned from shock outcomes. n = 41 participants. (B) Functional connectivity between striatal (Fig. 2A) and amygdala (A) ROIs, and the insula area in which the response to shocks correlated with learning bias (MNI coordinates [46 −32 20]). Error bars: 95% bootstrap CI, *P ≤ 0.05, NS: P > 0.1.