Representation of aversive prediction errors in the human periaqueductal gray

Abstract

Pain is a primary driver of learning and motivated action. It is also a target of learning, as nociceptive brain responses are shaped by learning processes. We combined an instrumental pain avoidance task with an axiomatic approach to assessing fMRI signals related to prediction errors (PEs), which drive reinforcement-based learning. We found that pain PEs were encoded in the periaqueductal gray (PAG), a structure important for pain control and learning in animal models. Axiomatic tests combined with dynamic causal modeling suggested that ventromedial prefrontal cortex, supported by putamen, provides an expected value-related input to the PAG, which then conveys PE signals to prefrontal regions important for behavioral regulation, including orbitofrontal, anterior mid-cingulate and dorsomedial prefrontal cortices. Thus, pain-related learning involves distinct neural circuitry, with implications for behavior and pain dynamics.

Figures

Figure 1

Pain avoidance learning task, behavioral and brain imaging results. (A) One experimental trial. Participants had 1.8 seconds to make their choice, after which their choice was displayed for 0.2 seconds. After an anticipation period of 4 seconds, participants either received a painful stimulation or nothing. The stimulation period was marked by a different fixation point. Trials were separated by a 6.6 – 11.4 seconds inter-trial interval. (B) Data from one participant. The blue and green lines depict the probability of pain associated with each option over the 150 trials (one of four possible pairs of random walks). Blue and green dots represent the selected option, and black triangles represent pain delivery. (C) Logistic regression model results (number or participants = 23). Probability of switching as a function of pain 1 – 6 trials back decays exponentially and is significantly different from zero at one (t(22) = 9.20, p

Figure 2

Pain avoidance learning model and axiomatic tests in ventral striatum and periaqueductal gray (PAG) regions of interest (ROI) (A) Regions encoding aversive prediction errors (in gray) should display higher activity for pain vs. No-Stimulus trials (axiom 1), higher activity for low expected probability of pain, regardless of outcome (axiom 2), and no difference in activity between highly predicted pain or No-Stimulus outcomes (axiom 3). Regions encoding the expected probability of avoidance (in green) should only display higher activity for low expected probability of pain, regardless of outcome. Aversive prediction errors result from the integration of pain-related information with prior expectations. This signal is then used to update future predictions. (B) Activity in a priori ventral striatum and PAG matter ROIs per quartile of expected probability of pain (number or participants = 23). Activity in the PAG satisfies all 3 axioms for aversive prediction errors (Axiom #1 (pain > no stimulus): t(22) = 3.67, p <0.05; Axiom # 2 (significance of slope for pain trials, sign permutation test): t(22)=– 2.05, p < 0.05; Axiom # 2 (significance of slope for no stimulus trials, sign permutation test): t(22)=–1.98, p < 0.05; Axiom # 3 (no difference between highly predicted pain or no stimulus outcomes, Bayesian analyses, odds in favor of the null: 5.48). By contrast, activity in the ventral striatum only satisfies axiom # 3 (no difference between highly predicted pain or no stimulus outcomes, Bayesian analyses, odds in favor of the null: 7.07). Asterisks with horizontal bars indicate significant differences between pain and no stimulus outcomes. Asterisks with vertical bars indicate significant slopes (* = p<0.05).

Figure 3

Results from the whole-brain conjunctive search (number or participants = 23). (A) Conjunction analysis of pain > no stimulus effects (axiom #1) and expected avoidance probability (1– pain probability) effects for both pain and no stimulus outcomes (axiom #2). Clusters used for the conjunction analysis were cluster-thresholded (p

Figure 4

Dynamic causal model (DCM) of aversive prediction errors at outcome onset (number or participants = 23). (A–B) This model was identified as the most likely of all the models tested (see Supplementary Fig. 3–6) by a Bayesian model selection (BMS) process. The regions included in the model were identified by the previous conjunction analysis (Fig. 3) as reflecting 1– expected probability of avoidance regardless of outcome (green), aversive prediction errors (black), or avoidance value updating (blue).