The Neural Basis of Aversive Pavlovian Guidance during Planning

Abstract

Important real-world decisions are often arduous as they frequently involve sequences of choices, with initial selections affecting future options. Evaluating every possible combination of choices is computationally intractable, particularly for longer multistep decisions. Therefore, humans frequently use heuristics to reduce the complexity of decisions. We recently used a goal-directed planning task to demonstrate the profound behavioral influence and ubiquity of one such shortcut, namely aversive pruning, a reflexive Pavlovian process that involves neglecting parts of the decision space residing beyond salient negative outcomes. However, how the brain implements this important decision heuristic and what underlies individual differences have hitherto remained unanswered. Therefore, we administered an adapted version of the same planning task to healthy male and female volunteers undergoing functional magnetic resonance imaging (fMRI) to determine the neural basis of aversive pruning. Through both computational and standard categorical fMRI analyses, we show that when planning was influenced by aversive pruning, the subgenual cingulate cortex was robustly recruited. This neural signature was distinct from those associated with general planning and valuation, two fundamental cognitive components elicited by our task but which are complementary to aversive pruning. Furthermore, we found that individual variation in levels of aversive pruning was associated with the responses of insula and dorsolateral prefrontal cortices to the receipt of large monetary losses, and also with subclinical levels of anxiety. In summary, our data reveal the neural signatures of an important reflexive Pavlovian process that shapes goal-directed evaluations and thereby determines the outcome of high-level sequential cognitive processes.SIGNIFICANCE STATEMENT Multistep decisions are complex because initial choices constrain future options. Evaluating every path for long decision sequences is often impractical; thus, cognitive shortcuts are often essential. One pervasive and powerful heuristic is aversive pruning, in which potential decision-making avenues are curtailed at immediate negative outcomes. We used neuroimaging to examine how humans implement such pruning. We found it to be associated with activity in the subgenual cingulate cortex, with neural signatures that were distinguishable from those covarying with planning and valuation. Individual variations in aversive pruning levels related to subclinical anxiety levels and insular cortex activation. These findings reveal the neural mechanisms by which basic negative Pavlovian influences guide decision-making during planning, with implications for disrupted decision-making in psychiatric disorders.

Keywords: aversive pruning; decision-making; fMRI; punishment; reward; subgenual cingulate cortex.

Copyright © 2017 the authors 0270-6474/17/3710216-15$15.00/0.

Figures

Figure 1.

Aversive pruning example and fMRI task design. A, Decision tree and monetary outcomes up to a depth of three from starting state 2. Purple- and orange-colored lines indicate pressing the left and right buttons, respectively. The totals earned for the two best paths (thicker lines; breaking even and losing 20 pence) are shown in blue and red. B, An example of disadvantageous aversive pruning. The red line shows the curtailment of the search within the decision tree upon encountering a large monetary loss (−70 pence), such that the more advantageous break-even sequence is not considered. C, Button presses and transitions within the maze. D, Monetary outcomes within the maze. E, Free plan trial. Beginning in a selected white box, participants had 9 s to plan a sequence of moves (3–5 moves, indicated at the top of the screen) to maximize income. Plusses and minuses below each box indicate the potential outcomes possible from moving from there but are not indicative of directionality. Colored sidebar arrows were included to match visual input with restricted plan trials. F, Restricted plan trial. Participants had 9 s to decide between two maze routes (green and blue), one of which provided higher net income. G, For restricted plan trials, the selection of either the blue or green route involved choosing either the left or right button. H, After entering their moves or path selection, participants were shown their selected path with the corresponding monetary outcome for each box-to-box transition for both free and restricted plan trials. Summed path totals were not shown.

Figure 2.

Initial model-free and computational model-based results, model comparison, and model validation analyses. A, Percentage of trials on which the correct sequence was chosen, split by whether it did not include a large loss (green: ONLL) or did (blue: OLL). Black dots represent individual performance and gray lines connect the two trial types. B, ONLL and OLL performance split by decision depth. C, Average likelihood of participants' choices. Chance model performance level is shown by the black dashed line; Lookahead represents optimal planning; Discount incorporates random stopping of the tree search; Pruning additionally incorporates a specific chance of stopping when a large loss (−70 pence) is encountered; and Pruning+Loss additionally incorporates individual reinforcement value sensitivities to account for loss aversion. D, Proportion of variance explained by the different models. E, Model evidence measured by group-level iBIC; red star indicates the best-performing (i.e., lower iBIC) model. F, Pruning parameters (values indicate the probability of continuing to evaluate the decision tree). Black dots in F show individual data (parameters taken from the Pruning+Loss model), connected by black dashed lines. G, Reinforcement sensitivity parameter estimates. H, Relationship between the trial-based measure of general planning ability, ONLL, and its computational equivalent γG. I, Relationship between the trial-based measure of aversive pruning (ONLL − OLL) and its computational equivalent, the difference between γG and γS. J, Comparison of ONLL and OLL correct trials between the observed data and the data generated from our winning model. K, Observed and generated data for each individual subject plotted for ONLL and OLL correct trials. L, The fraction of times the winning model gave the highest probability to the action chosen by the subject; red line shows chance level. Red and green error bars indicate one SE and 95% confidence intervals of the mean, respectively. ***p < 0.001.

Figure 3.

Neural responses during aversive pruning: model-based fMRI results. A, B, KL divergence value increased linearly with depth (A) and, based on participant behavior, was highest on trials classified as aversive pruning trials (B). C, Activation in pregenual ACC and SGC increased linearly with KL divergence value. D, There was an interaction between KL divergence value and difficulty in the SGC, with greater impact of the former on more difficult trials. Overlays are presented at a threshold of p < 0.005 (uncorrected). Error bars represent 1 SEM, and color bars indicate t values. N.S., Non-significant; ***p < 0.001.

Figure 4.

Neural responses to increasing difficulty and value and relationship between aversive pruning and loss receipt at outcome. A, Bilaterally, cerebellum (left), motor cortex (left), dorsal striatum (middle), and dorsolateral prefrontal cortex (right) activation increased linearly with task difficulty during the planning phase. Overlays are presented at a threshold of pWB < 0.05. B, VS (left) and mOFC (middle) activation increased linearly with the net monetary value during the outcome phase. Overlays are presented at a threshold of p < 0.005 (uncorrected), but VS and mOFC results survive voxel-level pWB < 0.05. Peak voxel mOFC activation to increasing reward (B, right) correlated with the sensitivity to a large rewards (+140 pence) parameter derived from our computational model. C, Contrasting feedback on the correct trial types (OLL vs ONLL correct) revealed responses in the right insula (left), and right DLPFC (left). Response in the insula was driven by increased activation during OLL correct outcomes (C, middle). The difference in insula activation between OLL and ONLL correct trials at outcome correlated with γG − γS, our computationally derived measure of overall aversive pruning (C, right). Overlays are presented at a threshold of p < 0.005 (uncorrected). Error bars represent 1 SEM, and color bars indicate t values.

Figure 5.

Confirmatory trial-based behavioral and fMRI results. A, Decision tree showing path selection starting from state 2 with three moves to go; line width is proportional to selection frequency. The optimal route (break-even, blue) and the suboptimal aversive pruning route (net income −20p, red) were selected with similar frequency. B, Aversive pruning percentage [aversive pruning/(aversive pruning + OLL error) * 100], split by depth. C, Mean trial earnings across the four conditions. The light red bar behind aversive pruning depicts the possible earnings if participants had performed optimally on the trials classified as aversive pruning. OLL error represents incorrect choices on OLL trials that could not be classified as aversive pruning. D, Reaction times for the first button press across trial types. E, Difficulty-related response in the SGC (left) contrasting aversive pruning trials against OLL correct trials. Overlay is presented at a threshold of p < 0.005 (uncorrected). The finding in the SGC was driven by a negative modulation by difficulty for OLL correct trials (p = 0.001), with no significant effect of difficulty on aversive pruning trials (p = 0.285, right). Error bars represent 1 SEM, and the color bar indicates t values. N.S., Non-significant; *p < 0.05; **p < 0.01; ***p < 0.001.