The neural correlates of subjective value during intertemporal choice

Abstract

Neuroimaging studies of decision-making have generally related neural activity to objective measures (such as reward magnitude, probability or delay), despite choice preferences being subjective. However, economic theories posit that decision-makers behave as though different options have different subjective values. Here we use functional magnetic resonance imaging to show that neural activity in several brain regions--particularly the ventral striatum, medial prefrontal cortex and posterior cingulate cortex--tracks the revealed subjective value of delayed monetary rewards. This similarity provides unambiguous evidence that the subjective value of potential rewards is explicitly represented in the human brain.

Figures

Figure 1

Intertemporal choice task. The sequence of events within a trial is shown. On each trial, subjects chose between an immediate and a delayed reward. The immediate reward was the same ($20) on every trial and was never presented visually. A red dot signaled the beginning of a trial, after which the delayed reward for that trial was presented for 2 s and then replaced again by the red dot. Subjects then had 6 s to consider their choice. Throughout the trial, subjects were required to hold down a button, and they indicated their decision by releasing or continuing to hold the button when the dot turned green. For half of the session, a button release indicated a choice of the delayed reward; for the other half, a button release indicated a choice of the immediate reward. If subjects released the button before the green light appeared, that trial was considered abandoned and removed from the analysis. The inter-trial interval was 2 s for behavioral sessions and 12 s for scanning sessions.

Figure 2

Subject-specific discount functions. (a–c) Choice data from three subjects during a single scanning session. Points are shaded according to the imposed delay to the delayed reward, and denote the fraction of times the subject chose the delayed reward over an immediate reward of $20 as a function of the objective amount of the delayed reward. The smooth curves are logistic functions fit to these data. Data from different delays are slightly offset so that all data are visible. (d–f) Indifference points, plotted as a function of the imposed delay to the delayed reward. Indifference points were estimated from the logistic fit in a–c as the amount for each delay at which the subject would choose the immediate and delayed rewards with equal frequency. The increase in indifference amounts with delay was fit by a line with a fixed intercept at $20. Delays are shaded as in a–c. (g–i) Indifference points from (d–f), divided into $20 to obtain a discount function. The decrease in subjective value with delay was fit with a single-parameter hyperbolic function. Delays are shaded as in a–c. Data from three subjects are shown (YH in a,d,g; JH in b,e,h; CH in c,f,i) to illustrate the observed heterogeneity in discount functions across subjects. CH was our most impulsive subject (k = 0.1189), YH was our most patient subject (k = 0.0005), and JH was near the median discount rate (k = 0.0097).

Figure 3

Group analysis showing areas in which activity is correlated with subjective value. (a) Areas in which neural activity was correlated with subjective value (during the 6–10-s window) in a random-effects group analysis, overlayed on the mean normalized anatomical image. Areas of correlation can be seen in the medial prefrontal cortex and posterior cingulate cortex (sagittal and axial images) and in the ventral striatum (coronal image). The color scale represents the t-value of the contrast testing for a significant effect of subjective value at time points 4–6 in the trial. (b–e) Activity in the ventral striatum, medial prefrontal cortex and posterior cingulate cortex was better correlated with subjective value than with (b) the objective amount of the delayed reward, (c) the inverse delay of the delayed reward, (d) the choice of the subject (chose delayed > chose immediate), or (e) the value of the delayed reward calculated using a single fixed discount rate for all subjects (k = 0.01, near the median for our subjects). Areas in which activity correlated with subjective value are shown in yellow, areas in which activity correlated with the other variables are shown in red, and areas of overlap are shown in orange. All maps are thresholded at P < 0.005 (uncorrected), spatial extent > 100 mm3. Data are shown in radiological convention, with the right hemisphere on the left.

Figure 4

Single-subject analyses showing areas in which activity correlated with subjective value. Data from five subjects are shown to illustrate that subjective value effects were evident for subjects spanning the entire range of behavioral discount rates. (a–e) Discount functions for five subjects, as measured behaviorally during a scanning session. These subjects include one of our more patient subjects (HM) and our most impulsive subject (CH). (f–j) Maps showing areas in which neural activity was correlated with subjective value in each subject. The sagittal overlay shows areas of activity in the medial prefrontal cortex, the coronal overlay areas of activity in the ventral striatum, and the axial overlay areas of activity in the posterior cingulate cortex. The color scale represents the t-value of the contrast testing for a significant effect of subjective value at time points 4–6 in the trial. These maps are thresholded at P < 0.01 (uncorrected), and the color scale ranges from this value to P < 0.05 (corrected for false discovery rate). Data are shown in radiological convention, with the right hemisphere on the left.

Figure 5

Single-subject time courses and neural discount functions. (a–c) Data from three subjects (HM, see Fig. 4a,f; RA, see Fig. 4c,h; and CH, see Fig. 4e,j) are shown. Data were averaged over all voxels that showed a correlation between activity and subjective value in the ventral striatum, medial prefrontal cortex and posterior cingulate cortex (from the individual-level analyses shown in Fig. 4), and then re-plotted as trial averages. Trial averages are color-coded by the imposed delay to the delayed reward. The 6–10-s window in which we observed significant effects is shown in gray. The largest standard error is shown on the right. The arrows indicate the point in the trial at which the delayed option was presented. (d–f) Data from a–c, summed over the 6–10-s window and re-plotted as a function of delay. The solid black line represents average predicted activity at each delay, from the fit of the subjective value regression using a subject-specific discount rate. Predicted activity is simply a scaled and shifted version of each subject’s behavioral discount function. This regression is also used to scale the y-axis across subjects (see Supplementary Methods).

Figure 6

Psychometric-neurometric comparisons. (a,d) A measure of the neural effect of delay is plotted against the subject’s behavioral discount rate for both (a) ROIs defined on the basis of the subjective value regression and (d) value ROIs defined in an unbiased manner. The neural effect of delay used is the ratio of the regression slope of neural activity against delay compared to that against amount. (b,e) The neural discount rate is plotted against the subject’s behavioral discount rate for both (b) subjective value ROIs and (e) unbiased value ROIs. The neural discount rate is estimated using a nonlinear version of the subjective value regression where k can vary. Colored lines show the robust linear fit for each ROI; black line shows the fit collapsed across all ROIs. Because of their skewed distribution, the ratio and neural and behavioral discount rates are log-transformed. All three data points from the most patient subject (bottom left) overlap in panels a,b,d and e. (c,f) Difference between neural discount rate and behavioral discount rate for both (c) subjective value ROIs and (f) unbiased value ROIs. Colored triangles indicate the median differences for each ROI. Panels a–c exclude two and d–f exclude one subject-ROI pair where an ROI could not be defined. Panel d also excludes two subject-ROI pairs where the correlation with amount was negative and e–f exclude five subject-ROI pairs where no discount rate accounted for a significant amount of variance in neural activity (see Methods).

Figure 7

Single-subject example of unbiased value ROIs and resulting neural discount functions. (a) ROIs in the ventral striatum, medial prefrontal cortex and posterior cingulate cortex are shown for one subject (HM, same subject shown in Fig. 4a,f and Fig. 5a,d), who demonstrated a relatively close psychometric-neurometric match in each region. Voxels were selected within these anatomically defined regions which showed either greater activity for trials involving the largest objective amount of the delayed reward than for trials involving the smallest amount, or greater activity for trials involving the shortest delay to the delayed reward than for those involving the longest delay. (b–d) A psychometric-neurometric comparison is shown for each ROI for this subject. Mean neural activity and standard error, summed over the 6–10-s window, are plotted as a function of the imposed delay to the delayed reward. The red line represents average predicted activity at each delay, from the fit of the subjective value regression using the subject’s behavioral discount rate (k = 0.0042). This regression is also used to scale the y-axis across ROIs (see Methods). The black line represents average predicted activity at each delay, from the fit of a nonlinear version of the subjective value regression where the discount rate is allowed to vary. The discount rates estimated from the neural data are 0.0052 in the ventral striatum, 0.0010 in the medial prefrontal cortex and 0.0099 in the posterior cingulate cortex.

Figure 8

Neural activity tracks subjective value, and not a more impulsive (β) or more patient (δ) estimate of value. (a,b) Difference between the neural (single exponential) discount rate and the behavioral (single exponential) discount rate, calculated separately for each ROI in each subject. The difference between neural and behavioral discount rates is centered on zero for both (a) ROIs defined on the basis of the subjective value regression and (b) value ROIs defined in an unbiased manner. (c,d) Difference between the neural (single exponential) discount rate and β, the steeper exponential from the sum of exponentials discount function estimated behaviorally. On average, β is larger than the neural discount rate for both (c) subjective value ROIs and (d) unbiased value ROIs. (e,f) Difference between the neural (single exponential) discount rate and δ, the shallower exponential from the sum of exponentials discount function estimated behaviorally. On average, the neural discount rate is larger than δ for both (e) subjective value ROIs and (f) unbiased value ROIs. Colored triangles in each panel indicate the medians for each ROI separately. These data exclude two subjects (six subject-ROI pairs) for which the fit of the β-δ model collapsed to a single exponential function (that is, β = δ). Panels b,d and f also exclude two subject-ROI pairs for which no discount rate accounted for a significant amount of variance in neural activity (see Methods).