From prediction error to incentive salience: mesolimbic computation of reward motivation

Abstract

Reward contains separable psychological components of learning, incentive motivation and pleasure. Most computational models have focused only on the learning component of reward, but the motivational component is equally important in reward circuitry, and even more directly controls behavior. Modeling the motivational component requires recognition of additional control factors besides learning. Here I discuss how mesocorticolimbic mechanisms generate the motivation component of incentive salience. Incentive salience takes Pavlovian learning and memory as one input and as an equally important input takes neurobiological state factors (e.g. drug states, appetite states, satiety states) that can vary independently of learning. Neurobiological state changes can produce unlearned fluctuations or even reversals in the ability of a previously learned reward cue to trigger motivation. Such fluctuations in cue-triggered motivation can dramatically depart from all previously learned values about the associated reward outcome. Thus, one consequence of the difference between incentive salience and learning can be to decouple cue-triggered motivation of the moment from previously learned values of how good the associated reward has been in the past. Another consequence can be to produce irrationally strong motivation urges that are not justified by any memories of previous reward values (and without distorting associative predictions of future reward value). Such irrationally strong motivation may be especially problematic in addiction. To understand these phenomena, future models of mesocorticolimbic reward function should address the neurobiological state factors that participate to control generation of incentive salience.

© 2012 The Author. European Journal of Neuroscience © 2012 Federation of European Neuroscience Societies and Blackwell Publishing Ltd.

Figures

Figure 1. Incentive salience distinguishes ‘wanting’, ‘liking’ and learning about the same reward

A cue’s learned associations (CS) or a UCS reward are each an input to potentially trigger ‘wanting’ (top) and ‘liking’ (bottom). Natural appetite or satiety states act as kappa factor in Zhang equation to modulate both ‘wanting’ and ‘liking’ for relevant reward UCS and CS. Dopamine drug and mesolimbic sensitization act more selectively to modulate only incentive salience because of the special dopamine relation to ‘wanting’ mechanisms. Re-drawn from Robinson & Berridge (1993), based on concepts from Toates (1986) and Bindra (1981).

Figure 2. Simulations of upshifts in CS temptation power

Zhang equations simulate actual enhancements in CS attractiveness induced by increases in kappa factor (Zhang et al., 2009). Multiplicative amplification of level of ‘wanting’ elicited by reward CS shown on left, induced by a new intoxication state (e.g., amphetamine) or by a mesolimbic sensitization state existing at the moment of cue re-encounter. Valence reversal from negatively aversive to positively ‘wanted’ shown at right, induced by sodium appetite, modulates incentive salience of CS previously associated with triple-seawater concentrated salty taste UCS. Simulated data based on (Wyvell & Berridge, 2001; Tindell et al., 2005; Smith et al., 2011; (Krieckhaus & Wolf, 1968; Fudim, 1978; Berridge & Schulkin, 1989; Stouffer & White, 2005; Tindell et al., 2009; Robinson & Berridge, 2010).

Figure 3. Dopamine/opioid amplification of neural signals for incentive salience

Neural signals of nucleus accumbens outputs were recorded in ventral pallidum. Two serial CSs had been previously learned by rats to predict sucrose UCS. Amphetamine (dopamine stimulation) or DAMGO (opioid stimulation) microinjections did not enhance prediction signal of upcoming reward elicited by first CS1 (which predicted CS2 and reward UCS with 100% certainty) (Smith et al., 2011). Drug stimulation did amplify >50% neural signals for incentive salience triggered by CS2 (associated with maximal motivation, though merely redundant predictor) microinjections. In a similar experiment (right), after Pavlovian training in a normal state, mesolimbic sensitization added incrementally to the acute effects of amphetamine on board in amplifying intensity of incentive salience triggered by sucrose CS (Tindell et al., 2005). Reprinted by permission.

Figure 4. Behavioral consequences of dopamine-amplified ‘wanting’: CS triggers higher pulses of ‘wanting’ for reward UCS

Cue-triggered ‘wanting’ for reward was assessed using Pavlovian-Instrumental transfer. Testing was in extinction (no sucrose UCS, so responding decreases as trial proceeded). Amphetamine microinjections in nucleus accumbens selectively amplified the peak height of cue-triggered pulses of increased motivation to obtain sucrose reward. Prior sensitization similarly produced selective amplification of CS+-triggered motivation to obtain reward (red; separate rats used for sensitization; sensitized rats tested here in the absence of amphetamine-on-board). The CS had never been paired with response of lever pressing prior to test, eliminating CS-press stimulus-response habit explanations. Amphetamine failed to enhance baseline levels of pressing (i.e., increasing pressing peak height, but not raising base plateau that peaks sit on). The pattern indicates dopamine stimulation did not increase any stable prediction of reward value throughout the session, but rather selectively amplified the phasic pulse of motivation elicited by each CS that lasted about a minute. Redrawn data from (Wyvell & Berridge, 2001); redrawn from (Zhang et al., 2009) by permission.

Figure 5. Normal learning by rats without dopamine: Pavlovian reward devaluation

After rats lost >98% of dopamine concentrations from nucleus accumbens and neostriatum, due to 6-OHDA lesions, a Pavlovian taste-aversion learning was used to devalue a sweet CS flavor. Tastes of a distinctive and originally palatable saccharin-polycose solution were associative paired as CS with injections of LiCl to induce nausea as UCS. On the first presentation, the CS taste elicited purely positive ‘liking’ facial expressions (e.g., rhythmic lip licking) in a taste reactivity test. The affective reactions of rats with dopamine-free brains were essentially as hedonically positive as normal control rats. After 3 Pavlovian training pairings with LiCl illness, affective expressions to the sweet CS taste were reversed in valence. Negative disgust reactions (e.g. gapes) were elicited by the CS after learning, indicating a learned shift to ‘disliking’. The magnitude of the learned devaluation was equivalent in dopamine-free 6-OHDA rats and normal control rats. Thus, rats with virtually no dopamine in their brains were still capable of learning a new value for a reward. Data redrawn from (Berridge & Robinson, 1998).