Affective bias as a rational response to the statistics of rewards and punishments
Erdem Pulcu, Michael Browning, Erdem Pulcu, Michael Browning
Abstract
Affective bias, the tendency to differentially prioritise the processing of negative relative to positive events, is commonly observed in clinical and non-clinical populations. However, why such biases develop is not known. Using a computational framework, we investigated whether affective biases may reflect individuals' estimates of the information content of negative relative to positive events. During a reinforcement learning task, the information content of positive and negative outcomes was manipulated independently by varying the volatility of their occurrence. Human participants altered the learning rates used for the outcomes selectively, preferentially learning from the most informative. This behaviour was associated with activity of the central norepinephrine system, estimated using pupilometry, for loss outcomes. Humans maintain independent estimates of the information content of distinct positive and negative outcomes which may bias their processing of affective events. Normalising affective biases using computationally inspired interventions may represent a novel approach to treatment development.
Keywords: computational Modelling; depression; human; learning; neuroscience; norepinepherine; pupilometry.
Conflict of interest statement
No competing interests declared.
Received travel expenses from Lundbeck for attending conferences.
Figures
References
- Aston-Jones G, Cohen JD. An integrative theory of locus coeruleus-norepinephrine function: adaptive gain and optimal performance. Annual Review of Neuroscience. 2005;28:403–450. doi: 10.1146/annurev.neuro.28.061604.135709.
- Behrens TE, Hunt LT, Woolrich MW, Rushworth MF. Associative learning of social value. Nature. 2008;456:245–249. doi: 10.1038/nature07538.
- Behrens TE, Woolrich MW, Walton ME, Rushworth MF. Learning the value of information in an uncertain world. Nature Neuroscience. 2007;10:1214–1221. doi: 10.1038/nn1954.
- Bradley BP, Mogg K, Williams R. Implicit and explicit memory for emotion-congruent information in clinical depression and anxiety. Behaviour Research and Therapy. 1995;33:755–770. doi: 10.1016/0005-7967(95)00029-W.
- Browning M, Behrens TE, Jocham G, O'Reilly JX, Bishop SJ. Anxious individuals have difficulty learning the causal statistics of aversive environments. Nature Neuroscience. 2015;18:590–596. doi: 10.1038/nn.3961.
- Browning M, Holmes EA, Charles M, Cowen PJ, Harmer CJ. Using attentional bias modification as a cognitive vaccine against depression. Biological Psychiatry. 2012;72:572–579. doi: 10.1016/j.biopsych.2012.04.014.
- Ciociola AA, Cohen LB, Kulkarni P, FDA-Related Matters Committee of the American College of Gastroenterology How drugs are developed and approved by the FDA: current process and future directions. The American Journal of Gastroenterology. 2014;109:620–623. doi: 10.1038/ajg.2013.407.
- Daw ND, Gershman SJ, Seymour B, Dayan P, Dolan RJ. Model-based influences on humans' choices and striatal prediction errors. Neuron. 2011;69:1204–1215. doi: 10.1016/j.neuron.2011.02.027.
- Eshel N, Roiser JP. Reward and punishment processing in depression. Biological Psychiatry. 2010;68:118–124. doi: 10.1016/j.biopsych.2010.01.027.
- Gotlib IH, Krasnoperova E, Yue DN, Joormann J. Attentional biases for negative interpersonal stimuli in clinical depression. Journal of Abnormal Psychology. 2004;113:127–135. doi: 10.1037/0021-843X.113.1.121.
- Jepma M, Murphy PR, Nassar MR, Rangel-Gomez M, Meeter M, Nieuwenhuis S. Catecholaminergic regulation of learning rate in a dynamic environment. PLoS Computational Biology. 2016;12:e1005171. doi: 10.1371/journal.pcbi.1005171.
- Joshi S, Li Y, Kalwani RM, Gold JI. Relationships between pupil diameter and neuronal activity in the locus coeruleus, colliculi, and cingulate cortex. Neuron. 2016;89:221–234. doi: 10.1016/j.neuron.2015.11.028.
- MacKay DJ. Information Theory, Inference and Learning Algorithms. Cambridge: Cambridge University Press; 2003.
- Mathews A, MacLeod C. Cognitive vulnerability to emotional disorders. Annual Review of Clinical Psychology. 2005;1:167–195. doi: 10.1146/annurev.clinpsy.1.102803.143916.
- Nassar MR, Rumsey KM, Wilson RC, Parikh K, Heasly B, Gold JI. Rational regulation of learning dynamics by pupil-linked arousal systems. Nature Neuroscience. 2012;15:1040–1046. doi: 10.1038/nn.3130.
- NICE . Treatment and management of depression in adults, including adults with a chronic physical health problem. London: NICE; 2009.
- Prelec D. The probability weighting function. Econometrica. 1998;66:497–527. doi: 10.2307/2998573.
- Rescorla RA, Wagner AR. A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement. In: Black A. H, Prokasy W. F, editors. Classiacal Conditioning II: Current Research and Theory. New York: Appleton-Centuary-Crofts; 1972. pp. 64–99.
- Rush AJ, Trivedi MH, Ibrahim HM, Carmody TJ, Arnow B, Klein DN, Markowitz JC, Ninan PT, Kornstein S, Manber R, Thase ME, Kocsis JH, Keller MB. The 16-item quick inventory of depressive symptomatology (QIDS), clinician rating (QIDS-C), and self-report (QIDS-SR): a psychometric evaluation in patients with chronic major depression. Biological Psychiatry. 2003;54:573–583. doi: 10.1016/S0006-3223(02)01866-8.
- Sharot T, Garrett N. Forming beliefs: why valence matters. Trends in Cognitive Sciences. 2016;20:25–33. doi: 10.1016/j.tics.2015.11.002.
- Spielberger CD, Gorsuch RL, Lushene RD. Manual for the State-Trait Anxiety Inventory. Palo Alto, CA: Consulting Psychologists Press; 1983.
- Sutton R, Barto AG. Reinforcement Learning. Cambridge, Massachusetts: MIT Press; 1998.
- Tversky A, Kahneman D. Advances in prospect theory: cumulative representation of uncertainty. Journal of Risk and Uncertainty. 1992;5:297–323. doi: 10.1007/BF00122574.
- Yechiam E, Telpaz A. To take risk is to face loss: a tonic pupillometry study. Frontiers in Psychology. 2011;2:344. doi: 10.3389/fpsyg.2011.00344.
- Yu AJ, Dayan P. Uncertainty, neuromodulation, and attention. Neuron. 2005;46:681–692. doi: 10.1016/j.neuron.2005.04.026.
Source: PubMed