Negative symptoms and the failure to represent the expected reward value of actions: behavioral and computational modeling evidence

Abstract

Context: Negative symptoms are a core feature of schizophrenia, but their pathogenesis remains unclear. Negative symptoms are defined by the absence of normal function. However, there must be a productive mechanism that leads to this absence.

Objective: To test a reinforcement learning account suggesting that negative symptoms result from a failure in the representation of the expected value of rewards coupled with preserved loss-avoidance learning.

Design: Participants performed a probabilistic reinforcement learning paradigm involving stimulus pairs in which choices resulted in reward or in loss avoidance. Following training, participants indicated their valuation of the stimuli in a transfer test phase. Computational modeling was used to distinguish between alternative accounts of the data.

Setting: A tertiary care research outpatient clinic.

Patients: In total, 47 clinically stable patients with a diagnosis of schizophrenia or schizoaffective disorder and 28 healthy volunteers participated in the study. Patients were divided into a high-negative symptom group and a low-negative symptom group.

Main outcome measures: The number of choices leading to reward or loss avoidance, as well as performance in the transfer test phase. Quantitative fits from 3 different models were examined.

Results: Patients in the high-negative symptom group demonstrated impaired learning from rewards but intact loss-avoidance learning and failed to distinguish rewarding stimuli from loss-avoiding stimuli in the transfer test phase. Model fits revealed that patients in the high-negative symptom group were better characterized by an "actor-critic" model, learning stimulus-response associations, whereas control subjects and patients in the low-negative symptom group incorporated expected value of their actions ("Q learning") into the selection process.

Conclusions: Negative symptoms in schizophrenia are associated with a specific reinforcement learning abnormality: patients with high-negative symptoms do not represent the expected value of rewards when making decisions but learn to avoid punishments through the use of prediction errors. This computational framework offers the potential to understand negative symptoms at a mechanistic level.

Conflict of interest statement

None of the other authors have any financial conflicts to report.

Figures

Figure 1. Example of Reinforcement Learning Task Stimuli and Feedback

Panel A illustrates the feedback delivered after a correct choice(indicated by a blue border) in the reward trials; Panel B illustrates the feedback delivered following an incorrect choice. Panel C illustrates the feedback delivered following a correct choice in the loss-avoidance trials; Panel D illustrates the feedback delivered following an incorrect choice.

Figure 2. Differences in Reinforcement learning among patients and healthy controls on 90% and 80% probability gain and loss-avoidance conditions

Panels A & B illustrate performance in the 90 and 80% gain conditions, respectively. Panels C & D illustrate performance in the 90 and 80% loss-avoidance conditions, respectively.

Figure 3. Performance on the Gain - Loss-Avoidance Difference Score among patient and healthy control groups

The difference score was calculated using Block 4 performance. Scores above zero indicate better learning from gains than from loss-avoidance, while scores below zero indicate better learning from loss-avoidance than from gains.

Figure 4. Observed and model simulation results for end acquisition and transfer performance in healthy controls and in patients

Panel A shows observed and Panel B shows simulated end acquisition performance across groups and illustrates how the modeled controls had a preference for learning from gains relative to losses, an effect which is reduced in LNS and absent in HNS patients. Panel C shows observed transfer performance while Panel D gives simulation results. Note that the simulations capture the reduced preference for Frequent Winners (FW) over Frequent Loss Avoiders (FLA) in HNS patients (the only significant difference in the behavioral analyses of the transfer pairs), coupled with a preserved preference for Frequent winners over Frequent Losers (FL) and Infrequent winners(IF). All groups and simulated groups show a preference for frequent loss avoiders over infrequent winners despite having lower expected value.

Figure 5. The relative contribution of Q-learning and Actor-Critic learning to behavioral choices

Panel A illustrates the greater contribution of Q-learning in healthy controls relative to the patient groups. Only the contrast between the HCs and HNS patients was statistically significant. Panel B illustrates the predicted performance in a model of pure actor-acritic (AC) or pure Q-learning (Q) in the two diagnostic transfer pairs. The Q model shows clear preference for Frequent Winners over Frequent Loss Avoiders, whereas the A-C model does not. The two models show opposite preferences for Frequent Loss Avoiders over Infrequent Winners. 1000 model simulations were run to generate these predictions, using parameters fit to the control group, but the pattern is robust to parameter changes.