Learning Predictive Statistics: Strategies and Brain Mechanisms

Abstract

When immersed in a new environment, we are challenged to decipher initially incomprehensible streams of sensory information. However, quite rapidly, the brain finds structure and meaning in these incoming signals, helping us to predict and prepare ourselves for future actions. This skill relies on extracting the statistics of event streams in the environment that contain regularities of variable complexity from simple repetitive patterns to complex probabilistic combinations. Here, we test the brain mechanisms that mediate our ability to adapt to the environment's statistics and predict upcoming events. By combining behavioral training and multisession fMRI in human participants (male and female), we track the corticostriatal mechanisms that mediate learning of temporal sequences as they change in structure complexity. We show that learning of predictive structures relates to individual decision strategy; that is, selecting the most probable outcome in a given context (maximizing) versus matching the exact sequence statistics. These strategies engage distinct human brain regions: maximizing engages dorsolateral prefrontal, cingulate, sensory-motor regions, and basal ganglia (dorsal caudate, putamen), whereas matching engages occipitotemporal regions (including the hippocampus) and basal ganglia (ventral caudate). Our findings provide evidence for distinct corticostriatal mechanisms that facilitate our ability to extract behaviorally relevant statistics to make predictions.SIGNIFICANCE STATEMENT Making predictions about future events relies on interpreting streams of information that may initially appear incomprehensible. Past work has studied how humans identify repetitive patterns and associative pairings. However, the natural environment contains regularities that vary in complexity from simple repetition to complex probabilistic combinations. Here, we combine behavior and multisession fMRI to track the brain mechanisms that mediate our ability to adapt to changes in the environment's statistics. We provide evidence for an alternate route for learning complex temporal statistics: extracting the most probable outcome in a given context is implemented by interactions between executive and motor corticostriatal mechanisms compared with visual corticostriatal circuits (including hippocampal cortex) that support learning of the exact temporal statistics.

Keywords: fMRI; learning; prediction; vision.

Copyright © 2017 Wang et al.

Figures

Figure 1.

Trial and sequence design. a, Trial design. 8–14 stimuli were presented sequentially, followed by a cue and the test display. b, Sequence design. Markov models comprised two levels of complexity. For the zero-order model (Level 0), different states (A–D) are assigned to four symbols with different frequencies. For the first-order model (Level 1), a diagram indicates states (circles) and transitional probabilities (black arrow: high probability, e.g., 80%; gray arrow: low probability, e.g., 20%). Transitional probabilities are shown in a four-by-four conditional probability matrix, with rows indicating temporal context and columns indicating the corresponding target. c, Experimental protocol. Observers underwent multiple days of behavioral training first with zero-order sequences and then with first-order sequences. For each level, observers completed three to five training sessions (an average of four sessions is shown for illustration purposes). Three fMRI scanning sessions were conducted before (Pre) and immediately after training per level (Post0, Post1).

Figure 2.

Behavioral performance. a, Mean PI across participants for test (open symbols) and training (solid symbols) blocks for Level 0 and Level 1. Data are fitted (least-squares nonlinear fit) across training blocks. Random guess baseline is indicated by dotted lines. b, Normalized PI during scanning. Data are shown before (gray bars) and after (black bars) training for each level. Error bars indicate SEM.

Figure 3.

fMRI results. a, GLM maps for the two-way interaction between scanning session (Pre, Post0, Post1) and sequence (structured vs random), at p < 0.005 (cluster threshold corrected). Only the first five volumes were included in the analysis that correspond to the presentation of sequence, the participants' prediction, and the test display presentation to avoid confounding the results by the participants' response. Similar results were observed at a more conservative threshold (p < 0.001), but small volume correction was necessary for small structures (i.e., putamen) at this threshold. b, PSC index (percentage signal change for structured sequences compared with random sequences) before and after training for Level 0 and Level 1. Data are shown for ROIs that showed a significant interaction between session (pretraining vs posttraining) and sequence (structured vs random). Error bars indicate SEM. Note that different number of runs were scanned before and after training (i.e., pretraining scan comprised three runs per level, whereas posttraining scans comprised nine runs per level). To compare equal amounts of data before and after training, we selected three of the nine runs from each posttraining scan; that is, we divided each session into two time periods and selected randomly one run per time period to match the order in which data were collected during the pretraining scan. Whole-brain voxelwise GLM analysis showed significant interactions for sequence (structured vs random) and scanning session (Pre, Post0, Post1) in the frontal, parietal, and subcortical regions, which is consistent with our main result.

Figure 4.

fMRI results controlled for differences in sequence entropy across levels. a, GLM maps (p < 0.001, cluster threshold corrected) for 2-way interaction between scanning session (Pre, Post0, Post1) and sequence (structured vs random) including entropy rate as a regressor. b, PSC index before and after training for Level 0 and Level 1. Error bars indicate SEM. Data are shown for ROIs that showed a significant interaction between session (pre vs posttraining) and sequence (structured vs random).

Figure 5.

fMRI results for sequence presentation and participants' prediction. a, PSC index for sequence presentation (volumes 1–2) and participant prediction (volumes 4–5) before and after training for Level 0 and Level 1. Data are shown for the representative ROIs from Figure 4b. Error bars indicate SEM. b, GLM maps for the 2-way interaction between scanning session and sequence at p < 0.005 (cluster threshold corrected) using only the volumes that correspond to sequence presentation.

Figure 6.

Strategy choice. Strategy choice is shown at the beginning (first two runs) and end (last two runs) of training for Level 0 (squares) and Level 1 (circles). Open symbols indicate individual participant data; closed symbols indicate mean date per level. Strategy choice was measured by comparing participant responses to two possible strategies: matching (i.e., predicting the presented target distribution) versus maximization (i.e., predicting the high probability targets per context). Negative values indicate a strategy closer to matching, whereas positive values indicate a strategy closer to maximization. Error bars indicate SEM.

Figure 7.

Brain activations correlating with matching. Covariance analysis shows significant (p < 0.05, cluster threshold corrected) negative correlations (R correlation coefficient) between individual strategy index and learning-dependent fMRI change (i.e., after vs before training) for Level 0 (a) and Level 1 (b). Whole-brain maps and plots show negative correlations between strategy index and PSC index change (posttraining vs pretraining) for representative ROIs, as derived from the covariance analysis (note that these correlation plots are only presented for demonstration purposes; no additional statistical analysis was performed in these ROIs after the covariance analysis to avoid circularity). Cd, Caudate: b, body; t, tail; Th, thalamus; PHG, parahippocampal gyrus; Hipp, hippocampus.

Figure 8.

Brain activations correlating with maximization. Covariance analysis shows significant (p < 0.05, cluster threshold corrected) positive correlations (R correlation coefficient) between individual strategy index and learning-dependent fMRI change (i.e., posttraining vs pretraining) for Level 0 (a) and Level 1 (b). Whole-brain maps and plots show positive correlations between strategy index and PSC index change (posttraining vs pretraining) for representative ROIs, as derived from the covariance analysis (note that these correlation plots are only presented for demonstration purposes; no additional statistical analysis was performed in these ROIs after the covariance analysis to avoid circularity).