Learning predictive statistics from temporal sequences: Dynamics and strategies

Abstract

Human behavior is guided by our expectations about the future. Often, we make predictions by monitoring how event sequences unfold, even though such sequences may appear incomprehensible. Event structures in the natural environment typically vary in complexity, from simple repetition to complex probabilistic combinations. How do we learn these structures? Here we investigate the dynamics of structure learning by tracking human responses to temporal sequences that change in structure unbeknownst to the participants. Participants were asked to predict the upcoming item following a probabilistic sequence of symbols. Using a Markov process, we created a family of sequences, from simple frequency statistics (e.g., some symbols are more probable than others) to context-based statistics (e.g., symbol probability is contingent on preceding symbols). We demonstrate the dynamics with which individuals adapt to changes in the environment's statistics-that is, they extract the behaviorally relevant structures to make predictions about upcoming events. Further, we show that this structure learning relates to individual decision strategy; faster learning of complex structures relates to selection of the most probable outcome in a given context (maximizing) rather than matching of the exact sequence statistics. Our findings provide evidence for alternate routes to learning of behaviorally relevant statistics that facilitate our ability to predict future events in variable environments.

Figures

Figure 1

Trial and sequence design. (a) Eight to 14 symbols were presented one at a time in a continuous stream followed by a cue and the test display. (b) Sequence design. For the zero-order model (Level 0): Different states (A, B, C, D) are assigned to four symbols with different probabilities. For first- (Level 1) and second- (Level 2) order models, diagrams indicate states (circles) and conditional probabilities (red arrow: high; gray arrow: low). Transitional probabilities were arranged in a 4 × 4 (Level 1) or 4 × 6 (Level 2) conditional-probability matrix.

Figure 2

Experiment 1: Behavioral performance. (a) Performance index for Group 0 (n = 19) across training (solid circles) blocks, pretraining test (Pre: open squares), and posttraining test (Post: open squares). The performance index expresses the absolute distance (proportion overlap) between the distribution of participant responses and the distribution of presented targets. Overall performance index is calculated as the weighted average across context probabilities. Data are fitted for participants who improved during training (black circles). Data are also shown for one participant who did not improve during training (Level 2, gray symbols). Error bars show standard error of the mean. (b) Response probabilities for individual targets (Level 0) or conditional probabilities of context–target contingencies (Levels 1 and 2) across training blocks. Red lines indicate targets or context–target contingencies with the highest (conditional) probability (i.e., 0.72 for Level 0 and 0.8 for Levels 1 and 2), blue lines indicate the second-highest (conditional) probabilities (i.e., 0.18 for Level 0 and 0.2 for Levels 1 and 2), and green lines indicate targets or context–target contingencies that appear rarely (i.e., 0.05) or not at all. For Level 2, first- and second-order contexts are presented separately (dashed vs. solid lines).

Figure 3

Experiment 1: Response tracking. (a) Functional clustering analysis (Group 0) showed two data clusters, indicated in red (Level 0: n = 13, Level 1: n = 14, Level 2: n = 11) versus blue (Level 0: n = 6, Level 1: n = 5, Level 2: n = 6). Mixture coefficient curves are shown for each individual participant; bold curves indicate sigmoid fits to each cluster. Data are also shown for two participants (black lines) who showed less than a 25% probability of extracting the correct context length at the end of training. (b) Learning predictive probabilities. ΔKL curves between the predictive mixture model for each level and baseline models across training blocks. ΔKL values above zero indicate that the participant responses approximated the Markov model that generated the sequences. Average data are shown per participant cluster (i.e., red vs. blue). Note: The smaller ΔKL values and error bars for Level 2 reflect small differences between Level 1 and Level 2 models; yet fast learners show higher values than zero, indicating that they are able to learn second-order context–target contingencies. Error bars show the standard error of the mean. (c) Strategy choice, as indicated by comparing (ΔKL) matching versus maximization for each participant per cluster (i.e., red vs. blue).

Figure 4

Experiment 2: Behavioral performance. Data for Group 1 (n = 8; Levels 1 and 2) and Group 2 (n = 12; Level 2). Performance index is shown across training (solid circles) blocks, pretraining test (Pre: open squares), and posttraining test (Post: open squares). Fitted data are shown for participants who improved during training (black circles). Data are also shown for participants (n = 4) in Group 2 who did not improve during training (Level 2, gray symbols). Error bars show standard error of the mean.

Figure 5

Strategies for learning context-based statistics. (a) Correlations of individual strategy index and learning rate for participants who improved at both Levels 1 and 2 during training in Group 0 and Group 1. (b) Correlation of individual strategy index between Level 1 and Level 2 for participants trained in Group 0 and Group 1. Negative strategy-index values indicate a strategy closer to matching, while positive values indicate a strategy closer to maximization.