Inferences on a multidimensional social hierarchy use a grid-like code

Abstract

Generalizing experiences to guide decision-making in novel situations is a hallmark of flexible behavior. Cognitive maps of an environment or task can theoretically afford such flexibility, but direct evidence has proven elusive. In this study, we found that discretely sampled abstract relationships between entities in an unseen two-dimensional social hierarchy are reconstructed into a unitary two-dimensional cognitive map in the hippocampus and entorhinal cortex. We further show that humans use a grid-like code in entorhinal cortex and medial prefrontal cortex for inferred direct trajectories between entities in the reconstructed abstract space during discrete decisions. These grid-like representations in the entorhinal cortex are associated with decision value computations in the medial prefrontal cortex and temporoparietal junction. Collectively, these findings show that grid-like representations are used by the human brain to infer novel solutions, even in abstract and discrete problems, and suggest a general mechanism underpinning flexible decision-making and generalization.

Conflict of interest statement

Competing interests

The authors declare no competing interests.

© 2021. The Author(s), under exclusive licence to Springer Nature America, Inc.

Figures

Extended Data Fig. 1. Multidimensional scaling (MDS)

Visualization of the group representation of the social hierarchy in a 2-D space using MDS on the neural activity extracted from the HC and EC ROIs. There were considerably fewer presentations of individuals at the rank positions 1 and 16, making their estimates less reliable, and so these less frequently sampled individuals were excluded in computing the MDS. The 2-D representations (top) can be factorized into two 1-D hierarchies: competence (middle) and popularity (bottom). The lines indicate the individuals at the same rank in the social hierarchy. The thicker the line, the higher the rank in the given dimension. a. The MDS computed from the mean pattern dissimilarity across participants after matching the number of samples per face. Numbers indicate face position as shown in the true hierarchy in the Model on the left. Blue colors correspond to competence the dimension and red to the popularity dimension. The distances and angles between the estimated individual locations in the HC and EC MDS are significantly correlated with the pairwise Euclidean distances (Spearman’s ρ = 0.84 for HC and ρ = 0.63 for EC) and cosine angles (ρ = 0.93 for HC and ρ = 0.71 for EC) in the true 4×4 social hierarchy structure, compared to random configurations (both p<0.01 compared to 1000 random permutations). b. The MDS estimated from the mean neural activity patterns including every presentation of the 14 face stimuli. Associated with Figure 3g and Extended Data Figure 2.

Extended Data Fig. 2. Representational similarity analysis (RSA) including all events of face stimuli presentation.

a. RSA including neural responses to all the events associated with 14 individuals in the social hierarchy. Consistent with the results of the RSA based on down-sampling shown in Figure 3c, we found effects of Euclidean distance on the pattern dissimilarity in the HC and EC but not in M1. The HC-EC system utilizes a 2-D relational cognitive map to represent the social hierarchy rather than representing 1-D map (τA of Euclidean distance (gray) > τA of one-dimensional rank distance in competence dimension (blue) and in popularity dimension (red)). **, pFWE<0.001 after correction for the number of bilateral ROIs (n=4) with the Bonferroni-Holm method; n.s., p>0.05, uncorrected. b. The dissimilarity between activity patterns estimated in bilateral HC and EC increases in proportion to the true pairwise Euclidean distance between individuals in the 2-D abstract social space (left, gray). The dissimilarity between activity patterns increases not only with the 1-D rank distance in the competence dimension (middle, blue) but also the 1-D rank distance in the popularity dimension (right, red). Methods and notation are identical to Figure 3. Box, lower and upper quartiles; line, median; whiskers, range of the data excluding outliers; +, the whiskers’ range of outliers.

Extended Data Fig. 3. The analysis procedure to examine the effects of grid-like codes

a. Identifying brain regions sensitive to hexagonal symmetry across the whole brain without aligning to the entorhinal grid orientation, φ. To identify hexagonally symmetric signals, we adopted previously developed methods from a previous study . We used a Z-transformed F-statistic to examine the BOLD activity modulated by any linear combination of sin(6θ) and cos(6θ). Hexagonal modulation was found in brain regions including in medial prefrontal cortex (mPFC; peak MNI coordinate, [x,y,z]=[2,66,−4], z=5.58), posterior cingulate cortex/precuneus (PCC; [2,−50, 36], z=3.19), bilateral posterior parietal cortex (PPC; [36,−46, 62], z=4.27 for right; [−38,−50, 50], z=3.81 for left), left lateral orbitofrontal cortex (OFC; [−42,44,−8], z=3.79), and retrosplenial cortex (RSC; [x,y,z] = [2,−54,28], z = 3.45) at a threshold pTFCE<0.05 (whole brain TFCE correction), as well as the right entorhinal cortex (EC; [26,−10, −40], z=2.80) at a threshold, pTFCE<0.05 (corrected within a priori anatomically defined EC region of interest (ROI) ,). For visualization purposes, the maps are thresholded at z>2.6 (p<0.005 uncorrected). b. We did not use the results of F-test for statistical inference but to functionally define ROIs in EC and mPFC for future tests. The ROIs were defined within the anatomically defined masks in the EC , and mPFC by including effects at a threshold of z>2.3, which corresponded to p<0.01. Using these independently defined ROIs, we then tested if the grid orientation estimated in the ROI was consistent across separate fMRI sessions in an unbiased way. It is important to note that the ROIs were defined from results of a statistically independent analysis, which was not dependent on the grid orientation. c. Illustrations of the cross-validation (CV) procedure. By splitting a dataset for estimating the grid orientation, φ, from another dataset to test for hexagonal modulation for inferred trajectories, φ, we could test for brain regions where activity was modulated by cos(6[θ−φ]) in alignment with the grid orientation estimated from the independent dataset for each participant. The CV was possible because the grid orientation, φ, is thought to be relatively stable but different across participants, whereas the direction of inferred trajectories varies only according to the position of F0, F1, and F2 (left panel). We performed a CV procedure both from fMRI sessions acquired within the same day (middle panel), and also from fMRI sessions acquired more than a week apart (right panel). d. We estimated the angle of the F1F2 vector (χ) and inputted the cos(6[χ − φ]) at the time of F2 presentation as an additional parametric regressor into GLM2. We did not find any brain areas encoding the angles of F1F2 vectors, except at a reduced threshold, in the bilateral somatosensory cortex ([x,y,z] = [−52,−14, 56], t=3.33 and [58,−8,50], t=3.17, p<0.005 uncorrected). e. Consistency of grid orientation in EC across sessions acquired more than a week apart. Concretely, in alignment with the EC grid orientation estimated from sessions acquired on a different day, we found hexadirectional modulation in a network of brain regions, including the mPFC ([−2, 36, −8], t=5.04), PCC ([6,− 58, 30], t=3.68), and TPJ ([−36, −64, 22], t=4.06) at our whole brain corrected threshold pTFCE<0.05, as well as in EC ([36, −10, 38], t=4,21) at pTFCE<0.05, small-volume-corrected in our a priori EC ROI (Supplementary Table 5b).

Extended Data Fig. 4. Hexagonal modulation for inferred trajectories only for the novel pairs when presented for the first-time

a. Among all the 88 (83) pairs (F0-F1 and F0-F2) presented during the partner selection task, those pairs that were not presented during behavioral training (always in the absence of feedback) but presented during fMRI for the first time are shown: 83 pairs in white were presented for the first-time for 4 participants; 88 pairs in white and gray were shown for the first-time for 17 participants. The grid effects were tested only for those pairs presented for the first time during the day1 scan. We extracted the mean activity and GP effects for each bin, restricted to when each pair was presented for the first time to participants. b and c. Associated with Figure 4b and c. The mean EC (left panel) and mPFC (middle panel) activity in 30° bins aligned to the EC grid orientation estimated from different blocks acquired in the same day’s scan (b) and the EC grid orientation acquired from a different day’s (day 2) scan (c) with six-fold symmetry. Right panel shows formal comparison of trajectories aligned and misaligned with both methods of computing the EC grid orientation. We found greater activity for the aligned pairs compared to the misaligned pairs to the EC grid orientation in EC and mPFC ROIs (one-sample t-test). d. Associated with Figure 5b. The GP effects in mPFC and bilateral TPJ are modulated by the grid alignment of the inferred trajectories aligned with the EC grid orientation. The GP effects are greater for the aligned pairs compared to misaligned pairs, even when they were presented for the first time (one-sample t-test). Box, lower and upper quartiles; line, median; whiskers, range of the data excluding outliers; +, the whisker’s range of outliers. **, p<0.01; *, p<0.05.

Extended Data Fig. 5. Individual differences in the relationship between the gridness and the effects of growth potential

a. In addition to the relationship between EC gridness (β cos(6[θ − Ψ])) and GP effects (βGP) in bilateral TPJ that we present in Figure 5c (upper left), we found a marginal positive correlation between the mPFC gridness and GP effects in mPFC and right TPJ (p<0.1) (upper middle). The gridness estimated in TPJ, however, does not correlate with their GP effects nor with the mPFC GP effect (p>0.1) (bottom left and middle). **, p<0.01; +, p<0.01. To further examine the relationship between the EC and mPFC gridness, we formally test which one better explains the GP effects in TPJ and mPFC. To address this question, we inputted the z-scored gridness of EC and mPFC into the same GLM to predict the GP effects in TPJ and mPFC. We found that the GP effect in bilateral TPJ was better explained by the EC gridness than the mPFC gridness (regression coefficient βEC=0.24** > βmPFC=0.17* for right TPJ; βEC=0.16* > βmPFC=0.13 for left TPJ; **, p<0.01 and *, p<0.05), and the GP effect in mPFC was better explained by the mPFC gridness than the EC gridness (βmPFC=0.09 (p=0.066) > βEC=0.01). Right: Colormap in matrix depicts regression coefficients for each regions’ gridness effect used to explain each regions’ GP effect. b. Left:Positive correlation between effects of differences in GP (|GP1−GP2|) in vmPFC during partner selection decisions and the EC gridness (r=0.43, p=0.05) but not mPFC gridness (r=−0.22, p=0.33). Right: Colormap shows regression coefficients for rEC and vmPFC gridness effects used to predict the vmPFC value difference effect. **, p<0.01; *, p<0.05; +, p<0.1. Box, lower and upper quartiles; line, median; whiskers, range of the data excluding outliers; +, the whiskers’ range of outliers.

Figure 1.. Behavioral training procedure and hypothesized neural representations.

a. A cognitive map of spatial (left) and non-spatial (right) relational structures allows for new direct routes or relationships to be inferred from directly experienced relationships, dramatically accelerating learning and decision capabilities through generalization. b. Participants learned the rank of 16 individuals organized into a 2-D social hierarchy defined by competence and popularity. Participants never saw neither 1-D or 2-D social hierarchies, but they could learn it piecemeal from dyadic comparisons in one dimension at a time during behavioral training. We hypothesized neural activity would be modulated hexadirectionally by the inferred trajectories over the 2-D social space, as predicted by a hexagonal grid organization. c. The inferred trajectories can be categorized as aligned and misaligned with the mean grid orientation, ϕ, which is different for each participant. θ1 and θ2 show examples of aligned and misaligned trajectories, respectively. Greater activity is predicted when the inferred trajectory is aligned compared to misaligned because it passes over more grid fields, which generates hexadirectional grid-like modulation. d. Behavioral training procedure. On day 1 and day 2 of behavioral training, participants learned ranks of 16 individuals (face stimuli) in each of two dimensions (competence or popularity) through binary decisions about the higher rank individual in a pair who differed by only one rank level in a given dimension, with each dimension learned on a different day. Within a day, the order of pairs compared was further randomized. After behavioral training, participants performed 3 blocks of the “partner selection task” twice during fMRI scanning, with a gap between sessions of at least one week. After the second session, participants were asked to place individuals according to their believed combined rank in a 2-D space for the first time (placement task).

Figure 2.. Partner Selection task.

a. Illustration of a trial of the partner selection task during fMRI. Participants were asked to make a binary decision by choosing a better business partner for a given individual (F0) between two (F1 and F2). The better partner is determined by the “growth potential (GP)” that each pair could expect from their collaboration. Participants could compute the GP of the F0 and F1 pair when F1 is shown and GP of the F0 and F2 pair when F2 is shown. Participants were subsequently asked to make a decision during F2 presentation. No feedback was given. b. To compute the GP of a pair, participants were instructed the GP corresponds to the higher “rank of the pair” in each dimension. Participants were further asked to weigh the ‘rank of the pair’ in both dimensions equally. Therefore, the GP corresponds to the area drawn by the higher rank of the two people in each dimension in the 2-D hierarchy (GPFOF1 (green rectangle) > GPFOF2 (red rectangle); F1 is the better partner for F0 in this example). We hypothesized people would infer direct trajectories over the mentally reconstructed 2-D space between the positions of F0 and each potential partner, F1 and F2, to compute the GP for each collaboration. We searched for neural evidence for hexadirectional modulation of inferred trajectories though the reconstructed cognitive map (θFOF1 at the time of F1 presentation and θFOF2 at the time of F2 presentation).

Figure 3.. Building a 2-D representation of a social hierarchy

a. Model representational dissimilarity matrix (RDM) computed from the pairwise Euclidean distances between individuals on the true 2-D social hierarchy. b. The extent to which the model RDM explained the pattern dissimilarity in ROIs. c. RSA based on equal sampling of 14 faces (by down-sampling). The rank correlation (τA) shows robust effects of Euclidean distance (gray) compared to the permuted baseline (1000 iterations; dashed line), but not in a control region, M1 (one-tailed Wilcoxon signed rank test). The τA of the Euclidean distance was significantly greater than the one-dimensional rank distance in each ROI (two-tailed Wilcoxon signed rank test) (Supplementary Table 2; pFWE<0.005, Bonferroni-Holm correction). d and e. The pattern dissimilarity in HC and EC increases in proportion to the pairwise Euclidean distance between individuals (gray), as well as with the two component 1-D rank distances (blue and red). For display purposes, the dissimilarity level was normalized to account for individual differences in scales (n=21). c, d, and e. Box, lower and upper quartiles; line, median; whiskers, range of the data excluding outliers; +, the whiskers’ range of outliers. f. Whole-brain searchlight RSA based on equal sampling of 14 faces at all events (F0, F1, and F2 presentations). g. Whole-brain searchlight RSA including all observations. The activity patterns in the HC, EC, mPFC/mOFC, and PCC/precuneus are explained by the model RDM for pairwise Euclidean distance (whole-brain TFCE correction, pFWE<0.05, except for HC and EC which was corrected using a priori ROIs [denoted by ✻]). h. Visualization of the group representation of the social hierarchy in a 2-D space using MDS on the neural activity extracted from the HC ROIs. The MDS (left) captures the true social hierarchy structure (right) better than random configurations (p<0.01 compared to 1000 random permutations; Extended Data Figure 1). i. Post-scan placement task. All participants successfully placed faces according to their ranks in a 2-D social hierarchy space. Each participant’s responses were rescaled according to their longest distances. Each dot indicates the face position placed by a single participant and colors indicate the ranks in the true social hierarchy.

Figure 4.. Hexagonal modulation for inferred trajectories.

a. Whole-brain parametric analysis showing hexagonal grid-like representation of inferred trajectories in alignment with the mean EC grid orientation at the time of F1 and F2 presentations. Significant effects are shown in EC (peak MNI coordinates, [x,y,z]=[22,−10,−28], t=4.11), mPFC ([−6,48,−4]; t=4.72), STS ([50,−40,4], t=4.05 for right; [−60,−24,−6], t=4.29 for left), TPJ ([46,−58,20],t=3.67 for right; [−56,−68,24], t=5.71 for left) (all pTFCE<0.05, whole-brain cluster corrected using threshold-free cluster enhancement (TFCE)), and EC (peak MNI coordinates, [x,y,z]=[22,−10,−28], t=4.11) (pFWE<0.05 TFCE correction within a priori anatomical ROI). The maps are displayed at a cluster-corrected threshold pTFCE< 0.05 over the whole brain for all brain regions, except for the EC, where we used a threshold of pTFCE< 0.05 corrected within anatomically defined ROI (denoted by ✻ next to EC), due to our strong a priori hypothesis of grid coding in EC ,(Supplementary Table 5). b. Six-fold modulation signals in the EC (left panel) and mPFC (right panel) ROIs aligned to the grid orientation in EC. The grid orientation was estimated from separate fMRI sessions acquired from the same day. The mean (±SE) z-scored activity is plotted separately for aligned (teal) and misaligned (gray) trajectories categorized into 12 equal bins of 30° according to the direction of inferred trajectories. The mean activity difference between aligned and misaligned trajectories was larger than zero for six-fold (p<0.01) but not for the other control periodicities (four-, five-, seven- and eight-fold; all p>0.05) (two-tailed one-sample t test), and the activity difference is greater for six-fold compared to the other periodicities (p<0.05; paired t test). c. Cross-day consistency of the grid orientation in EC (See Extended Data Figure 3e for whole brain analysis). The activity in EC and mPFC shows hexadirectional modulations for the inferred trajectories in alignment with the grid orientation in EC estimated from separate sessions acquired from a different day more than a week apart. This effect is also specific to the six-fold (p<0.01) periodicity (all p>0.05), suggesting that the mean grid angle in EC is consistent between sessions more than a week apart (Supplementary Table 6).

Figure 5.. Growth potential and value comparison.

a. Whole-brain map showing effects of the “growth potential” (GP) at the time of F1 and F2 presentation in a network of brain regions including the mPFC (peak MNI coordinates, [x,y,z]=[10,52,6], t=5.55) and bilateral TPJ ([54,−56,34], t=5.45) for right; [−54,−60,28], t=5.18) for left) (pTFCE<0.05 whole brain TFCE correction). The EC ([x,y,z]=[20,−4,−32], t=3.88) also showed GP effects. b. Mean (±SE) effects of GP (z-scored β) in mPFC and bilateral TPJ are modulated by the grid alignment of the inferred trajectories, θ, with six-fold periodicity aligned with the EC grid orientation, ϕ. Control analyses confirmed specificity for this six-fold periodicity (p<0.01) over control periodicities (effect of six-fold compared to other control periodicities, all p<0.05; Supplementary Table 6). c. Those participants with greater hexagonal modulation in EC show greater encoding of GP in independently defined bilateral TPJ ROIs (all p<0.05; Extended Data Figure 5). d. Whole-brain map contrasting the effects of GP (β GP) for trajectories aligned with the EC grid orientation, ϕ, compared to the misaligned trajectories (β GP aligned > β GP misaligned: pTFCE<0.05 whole brain TFCE correction). Contrasts are shown in mPFC, EC, STS, and TPJ. e. Mean (±SE) GP effects in anatomically defined mPFC, and bilateral TPJ ROIs (β GP). In all three areas, effects were greater for aligned compared to misaligned trajectories (all p<0.01), although the GP effects of both trajectories were significant (all p<0.01). b and e. Box, lower and upper quartiles; line, median; whiskers, range of the data excluding outliers; +, the whiskers’ range of outliers. f. Neural correlates of the relative decision value, |GP1-GP2|, during decision making, in vmPFC ([−8,54,−6]; t=5.07), HC ([−28,−4,−26]; t=4.53) (pTFCE<0.05), and bilateral EC ([22,−12,−26], t=5.10 for right; [−26,−16,−32], t=4.00 for left; pTFCE<0.05 TFCE corrected in a priori ROIs). ***, p<0.005; **, p<0.01; *, p<0.05. All the maps are displayed at a cluster-corrected threshold pTFCE < 0.05 over the whole brain for all brain regions, except for the EC, where we used a threshold of pTFCE < 0.05 corrected within our anatomically defined ROI (denoted by ✻ next to EC).