The similarity structure of distributed neural responses reveals the multiple representations of letters

Abstract

Most cognitive theories of reading and spelling posit modality-specific representations of letter shapes, spoken letter names, and motor plans as well as abstract, amodal letter representations that serve to unify the various modality-specific formats. However, fundamental questions remain regarding the very existence of abstract letter representations, the neuro-topography of the different types of letter representations, and the degree of cortical selectivity for orthographic information. We directly test quantitative models of the similarity/dissimilarity structure of distributed neural representations of letters using Multivariate Pattern Analysis-Representational Similarity Analysis (MVPA-RSA) searchlight methods to analyze the BOLD response recorded from single letter viewing. These analyses reveal a left hemisphere ventral temporal region selectively tuned to abstract letter representations as well as substrates tuned to modality-specific (visual, phonological and motoric) representations of letters. The approaches applied in this research address various shortcomings of previous studies that have investigated these questions and, therefore, the findings we report serve to advance our understanding of the nature and format of the representations that occur within the various sub-regions of the large-scale networks used in reading and spelling.

Keywords: Abstract letter identity; Letter representation; Multivariate Pattern Analysis (MVPA); Reading and spelling networks; Representational Similarity Analysis (RSA); Visual Word Form Area (VWFA).

Copyright © 2013 Elsevier Inc. All rights reserved.

Figures

Figure 1. The multiple formats of letter representation

Visual-Spatial letter representations are case-specific representations that are invariant across different fonts. Letter-Name representations correspond to the names of letters, while Motoric representations correspond to the basic motor sequences required to produce letter shapes. Abstract Letter Identities (ALIs) are amodal, abstract representations that lack visual form (they are font and case invariant), phonological content, or motor features. ALIs mediate translation between modality-specific formats. Word reading is based on ALIs that are accessed in response to processing specific visual letter shapes and are then used to search memory for the stored orthographic representations of familiar word forms.

Figure 2. Depiction of the four basic steps of the MVPA-RSA Searchlight Analysis

BOLD response was measured while participants viewed single (upper and lower case) letters during a symbol detection task. The search space for the analysis, depicted in purple, was derived from a functionally localized orthographic processing network.

Figure 3. Unthresholded Group Feature Sensitivity Maps depicting brain regions sensitive to each feature dimension

Each Group Feature Sensitivity Map was obtained from combining Individual Sensitivity maps (after Talairch normalization and Gaussian smoothing at 2 FWHM) and then carrying out a one-way, two-tailed, t-test at each voxel. These unthresholded t-maps depict the topography of feature-sensitivity for each of the four feature dimensions. Significant clusters after FDR correction are reported in Table 1.

Figure 4. Group ANOVA Selectivity Comparison Maps depicting brain regions selectively tuned to ALI and Visual-Spatial features

Each Group Selectivity Map was obtained from combining Individual Feature Sensitivity Maps (after Talairch normalization and Gaussian smoothing at 2 FWHM) and then carrying out four voxelwise ANOVA comparisons testing if activation from one feature dimension was greater than activation from the other 3. The maps are shown at a voxelwise threshold of f(3,32)>0. Only results from the Visual-Spatial and ALI feature dimensions are shown since the letter-name and motoric feature dimensions did not yield any significant clusters. See text for further details and Table 2 for information regarding clusters that were statistically significant after correction for multiple comparison.

Figure 5. Group Permutation Selectivity Maps depicting brain regions selectively tuned to ALI and Visual-Spatial features

Each group selectivity map was obtained by computing, at each voxel, the number of participants that exhibited selectivity to a single feature dimension. Selectivity was defined as a voxel in the Participant Feature Sensitivity Maps that exceeded p

Figure 6. Group Regression Map

Each group regression map was obtained by carrying out, at each searchlight volume, a regression analysis to determine the unique contribution of each feature dimension; the resulting beta weights for each dimension were then subjected to a one-way, two-tailed, t-test evaluation. Depicted are 4 horizontal slices from the unthresholded Group Regression Selectivity maps for each of the feature dimensions. Only the Abstract Letter Identity and Visual Spatial feature dimensions yielded significant clusters and these are reported in Table 3.

Figure 7. The significant clusters identified in each of the three selectivity analyses

For each of the maps, Visual-Spatial selectivity is depicted in green and Abstract Letter Identity is depicted in red. For the ANOVA comparison significant clusters (uncorrected voxelwise threshold p

Figure 8. The contribution of each feature dimension to the brain regions identified by the regression analysis as significant for ALI and Visual-Spatial similarity

The images depicts the clusters identified in the regression analysis as significant for Abstract Letter Identity (red) and Visual-Spatial Similarity (green). The graph depicts the regression t-values averaged across the voxels within each of the significant clusters.

Figure 9. Sensitivity of visually similar vs. visually dissimilar cross-case, same identity pairs within the ALI region (in red) identified in the regression analysis

The bar graph depicts the average t-value for voxels within the ALI-selective region for Group Sensitivity Maps based on a pRSM model in which the 6 most visually similar cross-case identity pairs are treated as ALIs and a pRSM in which the 6 most visually dissimilar cross-case identity pairs are treated as ALIs. The results indicate that visually similarity does not determine or influence responsivity to ALI.