The Importance of Considering Model Choices When Interpreting Results in Computational Neuroimaging

Abstract

Model-based analyses open exciting opportunities for understanding neural information processing. In a commentary published in eNeuro, Gardner and Liu (2019) discuss the role of model specification in interpreting results derived from complex models of neural data. As a case study, they suggest that one such analysis, the inverted encoding model (IEM), should not be used to assay properties of "stimulus representations" because the ability to apply linear transformations at various stages of the analysis procedure renders results "arbitrary." Here, we argue that the specification of all models is arbitrary to the extent that an experimenter makes choices based on current knowledge of the model system. However, the results derived from any given model, such as the reconstructed channel response profiles obtained from an IEM analysis, are uniquely defined and are arbitrary only in the sense that changes in the model can predictably change results. IEM-based channel response profiles should therefore not be considered arbitrary when the model is clearly specified and guided by our best understanding of neural population representations in the brain regions being analyzed. Intuitions derived from this case study are important to consider when interpreting results from all model-based analyses, which are similarly contingent upon the specification of the models used.

Keywords: computational neuroimaging; inverted encoding model; multivariate analysis; stimulus reconstruction.

Copyright © 2019 Sprague et al.

Figures

Figure 1.

Differences between conditions can be preserved across invertible linear transforms. A, We simulated voxel-level fMRI data where each voxel’s response was generated based on the sum of simulated responses across a population of simulated neurons with randomly centered tuning preferences and variable bandwidth (here, n = number of neurons, set to 100, although only 10 neural tuning functions are shown for clarity; see code on GitHub for full set of model parameters; https://github.com/tommysprague/iem_sim). Noise was added to the neural responses and then the gain factor (g) was applied to the data from each condition (condition 1: g = 1, condition 2: g = 1.8). For display purposes the noise (N) was set to 0 for panels A–C (following Gardner and Liu, 2019, their Fig. 3) and was set to 10 for panel D. B, We analyzed data using two different formats of channel basis functions, mirroring those used by Gardner and Liu (2019). Importantly, the two bases are related by an invertible linear transform (xform). C, Reconstructed channel response profiles differ in similar ways: condition 2 has a higher amplitude than condition 1, regardless of the basis set used, and the bimodal channel response profiles are related by the inverse of the linear transform that was used to create the bimodal basis in the first place (xform−1). D, Modeled gain compared to measured gain between conditions 2 and 1, computed using both the raised cosine basis set and the transformed bimodal version of the cosine basis set. Because there is not a straightforward way to quantify amplitude for the channel response profiles computed from the bimodal basis, we instead implemented a model-free quantification scheme in which we computed the ratio of the area under each channel response profile (i.e., ratio of area under the curve in condition 2 compared to condition 1).