A general decoding strategy explains the relationship between behavior and correlated variability
Amy M Ni, Chengcheng Huang, Brent Doiron, Marlene R Cohen, Amy M Ni, Chengcheng Huang, Brent Doiron, Marlene R Cohen
Abstract
Improvements in perception are frequently accompanied by decreases in correlated variability in sensory cortex. This relationship is puzzling because overall changes in correlated variability should minimally affect optimal information coding. We hypothesize that this relationship arises because instead of using optimal strategies for decoding the specific stimuli at hand, observers prioritize generality: a single set of neuronal weights to decode any stimuli. We tested this using a combination of multineuron recordings in the visual cortex of behaving rhesus monkeys and a cortical circuit model. We found that general decoders optimized for broad rather than narrow sets of visual stimuli better matched the animals' decoding strategy, and that their performance was more related to the magnitude of correlated variability. In conclusion, the inverse relationship between perceptual performance and correlated variability can be explained by observers using a general decoding strategy, capable of decoding neuronal responses to the variety of stimuli encountered in natural vision.
Keywords: neural coding; neuroscience; noise correlations; perception; rhesus macaque; visual attention.
Conflict of interest statement
AN, CH, BD, MC No competing interests declared
© 2022, Ni et al.
Figures
References
- Abbott LF, Dayan P. The effect of correlated variability on the accuracy of a population code. Neural Computation. 1999;11:91–101. doi: 10.1162/089976699300016827.
- Averbeck BB, Latham PE, Pouget A. Neural correlations, population coding and computation. Nature Reviews. Neuroscience. 2006;7:358–366. doi: 10.1038/nrn1888.
- Berkes P, Turner RE, Sahani M. A structured model of video reproduces primary visual cortical organisation. PLOS Computational Biology. 2009;5:e1000495. doi: 10.1371/journal.pcbi.1000495.
- Brainard DH. The psychophysics toolbox. Spatial Vision. 1997;10:433–436. doi: 10.1163/156856897X00357.
- Burgess AE, Wagner RF, Jennings RJ, Barlow HB. Efficiency of human visual signal discrimination. Science (New York, N.Y.) 1981;214:93–94. doi: 10.1126/science.7280685.
- Clery S, Cumming BG, Nienborg H. Decision-related activity in macaque V2 for fine disparity discrimination is not compatible with optimal linear readout. The Journal of Neuroscience. 2017;37:715–725. doi: 10.1523/JNEUROSCI.2445-16.2016.
- Cohen MR, Maunsell JHR. Attention improves performance primarily by reducing interneuronal correlations. Nature Neuroscience. 2009;12:1594–1600. doi: 10.1038/nn.2439.
- Cohen MR, Kohn A. Measuring and interpreting neuronal correlations. Nature Neuroscience. 2011;14:811–819. doi: 10.1038/nn.2842.
- Cohen MR, Maunsell JHR. Using neuronal populations to study the mechanisms underlying spatial and feature attention. Neuron. 2011;70:1192–1204. doi: 10.1016/j.neuron.2011.04.029.
- Cunningham JP, Yu BM. Dimensionality reduction for large-scale neural recordings. Nature Neuroscience. 2014;17:1500–1509. doi: 10.1038/nn.3776.
- Deneve S. Making decisions with unknown sensory reliability. Frontiers in Neuroscience. 2012;6:75. doi: 10.3389/fnins.2012.00075.
- Ecker AS, Berens P, Cotton RJ, Subramaniyan M, Denfield GH, Cadwell CR, Smirnakis SM, Bethge M, Tolias AS. State dependence of noise correlations in macaque primary visual cortex. Neuron. 2014;82:235–248. doi: 10.1016/j.neuron.2014.02.006.
- Goris RLT, Movshon JA, Simoncelli EP. Partitioning neuronal variability. Nature Neuroscience. 2014;17:858–865. doi: 10.1038/nn.3711.
- Gregoriou GG, Rossi AF, Ungerleider LG, Desimone R. Lesions of prefrontal cortex reduce attentional modulation of neuronal responses and synchrony in V4. Nature Neuroscience. 2014;17:1003–1011. doi: 10.1038/nn.3742.
- Gu Y, Liu S, Fetsch CR, Yang Y, Fok S, Sunkara A, DeAngelis GC, Angelaki DE. Perceptual learning reduces interneuronal correlations in macaque visual cortex. Neuron. 2011;71:750–761. doi: 10.1016/j.neuron.2011.06.015.
- Haefner RM, Gerwinn S, Macke JH, Bethge M. Inferring decoding strategies from choice probabilities in the presence of correlated variability. Nature Neuroscience. 2013;16:235–242. doi: 10.1038/nn.3309.
- Herrero JL, Gieselmann MA, Sanayei M, Thiele A. Attention-induced variance and noise correlation reduction in macaque V1 is mediated by NMDA receptors. Neuron. 2013;78:729–739. doi: 10.1016/j.neuron.2013.03.029.
- Howell DC. Statistical Methods for Psychology. Belmont, CA: Thomson Wadsworth; 2007.
- Huang C, Ruff DA, Pyle R, Rosenbaum R, Cohen MR, Doiron B. Circuit models of low-dimensional shared variability in cortical networks. Neuron. 2019;101:337–348. doi: 10.1016/j.neuron.2018.11.034.
- Huang C, Pouget A, Doiron B. Internally Generated Population Activity in Cortical Networks Hinders Information Transmission. bioRxiv. 2020 doi: 10.1101/2020.02.03.932723.
- Jeanne JM, Sharpee TO, Gentner TQ. Associative learning enhances population coding by inverting interneuronal correlation patterns. Neuron. 2013;78:352–363. doi: 10.1016/j.neuron.2013.02.023.
- Jin M, Beck JM, Glickfeld LL. Neuronal adaptation reveals a suboptimal decoding of orientation tuned populations in the mouse visual cortex. The Journal of Neuroscience. 2019;39:3867–3881. doi: 10.1523/JNEUROSCI.3172-18.2019.
- Kafashan M, Jaffe AW, Chettih SN, Nogueira R, Arandia-Romero I, Harvey CD, Moreno-Bote R, Drugowitsch J. Scaling of sensory information in large neural populations shows signatures of information-limiting correlations. Nature Communications. 2021;12:473. doi: 10.1038/s41467-020-20722-y.
- Kanashiro T, Ocker GK, Cohen MR, Doiron B. Attentional modulation of neuronal variability in circuit models of cortex. eLife. 2017;6:e23978. doi: 10.7554/eLife.23978.
- Kanitscheider I, Coen-Cagli R, Kohn A, Pouget A. Measuring fisher information accurately in correlated neural populations. PLOS Computational Biology. 2015a;11:e1004218. doi: 10.1371/journal.pcbi.1004218.
- Kanitscheider I, Coen-Cagli R, Pouget A. Origin of information-limiting noise correlations. PNAS. 2015b;112:E6973–E6982. doi: 10.1073/pnas.1508738112.
- Kaschube M, Schnabel M, Lowel S, Coppola DM, White LE, Wolf F. Universality in the evolution of orientation columns in the visual cortex. Science (New York, N.Y.) 2010;330:1113–1116. doi: 10.1126/science.1194869.
- Kersten D. Statistical efficiency for the detection of visual noise. Vision Research. 1987;27:1029–1040. doi: 10.1016/0042-6989(87)90016-2.
- Kohn A, Coen-Cagli R, Kanitscheider I, Pouget A. Correlations and neuronal population information. Annual Review of Neuroscience. 2016;39:237–256. doi: 10.1146/annurev-neuro-070815-013851.
- Lin IC, Okun M, Carandini M, Harris KD. The nature of shared cortical variability. Neuron. 2015;87:644–656. doi: 10.1016/j.neuron.2015.06.035.
- Lorteije JAM, Zylberberg A, Ouellette BG, De Zeeuw CI, Sigman M, Roelfsema PR. The formation of hierarchical decisions in the visual cortex. Neuron. 2015;87:1344–1356. doi: 10.1016/j.neuron.2015.08.015.
- Luo TZ, Maunsell JHR. Neuronal modulations in visual cortex are associated with only one of multiple components of attention. Neuron. 2015;86:1182–1188. doi: 10.1016/j.neuron.2015.05.007.
- Mayo JP, Maunsell JHR. Graded neuronal modulations related to visual spatial attention. The Journal of Neuroscience. 2016;36:5353–5361. doi: 10.1523/JNEUROSCI.0192-16.2016.
- Mitchell JF, Sundberg KA, Reynolds JH. Spatial attention decorrelates intrinsic activity fluctuations in macaque area V4. Neuron. 2009;63:879–888. doi: 10.1016/j.neuron.2009.09.013.
- Moreno-Bote R, Beck J, Kanitscheider I, Pitkow X, Latham P, Pouget A. Information-limiting correlations. Nature Neuroscience. 2014;17:1410–1417. doi: 10.1038/nn.3807.
- Nandy AS, Nassi JJ, Reynolds JH. Laminar organization of attentional modulation in macaque visual area V4. Neuron. 2017;93:235–246. doi: 10.1016/j.neuron.2016.11.029.
- Ni AM, Ruff DA, Alberts JJ, Symmonds J, Cohen MR. Learning and attention reveal a general relationship between population activity and behavior. Science (New York, N.Y.) 2018;359:463–465. doi: 10.1126/science.aao0284.
- Ni AM, Huang C, Doiron B, Cohen MR. GeneralDecoder. swh:1:rev:5056b409f2d943736b0478ff7ff38dd247b468b5Software Heritage. 2022
- Nirenberg S, Latham PE. Decoding neuronal spike trains: how important are correlations? PNAS. 2003;100:7348–7353. doi: 10.1073/pnas.1131895100.
- Panzeri S, Harvey CD, Piasini E, Latham PE, Fellin T. Cracking the neural code for sensory perception by combining statistics, intervention, and behavior. Neuron. 2017;93:491–507. doi: 10.1016/j.neuron.2016.12.036.
- Pelli DG. The VideoToolbox software for visual psychophysics: transforming numbers into movies. Spatial Vision. 1997;10:437–442.
- Pitkow X, Liu S, Angelaki DE, DeAngelis GC, Pouget A. How can single sensory neurons predict behavior? Neuron. 2015;87:411–423. doi: 10.1016/j.neuron.2015.06.033.
- Posner MI. Orienting of attention. The Quarterly Journal of Experimental Psychology. 1980;32:3–25. doi: 10.1080/00335558008248231.
- Rabinowitz NC, Goris RL, Cohen M, Simoncelli EP. Attention stabilizes the shared gain of V4 populations. eLife. 2015;4:e08998. doi: 10.7554/eLife.08998.
- Ruff DA, Cohen MR. Attention can either increase or decrease spike count correlations in visual cortex. Nature Neuroscience. 2014a;17:1591–1597. doi: 10.1038/nn.3835.
- Ruff DA, Cohen MR. Global cognitive factors modulate correlated response variability between V4 neurons. The Journal of Neuroscience. 2014b;34:16408–16416. doi: 10.1523/JNEUROSCI.2750-14.2014.
- Ruff DA, Cohen MR. Stimulus dependence of correlated variability across cortical areas. The Journal of Neuroscience. 2016;36:7546–7556. doi: 10.1523/JNEUROSCI.0504-16.2016.
- Ruff DA, Ni AM, Cohen MR. Cognition as a window into neuronal population space. Annual Review of Neuroscience. 2018;41:77–97. doi: 10.1146/annurev-neuro-080317-061936.
- Ruff DA, Cohen MR. Simultaneous multi-area recordings suggest that attention improves performance by reshaping stimulus representations. Nature Neuroscience. 2019;22:1669–1676. doi: 10.1038/s41593-019-0477-1.
- Rumyantsev OI, Lecoq JA, Hernandez O, Zhang Y, Savall J, Chrapkiewicz R, Li J, Zeng H, Ganguli S, Schnitzer MJ. Fundamental bounds on the fidelity of sensory cortical coding. Nature. 2020;580:100–105. doi: 10.1038/s41586-020-2130-2.
- Seitz A, Watanabe T. A unified model for perceptual learning. Trends in Cognitive Sciences. 2005;9:329–334. doi: 10.1016/j.tics.2005.05.010.
- Semedo JD, Zandvakili A, Machens CK, Yu BM, Kohn A. Cortical areas interact through a communication subspace. Neuron. 2019;102:249–259. doi: 10.1016/j.neuron.2019.01.026.
- Shadlen MN, Britten KH, Newsome WT, Movshon JA. A computational analysis of the relationship between neuronal and behavioral responses to visual motion. The Journal of Neuroscience. 1996;16:1486–1510.
- Shamir M, Sompolinsky H. Implications of neuronal diversity on population coding. Neural Computation. 2006;18:1951–1986. doi: 10.1162/neco.2006.18.8.1951.
- Verhoef BE, Maunsell JHR. Attention-related changes in correlated neuronal activity arise from normalization mechanisms. Nature Neuroscience. 2017;20:969–977. doi: 10.1038/nn.4572.
- Williamson RC, Cowley BR, Litwin-Kumar A, Doiron B, Kohn A, Smith MA, Yu BM. Scaling properties of dimensionality reduction for neural populations and network models. PLOS Computational Biology. 2016;12:e1005141. doi: 10.1371/journal.pcbi.1005141.
- Yan Y, Rasch MJ, Chen M, Xiang X, Huang M, Wu S, Li W. Perceptual training continuously refines neuronal population codes in primary visual cortex. Nature Neuroscience. 2014;17:1380–1387. doi: 10.1038/nn.3805.
- Zénon A, Krauzlis RJ. Attention deficits without cortical neuronal deficits. Nature. 2012;489:434–437. doi: 10.1038/nature11497.
Source: PubMed