Characterizing the effects of feature salience and top-down attention in the early visual system

Sonia Poltoratski, Sam Ling, Devin McCormack, Frank Tong, Sonia Poltoratski, Sam Ling, Devin McCormack, Frank Tong

Abstract

The visual system employs a sophisticated balance of attentional mechanisms: salient stimuli are prioritized for visual processing, yet observers can also ignore such stimuli when their goals require directing attention elsewhere. A powerful determinant of visual salience is local feature contrast: if a local region differs from its immediate surround along one or more feature dimensions, it will appear more salient. We used high-resolution functional MRI (fMRI) at 7T to characterize the modulatory effects of bottom-up salience and top-down voluntary attention within multiple sites along the early visual pathway, including visual areas V1-V4 and the lateral geniculate nucleus (LGN). Observers viewed arrays of spatially distributed gratings, where one of the gratings immediately to the left or right of fixation differed from all other items in orientation or motion direction, making it salient. To investigate the effects of directed attention, observers were cued to attend to the grating to the left or right of fixation, which was either salient or nonsalient. Results revealed reliable additive effects of top-down attention and stimulus-driven salience throughout visual areas V1-hV4. In comparison, the LGN exhibited significant attentional enhancement but was not reliably modulated by orientation- or motion-defined salience. Our findings indicate that top-down effects of spatial attention can influence visual processing at the earliest possible site along the visual pathway, including the LGN, whereas the processing of orientation- and motion-driven salience primarily involves feature-selective interactions that take place in early cortical visual areas.NEW & NOTEWORTHY While spatial attention allows for specific, goal-driven enhancement of stimuli, salient items outside of the current focus of attention must also be prioritized. We used 7T fMRI to compare salience and spatial attentional enhancement along the early visual hierarchy. We report additive effects of attention and bottom-up salience in early visual areas, suggesting that salience enhancement is not contingent on the observer's attentional state.

Keywords: fMRI; lateral geniculate nucleus; primary visual cortex; salience; visual attention.

Copyright © 2017 the American Physiological Society.

Figures

Fig. 1.
Fig. 1.
Annotated sample displays for both experiments. Feature-contrast salience is defined by orientation in A and by drifting motion direction in B (indicated by arrows, which were not present during the experiment). Dashed circles (likewise not present during the experiments) indicate the 2 locations at which the salient patch could appear and to which spatial attention could be directed. Although the task-relevant contrast decrement occurred at both of these locations, the observer was instructed to only perform the detection task on 1 patch, as indicated throughout each block by a small cue. In these examples, if the participant is cued to attend to the black cue (as labeled), in A, the left patch is attended/salient, whereas the right patch is unattended/nonsalient; in B, the left patch is attended/salient, and the right patch is unattended/salient. Gabor patch edges were Gaussian blurred and also spatially separated by a gap of 0.8°, yielding an effective spatial separation of ~1.8°. dva, Degrees of visual angle.
Fig. 2.
Fig. 2.
Results of experiment 1 for the 4 salience and attention conditions. A: magnitude of the attention and salience effects across ROIs, computed as the difference in percent signal change. Circles show the effect in individual subjects, and error bars depict SE (across subjects). In the LGN, only attention significantly modulated BOLD responses; neither the main effect of salience nor the interaction effect were significant. A significant effect of salience first emerged in V1 and is evident in each cortical ROI. Attention also modulated BOLD responses in V1 through hV4, but the 2 effects did not reliably interact in any region. B, top: mean ROI time courses time-locked to the beginning of each experimental block, which lasted 8 TRs and is demarcated by dotted lines. Bottom, the same data averaged across the block, offset to account for hemodynamic lag, and normalized by subtracting the mean response of the 2 TRs immediately preceding the block. Error bars on block averages depict SE in the mean BOLD response across subjects.
Fig. 3.
Fig. 3.
Examples of average fMRI time courses from representative subjects in experiment 1 (A and B) and experiment 2 (C and D). Error bars depict ±SE for each experimental condition, and each of the 4 panels shows data from a different individual. As expected, there is some variability between subjects; fMRI responses in the LGN are also more variable than those in early visual areas, partly because of the LGN’s smaller size and the presence of greater physiological noise in midbrain structures.
Fig. 4.
Fig. 4.
Magnitude of attention and salience effects plotted as a function of ROI size. We calculated the difference in BOLD response for attended minus unattended conditions and salient minus nonsalient conditions for a wide range of ROI sizes, ranging from 2 to 60 maximum voxels per hemisphere in the LGN and from 10 to 350 voxels per hemisphere in individual cortical visual areas. Error bars indicate SE across subjects. Our findings of positive salience and attention effects in each cortical ROI are highly consistent across a wide range of ROI sizes.
Fig. 5.
Fig. 5.
Mean BOLD amplitudes in experiment 2, in which salience was defined by the direction of drifting motion of Gabor patches. A: attention and salience effects (% signal change difference) across ROIs, with circles plotting individual subjects’ results and error bars showing SE across subjects. B: results in each condition averaged across the stimulus block and normalized by a prestimulus window for each condition. The pattern of results follows that of experiment 1: attentional enhancement is evident in each ROI, including the LGN, whereas salience modulated only cortical mean BOLD responses. In every region studied, the effects of attention and salience did not significantly interact.
Fig. 6.
Fig. 6.
Individual subject ANOVA results for experiment 1 (Expt 1; left) and experiment 2 (Expt 2; right). Each experimental session was analyzed independently, with experimental run as the repeated measure. Filled circles indicate significant effects of attention (black), salience (gray), or their interaction (cross) at P < 0.05 level. Subject labels (S1–S6) are arbitrary and unmatched between the 2 experiments. Although many subjects exhibited significant main effects of attention and salience, the interaction of these 2 factors was not significant in any ROI in any subject.

References

    1. Adesnik H, Bruns W, Taniguchi H, Huang ZJ, Scanziani M. A neural circuit for spatial summation in visual cortex. Nature 490: 226–231, 2012. doi:10.1038/nature11526.
    1. Alitto HJ, Usrey WM. Origin and dynamics of extraclassical suppression in the lateral geniculate nucleus of the macaque monkey. Neuron 57: 135–146, 2008. doi:10.1016/j.neuron.2007.11.019.
    1. Allman J, Miezin F, McGuinness E. Stimulus specific responses from beyond the classical receptive field: neurophysiological mechanisms for local-global comparisons in visual neurons. Annu Rev Neurosci 8: 407–430, 1985. doi:10.1146/annurev.ne.08.030185.002203.
    1. Angelucci A, Bressloff P. Contribution of feedforward, lateral and feedback connections to the classical receptive field center and extra-classical receptive field surround of primate V1 neurons. Prog Brain Res 154: 93–120, 2006. doi:10.1016/S0079-6123(06)54005-1.
    1. Bair W, Cavanaugh JR, Movshon JA. Time course and time-distance relationships for surround suppression in macaque V1 neurons. J Neurosci 23: 7690–7701, 2003.
    1. Beck D, Kastner S. Stimulus context modulates competition in human extrastriate cortex. Nat Neurosci 8: 1110–1116, 2005. doi:10.1038/nn1501.
    1. Blakemore C, Tobin EA. Lateral inhibition between orientation detectors in the cat’s visual cortex. Exp Brain Res 15: 439–440, 1972. doi:10.1007/BF00234129.
    1. Bogler C, Bode S, Haynes JD. Decoding successive computational stages of saliency processing. Curr Biol 21: 1667–1671, 2011. doi:10.1016/j.cub.2011.08.039.
    1. Bogler C, Bode S, Haynes JD. Orientation pop-out processing in human visual cortex. Neuroimage 81: 73–80, 2013. doi:10.1016/j.neuroimage.2013.05.040.
    1. Brainard DH. The Psychophysics Toolbox. Spat Vis 10: 433–436, 1997. doi:10.1163/156856897X00357.
    1. Brefczynski JA, DeYoe EA. A physiological correlate of the ‘spotlight’ of visual attention. Nat Neurosci 2: 370–374, 1999. doi:10.1038/7280.
    1. Cannon MW, Fullenkamp SC. Spatial interactions in apparent contrast: inhibitory effects among grating patterns of different spatial frequencies, spatial positions and orientations. Vision Res 31: 1985–1998, 1991. doi:10.1016/0042-6989(91)90193-9.
    1. Carandini M, Heeger DJ. Normalization as a canonical neural computation. Nat Rev Neurosci 13: 51–62, 2011.
    1. Carrasco M. Covert attention increases contrast sensitivity: psychophysical, neurophysiological and neuroimaging studies. In: Visual Perception: Fundamentals of Vision: Low and Mid-Level Processes in Perception (Progress in Brain Research), edited by Martinez-Conde S, Macknik SL, Martinez LM, Alonso JM, and Tse PU. Amsterdam: Elsevier Science, 2006, vol. 154A, p. 33–70.
    1. Cavanaugh JR, Bair W, Movshon JA. Nature and interaction of signals from the receptive field center and surround in macaque V1 neurons. J Neurophysiol 88: 2530–2546, 2002a. doi:10.1152/jn.00692.2001.
    1. Cavanaugh JR, Bair W, Movshon JA. Selectivity and spatial distribution of signals from the receptive field surround in macaque V1 neurons. J Neurophysiol 88: 2547–2556, 2002b. doi:10.1152/jn.00693.2001.
    1. Cheong SK, Tailby C, Solomon SG, Martin PR. Cortical-like receptive fields in the lateral geniculate nucleus of marmoset monkeys. J Neurosci 33: 6864–6876, 2013. doi:10.1523/JNEUROSCI.5208-12.2013.
    1. Coen-Cagli R, Dayan P, Schwartz O. Cortical surround interactions and perceptual salience via natural scene statistics. PLOS Comput Biol 8: e1002405, 2012. doi:10.1371/journal.pcbi.1002405.
    1. Coen-Cagli R, Kohn A, Schwartz O. Flexible gating of contextual influences in natural vision. Nat Neurosci 18: 1648–1655, 2015. doi:10.1038/nn.4128.
    1. Corbetta M, Shulman GL. Control of goal-directed and stimulus-driven attention in the brain. Nat Rev Neurosci 3: 201–215, 2002. doi:10.1038/nrn755.
    1. Cudeiro J, Sillito AM. Spatial frequency tuning of orientation-discontinuity-sensitive corticofugal feedback to the cat lateral geniculate nucleus. J Physiol 490: 481–492, 1996. doi:10.1113/jphysiol.1996.sp021159.
    1. DeAngelis GC, Freeman RD, Ohzawa I. Length and width tuning of neurons in the cat’s primary visual cortex. J Neurophysiol 71: 347–374, 1994.
    1. Dragoi V, Rivadulla C, Sur M. Foci of orientation plasticity in visual cortex. Nature 411: 80–86, 2001. doi:10.1038/35075070.
    1. Engel SA, Glover GH, Wandell BA. Retinotopic organization in human visual cortex and the spatial precision of functional MRI. Cereb Cortex 7: 181–192, 1997. doi:10.1093/cercor/7.2.181.
    1. Fecteau JH, Munoz DP. Salience, relevance, and firing: a priority map for target selection. Trends Cogn Sci 10: 382–390, 2006. doi:10.1016/j.tics.2006.06.011.
    1. Flevaris AV, Murray SO. Attention determines contextual enhancement versus suppression in human primary visual cortex. J Neurosci 35: 12273–12280, 2015. doi:10.1523/JNEUROSCI.1409-15.2015.
    1. Gandhi SP, Heeger DJ, Boynton GM. Spatial attention affects brain activity in human primary visual cortex. Proc Natl Acad Sci USA 96: 3314–3319, 1999. doi:10.1073/pnas.96.6.3314.
    1. Gilbert CD, Wiesel TN. Morphology and intracortical projections of functionally characterised neurones in the cat visual cortex. Nature 280: 120–125, 1979. doi:10.1038/280120a0.
    1. Gottlieb JP, Kusunoki M, Goldberg ME. The representation of visual salience in monkey parietal cortex. Nature 391: 481–484, 1998. doi:10.1038/35135.
    1. Harrison LM, Stephan KE, Rees G, Friston KJ. Extra-classical receptive field effects measured in striate cortex with fMRI. Neuroimage 34: 1199–1208, 2007. doi:10.1016/j.neuroimage.2006.10.017.
    1. Hastings C Jr, Mosteller F, Tukey JW, Winsor CP. Low moments for small samples: a comparative study of order statistics. Ann Math Stat 18: 413–426, 1947. doi:10.1214/aoms/1177730388.
    1. Henderson JM. Human gaze control during real-world scene perception. Trends Cogn Sci 7: 498–504, 2003. doi:10.1016/j.tics.2003.09.006.
    1. Honrubia V, Downey WL, Mitchell DP, Ward PH. Experimental studies on optokinetic nystagmus. II. Normal humans. Acta Otolaryngol 65: 441–448, 1968. doi:10.3109/00016486809120986.
    1. Hopf J-M, Noesselt T, Tempelmann C, Braun J, Schoenfeld MA, Heinze H-J. Popout modulates focal attention in the primary visual cortex. Neuroimage 22: 574–582, 2004. doi:10.1016/j.neuroimage.2004.01.031.
    1. Itti L, Koch C. Computational modelling of visual attention. Nat Rev Neurosci 2: 194–203, 2001a. doi:10.1038/35058500.
    1. Jeffreys SH. The Theory of Probability (3rd ed.). Oxford: Oxford University Press, 1961.
    1. Jehee JF, Brady DK, Tong F. Attention improves encoding of task-relevant features in the human visual cortex. J Neurosci 31: 8210–8219, 2011. doi:10.1523/JNEUROSCI.6153-09.2011.
    1. Jehee JF, Ling S, Swisher JD, van Bergen RS, Tong F. Perceptual learning selectively refines orientation representations in early visual cortex. J Neurosci 32: 16747–16753, 2012. doi:10.1523/JNEUROSCI.6112-11.2012.
    1. Jin DZ, Dragoi V, Sur M, Seung HS. Tilt aftereffect and adaptation-induced changes in orientation tuning in visual cortex. J Neurophysiol 94: 4038–4050, 2005. doi:10.1152/jn.00571.2004.
    1. Jones HE, Andolina IM, Ahmed B, Shipp SD, Clements JTC, Grieve KL, Cudeiro J, Salt TE, Sillito AM. Differential feedback modulation of center and surround mechanisms in parvocellular cells in the visual thalamus. J Neurosci 32: 15946–15951, 2012. doi:10.1523/JNEUROSCI.0831-12.2012.
    1. Jones HE, Andolina IM, Oakely NM, Murphy PC, Sillito AM. Spatial summation in lateral geniculate nucleus and visual cortex. Exp Brain Res 135: 279–284, 2000. doi:10.1007/s002210000574.
    1. Jones HE, Grieve KL, Wang W, Sillito AM. Surround suppression in primate V1. J Neurophysiol 86: 2011–2028, 2001.
    1. Joo SJ, Boynton GM, Murray SO. Long-range, pattern-dependent contextual effects in early human visual cortex. Curr Biol 22: 781–786, 2012. doi:10.1016/j.cub.2012.02.067.
    1. Kapadia MK, Ito M, Gilbert CD, Westheimer G. Improvement in visual sensitivity by changes in local context: parallel studies in human observers and in V1 of alert monkeys. Neuron 15: 843–856, 1995. doi:10.1016/0896-6273(95)90175-2.
    1. Kastner S, O’Connor DH, Fukui MM, Fehd HM, Herwig U, Pinsk MA. Functional imaging of the human lateral geniculate nucleus and pulvinar. J Neurophysiol 91: 438–448, 2004. doi:10.1152/jn.00553.2003.
    1. Kastner S, Ungerleider LG. Mechanisms of visual attention in the human cortex. Annu Rev Neurosci 23: 315–341, 2000. doi:10.1146/annurev.neuro.23.1.315.
    1. Kayser C, Körding KP, König P. Processing of complex stimuli and natural scenes in the visual cortex. Curr Opin Neurobiol 14: 468–473, 2004. doi:10.1016/j.conb.2004.06.002.
    1. Kincade JM, Abrams RA, Astafiev SV, Shulman GL, Corbetta M. An event-related functional magnetic resonance imaging study of voluntary and stimulus-driven orienting of attention. J Neurosci 25: 4593–4604, 2005. doi:10.1523/JNEUROSCI.0236-05.2005.
    1. Koene AR, Zhaoping L. Feature-specific interactions in salience from combined feature contrasts: evidence for a bottom-up saliency map in V1. J Vis 7: 6.1–6.14, 2007. doi:10.1167/7.7.6.
    1. Lamme VA. The neurophysiology of figure-ground segregation in primary visual cortex. J Neurosci 15: 1605–1615, 1995.
    1. Leventhal AG, Schall JD. Structural basis of orientation sensitivity of cat retinal ganglion cells. J Comp Neurol 220: 465–475, 1983. doi:10.1002/cne.902200408.
    1. Li CY, Li W. Extensive integration field beyond the classical receptive field of cat’s striate cortical neurons—classification and tuning properties. Vision Res 34: 2337–2355, 1994. doi:10.1016/0042-6989(94)90280-1.
    1. Li Z. A saliency map in primary visual cortex. Trends Cogn Sci 6: 9–16, 2002. doi:10.1016/S1364-6613(00)01817-9.
    1. Ling S, Pratte MS, Tong F. Attention alters orientation processing in the human lateral geniculate nucleus. Nat Neurosci 18: 496–498, 2015. doi:10.1038/nn.3967.
    1. Mazer JA, Gallant JL. Goal-related activity in V4 during free viewing visual search. Evidence for a ventral stream visual salience map. Neuron 40: 1241–1250, 2003. doi:10.1016/S0896-6273(03)00764-5.
    1. McAlonan K, Cavanaugh J, Wurtz RH. Guarding the gateway to cortex with attention in visual thalamus. Nature 456: 391–394, 2008. doi:10.1038/nature07382.
    1. McDonald JS, Seymour KJ, Schira MM, Spehar B, Clifford CW. Orientation-specific contextual modulation of the fMRI BOLD response to luminance and chromatic gratings in human visual cortex. Vision Res 49: 1397–1405, 2009. doi:10.1016/j.visres.2008.12.014.
    1. Melloni L, van Leeuwen S, Alink A, Müller NG. Interaction between bottom-up saliency and top-down control: how saliency maps are created in the human brain. Cereb Cortex 22: 2943–2952, 2012. doi:10.1093/cercor/bhr384.
    1. Naito T, Sadakane O, Okamoto M, Sato H. Orientation tuning of surround suppression in lateral geniculate nucleus and primary visual cortex of cat. Neuroscience 149: 962–975, 2007. doi:10.1016/j.neuroscience.2007.08.001.
    1. Nassi JJ, Lomber SG, Born RT. Corticocortical feedback contributes to surround suppression in V1 of the alert primate. J Neurosci 33: 8504–8517, 2013. doi:10.1523/JNEUROSCI.5124-12.2013.
    1. Nelson JI, Frost BJ. Orientation-selective inhibition from beyond the classic visual receptive field. Brain Res 139: 359–365, 1978. doi:10.1016/0006-8993(78)90937-X.
    1. Nothdurft HC. The role of features in preattentive vision: comparison of orientation, motion and color cues. Vision Res 33: 1937–1958, 1993. doi:10.1016/0042-6989(93)90020-W.
    1. Nurminen L, Angelucci A. Multiple components of surround modulation in primary visual cortex: multiple neural circuits with multiple functions? Vision Res 104: 47–56, 2014. doi:10.1016/j.visres.2014.08.018.
    1. O’Connor DH, Fukui MM, Pinsk MA, Kastner S. Attention modulates responses in the human lateral geniculate nucleus. Nat Neurosci 5: 1203–1209, 2002. doi:10.1038/nn957.
    1. Pelli DG. The VideoToolbox software for visual psychophysics: transforming numbers into movies. Spat Vis 10: 437–442, 1997. doi:10.1163/156856897X00366.
    1. Petrov Y, McKee SP. The effect of spatial configuration on surround suppression of contrast sensitivity. J Vis 6: 224–238, 2006. doi:10.1167/6.3.4.
    1. Rouder JN, Speckman PL, Sun D, Morey RD, Iverson G. Bayesian t tests for accepting and rejecting the null hypothesis. Psychon Bull Rev 16: 225–237, 2009. doi:10.3758/PBR.16.2.225.
    1. Saenz M, Buracas GT, Boynton GM. Global effects of feature-based attention in human visual cortex. Nat Neurosci 5: 631–632, 2002. doi:10.1038/nn876.
    1. Schallmo MP, Grant AN, Burton PC, Olman CA. The effects of orientation and attention during surround suppression of small image features: a 7 Tesla fMRI study. J Vis 16: 19, 2016. doi:10.1167/16.10.19.
    1. Schneider KA, Kastner S. Effects of sustained spatial attention in the human lateral geniculate nucleus and superior colliculus. J Neurosci 29: 1784–1795, 2009. doi:10.1523/JNEUROSCI.4452-08.2009.
    1. Shushruth S, Ichida JM, Levitt JB, Angelucci A. Comparison of spatial summation properties of neurons in macaque V1 and V2. J Neurophysiol 102: 2069–2083, 2009. doi:10.1152/jn.00512.2009.
    1. Shushruth S, Nurminen L, Bijanzadeh M, Ichida JM, Vanni S, Angelucci A. Different orientation tuning of near- and far-surround suppression in macaque primary visual cortex mirrors their tuning in human perception. J Neurosci 33: 106–119, 2013. doi:10.1523/JNEUROSCI.2518-12.2013.
    1. Silver MA, Kastner S. Topographic maps in human frontal and parietal cortex. Trends Cogn Sci 13: 488–495, 2009. doi:10.1016/j.tics.2009.08.005.
    1. Smith EL 3rd, Chino YM, Ridder WH 3rd, Kitagawa K, Langston A. Orientation bias of neurons in the lateral geniculate nucleus of macaque monkeys. Vis Neurosci 5: 525–545, 1990. doi:10.1017/S0952523800000699.
    1. Squire RF, Noudoost B, Schafer RJ, Moore T. Prefrontal contributions to visual selective attention. Annu Rev Neurosci 36: 451–466, 2013. doi:10.1146/annurev-neuro-062111-150439.
    1. Stettler DD, Das A, Bennett J, Gilbert CD. Lateral connectivity and contextual interactions in macaque primary visual cortex. Neuron 36: 739–750, 2002. doi:10.1016/S0896-6273(02)01029-2.
    1. Suematsu N, Naito T, Sato H. Relationship between orientation sensitivity and spatiotemporal receptive field structures of neurons in the cat lateral geniculate nucleus. Neural Netw 35: 10–20, 2012. doi:10.1016/j.neunet.2012.06.008.
    1. Vinje WE, Gallant JL. Sparse coding and decorrelation in primary visual cortex during natural vision. Science 287: 1273–1276, 2000. doi:10.1126/science.287.5456.1273.
    1. Wandell BA, Dumoulin SO, Brewer AA. Visual field maps in human cortex. Neuron 56: 366–383, 2007. doi:10.1016/j.neuron.2007.10.012.
    1. Xu X, Ichida J, Shostak Y, Bonds AB, Casagrande VA. Are primate lateral geniculate nucleus (LGN) cells really sensitive to orientation or direction? Vis Neurosci 19: 97–108, 2002. doi:10.1017/S0952523802191097.
    1. Zenger-Landolt B, Heeger DJ. Response suppression in v1 agrees with psychophysics of surround masking. J Neurosci 23: 6884–6893, 2003.
    1. Zhang X, Zhaoping L, Zhou T, Fang F. Neural activities in v1 create a bottom-up saliency map. Neuron 73: 183–192, 2012. doi:10.1016/j.neuron.2011.10.035.
    1. Zipser K, Lamme VA, Schiller PH. Contextual modulation in primary visual cortex. J Neurosci 16: 7376–7389, 1996.

Source: PubMed

Sponsor e collaboratori

Condizioni mediche

Interventi farmacologici

3
Sottoscrivi