Neural mechanisms of intermodal sustained selective attention with concurrently presented auditory and visual stimuli

Katja Saupe, Erich Schröger, Søren K Andersen, Matthias M Müller, Katja Saupe, Erich Schröger, Søren K Andersen, Matthias M Müller

Abstract

We investigated intermodal attention effects on the auditory steady-state response (ASSR) and the steady-state visual evoked potential (SSVEP). For this purpose, 40-Hz amplitude-modulated tones and a stream of flickering (7.5 Hz) random letters were presented concurrently. By means of an auditory or visual target detection task, participants' attention was directed to the respective modality for several seconds. Attention to the auditory stream led to a significant enhancement of the ASSR compared to when the visual stream was attended. This attentional modulation was located mainly in the right superior temporal gyrus. Vice versa, attention to the visual stream especially increased the second harmonic response of the SSVEP. This modulation was focused in the inferior occipital and lateral occipitotemporal gyrus of both hemispheres. To the best of our knowledge, this is the first demonstration of amplitude modulation of the ASSR and the SSVEP by intermodal sustained attention. Our results open a new avenue of research to understand the basic neural mechanisms of intermodal attention in the human brain.

Keywords: ASSR; SSVEP; human EEG; intermodal; supramodal; sustained attention.

Figures

Figure 1
Figure 1
Schematic illustration of one trial. (A) 40-Hz amplitude-modulated auditory stimulus with three targets included. Targets (30-Hz amplitude-modulated sequences with a duration of 200 ms) are indicated by a grey box. (B) Visual letter stream with a presentation rate of 7.5 Hz with two targets included, the letter ‘H’ (grey border). Each trial started with the presentation of the fixation cross for a randomized duration of 500–1000 ms. Auditory and visual stimulation started simultaneously and lasted for 3450 ms and 3467 ms, respectively. After the end of the visual letter stream a question mark appeared for 1500 ms before the next trial started.
Figure 2
Figure 2
Schematic illustration of the ASSR and SSVEP waveforms at the respective electrode positions. 40-Hz ASSR (black lines), 7.5-Hz SSVEP and 15-Hz SSVEP (grey lines) waveforms for one representative subject, extracted by moving window averages when one modality was either attended (solid line) or ignored (dotted line) averaged across the respective electrode clusters. Analyzed electrode clusters are indicated by the black (for ASSR) and grey (for SSVEP) circles.
Figure 3
Figure 3
Topographical distribution of the 40-Hz ASSR, the 7. 5-Hz SSVEP and the 15-Hz SSVEP (second harmonic response) averaged across all subjects. (A) Mean across conditions when one modality was either attended or ignored. (B) The difference (attend minus ignore) between conditions. Note different scales.
Figure 4
Figure 4
Signal amplitude of the 40-Hz ASSR averaged across all subjects. Amplitude was extracted for the conditions, when the auditory stream was either attended (solid line) or ignored (dotted line) by Fourier-Transformation, averaged across two individual adjacent electrodes, chosen from frontocentral channels exhibiting the highest 40-Hz amplitude in the mean across conditions.
Figure 5
Figure 5
Signal amplitude of the 7. 5-Hz SSVEP and its second and third harmonics (15 Hz and 22.5 Hz) averaged across all subjects. Amplitude was extracted for the conditions, when the visual stream was either attended (solid line) or ignored (dotted line) by Fourier-Transformation, averaged across the electrodes demonstrating the highest amplitude in the mean across conditions [O1, O2, PO7 and PO8, (A)] and the electrode position demonstrating the highest (but only marginal significant) difference between attentional conditions at 7.5 Hz [POz, (B)].
Figure 6
Figure 6
Statistical parametric maps of significant voxels of the inverse solution of the 40-Hz ASSR across all subjects for (A) the conditions when one modality was either attended or ignored and the difference values between conditions (attend minus ignore), and (B) for voxels, in which the deviation between conditions differed significantly between the right and left hemisphere. Scales represent t2-values (Hotelling, 1931). A significance threshold of α = 0.001, corrected for multiple comparisons, was applied. Note different scales.
Figure 7
Figure 7
Statistical parametric maps of significant voxels of the inverse solution of the 7. 5-Hz SSVEP (A) and the 15-Hz SSVEP [second harmonic, (B)] across all subjects for the conditions when one modality was either attended or ignored, and the difference values between conditions (attend minus ignore) for the second harmonic of the 7.5-Hz SSVEP. Scales represent t2-values (Hotelling, 1931). A significance threshold of α = 0.001, corrected for multiple comparisons, was applied. Note different scales.
Figure 8
Figure 8
Areas of overlapping attentional modulation between 40-Hz ASSR and 15-Hz SSVEP (second harmonic). Yellow marks represent voxels significantly different between conditions when one modality was attended or ignored for both, the ASSR and the 15-Hz SSVEP. A significance threshold of α = 0.0005, corrected for multiple comparisons, was applied.

References

    1. Alho K., Teder W., Lavikainen J., Näätänen R. (1994a). Strongly focused attention and auditory event-related potentials. Biol. Psychol. 38, 73.10.1016/0301-0511(94)90050-7
    1. Alho K., Woods D. L., Algazi A. (1994b). Processing of auditory stimuli during auditory and visual attention as revealed by event-related potentials. Psychophysiology 31, 469–47910.1111/j.1469-8986.1994.tb01050.x
    1. Alho K., Woods D. L., Algazi A., Näätänen R. (1992). Intermodal selective attention. II. Effects of attentional load on processing of auditory and visual stimuli in central space. Electroencephalogr. Clin. Neurophysiol. 82, 356.10.1016/0013-4694(92)90005-3
    1. Andersen S. K., Hillyard S. A., Muller M. M. (2008). Attention facilitates multiple stimulus features in parallel in human visual cortex. Curr. Biol. 18, 1006–100910.1016/j.cub.2008.06.030
    1. Arnott S. R., Alain C. (2002). Effects of perceptual context on event-related brain potentials during auditory spatial attention. Psychophysiology 39, 625–63210.1017/S0048577202394149
    1. Benton A. (1986). Reaction-time in brain disease – some reflections. Cortex 22, 129–140
    1. Bidet-Caulet A., Fischer C., Besle J., Aguera P. E., Giard M. H., Bertrand O. (2007). Effects of selective attention on the electrophysiological representation of concurrent sounds in the human auditory cortex. J. Neurosci. 27, 9252–926110.1523/JNEUROSCI.1402-07.2007
    1. Bisiach E., Vallar G. (1988). Hemineglect in humans. In Handbook of Neuropsychology. Boller F., Grafman J., eds (Elesevier, Amsterdam: ), pp. 195–222
    1. Bosch-Bayard J., Valdes-Sosa P., Virues-Alba T., Aubert-Vazquez E., John E. R., Harmony T., Riera-Diaz J., Trujillo-Barreto N. (2001). 3D statistical parametric mapping of EEG source spectra by means of Variable Resolution Electromagnetic Tomography (VARETA). Clin. Electroencephalogr. 32, 47–61
    1. Chatrian G. E., Lettich E., Nelson P. L. (1985). Ten percent electrode system for topographic studies of spontaneous and evoked EEG activities. Am. J. EEG. Technol. 25, 83–92
    1. Coull J. T., Nobre A. C. (1998). Where and when to pay attention: The neural systems for directing attention to spatial locations and to time intervals as revealed by both PET and fMRI. J. Neurosci. 18, 7426–7435
    1. Csibra G., Davis G., Spratling M. W., Johnson M. H. (2000). Gamma oscillations and object processing in the infant brain. Science 290, 1582–158510.1126/science.290.5496.1582
    1. de Ruiter M. B., Kok A., van der Schoot M. (1998). Effects of inter- and intramodal selective attention to non-spatial visual stimuli: an event-related potential analysis. Biol. Psychol. 49, 269.10.1016/S0301-0511(98)00046-5
    1. Delorme A., Makeig S. (2004). EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. J. Neurosci. Methods 134, 9–2110.1016/j.jneumeth.2003.10.009
    1. Ding J., Sperling G., Srinivasan R. (2006). Attentional modulation of SSVEP power depends on the network tagged by the flicker frequency. Cereb. Cortex 16, 1016–102910.1093/cercor/bhj044
    1. Eimer M., Schröger E. (1998). ERP effects of intermodal attention and cross-modal links in spatial attention. Psychophysiology 35, 313–32710.1017/S004857729897086X
    1. Evans A. C., Collins D. L., Mills S. R., Brown E. D., Kelly R. L., Peters T. M. (1993). 3D statistical neuroanatomical models from 305 MRI volumes. In Proceedings of the Nuclear Science Symposium and Medical Imaging Conference, San Francisco, CA, USA: IEEE Conference Record, pp. 1813–1817
    1. Fawcett I. P., Barnes G. R., Hillebrand A., Singh K. D. (2004). The temporal frequency tuning of human visual cortex investigated using synthetic aperture magnetometry. Neuroimage 21, 1542–155310.1016/j.neuroimage.2003.10.045
    1. Fuchs S., Andersen S. K., Gruber T., Müller M. M. (2008). Attentional bias of competitive interactions in neuronal networks of early visual processing in the human brain. Neuroimage 41, 1086.10.1016/j.neuroimage.2008.02.040
    1. Giabbiconi C. M., Trujillo-Barreto N. J., Gruber T., Müller M. M. (2007). Sustained spatial attention to vibration is mediated in primary somatosensory cortex. Neuroimage 35, 255–26210.1016/j.neuroimage.2006.11.022
    1. Griskova I., Morup M., Parnas J., Ruksenas O., Arnfred S. M. (2007). The amplitude and phase precision of 40-Hz auditory steady-state response depend on the level of arousal. Exp. Brain Res. 183, 133–13810.1007/s00221-007-1111-0
    1. Gutschalk A., Mase R., Roth R., Ille N., Rupp A., Hahnel S., Picton T. W., Scherg M. (1999). Deconvolution of 40-Hz steady-state fields reveals two overlapping source activities of the human auditory cortex. Clin. Neurophysiol. 110, 856–86810.1016/S1388-2457(99)00019-X
    1. Hansen J. C., Hillyard S. A. (1980). Endogenous brain potentials associated with selective auditory attention. Electroencephalogr. Clin. Neurophysiol. 49, 277–29010.1016/0013-4694(80)90222-9
    1. Hansen J. C., Hillyard S. A. (1983). Selective attention to multidimensional auditory-stimuli. J. Exp. Psychol. Hum. Percept. Perform. 9, 1–1910.1037/0096-1523.9.1.1
    1. Hansen J. C., Hillyard S. A. (1984). Effects of stimulation rate and attribute cuing on event-related potentials during selective auditory attention. Psychophysiology 21, 394–40510.1111/j.1469-8986.1984.tb00216.x
    1. Hari R., Hamalainen M., Joutsiniemi S. L. (1989). Neuromagnetic steady-state responses to auditory-stimuli. J. Acoust. Soc. Am. 86, 1033–103910.1121/1.398093
    1. Heilman K. M., Van Den Abell T. (1979). Right hemispheric dominance for mediating cerebral activation. Neuropsychologia 17, 315–32110.1016/0028-3932(79)90077-0
    1. Heilman K. M., Van Den Abell T. (1980). Right-hemisphere dominance for attention – mechanism underlying hemispheric asymmetries of inattention (neglect). Neurology 30, 327–330
    1. Herdman A. T., Lins O., Van Roon P., Stapells D. R., Scherg M., Picton T. W. (2002). Intracerebral sources of human auditory steady-state responses. Brain Topogr. 15, 69–8610.1023/A:1021470822922
    1. Herrmann C. S. (2001). Human EEG responses to 1–100 Hz flicker: resonance phenomena in visual cortex and their potential correlation to cognitive phenomena. Exp. Brain Res. 137, 346–35310.1007/s002210100682
    1. Heslenfeld D. (1998). Features and Attention in Vision: An Analysis of Electromagnetic Brain Responses, cited in: Talsma and Kok (2001). Amsterdam, University of Amsterdam
    1. Heslenfeld D. J., Kenemans J. L., Kok A., Molenaar P. C. M. (1997). Feature processing and attention in the human visual system: an overview. Biol. Psychol. 45, 183–21510.1016/S0301-0511(96)05228-3
    1. Hillyard S. A., Hink R. F., Schwent V. L., Picton T. W. (1973). Electrical signs of selective attention in human brain. Science 182, 177–18010.1126/science.182.4108.177
    1. Hillyard S. A., Hinrichs H., Tempelmann C., Morgan S. T., Hansen J. C., Scheich H., Heinze H. J. (1997). Combining steady-state visual evoked potentials and fMRI to localize brain activity during selective attention. Hum. Brain Mapp. 5, 287–29210.1002/(SICI)1097-0193(1997)5:4<287::AID-HBM14>;2-B
    1. Hotelling H. (1931). The generalization of Student's ratio. Ann. Math. Stat. 2, 360–37810.1214/aoms/1177732979
    1. Hötting K., Rösler F., Röder B. (2003). Crossmodal and intermodal attention modulate event-related brain potentials to tactile and auditory stimuli. Exp. Brain Res. 148, 26–3710.1007/s00221-002-1261-z
    1. Howes D., Boller F. (1975). Simple reaction-time – evidence for focal impairment from lesions of right hemisphere. Brain 98, 317–33210.1093/brain/98.2.317
    1. Jeeves M. A., Dixon N. F. (1970). Hemisphere differences in response rates to visual stimuli. Psychon. Sci. 20, 249–251
    1. Johnsrude I. S., Penhune V. B., Zatorre R. J. (2000). Functional specificity in the right human auditory cortex for perceiving pitch direction. Brain 123, 155–16310.1093/brain/123.1.155
    1. Junghöfer M., Elbert T., Tucker D. M., Rockstroh B. (2000). Statistical control of artifacts in dense array EEG/MEG studies. Psychophysiology 37, 523–53210.1017/S0048577200980624
    1. Linden R. D., Picton T. W., Hamel G., Campbell K. B. (1987). Human auditory steady-state evoked-potentials during selective attention. Electroencephalogr. Clin. Neurophysiol. 66, 145–15910.1016/0013-4694(87)90184-2
    1. Lutzenberger W., Pulvermuller F., Birbaumer N. (1994). Words and pseudowords elicit distinct patterns of 30-Hz EEG responses in humans. Neurosci. Lett. 176, 115–11810.1016/0304-3940(94)90884-2
    1. Morgan S. T., Hansen J. C., Hillyard S. A. (1996). Selective attention to stimulus location modulates the steady-state visual evoked potential. Proc. Natl. Acad. Sci. U.S.A. 93, 4770–477410.1073/pnas.93.10.4770
    1. Müller M. M., Andersen S., Trujillo N. J., Valdes-Sosa P., Malinowski P., Hillyard S. A. (2006). Feature-selective attention enhances color signals in early visual areas of the human brain. Proc. Natl. Acad. Sci. U.S.A. 103, 14250–1425410.1073/pnas.0606668103
    1. Müller M. M., Hübner R. (2002). Can the spotlight of attention be shaped like a doughnut? Evidence from steady-state visual evoked potentials. Psychol. Sci. 13, 119–124
    1. Müller M. M., Malinowski P., Gruber T., Hillyard S. A. (2003). Sustained division of the attentional spotlight. Nature 424, 309–31210.1038/nature01812
    1. Müller M. M., Picton T. W., Valdes-Sosa P., Riera J., Teder-Sälejärvi W. A., Hillyard S. A. (1998a). Effects of spatial selective attention on the steady-state visual evoked potential in the 20–28 Hz range. Brain Res. Cogn. Brain Res. 6, 249–26110.1016/S0926-6410(97)00036-0
    1. Müller M. M., Teder-Sälejärvi W., Hillyard S. A. (1998b). The time course of cortical facilitation during cued shifts of spatial attention. Nat. Neurosci. 1, 631–63410.1038/2865
    1. Müller N., Schlee W., Hartmann T., Lorenz I., Weisz N. (2009). Top-down modulation of the auditory steady-state response in a task-switch paradigm. Front. Hum. Neurosci. 3, 1–910.3389/neuro.09.001.2009
    1. Näätänen R. (1992). Attention and Brain function. Hillsdale, NJ, Lawrence Erlbaum Associates
    1. Oostenveld R., Praamstra P. (2001). The five percent electrode system for high-resolution EEG and ERP measurements. Clin. Neurophysiol. 112, 713–71910.1016/S1388-2457(00)00527-7
    1. Pantev C., Elbert T., Makeig S., Hampson S., Eulitz C., Hoke M. (1993). Relationship of transient and steady-state auditory evoked fields. Electroencephalogr. Clin. Neurophysiol./Evoked Potentials Section. 88, 389.10.1016/0168-5597(93)90015-H
    1. Pantev C., Roberts L. E., Elbert T., Ross B., Wienbruch C. (1996). Tonotopic organization of the sources of human auditory steady-state responses. Hear Res. 101, 62–7410.1016/S0378-5955(96)00133-5
    1. Pardo J. V., Fox P. T., Raichle M. E. (1991). Localization of a human system for sustained attention by positron emission Tomography. Nature 349, 61–6410.1038/349061a0
    1. Pastor M. A., Artieda J., Arbizu J., Marti-Climent J. M., Penuelas I., Masdeu J. C. (2002). Activation of human cerebral and cerebellar cortex by auditory stimulation at 40 Hz. J. Neurosci. 22, 10501–10506
    1. Pastor M. A., Valencia M., Artieda J., Alegre M., Masdeu J. C. (2007). Topography of cortical activation differs for fundamental and harmonic frequencies of the steady-state visual-evoked responses. An EEG and PET (H2O)-O-15 study. Cereb. Cortex 17, 1899–190510.1093/cercor/bhl098
    1. Patterson R. D., Uppenkamp S., Johnsrude I. S., Griffiths T. D. (2002). The processing of temporal pitch and melody information in auditory cortex. Neuron 36, 767–77610.1016/S0896-6273(02)01060-7
    1. Pei F., Pettet M. W., Norcia A. M. (2002). Neural correlates of object-based attention. J. Vis. 2, 588–59610.1167/2.9.1
    1. Perrin F., Pernier J., Bertrand O., Echallier J. F. (1989). Spherical splines for scalp potential and current-density mapping. Electroencephalogr. Clin. Neurophysiol. 72, 184–18710.1016/0013-4694(89)90180-6
    1. Picton T. W., John M. S., Dimitrijevic A., Purcell D. (2003). Human auditory steady-state responses. Int. J. Audiol. 42, 177–219
    1. Regan D. (1989). Human Brain Electrophysiology: Evoked Potentials and Evoked Magnetic Fields in Science and Medicine. New York, Elsevier
    1. Robin D. A., Tranel D., Damasio H. (1990). Auditory-perception of temporal and spectral events in patients with focal left and right cerebral-lesions. Brain Lang. 39, 539–55510.1016/0093-934X(90)90161-9
    1. Ross B., Borgmann C., Draganova R., Roberts L. E., Pantev C. (2000). A high-precision magnetoencephalographic study of human auditory steady-state responses to amplitude-modulated tones. J. Acoust. Soc. Am. 108, 679–69110.1121/1.429600
    1. Ross B., Herdman A. T., Pantev C. (2005). Right hemispheric laterality of human 40-Hz auditory steady-state responses. Cereb. Cortex 15, 2029–203910.1093/cercor/bhi078
    1. Ross B., Picton T. W., Herdman A. T., Pantev C. (2004). The effect of attention on the auditory steady-state response. Neurol. Clin. Neurophysiol. 2004, 22.
    1. Samson S., Ehrle N., Baulac M. (2001). Cerebral substrates for musical temporal processes. Biol. Found Music. 930, 166–178
    1. Saupe K., Widmann A., Bendixen A., Müller M. M., Schröger E. (2009). Effects of intermodal attention on the auditory steady-state response and the event-related potential. Psychophysiology 46, 321–32710.1111/j.1469-8986.2008.00765.x
    1. Skosnik P. D., Krishnan G. P., O'Donnell B. F. (2007). The effect of selective attention on the gamma-band auditory steady-state response. Neurosci. Lett. 420, 223–22810.1016/j.neulet.2007.04.072
    1. Srinivasan R., Bibi F. A., Nunez P. L. (2006). Steady-state visual evoked potentials: distributed local sources and wave-like dynamics are sensitive to flicker frequency. Brain Topogr. 18, 167–18710.1007/s10548-006-0267-4
    1. Talsma D., Doty T. J., Strowd R., Woldorff M. G. (2006). Attentional capacity for processing concurrent stimuli is larger across sensory modalities than within a modality. Psychophysiology 43, 541–54910.1111/j.1469-8986.2006.00452.x
    1. Talsma D., Kok A. (2001). Nonspatial intermodal selective attention is mediated by sensory brain areas: evidence from event-related potentials. Psychophysiology 38, 736–75110.1017/S0048577201000798
    1. Talsma D., Kok A. (2002). Intermodal spatial attention differs between vision and audition: an event-related potential analysis. Psychophysiology 39, 689–70610.1017/S004857720200152X
    1. Toffanin P., de Jong R., Johnson A., Martens S. (2009). Using frequency tagging to quantify attentional deployment in a visual divided attention task. Int. J. Psychophysiol. 72, 289–29810.1016/j.ijpsycho.2009.01.006
    1. Vidal J. R., Chaumon M., O'Regan J. K., Tallon-Baudry C. (2006). Visual grouping and the focusing of attention induce gamma-band oscillations at different frequencies in human magnetoencephalogram signals. J. Cogn. Neurosci. 18, 1850–186210.1162/jocn.2006.18.11.1850
    1. Vohn R., Fimm B., Weber J., Schnitker R., Thron A., Spijkers W., Willmes K., Sturm W. (2007). Management of attentional resources in within-modal and cross-modal divided attention tasks: an fMRI study. Hum. Brain Mapp. 28, 1267–127510.1002/hbm.20350
    1. Weintraub S., Mesulam M. M. (1987). Right cerebral-dominance in spatial attention – further evidence based on ipsilateral neglect. Arch. Neurol. 44, 621–625
    1. Whitehead R. (1991). Right-hemisphere processing superiority during sustained visual-attention. J. Cogn. Neurosci. 3, 329–33410.1162/jocn.1991.3.4.329
    1. Woods D. L., Alho K., Algazi A. (1992). Intermodal selective attention. I. Effects on event-related potentials to lateralized auditory and visual stimuli. Electroencephalogr. Clin. Neurophysiol. 82, 341.10.1016/0013-4694(92)90004-2
    1. Worsley K. J., Marrett S., Neelin P., Evans A. C. (1996). Searching scale space for activation in PET images. Hum. Brain Mapp. 4, 74–9010.1002/(SICI)1097-0193(1996)4:1<74::AID-HBM5>;2-M
    1. Yamaguchi S., Tsuchiya H., Kobayashi S. (1994). Electroencephalographic activity associated with shifts of visuospatial attention. Brain 117, 553–56210.1093/brain/117.3.553
    1. Zatorre R. J., Belin P. (2001). Spectral and temporal processing in human auditory cortex. Cereb. Cortex 11, 946–95310.1093/cercor/11.10.946

Source: PubMed

Sponsors and Collaborators

Medical Conditions

Drug Interventions

3
Subscribe