Anterior superior temporal sulcus is specialized for non-rigid facial motion in both monkeys and humans

Hui Zhang, Shruti Japee, Andrea Stacy, Molly Flessert, Leslie G Ungerleider, Hui Zhang, Shruti Japee, Andrea Stacy, Molly Flessert, Leslie G Ungerleider

Abstract

Facial motion plays a fundamental role in the recognition of facial expressions in primates, but the neural substrates underlying this special type of biological motion are not well understood. Here, we used fMRI to investigate the extent to which the specialization for facial motion is represented in the visual system and compared the neural mechanisms for the processing of non-rigid facial motion in macaque monkeys and humans. We defined the areas specialized for facial motion as those significantly more activated when subjects perceived the motion caused by dynamic faces (dynamic faces ​> ​static faces) than when they perceived the motion caused by dynamic non-face objects (dynamic objects ​> ​static objects). We found that, in monkeys, significant activations evoked by facial motion were in the fundus of anterior superior temporal sulcus (STS), which overlapped the anterior fundus face patch. In humans, facial motion activated three separate foci in the right STS: posterior, middle, and anterior STS, with the anterior STS location showing the most selectivity for facial motion compared with other facial motion areas. In both monkeys and humans, facial motion shows a gradient preference as one progresses anteriorly along the STS. Taken together, our results indicate that monkeys and humans share similar neural substrates within the anterior temporal lobe specialized for the processing of non-rigid facial motion.

Trial registration: ClinicalTrials.gov NCT00001360.

Keywords: Emotion processing; Facial motion; STS; fMRI.

Published by Elsevier Inc.

Figures

Fig. 1.
Fig. 1.
A) Monkeys viewed blocks of dynamic monkey faces, dynamic objects, static monkey faces and static objects in the main task. B) Humans viewed blocks of dynamic human faces, dynamic objects, static human faces and static objects in the main task. Each dynamic condition contained 12 different video clips, and each static condition contained 12 different images that were generated from the corresponding dynamic video clip. Each block lasted 24 s, with 16-sec baseline fixation periods between blocks. Within a block, each video clip (for dynamic conditions) or each image was presented for 2 s, with no blank periods between them. Motion energy in the dynamic face video clips was equivalent (or as close as possible) to that in the corresponding dynamic object videos for each block of the stimuli using an optic-flow algorithm.
Fig. 2.
Fig. 2.
A) Localization of facial motion selective regions (patches) by contrasting the fMRI response to motion caused by faces relative to motion caused by objects [(dynamic faces > static faces) - (dynamic objects > static objects)] in monkeys F, R, K and B. The location of these facial motion patches overlapped the AF face patches, for monkeys F, R and K bilaterally, and for monkey B in the right hemisphere. B) Localization of object motion selective regions by contrasting the fMRI response to motion caused by non-face objects relative to motion caused by faces [(dynamic objects > static objects) - (dynamic faces > static faces)] in each of the four monkeys. The object motion selective regions are mostly in the posterior and middle portions of STS in all 4 monkeys. Dashed white lines outline areas of static face-selective regions shown in Supplementary Figure S4A.
Fig. 3.
Fig. 3.
A) Localization of facial motion selective regions for a representative human subject by contrasting the fMRI response to motion caused by faces relative to motion caused by objects. This contrast identified three separate foci along the STS: anterior STS, middle STS and posterior STS. The posterior STS overlapped with the pSTS face-selective region. B) Localization of object motion selective regions for the representative human subject by contrasting the fMRI response to motion caused by objects relative to motion caused by faces. The identified object motion selective regions are mostly outside of human STS. Dashed white lines outline areas of static face-selective regions shown in Supplementary Figure S5A.
Fig. 4.
Fig. 4.
A-E) Percent signal change of the fMRI response amplitude for each category of visual stimuli (dynamic faces, static faces, dynamic objects, static objects) in the monkey STS regions-of-interest. AF: Anterior Fundus, MF: Middle Fundus, AL: Anterior Lateral, ML: Middle Lateral. F) The facial motion and object motion sensitivity for each monkey ROI. * indicate p

Fig. 5.

A-G) Percent signal change of…

Fig. 5.

A-G) Percent signal change of the fMRI response amplitude for each category of…

Fig. 5.
A-G) Percent signal change of the fMRI response amplitude for each category of visual stimuli (dynamic faces, static faces, dynamic objects, static objects) in the human regions-of-interest. aSTS: anterior Superior Temporal Sulcus, mSTS: middle Superior Temporal Sulcus, pSTS: posterior Superior Temporal Sulcus, aIT: anterior Inferior Temporal cortex, FFA: Fusiform Face Area, OFA: Occipital Face Area. H) The facial motion and object motion sensitivity for each human ROI. * indicate p

Fig. 6.

The PSC values of facial…

Fig. 6.

The PSC values of facial motion selectivity in the regions of: A) monkeys…

Fig. 6.
The PSC values of facial motion selectivity in the regions of: A) monkeys and B) humans. In monkeys, only AF showed significantly greater facial motion selectivity than zero. In humans, only aSTS and mSTS showed significantly greater facial motion selectivity than zero. * indicate p
Similar articles
Cited by
References
    1. Allison T, Puce A, McCarthy G, 2000. Social perception from visual cues: role of the STS region. Trends Cognit. Sci 4, 267–278. - PubMed
    1. Aggarwal JK, Cai Q, Liao W, Sabata B, 1998. Nonrigid motion analysis: articulated and elastic motion. Comput. Vis. Image Understand. 70 (2), 142–156.
    1. Ambadar Z, Schooler JW, Cohn JF, 2005. Deciphering the enigmatic face: the importance of facial dynamics in interpreting subtle facial expressions. Psychol. Sci 16, 403–410. - PubMed
    1. Andrews TJ, Ewbank MP, 2004. Distinct representations for facial identity and changeable aspects of faces in the human temporal lobe. Neuroimage 23, 905–913. - PubMed
    1. Axelrod V, Yovel G, 2013. The challenge of localizing the anterior temporal face area: a possible solution. Neuroimage 81, 371–380. - PubMed
Show all 61 references
Publication types
MeSH terms
Associated data
[x]
Cite
Copy Download .nbib
Format: AMA APA MLA NLM

NCBI Literature Resources

MeSH PMC Bookshelf Disclaimer

The PubMed wordmark and PubMed logo are registered trademarks of the U.S. Department of Health and Human Services (HHS). Unauthorized use of these marks is strictly prohibited.

Follow NCBI
Fig. 5.
Fig. 5.
A-G) Percent signal change of the fMRI response amplitude for each category of visual stimuli (dynamic faces, static faces, dynamic objects, static objects) in the human regions-of-interest. aSTS: anterior Superior Temporal Sulcus, mSTS: middle Superior Temporal Sulcus, pSTS: posterior Superior Temporal Sulcus, aIT: anterior Inferior Temporal cortex, FFA: Fusiform Face Area, OFA: Occipital Face Area. H) The facial motion and object motion sensitivity for each human ROI. * indicate p

Fig. 6.

The PSC values of facial…

Fig. 6.

The PSC values of facial motion selectivity in the regions of: A) monkeys…

Fig. 6.
The PSC values of facial motion selectivity in the regions of: A) monkeys and B) humans. In monkeys, only AF showed significantly greater facial motion selectivity than zero. In humans, only aSTS and mSTS showed significantly greater facial motion selectivity than zero. * indicate p
Similar articles
Cited by
References
    1. Allison T, Puce A, McCarthy G, 2000. Social perception from visual cues: role of the STS region. Trends Cognit. Sci 4, 267–278. - PubMed
    1. Aggarwal JK, Cai Q, Liao W, Sabata B, 1998. Nonrigid motion analysis: articulated and elastic motion. Comput. Vis. Image Understand. 70 (2), 142–156.
    1. Ambadar Z, Schooler JW, Cohn JF, 2005. Deciphering the enigmatic face: the importance of facial dynamics in interpreting subtle facial expressions. Psychol. Sci 16, 403–410. - PubMed
    1. Andrews TJ, Ewbank MP, 2004. Distinct representations for facial identity and changeable aspects of faces in the human temporal lobe. Neuroimage 23, 905–913. - PubMed
    1. Axelrod V, Yovel G, 2013. The challenge of localizing the anterior temporal face area: a possible solution. Neuroimage 81, 371–380. - PubMed
Show all 61 references
Publication types
MeSH terms
Associated data
[x]
Cite
Copy Download .nbib
Format: AMA APA MLA NLM
Fig. 6.
Fig. 6.
The PSC values of facial motion selectivity in the regions of: A) monkeys and B) humans. In monkeys, only AF showed significantly greater facial motion selectivity than zero. In humans, only aSTS and mSTS showed significantly greater facial motion selectivity than zero. * indicate p

References

    1. Allison T, Puce A, McCarthy G, 2000. Social perception from visual cues: role of the STS region. Trends Cognit. Sci 4, 267–278.
    1. Aggarwal JK, Cai Q, Liao W, Sabata B, 1998. Nonrigid motion analysis: articulated and elastic motion. Comput. Vis. Image Understand. 70 (2), 142–156.
    1. Ambadar Z, Schooler JW, Cohn JF, 2005. Deciphering the enigmatic face: the importance of facial dynamics in interpreting subtle facial expressions. Psychol. Sci 16, 403–410.
    1. Andrews TJ, Ewbank MP, 2004. Distinct representations for facial identity and changeable aspects of faces in the human temporal lobe. Neuroimage 23, 905–913.
    1. Axelrod V, Yovel G, 2013. The challenge of localizing the anterior temporal face area: a possible solution. Neuroimage 81, 371–380.
    1. Baker S, Matthews I, 2004. Lucas-Kanade 20 years on: a unifying framework. IJCV 53 (3), 221–255.
    1. Beauchamp MS, 2015. The social mysteries of the superior temporal sulcus. Trends Cognit. Sci 19 (9), 489–490.
    1. Beauchamp MS, Lee KE, Argall BD, Martin A, 2004. Integration of auditory and visual information about objects in superior temporal sulcus. Neuron 41 (5), 809–823.
    1. Beauchamp MS, Lee KE, Haxby JV, Martin A, 2002. Parallel visual motion processing streams for manipulable objects and human movements. Neuron 34 (1), 149–159.
    1. Beauchamp MS, Lee KE, Haxby JV, Martin A, 2003. FMRI responses to video and point-light displays of moving humans and manipulable objects. J. Cognit. Neurosci 15 (7), 991–1001.
    1. Beauchemin SS, Barron JL, 1995. The computation of optical flow. ACM Comput. Surv 27, 433–466.
    1. Bell AH, Hadj-Bouziane F, Frihauf JB, Tootell RB, Ungerleider LG, 2009. Object representations in the temporal cortex of monkeys and humans as revealed by functional magnetic resonance imaging. J. Neurophysiol 101, 688–700.
    1. Bernstein M, Yovel G, 2015. Two neural pathways of face processing: a critical evaluation of current models. Neurosci. Biobehav. Rev 55, 536–546.
    1. Bonda E, Petrides M, Ostry D, Evans A, 1996. Specific involvement of human parietal systems and the amygdala in the perception of biological motion. J. Neurosci 16, 3737–3744.
    1. Bruce V, Young A, 1990. Understanding face recognition. Br. J. Psychol 81, 361–380. Comment in.
    1. Bruhn A, Weickert J, Schnorr C, 2005. Lucas/Kanade meets Horn/Schunck: combining local and global optic flow methods. Int. J. Comput. Vis 61, 211–231.
    1. Cox RW, 1996. AFNI: software for analysis and visualization of functional magnetic resonance neuroimages. Comput. Biomed. Res 29, 162–173.
    1. Deen B, Koldewyn K, Kanwisher N, Saxe R, 2015. Functional organization of social perception and cognition in the superior temporal sulcus. Cerebr. Cortex 25 (11), 4596–4609.
    1. Desimone R, Ungerleider LG, 1986. Multiple visual areas in the caudal superior temporal sulcus of the macaque. J. Comp. Neurol 248, 164–189.
    1. Dubner R, Zeki SM, 1971. Response properties and receptive fields of cells in an anatomically defined region of the superior temporal sulcus in the monkey. Brain Res 35, 528–532.
    1. Duchaine B, Yovel G, 2015. A revised neural framework for face processing. Annu. Rev. Vis. Sci 1, 293–416.
    1. Fisher C, Freiwald WA, 2015a. Contrasting specializations for facial motion within the macaque face-processing system. Curr. Biol 25, 261–266.
    1. Fisher C, Freiwald WA, 2015b. Whole-agent selectivity within the macaque face-processing system. Proc. Natl. Acad. Sci. U. S. A 112, 14717–14722.
    1. Fleet DJ, Weiss Y, 2006. Optical flow estimation In: Paragios et al. Handbook of Mathematical Models in Computer Vision. Springer, pp. 239–257.
    1. Fox CJ, Iaria G, Barton JJ, 2009. Defining the face-processing network: optimization of the functional localizer in fMRI. Hum. Brain Mapp 30, 1637–1651.
    1. Furl N, Hadj-Bouziane F, Liu N, Averbeck BB, Ungerleider LG, 2012. Dynamic and static facial expressions decoded from motion-sensitive areas in the macaque monkey. J. Neurosci 32, 15952–15962.
    1. Hadj-Bouziane F, Bell AH, Knusten TA, Ungerleider LG, Tootell RB, 2008. Perception of emotional expressions is independent of face selectivity in monkey inferior temporal cortex. Proc. Natl. Acad. Sci. U.S.A 105, 5591–5596.
    1. Hadj-Bouziane F, Liu N, Bell AH, Gothard KM, Luh WM, Tootell RB, Murray EA, Ungerleider LG, 2012. Amygdala lesions disrupt modulation of functional MRI activity evoked by facial expression in the monkey inferior temporal cortex. Proc. Natl. Acad. Sci. U. S. A 109, E3640–E3648.
    1. Haxby JV, Hoffman EA, Gobbini MI, 2000. The distributed human neural system for face perception. Trends Cognit. Sci 4, 223–233.
    1. Harris RJ, Young AW, Andrews TJ, 2012. Morphing between expressions dissociates continuous from categorical representations of facial expression in the human brain. Proc. Natl. Acad. Sci. U.S.A 109, 21164–21169.
    1. Herrington JD, Nymberg C, Schultz RT, 2011. Biological motion task performance predicts superior temporal sulcus activity. Brain Cognit 77, 372–381.
    1. Jastorff J, Popivanov ID, Vogels R, Vanduffel W, Orban GA, 2012. Integration of shape and motion cues in biological motion processing in the monkey STS. Neuroimage 60 (2), 911–921.
    1. Kanwisher N, McDermott J, Chun MM, 1997. The fusiform face area: a module in human extrastriate cortex specialized for face perception. J. Neurosci 17, 4302–4311.
    1. Kanwisher N, Yovel G, 2006. The fusiform face area: a cortical region specialized for the perception of faces. Phil. Trans. Roy. Soc. Lond. B 361, 2109–2128.
    1. Knappmeyer B, Thornton IM, Bülthoff HH, 2003. The use of facial motion and facial form during the processing of identity. Vis. Res 43, 1921–1936.
    1. Knight B, Johnston A, 1997. The role of movement in face recognition. Vis. Cognit 4, 265–273.
    1. Kriegeskorte N, Formisano E, Sorger B, Goebel R, 2007. Individual faces elicit distinct response patterns in human anterior temporal cortex. Proc. Natl. Acad. Sci. U.S.A 104, 20600–20605.
    1. Lafer-Sousa R, Conway BR, 2013. Parallel, multi-stage processing of colors, faces and shapes in macaque inferior temporal cortex. Nat. Neurosci 16 (12), 1870–1878.
    1. Lafer-Sousa R, Conway BR, Kanwisher NG, 2016. Color-biased regions of the ventral visual pathway lie between face- and place-selective regions in humans, as in macaques. J. Neurosci 36 (5), 1682–1697.
    1. Lander K, Christie F, Bruce V, 1999. The role of movement in the recognition of famous faces. Mem. Cognit 27, 974–985.
    1. Liu N, Kriegeskorte N, Mur M, Hadj-Bouziane F, Luh WM, Tootell RB, Ungerleider LG, 2013. Intrinsic structure of visual exemplar and category representations in macaque brain. J. Neurosci 33, 11346–11360.
    1. Lucas BD, Kanade T, 1981. An iterative image registration technique with an application to stereo vision. In: Proc 7th Intl Joint Conf on Artificial Intelligence (IJCAI), 2, pp. 674–679, 1981.
    1. McCarthy G, Puce A, Gore JC, Allison T, 1997. Face-specific processing in the human fusiform gyrus. J. Cognit. Neurosci 9, 605–610.
    1. McMahon DB, Russ BE, Elnaiem HD, Kurnikova AI, Leopold DA, 2015. Single-unit activity during natural vision: diversity, consistency, and spatial sensitivity among AF face patch neurons. J. Neurosci 35, 5537–5548.
    1. O’Toole AJ, Roark DA, Abdi H, 2002. Recognizing moving faces: a psychological and neural synthesis. Trends Cognit. Sci 6, 261–266.
    1. Pitcher D, Dilks DD, Saxe RR, Triantafyllou C, Kanwisher N, 2011. Differential selectivity for dynamic versus static information in face-selective cortical regions. Neuroimage 56, 2356–2363.
    1. Polosecki P, Moeller S, Schweers N, Romanski LM, Tsao DY, Freiwald WA, 2013. Faces in motion: selectivity of macaque and human face processing areas for dynamic stimuli. J. Neurosci 33, 11768–11773.
    1. Puce A, Allison T, Bentin S, Gore JC, McCarthy G, 1998. Temporal cortex activation in humans viewing eye and mouth movements. J. Neurosci 18, 2188–2199.
    1. Roark DA, O’Toole AJ, Abdi H, Barrett SE, 2006. Learning the moves: the effect of familiarity and facial motion on person recognition across large changes in viewing format. Perception 35, 761–773.
    1. Schall S, Kiebel SJ, Maess B, von Kriegstein K, 2015. Voice identity recognition: functional division of the right STS and its behavioral relevance. J. Cognit. Neurosci 27 (2), 280–291.
    1. Schultz J, Brockhaus M, Bülthoff HH, Pilz KS, 2013. What the human brain likes about facial motion. Cerebr. Cortex 23, 1167–1178.
    1. Schultz J, Pilz KS, 2009. Natural facial motion enhances cortical responses to faces. Exp. Brain Res 194, 465–475.
    1. Sereno MI, Dale AM, Reppas JB, Kwong KK, Belliveau JW, Brady TJ, Rosen BR, Tootell RB, 1995. Borders of multiple visual areas in humans revealed by functional magnetic resonance imaging. Science 268, 889–893.
    1. Sereno MI, McDonald CT, Allman JM, 1994. Analysis of retinotopic maps in extrastriate cortex. Cerebr. Cortex 4, 601–620.
    1. Sliwa J, Freiwald WA, 2017. A dedicated network for social interaction processing in the primate brain. Science 356 (6339), 745–749.
    1. Trautmann SA, Fehr T, Herrmann M, 2009. Emotions in motion: dynamic compared to static facial expressions of disgust and happiness reveal more widespread emotion-specific activations. Brain Res 1284, 100–115.
    1. Tsao DY, Moeller S, Freiwald WA, 2008. Comparing face patch systems in macaques and humans. Proc. Natl. Acad. Sci. U. S. A 105, 19514–19519.
    1. Wehrle T, Kaiser S, Schmidt S, Scherer KR, 2000. Studying the dynamics of emotional expression using synthesized facial muscle movements. J. Pers. Soc. Psychol 78, 105–119.
    1. Willems RM, Peelen MV, Hagoort P, 2010. Cerebral lateralization of face-selective and body-selective visual areas depends on handedness. Cerebr. Cortex 20, 1719–1725.
    1. Yovel G, Tambini A, Brandman T, 2008. The asymmetry of the fusiform face area is a stable individual characteristic that underlies the left-visual-field superiority for faces. Neuropsychologia 46, 3061–3068.
    1. Zhu Q, Nelissen K, Van den Stock J, De Winter FL, Pauwels K, de Gelder B, Vanduffel W, Vandenbulcke M, 2013. Dissimilar processing of emotional facial expressions in human and monkey temporal cortex. Neuroimage 66, 402–411.

Source: PubMed

Sponsorit ja yhteistyökumppanit

Sairaudet

Huumeiden interventiot

3
Tilaa