The human amygdala parametrically encodes the intensity of specific facial emotions and their categorical ambiguity

Shuo Wang, Rongjun Yu, J Michael Tyszka, Shanshan Zhen, Christopher Kovach, Sai Sun, Yi Huang, Rene Hurlemann, Ian B Ross, Jeffrey M Chung, Adam N Mamelak, Ralph Adolphs, Ueli Rutishauser, Shuo Wang, Rongjun Yu, J Michael Tyszka, Shanshan Zhen, Christopher Kovach, Sai Sun, Yi Huang, Rene Hurlemann, Ian B Ross, Jeffrey M Chung, Adam N Mamelak, Ralph Adolphs, Ueli Rutishauser

Abstract

The human amygdala is a key structure for processing emotional facial expressions, but it remains unclear what aspects of emotion are processed. We investigated this question with three different approaches: behavioural analysis of 3 amygdala lesion patients, neuroimaging of 19 healthy adults, and single-neuron recordings in 9 neurosurgical patients. The lesion patients showed a shift in behavioural sensitivity to fear, and amygdala BOLD responses were modulated by both fear and emotion ambiguity (the uncertainty that a facial expression is categorized as fearful or happy). We found two populations of neurons, one whose response correlated with increasing degree of fear, or happiness, and a second whose response primarily decreased as a linear function of emotion ambiguity. Together, our results indicate that the human amygdala processes both the degree of emotion in facial expressions and the categorical ambiguity of the emotion shown and that these two aspects of amygdala processing can be most clearly distinguished at the level of single neurons.

Conflict of interest statement

The authors declare no competing financial interests.

Figures

Figure 1. Behavioural results.
Figure 1. Behavioural results.
(a) Task. A face was presented for 1 s followed by a question asking subjects to identify the facial emotion (fearful or happy). After a blank screen of 500 ms, subjects were then asked to indicate their confidence in their decision (‘1' for ‘very sure', ‘2' for ‘sure' or ‘3' for ‘unsure'). Faces are not shown to scale. (b) Sample stimuli of one female identity ranging from 0% fear/100% happy to 100% fear/0% happy. (c–q) Behavioural results. (c) Group average of psychometric curves. The psychometric curves show the proportion of trials judged as fearful as a function of morph levels (ranging from 0% fearful (100% happy; on the left) to 100% fearful (0% happy; on the right)). Shaded area denotes ±s.e.m. across subjects/sessions (n=3, 14, 15). The top bars illustrate the points with significant difference between amygdala lesion patients and neurosurgical patients (green; unpaired two-tailed t-test, P<0.05, corrected by FDR for Q<0.05) and between amygdala lesion patients and healthy controls (yellow). (d) Inflection point of the logistic function (xhalf). (e) Steepness of the psychometric curve (α). Individual values are shown on the left and average values are shown on the right. Error bars denote one s.e.m. across subjects/sessions. Asterisks indicate significant difference using unpaired two-tailed t-test. *P<0.05, and **P<0.01. NS: not significant (P>0.05). (f–q) Confidence ratings for lesion (f–i), neurosurgical (j–m) and control (n–q) subjects. (f,j,n) Explicit confidence ratings showed highest confidence for anchor faces and lowest for the most ambiguous (50% fear/50% happy) faces. (g,k,o) The RT for the fear/happy decision can be considered as an implicit measure of confidence because it showed a similar pattern as the explicit ratings. For the neural analysis, we grouped the seven morph levels into three levels of ambiguity (anchor, 30%/70% morph, 40–60% morph). Both explicit (h,l,p) and implicit (i,m,q) confidence measures varied systematically as a function of ambiguity. The behavioural patterns of all three subject groups were comparable. Error bars denote one s.e.m. across subjects/sessions.
Figure 2. fMRI results.
Figure 2. fMRI results.
(a) Fear levels were negatively correlated with the BOLD activity in the left amygdala. Here we used a functional amygdala ROI defined by the localizer task (see Methods section). The generated statistical parametric map was superimposed on anatomical sections of the standardized MNI T1-weighted brain template. Images are in neurological format with subject left on image left. R: right. (b) Parameter estimate (beta values) of the GLM for each fear level (Pearson correlation: r=−0.79, P=0.034). The colour scale denotes increasing degree of fear (cf. Fig. 1b). Error bars denote one s.e.m. across 19 subjects. (c) Ambiguity levels were correlated with the BOLD activity in the right amygdala (functional ROI defined by localizer task). (d) Time course of the BOLD response in the right amygdala (averaged across all voxels in the cluster) in units of TR (TR=2 s) relative to face onset. Error bars denote one s.e.m. across subjects. One-way repeated ANOVA at each TR: *P<0.05. (e) Parameter estimate of the GLM for each ambiguity level (one-way repeated-measure ANOVA, P=0.0025). Error bars denote one s.e.m. across subjects. Asterisks indicate significant difference between conditions using paired two-tailed t-test. *P<0.05. (f) Mean GLM parameter estimate of all voxels within the functional ROI for each side of the amygdala and for each aspect of the emotion coding. Error bars denote one s.e.m. across subjects. T-test against 0: **P<0.01, and +: P<0.1. (g) Peak voxel activity within the basolateral nuclei (BLA) and central nuclei (CeA) for each aspect of the emotion coding.
Figure 3. Emotion-tracking neurons.
Figure 3. Emotion-tracking neurons.
(a) Example neuron that increased its firing rate as a function of %fear (linear regression, P=0.024). (b) Example neuron that increased its firing rate as a function of %happy (P=0.032). Right shows the average firing rate for each morph level 250- to 1,750-ms post-stimulus onset. Grey lines represent linear fit. Error bars denote ±s.e.m. across trials. Waveforms for each unit are shown at the left. (c) Histogram of regression R2. Neurons that had a higher firing rate for fearful faces are shown on the right of 0, whereas neurons that had a higher firing rate for happy faces are shown on the left. Fear-tracking neurons are in red, happy-tracking neurons are in blue, whereas non-emotion-tracking neurons are in grey. (d) Slope of linear regression fit. Grey: non-emotion-tracking neurons. Red: fear-tracking neurons. Blue: happy-tracking neurons.
Figure 4. Ambiguity-coding neurons.
Figure 4. Ambiguity-coding neurons.
(a,b) Two example neurons that fire most to the anchors and least to the most ambiguous stimuli (linear regression: P<0.05). Each raster (top), PSTH (middle) and average firing rate (bottom) is colour coded according to ambiguity levels as indicated. Trials are aligned to face stimulus onset (left grey bar, fixed 1 s duration) and sorted by RT (black line). PSTH bin size is 250 ms. Shaded area and error bars denote ±s.e.m. across trials. Asterisk indicates a significant difference between the conditions in that bin (P<0.05, one-way ANOVA, Bonferroni-corrected). Bottom left shows the average firing rate for each morph level 250- to 1,750-ms post-stimulus-onset. Bottom right shows the average firing rate for each ambiguity level 250- to 1,750-ms post-stimulus onset. Asterisks indicate significant difference between levels of ambiguity using unpaired two-tailed t-test. **P<0.01 and +: P<0.1. Waveforms for each unit are shown at the top of the raster plot. (c,d) Average normalized firing rate of ambiguity-coding neurons that increased (n=29) and decreased (n=3) their firing rate for the least ambiguous faces, respectively. Asterisk indicates a significant difference between the conditions in that bin (P<0.05, one-way ANOVA, Bonferroni-corrected). (e,f) Mean normalized firing rate at each morph level (e) and ambiguity level (f) for 29 units that increased their spike rate for less ambiguous faces. (g,h) Mean normalized firing rate at each morph level (g) and ambiguity level (h) for 3 units that increased their spike rate for more ambiguous faces. Normalized firing rate for each unit (left) and mean±s.e.m. across units (right) are shown at each level. Asterisks indicate significant difference between conditions using paired two-tailed t-test. ***P<0.001. (i) Histogram of AUC values for ambiguity-coding neurons (orange) and unselected neurons that are neither ambiguity coding nor emotion tracking (grey). (j) Histogram of AUC values for emotion-tracking neurons (purple) and unselected neurons that are neither ambiguity coding nor emotion tracking (grey). (k) Cumulative distribution of the AUC values. (i–k) Ambiguity-coding neurons did not differentiate fearful versus happy emotions with anchor faces (similar AUC values as unselected neurons) but emotion-coding neurons did (greater AUC values than unselected neurons).
Figure 5. Population analysis of all recorded…
Figure 5. Population analysis of all recorded neurons (n=234) using a regression model and effect size metric ω2.
(a,b) Time course of the effect size averaged across all neurons. (a) Morph levels. (b) Ambiguity levels. Trials are aligned to face stimulus onset (left grey bar, fixed 1 s duration). Bin size is 500 ms and step size is 100 ms. Neurons from permutation were averaged across 500 runs. Shaded area denotes ±s.e.m. across neurons. Dashed horizontal lines indicate the 95% confidence interval estimated from the permuted distribution. Asterisk indicates a significant difference between the observed average across neurons versus permuted average across neurons in that bin (permutation P<0.05, Bonferroni-corrected). (c–l) Summary of the effect size across all runs. Effect size was computed in a 1.5-s window starting 250 ms after stimulus onset (single fixed window, not a moving window) and was averaged across all neurons for each run. Grey and red vertical lines indicate the chance mean effect size and the observed effect size, respectively. The observed effect size was well above 0, whereas the mean effect size in the permutation test was near 0. (c) Regression model for morph levels (permutation P<0.001). (d) Regression model for ambiguity levels (P<0.001). (e–g) Regression model for decision of emotion (fear or happy) with (e) all faces (P<0.001), (f) all ambiguous faces (all morphed faces; P<0.001), (g) the most ambiguous faces (40–60% morph; P=0.006) and (h) the most ambiguous faces with equal number of fear/happy responses for each identity (P=0.002). (i–l) Regression model for confidence judgment (i) with all faces (P<0.001), (j) at the 30% fear/70% happy morph level (P<0.001), (k) at the 50% fear/50% happy morph level (P=0.026) and (l) at the 30% fear/70% happy morph level (P=0.028).

References

    1. Adolphs R. Fear, faces, and the human amygdala. Curr. Opin. Neurobiol. 18, 166–172 (2008).
    1. Adolphs R., Tranel D., Damasio H. & Damasio A. Impaired recognition of emotion in facial expressions following bilateral damage to the human amygdala. Nature 372, 669–672 (1994).
    1. Rutishauser U., Mamelak A. N. & Adolphs R. The primate amygdala in social perception–insights from electrophysiological recordings and stimulation. Trends Neurosci. 38, 295–306 (2015).
    1. Calder A. J. Facial emotion recognition after bilateral amygdala damage: differentially severe impairment of fear. Cogn. Neuropsychol. 13, 699–745 (1996).
    1. Broks P. et al.. Face processing impairments after encephalitis: amygdala damage and recognition of fear. Neuropsychologia 36, 59–70 (1998).
    1. Adolphs R. et al.. Recognition of facial emotion in nine individuals with bilateral amygdala damage. Neuropsychologia 37, 1111–1117 (1999).
    1. Morris J. S. et al.. A differential neural response in the human amygdala to fearful and happy facial expressions. Nature 383, 812–815 (1996).
    1. Phillips M. L. et al.. Neural responses to facial and vocal expressions of fear and disgust. Proc. R. Soc. Lond. B Biol. Sci. 265, 1809–1817 (1998).
    1. Whalen P. J. et al.. Human amygdala responsivity to masked fearful eye whites. Science 306, 2061 (2004).
    1. Mende-Siedlecki P., Verosky S. C., Turk-Browne N. B. & Todorov A. Robust selectivity for faces in the human amygdala in the absence of expressions. J. Cogn. Neurosci. 25, 2086–2106 (2013).
    1. Fried I., MacDonald K. A. & Wilson C. L. Single neuron activity in human hippocampus and amygdala during recognition of faces and objects. Neuron 18, 753–765 (1997).
    1. Viskontas I. V., Quiroga R. Q. & Fried I. Human medial temporal lobe neurons respond preferentially to personally relevant images. Proc. Natl Acad. Sci. 106, 21329–21334 (2009).
    1. Rutishauser U. et al.. Single-unit responses selective for whole faces in the human amygdala. Curr. Biol. 21, 1654–1660 (2011).
    1. Quian Quiroga R., Kraskov A., Mormann F., Fried I. & Koch C. Single-cell responses to face adaptation in the human medial temporal lobe. Neuron 84, 363–369 (2014).
    1. Fitzgerald D. A., Angstadt M., Jelsone L. M., Nathan P. J. & Phan K. L. Beyond threat: Amygdala reactivity across multiple expressions of facial affect. Neuroimage 30, 1441–1448 (2006).
    1. Rolls E. Neurons in the cortex of the temporal lobe and in the amygdala of the monkey with responses selective for faces. Hum. Neurobiol. 3, 209–222 (1984).
    1. Leonard C. M., Rolls E. T., Wilson F. A. & Baylis G. C. Neurons in the amygdala of the monkey with responses selective for faces. Behav. Brain Res. 15, 159–176 (1985).
    1. Gothard K. M., Battaglia F. P., Erickson C. A., Spitler K. M. & Amaral D. G. Neural responses to facial expression and face identity in the monkey amygdala. J. Neurophysiol. 97, 1671–1683 (2007).
    1. Hoffman K. L., Gothard K. M., Schmid M. C. & Logothetis N. K. Facial-expression and gaze-selective responses in the monkey amygdala. Curr. Biol. 17, 766–772 (2007).
    1. Mosher C. P., Zimmerman P. E. & Gothard K. M. Neurons in the monkey amygdala detect eye contact during naturalistic social interactions. Curr. Biol. 24, 2459–2464 (2014).
    1. Mattavelli G. et al.. Neural responses to facial expressions support the role of the amygdala in processing threat. Soc. Cogn. Affect. Neurosci. 9, 1684–1689 (2014).
    1. Terburg D. et al.. Hypervigilance for fear after basolateral amygdala damage in humans. Transl. Psychiatry 2, e115 (2012).
    1. van Honk J., Terburg D., Thornton H., Stein D. J. & Morgan B. in Living Without an Amygdala eds Amaral D. G., Adolphs R. 334–363Guilford Press (2016).
    1. Whalen P. J. Fear, vigilance, and ambiguity: initial neuroimaging studies of the human amygdala. Curr. Dir. Psychol. Sci. 7, 177–188 (1998).
    1. Adams R. B., Gordon H. L., Baird A. A., Ambady N. & Kleck R. E. Effects of gaze on amygdala sensitivity to anger and fear faces. Science 300, 1536 (2003).
    1. Roesch M. R., Calu D. J., Esber G. R. & Schoenbaum G. Neural correlates of variations in event processing during learning in basolateral amygdala. J. Neurosci. 30, 2464–2471 (2010).
    1. Freeman J. B., Stolier R. M., Ingbretsen Z. A. & Hehman E. A. Amygdala responsivity to high-level social information from unseen faces. J. Neurosci. 34, 10573–10581 (2014).
    1. Said C. P., Baron S. G. & Todorov A. Nonlinear amygdala response to face trustworthiness: contributions of high and low spatial frequency information. J. Cogn. Neurosci. 21, 519–528 (2009).
    1. Todorov A., Baron S. G. & Oosterhof N. N. Evaluating face trustworthiness: a model based approach. Soc. Cogn. Affect. Neurosci. 3, 119–127 (2008).
    1. Said C. P., Dotsch R. & Todorov A. The amygdala and FFA track both social and non-social face dimensions. Neuropsychologia 48, 3596–3605 (2010).
    1. Costafreda S. G., Brammer M. J., David A. S. & Fu C. H. Y. Predictors of amygdala activation during the processing of emotional stimuli: a meta-analysis of 385 PET and fMRI studies. Brain Res. Rev. 58, 57–70 (2008).
    1. Adolphs R. in Living Without an Amygdala eds Amaral D. G., Adolphs R. 276–305Guilford Press (2016).
    1. Bijanki K. R. et al.. Case report: stimulation of the right amygdala induces transient changes in affective bias. Brain Stimul. 7, 690–693 (2014).
    1. Mormann F. et al.. Latency and selectivity of single neurons indicate hierarchical processing in the human medial temporal lobe. J. Neurosci. 28, 8865–8872 (2008).
    1. Wang S. et al.. Neurons in the human amygdala selective for perceived emotion. Proc. Natl Acad. Sci. 111, E3110–E3119 (2014).
    1. Mormann F. et al.. Neurons in the human amygdala encode face identity, but not gaze direction. Nat. Neurosci. 18, 1568–1570 (2015).
    1. Tyszka J. M. & Pauli W. M. In vivo delineation of subdivisions of the human amygdaloid complex in a high-resolution group template. Hum. Brain Mapp. 37, 3979–3998 (2016).
    1. Maier A. et al.. Divergence of fMRI and neural signals in V1 during perceptual suppression in the awake monkey. Nat. Neurosci. 11, 1193–1200 (2008).
    1. Sterzer P. & Kleinschmidt A. A neural basis for inference in perceptual ambiguity. Proc. Natl Acad. Sci. 104, 323–328 (2007).
    1. Hsu M., Bhatt M., Adolphs R., Tranel D. & Camerer C. F. Neural systems responding to degrees of uncertainty in human decision-making. Science 310, 1680–1683 (2005).
    1. Herry C. et al.. Processing of temporal unpredictability in human and animal amygdala. J. Neurosci. 27, 5958–5966 (2007).
    1. Buchanan T. W., Tranel D., Adolphs R. in The Human Amygdala eds Whalen P. W., Phelps L. 289–320Oxford University Press (2009).
    1. LeDoux J. The amygdala. Curr. Biol. 17, R868–R874 (2007).
    1. Aggleton J. P., Burton M. J. & Passingham R. E. Cortical and subcortical afferents to the amygdala of the rhesus monkey (Macaca mulatta). Brain Res. 190, 347–368 (1980).
    1. Davis M. The role of the amygdala in fear and anxiety. Annu. Rev. Neurosci. 15, 353–375 (1992).
    1. Fudge J. L. & Tucker T. Amygdala projections to central amygdaloid nucleus subdivisions and transition zones in the primate. Neuroscience 159, 819–841 (2009).
    1. Phelps E. A. & LeDoux J. E. Contributions of the amygdala to emotion processing: from animal models to human behavior. Neuron 48, 175–187 (2005).
    1. Haubensak W. et al.. Genetic dissection of an amygdala microcircuit that gates conditioned fear. Nature 468, 270–276 (2010).
    1. Tye K. M. et al.. Amygdala circuitry mediating reversible and bidirectional control of anxiety. Nature 471, 358–362 (2011).
    1. Mormann F. et al.. A category-specific response to animals in the right human amygdala. Nat. Neurosci. 14, 1247–1249 (2011).
    1. Ji G. & Neugebauer V. Hemispheric lateralization of pain processing by amygdala neurons. J. Neurophysiol. 102, 2253–2264 (2009).
    1. Hardee J. E., Thompson J. C. & Puce A. The left amygdala knows fear: laterality in the amygdala response to fearful eyes. Soc. Cogn. Affect. Neurosci. 3, 47–54 (2008).
    1. Berl M. M. et al.. Seizure focus affects regional language networks assessed by fMRI. Neurology 65, 1604–1611 (2005).
    1. Hofer P. A. Urbach-Wiethe disease (lipoglycoproteinosis; lipoid proteinosis; hyalinosis cutis et mucosae). A review. Acta Derm. Venereol. Suppl. (Stockh.) 53, 1–52 (1973).
    1. Becker B. et al.. Fear processing and social networking in the absence of a functional amygdala. Biol. Psychiatry 72, 70–77 (2012).
    1. Smith M. L., Cottrell G. W., Gosselin F. & Schyns P. G. Transmitting and decoding facial expressions. Psychol. Sci. 16, 184–189 (2005).
    1. Roy S. et al.. A dynamic facial expression database. J. Vis. 7, 944–944 (2007).
    1. Willenbockel V. et al.. Controlling low-level image properties: the SHINE toolbox. Behav. Res. Methods 42, 671–684 (2010).
    1. Burnham K. P. & Anderson D. R. Model Selection and Multimodel Inference: A Practical Information-Theoretic Approach 2nd edn Springer-Verlag (2002).
    1. Rutishauser U. et al.. Representation of retrieval confidence by single neurons in the human medial temporal lobe. Nat. Neurosci. 18, 1041–1050 (2015).
    1. Olejnik S. & Algina J. Generalized eta and omega squared statistics: measures of effect size for some common research designs. Psychol. Methods 8, 434–447 (2003).
    1. Hentschke H. & Stuttgen M. C. Computation of measures of effect size for neuroscience data sets. Eur. J. Neurosci. 34, 1887–1894 (2011).
    1. Rutishauser U., Schuman E. M. & Mamelak A. N. Online detection and sorting of extracellularly recorded action potentials in human medial temporal lobe recordings, in vivo. J. Neurosci. Methods 154, 204–224 (2006).
    1. Jenkinson M. & Smith S. A global optimisation method for robust affine registration of brain images. Med. Image Anal. 5, 143–156 (2001).
    1. Bookstein F. L. Principal warps: thin-plate splines and the decomposition of deformations. IEEE Trans. Pattern Anal. Mach. Intel. 11, 567–585 (1989).
    1. Oya H., Kawasaki H., Dahdaleh N. S., Wemmie J. A. & Howard Iii M. A. Stereotactic atlas-based depth electrode localization in the human amygdala. Stereotact. Funct. Neurosurg. 87, 219–228 (2009).
    1. Yushkevich P. A. et al.. User-guided 3D active contour segmentation of anatomical structures: significantly improved efficiency and reliability. Neuroimage 31, 1116–1128 (2006).
    1. Hariri A. R. et al.. Serotonin transporter genetic variation and the response of the human amygdala. Science 297, 400–403 (2002).
    1. Penny W. & Holmes A. in Human Brain Function eds Frackowiak R. S. al.. 843–850Elsevier (2004).
    1. Tzourio-Mazoyer N. et al.. Automated anatomical labeling of activations in SPM using a macroscopic anatomical parcellation of the MNI MRI single-subject brain. Neuroimage 15, 273–289 (2002).

Source: PubMed

Patrocinadores e Colaboradores

Condições médicas

Intervenções de drogas

3
Se inscrever