Listeners' perceptions of the certainty and honesty of a speaker are associated with a common prosodic signature
Louise Goupil, Emmanuel Ponsot, Daniel Richardson, Gabriel Reyes, Jean-Julien Aucouturier, Louise Goupil, Emmanuel Ponsot, Daniel Richardson, Gabriel Reyes, Jean-Julien Aucouturier
Abstract
The success of human cooperation crucially depends on mechanisms enabling individuals to detect unreliability in their conspecifics. Yet, how such epistemic vigilance is achieved from naturalistic sensory inputs remains unclear. Here we show that listeners' perceptions of the certainty and honesty of other speakers from their speech are based on a common prosodic signature. Using a data-driven method, we separately decode the prosodic features driving listeners' perceptions of a speaker's certainty and honesty across pitch, duration and loudness. We find that these two kinds of judgments rely on a common prosodic signature that is perceived independently from individuals' conceptual knowledge and native language. Finally, we show that listeners extract this prosodic signature automatically, and that this impacts the way they memorize spoken words. These findings shed light on a unique auditory adaptation that enables human listeners to quickly detect and react to unreliability during linguistic interactions.
Conflict of interest statement
The authors declare no competing interests.
Figures
References
- Sperber D, et al. Epistemic vigilance. Mind Lang. 2010;25:359–393. doi: 10.1111/j.1468-0017.2010.01394.x.
- Vrij, A., Hartwig, M. & Granhag, P. A. Reading lies: nonverbal communication and deception. Annu. Rev. Psychol. 70, 295–317 (2019).
- ten Brinke L, Vohs KD, Carney DR. Can ordinary people detect deception after all? Trends Cogn. Sci. 2016;20:579–588. doi: 10.1016/j.tics.2016.05.012.
- Bahrami B, et al. Optimally interacting minds. Science. 2010;329:1081–1085. doi: 10.1126/science.1185718.
- Poulin-Dubois D, Brosseau-Liard P. The developmental origins of selective social learning. Curr. Dir. Psychol. Sci. 2016;25:60–64. doi: 10.1177/0963721415613962.
- Mercier, H. Not Born Yesterday (Princeton University Press, 2020).
- de Haan, F. The relation between modality and evidentiality. Linguist. Berichte9, 201–216 (2001).
- Fusaroli R, et al. Coming to terms. Psychol. Sci. 2012;23:931–939. doi: 10.1177/0956797612436816.
- Roseano, P., González, M., Borràs-Comes, J. & Prieto, P. Communicating epistemic stance: how speech and gesture patterns reflect epistemicity and evidentiality. Discourse Process. 53, 135–174 (2016).
- Bang D, et al. Confidence matching in group decision-making. Nat. Hum. Behav. 2017;1:1–7. doi: 10.1038/s41562-017-0117.
- Goupil, L. & Kouider, S. Developing a reflective mind: from core metacognition to explicit self-reflection. Curr. Dir. Psychol. Sci. 28, 403–408 (2019).
- Shea N, et al. Supra-personal cognitive control and metacognition. Trends Cogn. Sci. 2014;18:186–193. doi: 10.1016/j.tics.2014.01.006.
- Brennan SE, Williams M. The feeling of another′s knowing: prosody and filled pauses as cues to listeners about the metacognitive states of speakers. J. Mem. Lang. 1995;34:383–398. doi: 10.1006/jmla.1995.1017.
- Jiang X, Pell MD. The sound of confidence and doubt. Speech Commun. 2017;88:106–126. doi: 10.1016/j.specom.2017.01.011.
- Holtgraves T, Lasky B. Linguistic power and persuasion. J. Lang. Soc. Psychol. 1999;18:196–205. doi: 10.1177/0261927X99018002004.
- Van Zant AB, Berger J. How the voice persuades. J. Pers. Soc. Psychol. 2019;118:661–682. doi: 10.1037/pspi0000193.
- Jiang X, Pell MD. The feeling of another’s knowing: how ‘mixed messages’ in speech are reconciled. J. Exp. Psychol. Hum. Percept. Perform. 2016;42:1412–1428. doi: 10.1037/xhp0000240.
- Goupil, L. & Aucouturier, J.-J. Event-related prosody reveals distinct acoustic manifestations of accuracy and confidence in speech. Preprint at 10.31234/ (2019).
- Kimble CE, Seidel SD. Vocal signs of confidence. J. Nonverbal Behav. 1991;15:99–105. doi: 10.1007/BF00998265.
- Jiang X, Pell MD. Neural responses towards a speaker’s feeling of (un)knowing. Neuropsychologia. 2016;81:79–93. doi: 10.1016/j.neuropsychologia.2015.12.008.
- Dezecache G, Mercier H, Scott-Phillips TC. An evolutionary approach to emotional communication. J. Pragmat. 2013;59:221–233. doi: 10.1016/j.pragma.2013.06.007.
- Wharton, T. Pragmatics and Non-Verbal Communication (Cambridge University Press, 2009).
- Gussenhoven, C. Intonation and interpretation: phonetics and phonology. In 2002 Proc. First International Conference on Speech Prosody 47–57 (ISCA, 2002).
- Lewis, D. Convention (Harvard University Press, 1969).
- Grice, H. P. Meaning (Philosophical Review, 1957).
- Ponsot, E., Burred, J. J., Belin, P. & Aucouturier, J.-J. Cracking the social code of speech prosody using reverse correlation. Proc. Natl. Acad. Sci. USA115, 3972–3977 (2018).
- Armstrong MM, Lee AJ, Feinberg DR. A house of cards: bias in perception of body size mediates the relationship between voice pitch and perceptions of dominance. Anim. Behav. 2019;147:43–51. doi: 10.1016/j.anbehav.2018.11.005.
- Juslin, P. N., Laukka, P. & Bänziger, T. The mirror to our soul? Comparisons of spontaneous and posed vocal expression of emotion. J. Nonverbal Behav. 10.1007/s10919-017-0268-x (2018).
- Scherer, K. R. in Chinese Spoken Language Processing. ISCSLP 2006. Lecture Notes in Computer Science (eds Huo, Q., Ma, B., Chng, E. S., Li, H.) 13–14 (Springer, Berlin, 2006).
- Yap, T. F. Speech Production Under Cognitive Load: Effects and Classification. PhD thesis, The University of New South Wales (2012).
- Giddens, C. L., Barron, K. W., Byrd-Craven, J., Clark, K. F. & Winter, A. S. Vocal indices of stress: a review. J. Voice10.1016/j.jvoice.2012.12.010 (2013).
- Scherer, K. R., Grandjean, D., Johnstone, T., Klasmeyer, G. & Bänziger, T. Acoustic correlates of task load and stress. in 7th International Conference on Spoken Language Processing, ICSLP 2002 (ICSLP, 2002).
- Berthold, A. & Jameson, A. Interpreting symptoms of cognitive load in speech input. in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 235–244 (Springer Verlag, 1999).
- Le, P. N. The Use of Spectral Information in the Development of Novel Techniques for Speech-Based Cognitive Load Classification. PhD thesis, The University of New South Wales (2012).
- Ackerman R, Zalmanov H. The persistence of the fluency-confidence association in problem solving. Psychon. Bull. Rev. 2012;19:1187–1192. doi: 10.3758/s13423-012-0305-z.
- Proust, J. in Foundations of Metacognition (eds. Beran, M. J., Brandl, J. L., Perner, J. & Proust, J.) 234–251 (Oxford University Press, 2012).
- Zuckerman M, DePaulo BM, Rosenthal R. Verbal and nonverbal communication of deception. Adv. Exp. Soc. Psychol. 1981;14:1–59. doi: 10.1016/S0065-2601(08)60369-X.
- Proust, J. in Embodied Communication in Humans and Machines Ch. 15 (Oxford University Press, 2012).
- Villar G, Arciuli J, Paterson H. Vocal pitch production during lying: beliefs about deception matter. Psychiatry Psychol. Law. 2013;20:123–132. doi: 10.1080/13218719.2011.633320.
- Bond CF, DePaulo BM. Accuracy of deception judgments. Pers. Soc. Psychol. Rev. 2006;10:214–234. doi: 10.1207/s15327957pspr1003_2.
- DePaulo BM, et al. Cues to deception. Psychol. Bull. 2003;129:74–118. doi: 10.1037/0033-2909.129.1.74.
- Fish K, Rothermich K, Pell MD. The sound of (in)sincerity. J. Pragmat. 2017;121:147–161. doi: 10.1016/j.pragma.2017.10.008.
- Spence, K., Arciuli, J. & Villar, G. The role of pitch and speech rate as markers of deception in Italian speech. Frontiers in Psychology. 3, 453 (2012).
- Adolphs, R., Nummenmaa, L., Todorov, A. & Haxby, J. V. Data-driven approaches in the investigation of social perception. Philos. Trans. R. Soc. B Biol. Sci.10.1098/rstb.2015.0367 (2016).
- Jack RE, Schyns PG. Toward a social psychophysics of face communication. Annu. Rev. Psychol. 2017 doi: 10.1146/annurev-psych-010416-044242.
- Guyer JJ, Fabrigar LR, Vaughan-Johnston TI. Speech rate, intonation, and pitch: investigating the bias and cue effects of vocal confidence on persuasion. Personal. Soc. Psychol. Bull. 2018;45:389–405. doi: 10.1177/0146167218787805.
- Murray, R. F. Classification images: a review. J. Vis.10.1167/11.5.2 (2011). 10.1167/11.5.1
- Green, D. M. Consistency of auditory detection judgments. Psychol. Rev. (1964).
- Burgess, A. E. & Colborne, B. Visual signal detection. IV. Observer inconsistency. J. Opt. Soc. Am. A5, 617–627 (1988).
- Neri, P. How inherently noisy is human sensory processing? Psychon. Bull. Rev. 17, 802–808 (2010).
- Ponsot E, Arias P, Aucouturier J-J. Uncovering mental representations of smiled speech using reverse correlation. J. Acoust. Soc. Am. 2018;143:EL19–EL24. doi: 10.1121/1.5020989.
- de Gardelle V, Mamassian P. Weighting mean and variability during confidence judgments. PLoS ONE. 2015;10:e0120870. doi: 10.1371/journal.pone.0120870.
- McCurdy LY, et al. Anatomical coupling between distinct metacognitive systems for memory and visual perception. J. Neurosci. 2013;33:1897–1906. doi: 10.1523/JNEUROSCI.1890-12.2013.
- Laukka, P. & Elfenbein, H. A. Cross-cultural emotion recognition and in-group advantage in vocal expression: a meta-analysis. Emot. Rev. 10.1177/175407391989729 (2020).
- Chen, A. & Gussenhoven, C. Language-dependence in the signalling of attitude in speech. in Proc. Workshop on the Subtle Expressivity of Emotion (2003).
- Shochi, T., Rilliard, A., Aubergé, V. & Erickson, D. Intercultural perception of English, French and Japanese social affective prosody. Linguistic Insights Stud. Lang. Commun. 97, 31–60 (2009).
- Jack, R. E., Garrod, O. G. B., Yu, H., Caldara, R. & Schyns, P. G. Facial expressions of emotion are not culturally universal. Proc. Natl. Acad. Sci. USA (2012). 10.1073/pnas.1200155109
- House, D., Karlsson, A. & Svantesson, J.-O. When epistemic meaning overrides the constraints of lexical tone: a case from Kammu. in Satelite Workshop at TIE 2016 (Lund University, 2016).
- Vlassova A, Donkin C, Pearson J. Unconscious information changes decision accuracy but not confidence. Proc. Natl Acad. Sci. USA. 2014;111:16214–16218. doi: 10.1073/pnas.1403619111.
- Arnal LH, Flinker A, Kleinschmidt A, Giraud AL, Poeppel D. Human screams occupy a privileged niche in the communication soundscape. Curr. Biol. 2015;25:2051–2056. doi: 10.1016/j.cub.2015.06.043.
- Scott-Phillips, T. C. Speaking Our Minds: Why Human Communication is Different, and How Language Evolved to Make it Special (Palgrave Macmillan, 2015).
- Nencheva, M., Piazza, E. A. & Lew-Williams, C. The moment-to-moment pitch dynamics of child-directed speech shape toddlers’ attention and learning. Dev. Sci. 10.1111/desc.12997 (2020).
- Crivelli C, Fridlund AJ. Facial displays are tools for social influence. Trends Cogn. Sci. 2018;22:388–399. doi: 10.1016/j.tics.2018.02.006.
- Pleskac TJ, Busemeyer JR. Two-stage dynamic signal detection: a theory of choice, decision time, and confidence. Psychol. Rev. 2010;117:864–901. doi: 10.1037/a0019737.
- Patel D, Fleming SM, Kilner JM. Inferring subjective states through the observation of actions. Proc. R. Soc. B Biol. Sci. 2012;279:4853–4860. doi: 10.1098/rspb.2012.1847.
- Brooks JA, Freeman JB. Conceptual knowledge predicts the representational structure of facial emotion perception. Nat. Hum. Behav. 2018;2:581–591. doi: 10.1038/s41562-018-0376-6.
- Greenberg, D. M., Warrier, V., Allison, C. & Baron-Cohen, S. Testing the empathizing–systemizing theory of sex differences and the extreme male brain theory of autism in half a million people. Proc. Natl. Acad. Sci. USA115, 12152–12157 (2018).
- Capraro V. Gender differences in lying in sender-receiver games: a meta-analysis. Judgm. Decis. Mak. 2017;13:345–355.
- Burgoon, J. K., Buller, D. B., Blair, J. P. & Tilley, P. in Sex Differences and Similarities in Communication (eds Dindia, K. & Canary, D. J.) 263–280 (Lawrence Erlbaum Associates Publishers, 2006).
- Sutherland, C. A. M. et al. Individual differences in trust evaluations are shaped mostly by environments, not genes. Proc. Natl. Acad. Sci. USA117, 10218–10224 (2020).
- Mahrholz, G., Belin, P. & McAleer, P. Judgements of a speaker’s personality are correlated across differing content and stimulus type. PLoS ONE13, e0204991 (2018).
- Sumner M, Kim SK, King E, McGowan KB. The socially weighted encoding of spoken words: a dual-route approach to speech perception. Front. Psychol. 2014;4:1015. doi: 10.3389/fpsyg.2013.01015.
- Kassin SM. Paradigm shift in the study of human lie-detection: bridging the gap between science and practice. J. Appl. Res. Mem. Cogn. 2012;1:118–119. doi: 10.1016/j.jarmac.2012.04.009.
- Lazer, D. M. J. et al. The science of fake news. Science359, 1094–1096 (2018).
- Burred, J. J., Ponsot, E., Goupil, L., Liuni, M. & Aucouturier, J. J. Cleese: an open-source audio-transformation toolbox for data-driven experiments in speech and music cognition. PLoS ONE14, e0205943 (2019).
- Kuznetsova, A., Brockhoff, P. B. & Christensen, H. B. lmerTest: tests for random and fixed effects for linear mixed effect models (lmer objects of lme4 package). R package version 2.0-3 (2014).
- Fleming SM, Lau HC. How to measure metacognition. Front. Hum. Neurosci. 2014;8:1–9. doi: 10.3389/fnhum.2014.00443.
- Peirce JW. PsychoPy-Psychophysics software in Python. J. Neurosci. Methods. 2007;162:8–13. doi: 10.1016/j.jneumeth.2006.11.017.
Source: PubMed