The Impact of Auditory Spectral Resolution on Listening Effort Revealed by Pupil Dilation

Abstract

Objectives: This study measured the impact of auditory spectral resolution on listening effort. Systematic degradation in spectral resolution was hypothesized to elicit corresponding systematic increases in pupil dilation, consistent with the notion of pupil dilation as a marker of cognitive load.

Design: Spectral resolution of sentences was varied with two different vocoders: (1) a noise-channel vocoder with a variable number of spectral channels; and (2) a vocoder designed to simulate front-end processing of a cochlear implant, including peak-picking channel selection with variable synthesis filter slopes to simulate spread of neural excitation. Pupil dilation was measured after subject-specific luminance adjustment and trial-specific baseline measures. Mixed-effects growth curve analysis was used to model pupillary responses over time.

Results: For both types of vocoder, pupil dilation grew with each successive degradation in spectral resolution. Within each condition, pupillary responses were not related to intelligibility scores, and the effect of spectral resolution on pupil dilation persisted even when only analyzing trials in which responses were 100% correct.

Conclusions: Intelligibility scores alone were not sufficient to quantify the effort required to understand speech with poor resolution. Degraded spectral resolution results in increased effort required to understand speech, even when intelligibility is at 100%. Pupillary responses were a sensitive and highly granular measurement to reveal changes in listening effort. Pupillary responses might potentially reveal the benefits of aural prostheses that are not captured by speech intelligibility performance alone as well as the disadvantages that are overcome by increased listening effort.

Figures

Figure 1

Change in pupil dilation over time as a function of spectral resolution (number of vocoder channels) in Experiment 1. Time (ms) is plotted relative to stimulus offset. The boxed area represents the time window traditionally used for aggregated analysis. Left panel: data from all trials; Right panel: data only from trials where the entire sentence was identified correctly.

Figure 2

Change in pupil dilation as a function of spectral resolution in Experiment 1. Data represent mean (bar height) and maximum (point) pupil dilation during the time window beginning 500 ms before and ending 2000 ms after stimulus offset (see Figure 1). Error bars represent +/− 1 standard error.

Figure 3

Model fits (open lines) overlaid on aggregated data (points with +/− standard error lines) representing change in pupil diameter over time in Experiment 1. Time (ms) is plotted relative to stimulus offset.

Figure 4

Maximum pupil dilation (x-axis) and word intelligibility (y-axis) for each condition in Experiment 1. Regression lines are dashed when extrapolated beyond the observed data range.

Figure 5

Change in pupil dilation over time as a function of spectral resolution (vocoder carrier filter slope) in Experiment 2. Time (ms) is plotted relative to stimulus offset. The boxed area represents the time window traditionally used for aggregated analysis. Left panel: data from all trials; Right panel: data only from trials where the entire sentence was identified correctly.

Figure 6

Change in pupil dilation as a function of spectral resolution in Experiment 2. Data represent mean (bar height) and maximum (point) pupil dilation during the time window beginning 500 ms before and ending 2000 ms after stimulus offset (see Figure 5). Error bars represent +/− 1 standard error.

Figure 7

Model fits (open lines) overlaid on aggregated data (points with +/− standard error lines) representing change in pupil diameter over time in Experiment 2. Time (ms) is plotted relative to stimulus offset.

Figure 8

Maximum pupil dilation (x-axis) and word intelligibility (y-axis) for each condition in Experiment 2. Regression lines are dashed when extrapolated beyond the observed data range.

Figure 9

Maximum pupil size elicited by various tasks. The first 2 columns show data from the present study. Digit span memory data aggregated by Beatty (1982) are from Ahern (1978), Kahneman & Beatty (1966), Kahneman et al (1968), and Peavler (1974). Masked sentence intelligibility scores for young normal hearing (YNH) listeners for 50, 71 and 84% are from Zekveld et al. (2010), and 29% is from Zekveld et al. (2013). Data for different masker types to elicit 50% sentence recognition thresholds (SRT) are from Koelewijn et al. (2012). Data for mental multiplication are from Ahern and Beatty (1979). Data for pain are from Höfle et al. (2008).