Development and validation of the AzBio sentence lists

Abstract

Objectives: The goal of this study was to create and validate a new set of sentence lists that could be used to evaluate the speech perception abilities of hearing-impaired listeners and cochlear implant (CI) users. Our intention was to generate a large number of sentence lists with an equivalent level of difficulty for the evaluation of performance over time and across conditions.

Design: The AzBio sentence corpus includes 1000 sentences recorded from two female and two male talkers. The mean intelligibility of each sentence was estimated by processing each sentence through a five-channel CI simulation and calculating the mean percent correct score achieved by 15 normal-hearing listeners. Sentences from each talker were sorted by percent correct score, and 165 sentences were selected from each talker and were then sequentially assigned to 33 lists, each containing 20 sentences (5 sentences from each talker). List equivalency was validated by presenting all lists, in random order, to 15 CI users.

Results: Using sentence scores from the CI simulation study produced 33 lists of sentences with a mean score of 85% correct. The results of the validation study with CI users revealed no significant differences in percent correct scores for 29 of the 33 sentence lists. However, individual listeners demonstrated considerable variability in performance on the 29 lists. The binomial distribution model was used to account for the inherent variability observed in the lists. This model was also used to generate 95% confidence intervals for one and two list comparisons. A retrospective analysis of 172 instances where research subjects had been tested on two lists within a single condition revealed that 94% of results were accurately contained within these confidence intervals.

Conclusions: The use of a five-channel CI simulation to estimate the intelligibility of individual sentences allowed for the creation of a large number of sentence lists with an equivalent level of difficulty. The results of the validation procedure with CI users found that 29 of 33 lists allowed scores that were not statistically different. However, individual listeners demonstrated considerable variability in performance across lists. This variability was accurately described by the binomial distribution model and was used to estimate the magnitude of change required to achieve statistical significance when comparing scores from one and two lists per condition. Fifteen sentence lists have been included in the AzBio Sentence Test for use in the clinical evaluation of hearing-impaired listeners and CI users. An additional eight sentence lists have been included in the Minimum Speech Test Battery to be distributed by the CI manufacturers for the evaluation of CI candidates.

Figures

Figure 1

Intelligibility estimates of the 33 sentence lists. Symbols represent the mean percent correct score of a single sentence presented to 15 normal-hearing subjects listening to a 5-channel cochlear implant simulation. The mean percent correct score for each list is indicated by a horizontal bar.

Figure 2

List scores for 15 CI listeners. The absolute percent correct score for each of the 33 tested lists is shown as a closed circle. The mean level of performance for each listener is indicated by a horizontal line.

Figure 3

Normalized scores for 15 CI listeners on all 33 sentence lists. Symbols represent an individual listener’s list score relative to their overall mean level of performance. Positive values indicate better than average performance and negative values indicate below average performance. The average normalized score for each list is shown as a horizontal bar.

Figure 4

Predicted variability of materials as a function of the mean percent correct scores. The predicted standard deviation (solid line) and 95% confidence intervals (dashed lines) are based on 29 list scores, with each list containing 40 items. Symbols represent the mean and standard deviation of scores from 15 cochlear implant listeners on 29 lists of AzBio sentences. For each subject, all lists were presented in quiet (circles) or in noise at a single signal-to-noise ratio (triangles) to prevent ceiling or floor effects.

Figure 5

Comparison of 172 instances where individual cochlear implant listeners (n=66) were tested on two lists within the same listening condition. Solid lines represent the upper and lower 95% confidence intervals for single list comparisons. Symbols represent scores on the first (A) and second (B) list tested within the same condition. Scores falling outside of the 95% confidence interval would be incorrectly labeled as significantly different.

Figure 6

Normalized scores for 15 CI listeners on the 15 sentence lists included in the AzBio Sentence Test. Symbols represent an individual listener’s list score relative to their mean level of performance on all 15 lists. Positive values indicate better than average performance and negative values indicate below average performance. The average normalized score for each list is shown as a horizontal bar.

Figure 7

Normalized scores for 15 CI listeners on the 8 sentence lists included in the Minimum Speech Test Battery. Symbols represent an individual listener’s list score relative to their own mean level of performance on the 8 lists. Positive values indicate better than average performance and negative values indicate below average performance. The average normalized score for each list is shown as a horizontal bar.