Sonification as a possible stroke rehabilitation strategy

Abstract

Despite cerebral stroke being one of the main causes of acquired impairments of motor skills worldwide, well-established therapies to improve motor functions are sparse. Recently, attempts have been made to improve gross motor rehabilitation by mapping patient movements to sound, termed sonification. Sonification provides additional sensory input, supplementing impaired proprioception. However, to date no established sonification-supported rehabilitation protocol strategy exists. In order to examine and validate the effectiveness of sonification in stroke rehabilitation, we developed a computer program, termed "SonicPointer": Participants' computer mouse movements were sonified in real-time with complex tones. Tone characteristics were derived from an invisible parameter mapping, overlaid on the computer screen. The parameters were: tone pitch and tone brightness. One parameter varied along the x, the other along the y axis. The order of parameter assignment to axes was balanced in two blocks between subjects so that each participant performed under both conditions. Subjects were naive to the overlaid parameter mappings and its change between blocks. In each trial a target tone was presented and subjects were instructed to indicate its origin with respect to the overlaid parameter mappings on the screen as quickly and accurately as possible with a mouse click. Twenty-six elderly healthy participants were tested. Required time and two-dimensional accuracy were recorded. Trial duration times and learning curves were derived. We hypothesized that subjects performed in one of the two parameter-to-axis-mappings better, indicating the most natural sonification. Generally, subjects' localizing performance was better on the pitch axis as compared to the brightness axis. Furthermore, the learning curves were steepest when pitch was mapped onto the vertical and brightness onto the horizontal axis. This seems to be the optimal constellation for this two-dimensional sonification.

Keywords: auditory-motor integration; music perception; pitch perception; sonification; stroke rehabilitation; timbre perception; validation of rehabilitation method.

Figures

Figure 1

Invisible overlaid 7 × 7 matrix of the sound parameters mapped onto a plane. Condition 1, in which pitch was mapped onto the y axis and brightness onto the x axis is shown. In condition 2 the square parameter grid was rotated clockwise by 90° putting pitch onto the x axis and brightness onto the y axis.

Figure 2

Schematic timeline of one experimental block. In the left lower quadrant, the exposure phase when the sound stimulus was presented is shown. This was followed by the “finding-phase” in which the subject searched the origin of the previously heard but on the screen invisible target sound stimulus using the sonified mouse movement as feedback. Finally the subjects clicked at the supposed target position.

Figure 3

Boxplots of participant's trial duration times for condition 1. The 50 trials were binned into 5 bins of 10 trials as shown on the x axis. Participant's trial duration times vary around 5000 ms as depicted on the y axis. There is a significant decrease of participant's trial duration times over time (**p < 0.001) when comparing Bin #1 and Bin #5.

Figure 4

Boxplots of participant's trial duration times for condition 2. The 50 trials were binned into 5 bins of 10 trials as shown on the x axis. Trial duration times vary around 5000 ms as depicted on the y axis. There is no significant change over time.

Figure 5

Learning curves for condition 1. The y axis displays the mean city-block distance between participants' clicks and the target for x and y position of the mouse. For the x axis, the 50 trials of each participant within a block were binned into 5 bins of 10 trials and averaged across participants. The error bars display the lower 99 % confidence boundary below participants' mean click-to-target distance in the respective trial bin. Participants showed a significant decrease of click-to-target distance for both dimensions, i.e., pitch (red, dashed) and brightness (green, solid) in the course of the condition. **p < 0.001.

Figure 6

Learning curves for condition 2. The y axis displays the mean city-block distance between participants' clicks and the target for x and y position of the mouse. The 50 trials were binned into 5 bins of 10 trials as shown on the x axis. The error bars display the lower boundary of a 99 % confidence interval below participants mean click-to-target distance in the corresponding trial bin. Participants showed a significant decrease of click-to-target distance over time for the dimension pitch (red, dashed) but not for brightness (green, solid). **p < 0.01.