Using speech recognition to enhance the Tongue Drive System functionality in computer access

Abstract

Tongue Drive System (TDS) is a wireless tongue operated assistive technology (AT), which can enable people with severe physical disabilities to access computers and drive powered wheelchairs using their volitional tongue movements. TDS offers six discrete commands, simultaneously available to the users, for pointing and typing as a substitute for mouse and keyboard in computer access, respectively. To enhance the TDS performance in typing, we have added a microphone, an audio codec, and a wireless audio link to its readily available 3-axial magnetic sensor array, and combined it with a commercially available speech recognition software, the Dragon Naturally Speaking, which is regarded as one of the most efficient ways for text entry. Our preliminary evaluations indicate that the combined TDS and speech recognition technologies can provide end users with significantly higher performance than using each technology alone, particularly in completing tasks that require both pointing and text entry, such as web surfing.

Figures

Fig. 1

Various components of the multimodal Tongue Drive System (mTDS) prototype.

Fig. 2

Block diagram of multimodal Tongue Drive System headset

Fig. 3

Evaluation results of using the TDS, Dragon, and multimodal TDS (mTDS) to complete a web browsing task.