Enhancing brain-machine interface (BMI) control of a hand exoskeleton using electrooculography (EOG)

Matthias Witkowski, Mario Cortese, Marco Cempini, Jürgen Mellinger, Nicola Vitiello, Surjo R Soekadar, Matthias Witkowski, Mario Cortese, Marco Cempini, Jürgen Mellinger, Nicola Vitiello, Surjo R Soekadar

Abstract

Background: Brain-machine interfaces (BMIs) allow direct translation of electric, magnetic or metabolic brain signals into control commands of external devices such as robots, prostheses or exoskeletons. However, non-stationarity of brain signals and susceptibility to biological or environmental artifacts impede reliable control and safety of BMIs, particularly in daily life environments. Here we introduce and tested a novel hybrid brain-neural computer interaction (BNCI) system fusing electroencephalography (EEG) and electrooculography (EOG) to enhance reliability and safety of continuous hand exoskeleton-driven grasping motions.

Findings: 12 healthy volunteers (8 male, mean age 28.1 ± 3.63y) used EEG (condition #1) and hybrid EEG/EOG (condition #2) signals to control a hand exoskeleton. Motor imagery-related brain activity was translated into exoskeleton-driven hand closing motions. Unintended motions could be interrupted by eye movement-related EOG signals. In order to evaluate BNCI control and safety, participants were instructed to follow a visual cue indicating either to move or not to move the hand exoskeleton in a random order. Movements exceeding 25% of a full grasping motion when the device was not supposed to be moved were defined as safety violation. While participants reached comparable control under both conditions, safety was frequently violated under condition #1 (EEG), but not under condition #2 (EEG/EOG).

Conclusion: EEG/EOG biosignal fusion can substantially enhance safety of assistive BNCI systems improving their applicability in daily life environments.

Figures

Figure 1
Figure 1
Hand exoskeleton for grasping motions. The illustrated device was developed by The BioRobotics Institute (Scuola Superiore Sant’Anna, Pisa, Italy) to perform opening and closing motions of a hand [2]. A) full opening position. B) full closing position.
Figure 2
Figure 2
Biosignals recorded by electroencephalography (EEG) and electrooculography (EOG) were used to control a hand-exoskeleton allowing for grasping motions. While under condition #1 only EEG signals were used for hand exoskeleton control, both EEG and EOG signals were used during condition #2.
Figure 3
Figure 3
Experimental design: after calibration, all participants controlled the BNCI system under two conditions. During condition #1, EEG was used, while during condition #2 merged EEG and EOG signals were used for BNCI control of the hand exoskeleton. During EEG calibration, either a red square (indicating to rest) or green square (indicating to engage in motor-imagery) was shown. For EOG calibration, participants were asked to either look to the left (blue arrow to the left) or to the right (blue arrow to the right). For evaluation of BNCI control, a visual cue indicated not to move (red square) or to close the hand exoskeleton (green square) over 6 minutes in a random order. Visual indications were separated by inter-trial-intervals (ITIs) of 4-6 sec.
Figure 4
Figure 4
Illustration of event-related desynchronization (ERD) of electroencephalographic (EEG) sensorimotor rhythm activity (SMR, 8-15Hz) related to motor imagery of hand closing motions in a representative participant during calibration. The red line indicates ERD during the instruction to rest (red square presentations), while the black line indicates ERD during the instruction to imagine hand-closing motions (green square presentations). ERD was calculated relative to a reference period at -1.5 to -0.5 s before the visual cue. The 95% confidence levels are shown as red and green areas, respectively. The discrimination threshold for detection of motor imagery-related ERD for BNCI control is indicated as red dotted line.
Figure 5
Figure 5
Hand exoskeleton-closing motions in % relative to a full closing motion during EEG control (condition #1, left side) and hybrid EEG/EOG BNCI control (condition #2, right side) averaged across all participants while green or red squares were presented. Participants were instructed to close the hand exoskeleton during green square presentations (black circles/crosses), and not to move during red square presentations (red circles/crosses). All participants were able to successfully close the device and reached successful control during green square presentations. However, during condition #1, the safety threshold (set at 25% closing motions during red square presentations) was often exceeded, but only once under condition #2.

References

    1. Millán JDR, Rupp R, Müller-Putz GR, Murray-Smith R, Giugliemma C, Tangermann M, Vidaurre C, Cincotti F, Kübler A, Leeb R, Neuper C, Müller KR, Mattia D. Combining brain-computer interfaces and assistive technologies: state-of-the-Art and challenges. Front Neurosci. 2010;4:161.
    1. Cempini M, Cortese M, Vitiello N. A powered finger-thumb wearable hand exoskeleton with self-aligning joint axes. IEEE/ASME Transact Mechatron. 2014;20:705–716. doi: 10.1109/TMECH.2014.2315528.
    1. de Almeida Ribeiro PR, Brasil FL, Witkowski M, Shiman F, Cipriani C, Vitiello N, Carrozza MC, Soekadar SR. Controlling assistive machines in paralysis using brain waves and other biosignals. Adv Hum-Comput Interact. 2013;2013(2):1–9. doi: 10.1155/2013/369425.
    1. Lu CW, Patil PG, Chestek CA. Current challenges to the clinical translation of brain machine interface technology. Int Rev Neurobiol. 2011;107:137–160. doi: 10.1016/B978-0-12-404706-8.00008-5.
    1. Blankertz B, Dornhege G, Schäfer C, Krepki R, Kohlmorgen J, Müller KR, Kunzmann V, Losch F, Curio G. Boosting bit rates and error detection for the classification of fast-paced motor commands based on single-trial EEG analysis. IEEE Trans Neural Syst Rehabil Eng. 2003;11:127–131. doi: 10.1109/TNSRE.2003.814456.
    1. McFarland DJ, Anderson CW, Müller KR, Schlögl A, Krusienski DJ. BCI Meeting 2005--workshop on BCI signal processing: feature extraction and translation. IEEE Trans Neural Syst Rehabil Eng. 2006;14(2):135–138. doi: 10.1109/TNSRE.2006.875637.
    1. Pfurtscheller G, Solis-Escalante T, Ortner R, Linortner P, Müller-Putz GR. Self-paced operation of an SSVEP-Based orthosis with and without an imagery-based “brain switch:” a feasibility study towards a hybrid BCI. IEEE Trans Neural Syst Rehabil Eng. 2010;18:409–414. doi: 10.1109/TNSRE.2010.2040837.
    1. Yong X, Fatourechi M, Ward RK, Birch GE. The design of a point-and-click system by integrating a self-paced brain–computer interface with an Eye-tracker. IEEE J Emerg Sel Top Circ Syst. 2011;4:590–602. doi: 10.1109/JETCAS.2011.2175589.
    1. Punsawad Y, Wongsawat Y, Parnichkun M. Hybrid EEG-EOG brain-computer interface system for practical machine control. Conf Proc IEEE Eng Med Biol Soc. 2010;2010:1360–1363.
    1. Wang H, Li Y, Long J, Yu T, Gu Z. An asynchronous wheelchair control by hybrid EEG–EOG brain–computer interface. Cognit Neurodynamics. 2014;8(5):399–409. doi: 10.1007/s11571-014-9296-y.
    1. Oldfield RC. The assessment and analysis of handedness: the Edinburgh inventory. Neuropsychologia. 1971;9(1):97–113. doi: 10.1016/0028-3932(71)90067-4.
    1. Schalk G. Effective brain-computer interfacing using BCI2000. Conf Proc IEEE Eng Med Biol Soc. 2009;2009:5498–5501.
    1. Pfurtscheller G, Aranibar A. Evaluation of event-related desynchronization (ERD) preceding and following voluntary self-paced movement. Electroencephalogr Clin Neurophysiol. 1979;46(2):138–146. doi: 10.1016/0013-4694(79)90063-4.
    1. Soekadar SR, Witkowski M, Mellinger J, Ramos A, Birbaumer N, Cohen LG. ERD-based online brain-machine interfaces (BMI) in the context of neurorehabilitation: optimizing BMI learning and performance. IEEE Trans Neural Syst Rehabil Eng. 2011;19(5):542–549. doi: 10.1109/TNSRE.2011.2166809.
    1. Sakurada T, Kawase T, Takano K, Komatsu T, Kansaku K. A BMI-based occupational therapy assist suit: asynchronous control by SSVEP. Front Neurosci. 2013;7:172. doi: 10.3389/fnins.2013.00172.
    1. Soekadar SR, Witkowski M, Birbaumer N, Cohen LG. Cereb Cortex. 2014. Enhancing Hebbian learning to control brain oscillatory activity.
    1. Barachant A, Bonnet S, Congedo M, Jutten C. Multiclass brain-computer interface classification by Riemannian geometry. IEEE Trans Biomed Eng. 2012;59(4):920–928. doi: 10.1109/TBME.2011.2172210.
    1. Delgado Saa JF, Çetin M. Discriminative methods for classification of asynchronous imaginary motor tasks from EEG data. IEEE Trans Neural Syst Rehabil Eng. 2013;21(5):716–724. doi: 10.1109/TNSRE.2013.2268194.

Source: PubMed

Sponsorit ja yhteistyökumppanit

Sairaudet

Huumeiden interventiot

3
Tilaa