Sensory Integration in Human Movement: A New Brain-Machine Interface Based on Gamma Band and Attention Level for Controlling a Lower-Limb Exoskeleton

Abstract

Brain-machine interfaces (BMIs) can improve the control of assistance mobility devices making its use more intuitive and natural. In the case of an exoskeleton, they can also help rehabilitation therapies due to the reinforcement of neuro-plasticity through repetitive motor actions and cognitive engagement of the subject. Therefore, the cognitive implication of the user is a key aspect in BMI applications, and it is important to assure that the mental task correlates with the actual motor action. However, the process of walking is usually an autonomous mental task that requires a minimal conscious effort. Consequently, a brain-machine interface focused on the attention to gait could facilitate sensory integration in individuals with neurological impairment through the analysis of voluntary gait will and its repetitive use. This way the combined use of BMI+exoskeleton turns from assistance to restoration. This paper presents a new brain-machine interface based on the decoding of gamma band activity and attention level during motor imagery mental tasks. This work also shows a case study tested in able-bodied subjects prior to a future clinical study, demonstrating that a BMI based on gamma band and attention-level paradigm allows real-time closed-loop control of a Rex exoskeleton.

Keywords: EEG; Stockwell Transform; brain-machine interface; gamma band; human movement; lower-limb exoskeleton; motor imagery; sensory integration.

Copyright © 2020 Ortiz, Ferrero, Iáñez, Azorín and Contreras-Vidal.

Figures

Figure 1

Electrode configuration for the experiments. Sixty of the electrodes were used for EEG recording. Four of the electrodes of the first bundle were used for assessing eye artifacts. Ground and reference were positioned on left and right ear, respectively.

Figure 2

Structure of a session with only opened-loop control per paradigm of control. The trials were registered and computed for a determined paradigm of control (MI or MI+att) in groups of five trials. Each session consisted of 10 training trials, which were recalculated for the other paradigm of control in a pseudo-analysis (1 − 5 and 6 − 10). The model used for testing each trial in opened-loop included the previous n-1 trials up to a maximum of four trials.

Figure 3

Structure of a session with closed-loop control per paradigm of control. The subject performed the whole diagram once per each paradigm of control (MI and MI+att). Each paradigm consisted of five training and five tests trials, so each session consisted of 20 trials. The test trials were tested with the specific trained model.

Figure 4

Times of the mental events during the experimental trials. Both trials included a previous time of 15 s followed by a 10 s rest period. In the case of training trials, this period was followed by walk/mathematical count/stop events. Test trials for testing did not include the mathematical count event and had an extended final stop event to allow the Rex to stop. As first and last steps of the exoskeleton had a variable time, extra windows of time of 5 s for the start and 4 s for the stop were considered.

Figure 5

Experimental protocol for the training trials. Each event started/finished with an acoustic cue (red/blue/green arrows for start, stop, counting events). The figure also shows the Rex status during the trial (Standing/transition step to walk or stop/Normal walking). First 15 s were not considered for analysis and were used to allow the artifact removal algorithm to converge. As it can be seen there is a hardware lag in the status of the Rex since a start or stop cue is issued and the Rex changes its status.

Figure 6

Scheme of the full processing of an epoch since it is acquired and an output command decision is interpreted. The processing is carried out in the same way for the pseudo and the online analysis in real time. The only difference is that during pseudo-online analysis the command decision is not sent to the exoskeleton. Attention paradigm part is in green while MI paradigm is in beige.

Figure 7

Output information of the fifth opened-loop training trial of subject S1. Up image shows the output of the MI paradigm features for electrode Cz. Center image shows the MI classifier output. Down image shows the attention classifier output. Mental tasks are color coded (Blue for rest, red for walk and green for count). MI and attention levels are shown for each paradigm in bars ranged from 0 to 1. A high level is considered when the bars are above 0.5 (green dotted) and a medium level when it is above 0.25 (golden dotted). Classifier outputs are shown as •, and exoskeleton commands as •. Rex command status is represented as a thicked black line, which would represent the status of the exoskeleton in the case the commands could be issued instantaneously without any hardware lag. The exoskeleton was commanded for this example using the MI+att paradigm based on the periods of high and medium MI and attention.

Figure 8

Output information of the first closed-loop test trial of subject S2. Information is presented in a similar way to Figure 7. In the upper image it is also included the actual movement status of the exoskeleton caused by the EXO commands. As it can be seen, there is a certain lag between the EXO command status and the actual movement due to the hardware (difference between the thicked black line and the thin black line in the upper image). The exoskeleton was commanded for this example using just the MI paradigm, neglecting the information provided by the attention paradigm. The combination of both paradigms would have result in a shorter movement following the rules in subsection 2.3.4 issuing the activation command in the 33 s approximately instead of the 30.5 s.

Figure 9

Example of an erroneous classification in an opened-loop trial. The image corresponds to the training trial 5 of subject S4. The poor %MI classification (52.8%) produced three FP and an Acc of 37.5%. If the attention paradigm had been considered, the three FP marked by the arrows would not have been computed. However, the exoskeleton would have been activated for a shorter period of time (between 33 and 38 s), stopping before the count event starts. This would have had a 50% Acc as the stop would have been commanded before the MI period ended. Compare Table 1 results for both paradigms of training trial 5 of S4.