Towards Rehabilitation Robotics: Off-the-Shelf BCI Control of Anthropomorphic Robotic Arms

Alkinoos Athanasiou, Ioannis Xygonakis, Niki Pandria, Panagiotis Kartsidis, George Arfaras, Kyriaki Rafailia Kavazidi, Nicolas Foroglou, Alexander Astaras, Panagiotis D Bamidis, Alkinoos Athanasiou, Ioannis Xygonakis, Niki Pandria, Panagiotis Kartsidis, George Arfaras, Kyriaki Rafailia Kavazidi, Nicolas Foroglou, Alexander Astaras, Panagiotis D Bamidis

Abstract

Advances in neural interfaces have demonstrated remarkable results in the direction of replacing and restoring lost sensorimotor function in human patients. Noninvasive brain-computer interfaces (BCIs) are popular due to considerable advantages including simplicity, safety, and low cost, while recent advances aim at improving past technological and neurophysiological limitations. Taking into account the neurophysiological alterations of disabled individuals, investigating brain connectivity features for implementation of BCI control holds special importance. Off-the-shelf BCI systems are based on fast, reproducible detection of mental activity and can be implemented in neurorobotic applications. Moreover, social Human-Robot Interaction (HRI) is increasingly important in rehabilitation robotics development. In this paper, we present our progress and goals towards developing off-the-shelf BCI-controlled anthropomorphic robotic arms for assistive technologies and rehabilitation applications. We account for robotics development, BCI implementation, and qualitative assessment of HRI characteristics of the system. Furthermore, we present two illustrative experimental applications of the BCI-controlled arms, a study of motor imagery modalities on healthy individuals' BCI performance, and a pilot investigation on spinal cord injured patients' BCI control and brain connectivity. We discuss strengths and limitations of our design and propose further steps on development and neurophysiological study, including implementation of connectivity features as BCI modality.

Figures

Figure 1
Figure 1
Current generation of Mercury robotic arm: (a) the robotic arm in position during an illustrative experiment, (b) the 3D-printed gripper (in focus circle), and (c) schematic of the 8 DoFs of the robotic arm. Mercury arms are house-built, of low cost, and anthropomorphic.
Figure 2
Figure 2
Schematic of Brain-Computer Interface loop: using off-the-shelf EEG-BCI for control of house-built robotic arms.
Figure 3
Figure 3
Overview of the experimental setup in the Thess-AHALL Living Lab. The figure is modified with authors' permission [46].
Figure 4
Figure 4
The training procedure of the qualitative assessment experiment.
Figure 5
Figure 5
Overview of robotic arm control trials during the qualitative assessment experiment.
Figure 6
Figure 6
28-year-old female SCI patient participating in the pilot investigation: (a) 1st part of the experiment, 128-channel EEG recording during oddball presentation of multiple limb movements (visual imagery); (b) 2nd part of the experiment, control of robotic arms using a commercial EEG-BCI headset, employing mental rehearsal of movements (kinesthetic imagery).
Figure 7
Figure 7
Regions of interest (ROIs) for connectivity analysis at the cortical level: (a) midline surface, left hemisphere, (b) top view, both hemispheres, and (c) lateral view, right hemisphere. (1): SAC, (2): S1F, (3): S1H, (4): S2, (5): CMA, (6): M1F, (7): M1H, (8): M1L, (9): SMA, (10): pSMA, (11): PMd, and (12): PMv.
Figure 8
Figure 8
KMI against VMI success scores for 24 participants above action power threshold. Most participants performed better with VMI but the difference was not statistically significant.
Figure 9
Figure 9
Performance in BCI control of patient and healthy participants in comparison to (a) their g-SCIM-III total score, (b) VVIQ score, and (c) Godspeed total score. Also (d) evaluation of the robotic arms in each separate Godspeed subcategory by participant.
Figure 10
Figure 10
Functional connectivity networks formed in alpha brainwave band during different visual motor imagery tasks performed by an SCI patient and a sex and age matched healthy control participant (connections > 60% max power displayed).

References

    1. Hochberg L. R., Bacher D., Jarosiewicz B., et al. Reach and grasp by people with tetraplegia using a neurally controlled robotic arm. Nature. 2012;485(7398):372–375. doi: 10.1038/nature11076.
    1. Collinger J. L., Wodlinger B., Downey J. E., et al. High-performance neuroprosthetic control by an individual with tetraplegia. The Lancet. 2013;381(9866):557–564. doi: 10.1016/S0140-6736(12)61816-9.
    1. Donati A. R. C., Shokur S., Morya E., et al. Long-term training with a brain-machine interface-based gait protocol induces partial neurological recovery in paraplegic patients. Scientific Reports. 2016;6(1):p. 30383. doi: 10.1038/srep30383.
    1. Alam M., Rodrigues W., Pham B. N., Thakor N. V. Brain-machine interface facilitated neurorehabilitation via spinal stimulation after spinal cord injury: Recent progress and future perspectives. Brain Research. 2016;1646:25–33. doi: 10.1016/j.brainres.2016.05.039.
    1. Daly J. J., Wolpaw J. R. Brain-computer interfaces in neurological rehabilitation. The Lancet Neurology. 2008;7(11):1032–1043. doi: 10.1016/S1474-4422(08)70223-0.
    1. Anderson K. D., Acuff M. E., Arp B. G., et al. United States (US) multi-center study to assess the validity and reliability of the Spinal Cord Independence Measure (SCIM III) Spinal Cord. 2011;49(8):880–885. doi: 10.1038/sc.2011.20.
    1. Nardone R., Höller Y., Brigo F., et al. Functional brain reorganization after spinal cord injury: Systematic review of animal and human studies. Brain Research. 2013;1504:58–73. doi: 10.1016/j.brainres.2012.12.034.
    1. Kaushal M., Oni-Orisan A., Chen G., et al. Evaluation of whole-brain resting-state functional connectivity in spinal cord injury: a large-scale network analysis using network-based statistic. Journal of Neurotrauma. 2017;34(6):1278–1282. doi: 10.1089/neu.2016.4649.
    1. De Vico Fallani F., Rodrigues F. A., Da Fontoura Costa L., et al. Multiple pathways analysis of brain functional networks from EEG signals: an application to real data. Brain Topography. 2011;23(4):344–354. doi: 10.1007/s10548-010-0152-z.
    1. Lee B., Liu C. Y., Apuzzo M. L. J. A primer on brain-machine interfaces, concepts, and technology: A key element in the future of functional neurorestoration. World Neurosurgery. 2013;79(3-4):457–471. doi: 10.1016/j.wneu.2013.01.078.
    1. Athanasiou A., Klados M. A., Foroglou N., Kavazidi K. R., Polyzoidis K., Bamidis P. D. Reorganization of brain networks after spinal cord injury: a qualitative synthesis of the literature. Front. Hum. Neurosci. Conference Abstract: SAN2016 Meeting; 2016.
    1. Lee J.-H., Ryu J., Jolesz F. A., Cho Z.-H., Yoo S.-S. Brain-machine interface via real-time fMRI: Preliminary study on thought-controlled robotic arm. Neuroscience Letters. 2009;450(1):1–6. doi: 10.1016/j.neulet.2008.11.024.
    1. Yanagisawa T., Hirata M., Saitoh Y., et al. Real-time control of a prosthetic hand using human electrocorticography signals: Technical note. Journal of Neurosurgery. 2011;114(6):1715–1722. doi: 10.3171/2011.1.JNS101421.
    1. Nicolas-Alonso L. F., Gomez-Gil J. Brain computer interfaces, a review. Sensors. 2012;12(2):1211–1279. doi: 10.3390/s120201211.
    1. Galán F., Nuttin M., Lew E., et al. A brain-actuated wheelchair: asynchronous and non-invasive Brain-computer interfaces for continuous control of robots. Clinical Neurophysiology. 2008;119(9):2159–2169. doi: 10.1016/j.clinph.2008.06.001.
    1. Meng J., Zhang S., Bekyo A., Olsoe J., Baxter B., He B. Noninvasive electroencephalogram based control of a robotic arm for reach and grasp tasks. Scientific Reports. 2016;6(1, article 38565) doi: 10.1038/srep38565.
    1. Hamedi M., Salleh S.-H., Noor A. M. Electroencephalographic motor imagery brain connectivity analysis for BCI: a review. Neural Computation. 2016;28(6):999–1041. doi: 10.1162/neco_a_00838.
    1. Benz H. L., Zhang H., Bezerianos A., et al. Connectivity analysis as a novel approach to motor decoding for prosthesis control. IEEE Transactions on Neural Systems and Rehabilitation Engineering. 2012;20(2):143–152. doi: 10.1109/TNSRE.2011.2175309.
    1. Zhang H., Chavarriaga R., Millán J. D. R. Towards Implementation of motor imagery using brain connectivity features. 2014:1–4.
    1. Ranky G. N., Adamovich S. Analysis of a commercial EEG device for the control of a robot arm. Proceedings of the 2010 IEEE 36th Annual Northeast Bioengineering Conference (NEBEC); 2010; New York, NY, USA. pp. 1–2.
    1. Astaras A., Moustakas N., Athanasiou A., Gogoussis A. Towards brain-computer interface control of a 6-degree-of-freedom robotic arm using dry EEG electrodes. Advances in Human-Computer Interaction. 2013;2013:6. doi: 10.1155/2013/641074.641074
    1. Xu L., Zhang H., Hui M., et al. Motor execution and motor imagery: A comparison of functional connectivity patterns based on graph theory. Neuroscience. 2014;261:184–194. doi: 10.1016/j.neuroscience.2013.12.005.
    1. Hétu S., Grégoire M., Saimpont A., et al. The neural network of motor imagery: an ALE meta-analysis. Neuroscience and Biobehavioral Reviews. 2013;37(5):930–949. doi: 10.1016/j.neubiorev.2013.03.017.
    1. Guillot A., Collet C., Nguyen V. A., Malouin F., Richards C., Doyon J. Brain activity during visual versus kinesthetic imagery: an fMRI study. Human Brain Mapping. 2009;30(7):2157–2172. doi: 10.1002/hbm.20658.
    1. Saimpont A., Lafleur M. F., Malouin F., Richards C. L., Doyon J., Jackson P. L. The comparison between motor imagery and verbal rehearsal on the learning of sequential movements. Frontiers in Human Neuroscience. 2013;7 doi: 10.3389/fnhum.2013.00773.
    1. Mizuguchi N., Nakamura M., Kanosue K. Task-dependent engagements of the primary visual cortex during kinesthetic and visual motor imagery. Neuroscience Letters. 2017;636:108–112. doi: 10.1016/j.neulet.2016.10.064.
    1. Harwin W. S., Rahman T., Foulds R. A. A Review of design issues in rehabilitation robotics with reference to north american research. IEEE Transactions on Rehabilitation Engineering. 1995;3(1):3–13. doi: 10.1109/86.372887.
    1. Kubben P., Pouratian N. An open-source and cross-platform framework for brain computer interface-guided robotic arm control. Surgical Neurology International. 2012;3(1):p. 149. doi: 10.4103/2152-7806.104743.
    1. Chang P. H., Park H. S. Development of a robotic arm for handicapped people: A task-oriented design approach. Autonomous Robots. 2003;15(1):81–92. doi: 10.1023/A:1024449018826.
    1. Sharkey N. The programmable robot of ancient Greece. 2007,
    1. Nageotte F., Zanne P., Doignon C., De Mathelin M. Stitching planning in laparoscopic surgery: towards robot-assisted suturing. International Journal of Robotics Research. 2009;28(10):1303–1321. doi: 10.1177/0278364909101786.
    1. Kassahun Y., Yu B., Tibebu A. T., et al. Surgical robotics beyond enhanced dexterity instrumentation: a survey of machine learning techniques and their role in intelligent and autonomous surgical actions. International Journal of Computer Assisted Radiology and Surgery. 2016;11(4):553–568. doi: 10.1007/s11548-015-1305-z.
    1. Siqueira-Batista R., Souza C. R., Maia P. M., Siqueira S. L. Robotic surgery: bioethical aspects. ABCD. Arquivos Brasileiros de Cirurgia Digestiva (São Paulo) 2016;29(4):287–290.
    1. Fong T., Nourbakhsh I., Dautenhahn K. A survey of socially interactive robots. Robotics and Autonomous Systems. 2003;42(3-4):143–166. doi: 10.1016/S0921-8890(02)00372-X.
    1. Dautenhahn K. Methodology & themes of human-robot interaction: a growing research field. International Journal of Advanced Robotic Systems. 2007;4(1):103–108.
    1. Bartneck C., Kulić D., Croft E., Zoghbi S. Measurement instruments for the anthropomorphism, animacy, likeability, perceived intelligence, and perceived safety of robots. International Journal of Social Robotics. 2009;1(1):71–81. doi: 10.1007/s12369-008-0001-3.
    1. Christ O., Reiner M. Perspectives and possible applications of the rubber hand and virtual hand illusion in non-invasive rehabilitation: Technological improvements and their consequences. Neuroscience and Biobehavioral Reviews. 2014;44:33–44. doi: 10.1016/j.neubiorev.2014.02.013.
    1. Kooijmans T., Kanda T., Bartneck C., Ishiguro H., Hagita N. Accelerating robot development through integral analysis of human-robot interaction. IEEE Transactions on Robotics. 2007;23(5):1001–1012. doi: 10.1109/TRO.2007.906263.
    1. Moustakas N., Athanasiou A., Kartsidis P., Bamidis P. D., Astaras A. Development and user assessment of a body-machine interface for a hybrid-controlled 6-degree of freedom robotic arm (MERCURY). Proceedings of the 13th Mediterranean Conference on Medical and Biological Engineering and Computing 2013, MEDICON 2013; September 2013; pp. 65–68.
    1. Moustakas N., Kartsidis P., Athanasiou A., Astaras A., Bamidis P. D. Development of MERCURY version 2.0 robotic arms for rehabilitation applications. Proceedings of the 8th ACM International Conference on PErvasive Technologies Related to Assistive Environments (PETRA '15); July 2015;
    1. Brainwave control of a wearable robotic arm for rehabilitation and neurophysiological study in cervical spine injury (CSI:Brainwave) .
    1. Moustakas N., Athanasiou A., Bamidis P., Kalfas A., Astaras A., Gogoussis A. A Hybrid body-machine and brain-computer interface for a 6-degree of freedom robotic arm (Mercury). Proceedings of the 5th Panhellenic Conference on Biomedical Technology Conference; 2013; p. p. 108.
    1. Kartsidis P., Moustakas N., Athanasiou A., Astaras A., Bamidis P. D., Carlo M. Comparative analysis of perceived psychometric characteristics : pilot study of the "Mercury" 6-degree of freedom robotic arm. Proceedings of the 7th Electrical and Computer Engineering Student Conference; 2014; pp. 179–181.
    1. Astaras A., Athanasiou A., Alexandrou A., Kartsidis P., Moustakas N., Bamidis P. D. Double-blind greek translation and online implementation of the Godspeed robotics questionnaire. Proceedings of the 6th Panhellenic Conference on Biomedical Technology Conference; 2015; p. p. 34.
    1. Perdikis S., Leeb R., Millan J. D. R. Subject-oriented training for motor imagery brain-computer interfaces. Proceedings of the 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2014; August 2014; pp. 1259–1262.
    1. Konstantinidis E. I., Bamidis P. D. Density based clustering on indoor kinect location tracking: A new way to exploit active and healthy aging living lab datasets. Proceedings of the 2015 IEEE 15th International Conference on Bioinformatics and Bioengineering (BIBE); November 2015; pp. 1–6.
    1. Konstantinidis E. I., Billis A., Bratsas C., Siountas A., Bamidis P. D. Thessaloniki active and healthy ageing living lab: The roadmap from a specific project to a living lab towards openness. Proceedings of the 9th ACM International Conference on PErvasive Technologies Related to Assistive Environments (PETRA '2016); July 2016; pp. 1–4.
    1. Konstantinidis E. I., Antoniou P. E., Bamparopoulos G., Bamidis P. D. A lightweight framework for transparent cross platform communication of controller data in ambient assisted living environments. Information Sciences. 2015;300(1):124–139. doi: 10.1016/j.ins.2014.10.070.
    1. Oostenveld R., Praamstra P. The five percent electrode system for high-resolution EEG and ERP measurements. Clinical Neurophysiology. 2001;112(4):713–719. doi: 10.1016/S1388-2457(00)00527-7.
    1. Arfaras G., Athanasiou A., Pandria N., et al. Visual versus kinesthetic motor imagery for BCI control of robotic arms (Mercury 2.0). Proceedings of the 30th IEEE International Symposium on Computer-Based Medical Systems; 2017; pp. 440–445.
    1. Shapiro S. S., Wilk M. B. An analysis of variance test for normality: Complete samples. Biometrika. 1965;52(3-4):591–611. doi: 10.1093/biomet/52.3-4.591.
    1. Öztuna D., Elhan A. H., Tüccar E. Investigation of four different normality tests in terms of type 1 error rate and power under different distributions. Turkish Journal of Medical Sciences. 2006;36(3):171–176.
    1. Athanasiou A., Arfaras G., Xygonakis I., et al. Commercial BCI Control and functional brain networks in spinal cord injury: a proof-of-concept. Proceedings of the 30th IEEE International Symposium on Computer-Based Medical Systems; 2017; pp. 262–267.
    1. Kirshblum S. C., Burns S. P., Biering-Sorensen F., et al. International standards for neurological classification of spinal cord injury (revised 2011) Journal of Spinal Cord Medicine. 2011;34(6):535–546. doi: 10.1179/204577211X13207446293695.
    1. Itzkovich M., Gelernter I., Biering-Sorensen F., et al. The Spinal Cord Independence Measure (SCIM) version III: Reliability and validity in a multi-center international study. Disability and Rehabilitation. 2007;29(24):1926–1933. doi: 10.1080/09638280601046302.
    1. Athanasiou A., Alexandrou A., Paraskevopoulos E., Foroglou N., Prassas A., Bamidis P. D. Towards a Greek adaptation of the Spinal Cord Independence Measure (SCIM). Proceedings of the 15th European Congress of Neurosurgery (EANS 14); 2015; pp. 181–184.
    1. Marks D. F. Visual imagery differences in the recall of pictures. British Journal of Psychology. 1973;64(1):17–24.
    1. Beck A. T., Steer R. A., Garbin M. G. Psychometric properties of the Beck Depression Inventory: twenty-five years of evaluation. Clinical Psychology Review. 1988;8(1):77–100. doi: 10.1016/0272-7358(88)90050-5.
    1. Giannakou M., Roussi P., Kosmides M.-E., Kiosseoglou G., Adamopoulou A., Garyfallos G. Adaptation of the beck depression inventory-II to greek population. Hellenic Journal of Psychology. 2013;10(2):120–146.
    1. Petersen W. Society and the Adolescent Self-Image. Morris Rosenberg. Princeton University Press, Princeton, N.J., 1965. xii + 326 pp. $6.50. Science. 1965;148(3671):p. 804. doi: 10.1126/science.148.3671.804.
    1. Galanou C., Galanakis M., Alexopoulos E., Darviri C. Rosenberg Self-Esteem Scale Greek Validation on Student Sample. Psychology. 2014;05(08):819–827. doi: 10.4236/psych.2014.58093.
    1. Ludwig K. A., Miriani R. M., Langhals N. B., Joseph M. D., Anderson D. J., Kipke D. R. Using a common average reference to improve cortical neuron recordings from microelectrode arrays. Journal of Neurophysiology. 2009;101(3):1679–1689. doi: 10.1152/jn.90989.2008.
    1. Tadel F., Baillet S., Mosher J. C., Pantazis D., Leahy R. M. Brainstorm: a user-friendly application for MEG/EEG analysis. Computational Intelligence and Neuroscience. 2011;2011:13. doi: 10.1155/2011/879716.879716
    1. Hallez H., Vanrumste B., Grech R., et al. Review on solving the forward problem in EEG source analysis. Journal of NeuroEngineering and Rehabilitation. 2007;4(1):p. 46. doi: 10.1186/1743-0003-4-46.
    1. Grech R., Cassar T., Muscat J., et al. Review on solving the inverse problem in EEG source analysis. Journal of NeuroEngineering and Rehabilitation. 2008;5(1):p. 25. doi: 10.1186/1743-0003-5-25.
    1. Holmes C. J., Hoge R., Collins L., Woods R., Toga A. W., Evans A. C. Enhancement of MR images using registration for signal averaging. Journal of Computer Assisted Tomography. 1998;22(2):324–333.
    1. Evans A., Collins D., Mills S., Brown E., Kelly R., Peters T. 3D statistical neuroanatomical models from 305 MRI volumes. Proceedings of the 1993 IEEE Conference Record Nuclear Science Symposium and Medical Imaging Conference; 1993; San Francisco, CA, USA. pp. 1813–1817.
    1. Gramfort A., Papadopoulo T., Olivi E., Clerc M. OpenMEEG: Opensource software for quasistatic bioelectromagnetics. BioMedical Engineering Online. 2010;9(1):p. 45. doi: 10.1186/1475-925X-9-45.
    1. Pascual-Marqui R. D. Standardized low-resolution brain electromagnetic tomography (sLORETA): technical details. Methods and Findings in Experimental and Clinical Pharmacology. 2002;24(Supplement D):5–12.
    1. Edelman B. J., Baxter B., He B. EEG source imaging enhances the decoding of complex right-hand motor imagery tasks. IEEE Transactions on Biomedical Engineering. 2016;63(1):4–14. doi: 10.1109/TBME.2015.2467312.
    1. Athanasiou A., Klados M. A., Styliadis C., Foroglou N., Polyzoidis K., Bamidis P. D. Investigating the role of alpha and beta rhythms in functional motor networks. Neuroscience. 2016 doi: 10.1016/j.neuroscience.2016.05.044.
    1. Chicharro D. On the spectral formulation of Granger causality. Biological Cybernetics. 2011;105(5-6):331–347. doi: 10.1007/s00422-011-0469-z.
    1. Greek version of the Godspeed Questionnaire Series Available. .
    1. Destephe M., Brandao M., Kishi T., Zecca M., Hashimoto K., Takanishi A. Walking in the uncanny valley: Importance of the attractiveness on the acceptance of a robot as a working partner. Frontiers in Psychology. 2015;6, article no. 204 doi: 10.3389/fpsyg.2015.00204.
    1. Weiss A., Bartneck C. Meta analysis of the usage of the Godspeed Questionnaire Series. Proceedings of the 2015 24th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN); September 2015; pp. 381–388.
    1. Giuliani M., Petrick R. P. A., Foster M. E., et al. Comparing task-based and socially intelligent behaviour in a robot bartender. Proceedings of the 2013 15th ACM International Conference on Multimodal Interaction (ICMI '13); December 2013; pp. 263–270.
    1. Dawis R. V. Scale Construction. Journal of Counseling Psychology. 1987;34(4):481–489. doi: 10.1037/0022-0167.34.4.481. doi: 10.1037//0022-0167.34.4.481.
    1. Boone H. N. J., Boone D. A. Analyzing Likert data. Journal of Extension. 2012;50(2):p. 30.
    1. Van Den Broek S. P., Reinders F., Donderwinkel M., Peters M. J. Volume conduction effects in EEG and MEG. Electroencephalography and Clinical Neurophysiology. 1998;106(6):522–534. doi: 10.1016/S0013-4694(97)00147-8.
    1. Salehi S. S. M., Moghadamfalahi M., Quivira F., Piers A., Nezamfar H., Erdogmus D. Decoding Complex Imagery Hand Gestures. 2017, .
    1. Spüler M. A high-speed brain-computer interface (BCI) using dry EEG electrodes. PLOS One. 2017 doi: 10.1371/journal.pone.0172400.

Source: PubMed

Спонсоры и соавторы

Заболевания

Медикаментозные вмешательства

Подписаться