Motor cortical activity changes during neuroprosthetic-controlled object interaction
John E Downey, Lucas Brane, Robert A Gaunt, Elizabeth C Tyler-Kabara, Michael L Boninger, Jennifer L Collinger, John E Downey, Lucas Brane, Robert A Gaunt, Elizabeth C Tyler-Kabara, Michael L Boninger, Jennifer L Collinger
Abstract
Brain-computer interface (BCI) controlled prosthetic arms are being developed to restore function to people with upper-limb paralysis. This work provides an opportunity to analyze human cortical activity during complex tasks. Previously we observed that BCI control became more difficult during interactions with objects, although we did not quantify the neural origins of this phenomena. Here, we investigated how motor cortical activity changed in the presence of an object independently of the kinematics that were being generated using intracortical recordings from two people with tetraplegia. After identifying a population-wide increase in neural firing rates that corresponded with the hand being near an object, we developed an online scaling feature in the BCI system that operated without knowledge of the task. Online scaling increased the ability of two subjects to control the robotic arm when reaching to grasp and transport objects. This work suggests that neural representations of the environment, in this case the presence of an object, are strongly and consistently represented in motor cortex but can be accounted for to improve BCI performance.
Trial registration: ClinicalTrials.gov NCT01364480 NCT01894802.
Conflict of interest statement
The authors declare that they have no competing interests.
Figures
References
- Anderson KD, Cowan RE, Horsewell J. Facilitators and barriers to spinal cord injury clinical trial participation: Multi-national perspective of people living with spinal cord injury. J. Neurotrauma. 2016;33:493–499. doi: 10.1089/neu.2015.4064.
- Collinger JL, et al. Functional priorities, assistive technology, and brain-computer interfaces after spinal cord injury. J. Rehabil. Res. Dev. 2013;50:145–160. doi: 10.1682/JRRD.2011.11.0213.
- Lo C, et al. Functional priorities in persons with spinal cord injury: Using discrete choice experiments to determine preferences. J. Neurotrauma. 2016;33:1958–1968. doi: 10.1089/neu.2016.4423.
- Collinger JL, et al. High-performance neuroprosthetic control by an individual with tetraplegia. Lancet. 2013;381:557–564. doi: 10.1016/S0140-6736(12)61816-9.
- Aflalo T, et al. Neurophysiology. Decoding motor imagery from the posterior parietal cortex of a tetraplegic human. Science. 2015;348:906–910.
- Hochberg LR, et al. Reach and grasp by people with tetraplegia using a neurally controlled robotic arm. Nature. 2012;485:372–375. doi: 10.1038/nature11076.
- Vogel J, et al. An assistive decision-and-control architecture for force-sensitive hand–arm systems driven by human–machine interfaces. Int. J. Rob. Res. 2015;34:763–780. doi: 10.1177/0278364914561535.
- Wodlinger B, et al. Ten-dimensional anthropomorphic arm control in a human brain-machine interface: Difficulties, solutions, and limitations. J. Neural. Eng. 2015;12:016011. doi: 10.1088/1741-2560/12/1/016011.
- Wang W, Chan SS, Heldman DA, Moran DW. Motor cortical representation of hand translation and rotation during reaching. J. Neurosci. 2010;30:958–962. doi: 10.1523/JNEUROSCI.3742-09.2010.
- Wang W, Chan SS, Heldman DA, Moran DW. Motor cortical representation of position and velocity during reaching. J. Neurophysiol. 2007;97:4258–4270. doi: 10.1152/jn.01180.2006.
- Paninski L, Fellows MR, Hatsopoulos NG, Donoghue JP. Spatiotemporal tuning of motor cortical neurons for hand position and velocity. J. Neurophysiol. 2004;91:515–532. doi: 10.1152/jn.00587.2002.
- Kakei, S., Hoffman, D. S. & Strick, P. L. Muscle and movement representations in the primary motor cortex. Science 285, 10.1126/science.285.5436.2136 (1999).
- Moran DW, Schwartz AB. Motor cortical activity during drawing movements: Population representation during spiral tracing. J. Neurophysiol. 1999;82:2693–2704.
- Fu, Q. G., Flament, D., Colt, J. D. & Ebner, T. J. Temporal encoding of movement kinematics in the discharge of primate primary motor and premotor neurons. J. Neurophysiol. 73 (1995).
- Schwartz AB, Kettner RE, Georgopoulos AP. Primate motor cortex and free arm movements to visual targets in three-dimensional space. I. Relations between single cell discharge and direction of movement. J. Neurosci. 1988;8:2913–2927.
- Kettner RE, Schwartz AB, Georgopoulos AP. Primate motor cortex and free arm movements to visual targets in three-dimensional space. Iii. Positional gradients and population coding of movement direction from various origins. J. Neurosci. 1988;8:2938–2947.
- Georgopoulos AP, Kettner RE, Schwartz AB. Primate motor cortex and free arm movements to visual targets in three- dimensional space. Ii. Coding of the direction of movement by a neuronal population. J. Neurosci. 1988;8:2928–2937.
- Georgopoulos AP, Schwartz AB, Kettner RE. Neuronal population coding of movement direction. Science. 1986;233:1416–1419. doi: 10.1126/science.3749885.
- Velliste M, et al. Cortical control of a prosthetic arm for self-feeding. Nature. 2008;453:1098–1101. doi: 10.1038/nature06996.
- Downey JE, et al. Blending of brain-machine interface and vision-guided autonomous robotics improves neuroprosthetic arm performance during grasping. J. Neuroeng. Rehabil. 2016;13:28. doi: 10.1186/s12984-016-0134-9.
- Rouse AG, Schieber MH. Spatiotemporal distribution of location and object effects in primary motor cortex neurons during reach-to-grasp. J. Neurosci. 2016;36:10640–10653. doi: 10.1523/JNEUROSCI.1716-16.2016.
- Hendrix CM, Mason CR, Ebner TJ. Signaling of grasp dimension and grasp force in dorsal premotor cortex and primary motor cortex neurons during reach to grasp in the monkey. J. Neurophysiol. 2009;102:132–145. doi: 10.1152/jn.00016.2009.
- Mason CR, Gomez JE, Ebner TJ. Primary motor cortex neuronal discharge during reach-to-grasp: Controlling the hand as a unit. Arch. Ital. Biol. 2002;140:229–236.
- Schaffelhofer S, Sartori M, Scherberger H, Farina D. Musculoskeletal representation of a large repertoire of hand grasping actions in primates. IEEE Trans. Neural Syst. Rehabil. Eng. 2015;23:210–220. doi: 10.1109/TNSRE.2014.2364776.
- Schaffelhofer S, Agudelo-Toro A, Scherberger H. Decoding a wide range of hand configurations from macaque motor, premotor, and parietal cortices. J. Neurosci. 2015;35:1068–1081. doi: 10.1523/JNEUROSCI.3594-14.2015.
- Menz VK, Schaffelhofer S, Scherberger H. Representation of continuous hand and arm movements in macaque areasm1, f5, and aip: A comparative decoding study. J. Neural. Eng. 2015;12:056016. doi: 10.1088/1741-2560/12/5/056016.
- Barde LH, Buxbaum LJ, Moll AD. Abnormal reliance on object structure in apraxics’ learning of novel object-related actions. J. Int. Neuropsychol. Soc. 2007;13:997–1008. doi: 10.1017/S1355617707070981.
- Buxbaum LJ, Kyle K, Grossman M, Coslett B. Left inferior parietal representations for skilled hand-object interactions: Evidence from stroke and corticobasal degeneration. Cortex. 2007;43:411–423. doi: 10.1016/S0010-9452(08)70466-0.
- Homer ML, et al. Adaptive offset correction for intracortical brain-computer interfaces. IEEE Trans. Neural. Syst. Rehabil. Eng. 2014;22:239–248. doi: 10.1109/TNSRE.2013.2287768.
- Jarosiewicz, B. et al. Virtual typing by people with tetraplegia using a self-calibrating intracortical brain-computer interface. Sci. Transl. Med. 7, 313ra179, 10.1126/scitranslmed.aac7328 (2015).
- Li Z, O’Doherty JE, Lebedev MA, Nicolelis MA. Adaptive decoding for brain-machine interfaces through bayesian parameter updates. Neural Comput. 2011;23:3162–3204. doi: 10.1162/NECO_a_00207.
- Heliot, R., Venkatraman, S. & Carmena, J. M. Decoder remapping to counteract neuron loss in brain-machine interfaces. 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC'10, 1670–1673 (2010).
- Velliste M, et al. Motor cortical correlates of arm resting in the context of a reaching task and implications for prosthetic control. J. Neurosci. 2014;34:6011–6022. doi: 10.1523/JNEUROSCI.3520-13.2014.
- Thoroughman KA, Shadmehr R. Electromyographic correlates of learning an internal model of reaching movements. J. Neurosci. 1999;19:8573–8588.
- Schaffelhofer, S. & Scherberger, H. Object vision to hand action in macaque parietal, premotor, and motor cortices. elife5, e15278, 10.7554/eLife.15278 (2016).
- Townsend BR, Subasi E, Scherberger H. Grasp movement decoding from premotor and parietal cortex. J. Neurosci. 2011;31:14386–14398. doi: 10.1523/JNEUROSCI.2451-11.2011.
- Fluet MC, Baumann MA, Scherberger H. Context-specific grasp movement representation in macaque ventral premotor cortex. J. Neurosci. 2010;30:15175–15184. doi: 10.1523/JNEUROSCI.3343-10.2010.
- Shenoy KV, Sahani M, Churchland MM. Cortical control of arm movements: A dynamical systems perspective. Annu Rev Neurosci. 2013;36:337–359. doi: 10.1146/annurev-neuro-062111-150509.
- Michaels JA, Dann B, Scherberger H. Neural population dynamics during reaching are better explained by a dynamical system than representational tuning. Nature. 2016;487:51–56.
- Sussillo D, et al. A recurrent neural network for closed-loop intracortical brain-machine interface decoders. J Neural Eng. 2012;9:026027. doi: 10.1088/1741-2560/9/2/026027.
- Friedenberg DA, et al. Neuroprosthetic-enabled control of graded arm muscle contraction in a paralyzed human. Sci. Rep. 2017;7:8386. doi: 10.1038/s41598-017-08120-9.
- Ajiboye AB, et al. Restoration of reaching and grasping movements through brain-controlled muscle stimulation in a person with tetraplegia: A proof-of-concept demonstration. Lancet. 2017;389:1821–1830. doi: 10.1016/S0140-6736(17)30601-3.
- Ethier C, Oby ER, Bauman MJ, Miller LE. Restoration of grasp following paralysis through brain-controlled stimulation of muscles. Nature. 2012;485:368–371. doi: 10.1038/nature10987.
- Boninger M, et al. Neuroprosthetic control and tetraplegia – authors’reply. Lancet. 2013;381:1900–1901. doi: 10.1016/S0140-6736(13)61154-X.
- Kirshblum SC, et al. Reference for the 2011 revision of the international standards for neurological classification of spinal cord injury. J. Spinal Cord Med. 2011;34:547–554. doi: 10.1179/107902611X13186000420242.
- Flesher, S. N. et al. Intracortical microstimulation of human somatosensory cortex. Sci. Transl. Med. 8, 361ra141, 10.1126/scitranslmed.aaf8083 (2016).
- Johannes M, Bigelow J. An overview of the developmental process for the modular prosthetic limb. Johns Hopkins APL Tech. Dig. 2011;30:201–216.
- Mathiowetz V, Volland G, Kashman N, Weber K. Adult norms for the box and block test of manual dexterity. Am. J. Occup. Ther. 1985;39:386–391. doi: 10.5014/ajot.39.6.386.
Source: PubMed