Dynamic Stimulation of Visual Cortex Produces Form Vision in Sighted and Blind Humans

Abstract

A visual cortical prosthesis (VCP) has long been proposed as a strategy for restoring useful vision to the blind, under the assumption that visual percepts of small spots of light produced with electrical stimulation of visual cortex (phosphenes) will combine into coherent percepts of visual forms, like pixels on a video screen. We tested an alternative strategy in which shapes were traced on the surface of visual cortex by stimulating electrodes in dynamic sequence. In both sighted and blind participants, dynamic stimulation enabled accurate recognition of letter shapes predicted by the brain's spatial map of the visual world. Forms were presented and recognized rapidly by blind participants, up to 86 forms per minute. These findings demonstrate that a brain prosthetic can produce coherent percepts of visual forms.

Trial registration: ClinicalTrials.gov NCT03344848.

Keywords: blind; brain; dynamic; electrical; electrodes; human; phosphene; prosthetic; stimulation; visual.

Conflict of interest statement

Declaration of interests W.H.B., N.P., and D.Y. receive research funding from Second Sight Medical Products, a manufacturer of visual cortical prosthetics. N.P. is also a consultant to Second Sight. A provisional patent application describing dynamic current steering (serial no. 62/638,365) was filed with the US Patent and Trademark Office on March 5, 2018, entitled “Systems and Computer-Implemented Methods of Conveying a Visual Image to a Blind Subject Fitted with a Visual Prosthesis.”

Published by Elsevier Inc.

Figures

Figure 1.. Stimulation paradigms for visual cortical prosthetics.

(A) To convey a letter through touch, one could use static tactile stimulation to press multiple probes (blue dots) into the palm of the hand, forming the shape of a static letter. However, this results in an amorphous percept (blob in thought bubble). (B) Alternately, one could use dynamic tactile stimulation to trace the shape of the letter dynamically using a single probe (single blue dot) traced across the palm in a sequence (black line with arrow) that matches the desired shape, producing a coherent letter percept (“Z” in thought bubble). (C) In a cortical visual prosthetic, an electrode grid is implanted over visual cortex. For static electrical stimulation, current is delivered concurrently to some electrodes (blue circles) but not others (gray circles), resulting in an amorphous percept. (D) For dynamic current steering, current is delivered to the electrode grid in a temporal sequence that matches the desired shape (white arrows), resulting in a coherent visual percept. Current is delivered in sequence to physical electrodes (dark blue circles) and virtual electrodes (light blue ovals) created by current steering (delivering current simultaneously to adjacent physical electrodes).

Figure 2.. Effectiveness of current steering in a blind participant.

(A) A medial view of a cortical surface model of the participant’s occipital lobe (dashed line indicates calcarine sulcus). Two subdural electrodes are shown as a red and a blue disc (labeled F01 and F03). (B) Current steering was implemented by delivering varying amount of current to the two electrodes. In the first condition, maximum current was delivered to electrode F01 and no current was delivered to electrode F03; in successive conditions, the amount of current to F01 was decreased and the amount of current delivered to F03 was increased, until in the fifth condition, no current was delivered to F01 and maximal current was delivered to F03. Colored circles underneath each condition correspond to phosphene locations in (D). (C) In each condition, current was delivered in a single 100 ms pulse train with 6 pulses per train (pulse frequency of 60 Hz). The pulse train for the third stimulation condition in (B) is shown, with equivalent current amplitudes for F01 and F03. For other conditions, the current amplitude delivered to the electrodes differed, but the timing of the pulse trains was identical. (D) The participant used a touch screen to report the location of the visual percept resulting from each stimulation condition. Ten trials of each condition were performed. The colored circle shows the average location of all reports for that condition, the dashed line shows the 95% spatial confidence interval fit with an ellipse. Colors of each circle and dashed line correspond to condition colors shown in (B).

Figure 3.. Effectiveness of dynamic current steering in a blind participant.

(A) A medial view of a cortical surface model of the participant’s occipital lobe (dashed line indicates calcarine sulcus; same participant as Figure 2). Five subdural electrodes are shown as blue discs (letters show electrode labels). (B) Dynamic current steering was implemented by delivering time-varying amounts of current to the electrodes. (C) The participant reported the percept of a point traversing the visual field in a horizontal direction.

Figure 4.. Dynamic stimulation produces perception of letter forms in sighted participants.

(A) Medial view of the left occipital lobe of sighted participant YBN. Blue circles show the 24 electrodes contained in a grid implanted inferior to the calcarine sulcus (dashed black line). Black numbers label electrodes and correspond to phosphene numbers in (B). (B) To generate receptive field maps, the participant fixated while mapping stimuli were presented (see Supplementary Figure 1). The blue circles show the location of the receptive field centers for each electrode relative to the fixation point (+). (C) Pulse diagram for dynamic stimulation (without current steering). The timing of the pulses to the first three electrodes in a sequence are shown; pulses to successive electrodes occurred with the same timing. For each electrode, a stimulation current was used that produced a reliable phosphene when that electrode was stimulated in isolation, ranging from 1.2 to 1.5 mA for different electrodes. (D) Dynamic stimulation of selected electrodes was used to generate visual percepts of four different letter forms. For each letter form, the left panel shows the stimulated electrodes (bold circles) and the direction of the temporal sequence of stimulation (arrow). The right panel shows the participant’s actual drawing of the visual percept and the verbal label used to identify it. (E) Medial view of the left occipital lobe of sighted participant YAY. Blue circles show the location of stimulated electrodes relative to calcarine sulcus (dashed black line). (F) Location of individual phosphenes. The participant fixated while electrical stimulation was delivered to one electrode at a time. The participant drew each phosphene on a touchscreen (bold ellipses, numbered by the corresponding electrode). (G) Pulse diagram for dynamic stimulation with current steering. The timing of the pulses to the first three electrodes in a sequence are shown; pulses to successive electrodes occurred with the same timing. Baseline stimulation currents for each electrode ranged from 0.7 mA to 1.5 mA. (H) The phosphene map was used to design a stimulation sequence to produce the visual percept of the letter “Z”. The black arrows show the temporal sequence of stimulated electrodes (black numbers) and virtual electrodes located between the physical electrodes (red dots). The participant drew the pattern they perceived on the touchscreen (blue line in middle panel). Right panel shows still frame from a video of the participant drawing, see Supplementary Video 1 for full video.

Figure 5.. Dynamic stimulation tested in blind participant BAA.

(A) Still frame from a video of the participant drawing, see Supplementary Video 2 for full video. The participant placed the index finger of the left hand on a tactile fixation point in the middle of the touch screen and used the index finger of the right hand to trace the visual percept on the touch screen. (B) Medial view of a surface model of the participant’s right occipital lobe. Dashed line shows calcarine sulcus, circles show electrode locations and numbers. (C) Blue ellipses show participant drawing of phosphenes created by stimulation of individual electrodes, with numbers corresponding to electrodes in (B). Crosshairs show location of tactile fixation point. (D) Dynamic stimulation pulse diagram. The timing of the pulses to the first three electrodes in a sequence are shown; pulses to successive electrodes occurred with the same timing. Stimulation current for all electrodes was 2 mA. (E) Seven different letter-like shapes created by seven different dynamic stimulation patterns. The left panel for each shape shows the temporal sequence of stimulation (electrodes indicated by numbers connected by black arrows). The right panel for each shape shows the participant drawings with each stimulation pattern. Each line illustrates a separate trial (randomly interleaved), colored in different shades of blue for visibility. The letter in quotation marks shows the participant mnemonic for that pattern (backwards “G”; backwards “N”; backwards “R”; upside-down “U”; upside-down “V”; “W”; “Z”). (F) Quantification of the drawings produced by the participant for each trial of each stimulation pattern using multidimensional scaling analysis. Each letter corresponds to a single trial of the corresponding stimulation pattern. Boldface letters show the centroid of each cluster from a k-means analysis.

Figure 6.. Dynamic stimulation tested in blind participant 03–281.

(A) Still frame from a video of the participant drawing, see Supplementary Video 3 for full video. The participant placed the index finger of the left hand on a tactile fixation point and used the index finger of the right hand to trace the visual percept on the touch screen. (B) Medial view of a surface model of the participant’s right occipital lobe. Dashed line shows calcarine sulcus, circles show electrode locations and labels. (C) Blue ellipses show participant drawing of phosphenes created by stimulation of individual electrodes, with labels corresponding to electrodes in (B). (D) Dynamic stimulation pulse diagram. The timing of the pulses to the first three electrodes in a sequence are shown; pulses to successive electrodes occurred with the same timing. Currents ranged between 3.5 and 5.8 mA per electrode. (E) Four different letter-like shapes created by four different dynamic stimulation patterns. The left panel for each shape shows the temporal sequence of stimulation (electrodes indicated by numbers connected by black arrows). The right panel for each shape shows the participant drawing for each stimulation pattern along with verbal label. (F) Participant drawings for ten different trials of each stimulation pattern, each trial indicated with a different color. (G) Quantification of the drawings produced by the participant for each trial of each stimulation pattern using multidimensional scaling analysis. Each letter corresponds to a single trial of the corresponding stimulation pattern. Boldface letters show the centroid of each cluster from a k-means analysis. (H) Accuracy of form identification at two different presentation rates. Left bar: two forms (a downward line and an upward line) were presented in random order at 85 forms per minute and verbally identified by the participant. Right bar: three forms (“C”, “U”, backwards “C”) were presented at 30 forms per minute. Error bars show 95% confidence interval from binomial distribution. See Supplementary Video 4 for full video.