Environmental Localization Mapping and Guidance for Visual Prosthesis Users (SLAM)

This study is driven by the hypothesis that independent navigation by blind users of visual prosthetic devices can be greatly aided by use of an autonomous navigational aid that provides information about the environment and guidance for navigation through multimodal sensory cues. For this study, the investigators developed a navigation system that uses on-board sensing to map the user's environment and compute navigable paths to desired destinations in real-time. Information regarding obstacles and directional guidance is communicated to the user via a combination of sensory modalities including limited vision (through the user's visual prosthesis), haptic, and audio cues. This study evaluates how effectively this navigational aid improves prosthetic vision users' ability to perform navigational tasks. The participants for this study include both retinal prosthesis users of the Argus II Retinal Prosthesis System (Argus II) and normally sighted individuals who use a virtual reality headset to simulate the limited vision of the Argus II system.

Study Overview

• Status: Completed
• Conditions: Retinitis Pigmentosa; Visual Impairment; Visual Prosthesis
• Intervention / Treatment: Device: Navigation system mode: Argus Vision; Device: Navigation system mode: Depth Vision; Device: Navigation system mode: Depth Vision with Haptic / Audio; Device: Navigation system mode: Haptic / Audio; Device: Navigation system mode: High Field-of-View Depth Vision; Device: Distance test vision mode: Low Resolution / Low Field-of-View; Device: Distance test vision mode: Low Resolution / High Field-of-View; Device: Distance test vision mode: High Resolution / Low Field-of-View; Device: Distance test vision mode: High Resolution / High Field-of-View

Detailed Description

About 1.3 million Americans aged 40 and older are legally blind, a majority because of diseases with onset later in life, such as glaucoma and age-related macular degeneration. Second Sight Medical Products (SSMP) has developed the world's first FDA approved retinal implant, Argus II, intended to restore some functional vision for people suffering from retinitis pigmentosa (RP).

In this era of smart devices, generic navigation technology, such as GPS mapping apps for smartphones, can provide directions to help guide a blind user from point A to point B. However, these navigational aids do little to enable blind users to form an egocentric understanding of the surroundings, are not suited to navigation indoors, and do nothing to assist in avoiding obstacles to mobility. The Argus II, on the other hand, provides blind users with a limited visual representation of the users surroundings that improves users' ability to orient themselves and traverse obstacles, yet lacks features for high-level navigation and semantic interpretation of the surroundings. The proposed study aims to address these limitations of the Argus II through a synergy of state-of-the-art simultaneous localization and mapping (SLAM) and scene recognition technologies.

This study is driven by the hypothesis that independent navigation by blind users of visual prosthetic devices can be greatly aided by use of an autonomous navigational aid that provides information about the environment and guidance for navigation through multimodal sensory cues. The investigators developed a navigation system that uses on-board sensing and SLAM-based algorithms to continuously construct a map of the user's environment and locate the user within that map in real-time. On-board path planning algorithms compute optimal navigation routes to reach desired destinations based on the constructed map. The system then communicates obstacle locations and navigational cues to the user while navigating via a combination of sensory modalities. The participants for this study include blind Argus II users, who use their retinal implant for vision, and normally sighted individuals, who use a virtual reality headset to simulate the limited vision of a retinal prosthesis.

The sensory modalities used by the navigational aid to communicate information back to the user include:

• Limited vision is provided via the user's visual prosthesis, with the Argus II retinal implant supporting an image size of 10 x 6 pixels spanning approximately 18 x 11 degrees field-of-view. Images sent to the visual implant are derived from video frames provided by forward-facing cameras integrated within headgear worn by the user. Three forms of vision feedback are evaluated in this study including: 1) the standard vision output of the Argus II (which uses texture-based processing including difference-of-Gaussian and contrast enhancement filters), 2) an enhanced depth-based vision mode that uses the depth sensing capabilities of the navigational aid to highlight above-ground obstacles, and 3) a high field of view depth-based vision mode that doubles the pixel size and field of view of the visual feedback in each dimension. The depth-vision modes display only above-ground obstacles with increasing brightness relative to decreasing distance from the user; the ground and obstacles beyond a threshold distance are not displayed in order to declutter the visual scene. The high field-of-view depth-vision mode is only utilized by normally sighted participants, as this mode exceeds the vision capabilities of the Argus II implant.
• Haptic cues indicate the direction in which the user should advance in order to follow the path computed by the navigational aid to reach a target destination. The haptic cues are generated by vibrators situated at five positions located left-to-right along the users forehead, which are built into user-worn headgear. The five vibration points indicate for the user to turn in directions: "far left", "slight left", "straight ahead", "slight right", and "far right".
• Audio cues provide an audible alert when the user approaches an obstacle within 1.5 feet and provide verbal updates on the remaining distance to reach the destination along the path computed by the navigational aid.

This study compares participants' performance in completing navigation tasks using five different modes and combinations of the foregoing sensory modalities as follows: 1) Argus vision, 2) depth vision, 3) depth vision with haptic and audio, 4) haptic and audio (without vision), and 5) high field-of-view depth vision.

The navigation tasks performed by the participants using these modalities include navigating through a dense obstacle field and navigating between rooms within an indoor facility that requires successful traversal of non-trivial paths.

In addition, a third experiment evaluates the effect of resolution and field-of-view of the retinal implant upon participants' ability to visually discern relative distances to different obstacles based on optical flow patterns induced by the participant's motion when approaching obstacles situated at different distances ahead of the user. For this experiment, the following four vision settings are evaluated: 1) low resolution / low field-of-view, 2) low resolution / high field-of-view, 3) high resolution / low field-of-view, and 4) high resolution / high field-of-view. The "low" settings correspond to the values of the Argus II system, whereas the "high" settings corresponding to a doubling of the "low" values along each dimension. For Argus user participants, only the low resolution / low field-of-view setting is evaluated since the Argus II retinal implant is incapable of supporting the higher vision settings.

• Study Type: Interventional
• Enrollment (Actual): 26
• Phase: Not Applicable

Contacts and Locations

Study Locations

• United States
  • Maryland
    • Baltimore, Maryland, United States, 21205
      • Johns Hopkins Medicine - Wilmer Eye Institute
    • Laurel, Maryland, United States, 20723
      • Johns Hopkins Applied Physics Laboratory

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: 18 years and older (Adult, Older Adult)
• Accepts Healthy Volunteers: Yes

Description

Criteria for inclusion of normally sighted individuals:

• Subject speaks English;
• Subjects must be an adult (at least 18 years of age);
• Subject has the cognitive and communication ability to participate in the study (i.e., follow spoken directions, perform tests, and give feedback);
• Subject is willing to conduct psychophysics testing up to 4-6 hours per day of testing on 3-5 consecutive days;
• Subject has visual acuity of 20/40 or better (corrected);
• Subject is capable of understanding participant information materials and giving written informed consent.
• Subject is able to walk unassisted

Criteria for inclusion of Argus II users:

The inclusion criteria for the study are the following:

• Subject is at least 25 years of age;
• Subject has been implanted with the Argus II system;
• Subject's eye has healed from surgery and the surgeon has cleared the subject for programming;
• Subject has the cognitive and communication ability to participate in the study (i.e., follow spoken directions, perform tests, and give feedback);
• Subject is willing to conduct psychophysics testing up to 4-6 hours per day of testing on 3-5 consecutive days;
• Subject is capable of understanding patient information materials and giving written informed consent;
• Subject is able to walk unassisted.

Exclusion criteria for all subjects is the following:

• Subject is unwilling or unable to travel to testing facility for at least 3 days of testing within a one-week timeframe;
• Subject does not speak English;
• Subject has language or hearing impairment.

Study Plan

How is the study designed?

Design Details

• Primary Purpose: Basic Science
• Allocation: Non-Randomized
• Interventional Model: Single Group Assignment
• Masking: None (Open Label)

Number of Arms

2

Arms and Interventions

Participant Group / Arm	Intervention / Treatment
Experimental: Evaluation of normally sighted participants using a simulated visual prosthesis; All normally sighted participants will be assigned to this arm of the study. This study group will perform all experiments using a virtual reality headset (Oculus Go) to simulate the limited vision of a retinal prosthesis system. This study group will include all interventions, including those that both match and exceed the visual performance of the Argus II system.	Device: Navigation system mode: Argus Vision; This intervention uses the navigational aid with the output sensory modalities configured as follows: Vision: Argus mode; Haptics: none; Audio: none Since this mode only provides the standard Argus vision, it is equivalent to using the base Argus II system without the navigational aid.; Device: Navigation system mode: Depth Vision; This intervention uses the navigational aid with the output sensory modalities configured as follows: Vision: Depth mode (at Argus II resolution and field-of-view); Haptics: none; Audio: none; Device: Navigation system mode: Depth Vision with Haptic / Audio; This intervention uses the navigational aid with the output sensory modalities configured as follows: Vision: Depth mode (at Argus II resolution and field-of-view); Haptics: yes; Audio: yes; Device: Navigation system mode: Haptic / Audio; This intervention uses the navigational aid with the output sensory modalities configured as follows: Vision: none; Haptics: yes; Audio: yes Since this mode does not provide any visual feedback, it is equivalent to using navigational aid completely blind without a visual prosthesis.; Device: Navigation system mode: High Field-of-View Depth Vision; This intervention uses the navigational aid with the output sensory modalities configured as follows: Vision: High Field-of-View Depth mode (at twice the Argus II resolution and field-of-view along each dimension); Haptics: none; Audio: none; Device: Distance test vision mode: Low Resolution / Low Field-of-View; This intervention performs the relative distance to two obstacles test with the user's vision output configured as follows: Resolution: Low; Field-of-View: Low; Device: Distance test vision mode: Low Resolution / High Field-of-View; This intervention performs the relative distance to two obstacles test with the user's vision output configured as follows: Resolution: Low; Field-of-View: High; Device: Distance test vision mode: High Resolution / Low Field-of-View; This intervention performs the relative distance to two obstacles test with the user's vision output configured as follows: Resolution: High; Field-of-View: Low; Device: Distance test vision mode: High Resolution / High Field-of-View; This intervention performs the relative distance to two obstacles test with the user's vision output configured as follows: Resolution: High; Field-of-View: High
Experimental: Evaluation of blind participants using the Argus II retinal prosthesis; All blind participants who have been implanted with the Argus II Retinal Prosthesis System will be assigned to this arm of the study. This study group will be limited to performing a subset of the interventions including only those that do not exceed the visual performance of the Argus II system.	Device: Navigation system mode: Argus Vision; This intervention uses the navigational aid with the output sensory modalities configured as follows: Vision: Argus mode; Haptics: none; Audio: none Since this mode only provides the standard Argus vision, it is equivalent to using the base Argus II system without the navigational aid.; Device: Navigation system mode: Depth Vision; This intervention uses the navigational aid with the output sensory modalities configured as follows: Vision: Depth mode (at Argus II resolution and field-of-view); Haptics: none; Audio: none; Device: Navigation system mode: Depth Vision with Haptic / Audio; This intervention uses the navigational aid with the output sensory modalities configured as follows: Vision: Depth mode (at Argus II resolution and field-of-view); Haptics: yes; Audio: yes; Device: Navigation system mode: Haptic / Audio; This intervention uses the navigational aid with the output sensory modalities configured as follows: Vision: none; Haptics: yes; Audio: yes Since this mode does not provide any visual feedback, it is equivalent to using navigational aid completely blind without a visual prosthesis.; Device: Distance test vision mode: Low Resolution / Low Field-of-View; This intervention performs the relative distance to two obstacles test with the user's vision output configured as follows: Resolution: Low; Field-of-View: Low

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Duration of Time to Complete Navigational Tasks as Assessed by the Mean Duration of Travel in Seconds Normalized by the Nominal Path Distance in Meters	Participants will be asked to complete the navigation tasks of crossing an obstacle field approximately 6 meters in length and navigating to a target destination located about 20 - 30 meters away. The normalized duration of time to complete each navigation task will be measured in seconds. To account for variability in the path distances of different trials, the measured time will be normalized by a reference (nominal) path distance in meters associated with the configuration of each trial. The mean normalized time across all trials will be computed for each intervention.	Up to 7 minutes for each navigation task, on Day 1 or Day 2
Distance Traversed to Complete Navigational Tasks as Assessed by the Mean Distance Traveled in Meters Normalized by the Nominal Path Distance in Meters	Participants will be asked to complete the navigation tasks of crossing an obstacle field approximately 6 meters in length and navigating to a target destination located about 20 - 30 meters away. The normalized distance traversed to complete each navigation task will be measured. To account for variability in the path distances of different trials, the measured distance will be normalized by a reference (nominal) path distance in meters associated with the configuration of each trial. The mean normalized distance across all trials will be computed for each intervention.	Up to 7 minutes for each navigation task, on Day 1 or Day 2
Obstacle Avoidance While Completing Navigational Tasks as Assessed by the Mean Number of Contacts With Obstacles Normalized by the Nominal Path Distance in Meters	Participants will be asked to complete the navigation tasks of crossing an obstacle field approximately 6 meters in length and navigating to a target destination located about 20 - 30 meters away. The number of contacts with obstacles encountered while completing each navigation task will be measured. To account for variability in the path distances of different trials, the number of contacts will be normalized by a reference (nominal) path distance in meters associated with the configuration of each trial. The mean normalized number of contacts with obstacles across all trials will be computed for each intervention.	Up to 7 minutes for each navigation task, on Day 1 or Day 2
Success in Completing Navigational Tasks as Assessed by the Mean Percentage of Trials Where Participants Successfully Reached the Intended Destination	Participants will be asked to navigate to a target destination located about 20 - 30 meters away. We will measure success rate as the mean number of trials for which each participants successfully reached the intended destination divided by the total number of trials performed by that participant for each intervention.	Up to 7 minutes for each navigation task, on Day 1 or Day 2
Accuracy of Relative Distance Judgments as Assessed by the Percentage of Correct Responses	Participants will be presented with two physical objects and asked to identify which of the objects is the nearer one after walking a controlled distance down a centerline oriented between the objects. We will measure accuracy as the percentage of correct responses (the total number of correct responses divided by the total number of trials times 100).	Up to 60 seconds for each discrimination task, on Day 1 or Day 2

Collaborators and Investigators

• Sponsor: Johns Hopkins University
• Collaborators: National Eye Institute (NEI); Carnegie Mellon University
• Investigators: Principal Investigator: Seth Billings, Ph.D., Johns Hopkins University

Study record dates

Study Major Dates

• Study Start (Actual): March 16, 2021
• Primary Completion (Actual): September 30, 2024
• Study Completion (Actual): September 30, 2024

Study Registration Dates

• First Submitted: April 21, 2020
• First Submitted That Met QC Criteria: April 21, 2020
• First Posted (Actual): April 24, 2020

Study Record Updates

• Last Update Posted (Actual): March 25, 2025
• Last Update Submitted That Met QC Criteria: February 18, 2025
• Last Verified: February 1, 2025

More Information

Terms related to this study

• Keywords: Retinal Prosthesis; Argus II
• Additional Relevant MeSH Terms: Neurologic Manifestations; Nervous System Diseases; Genetic Diseases, Inborn; Eye Diseases; Eye Diseases, Hereditary; Sensation Disorders; Retinal Diseases; Retinal Dystrophies; Retinal Degeneration; Vision, Low; Retinitis; Retinitis Pigmentosa; Vision Disorders
• Other Study ID Numbers: IRB00228932; 1R01EY029741-01A1 (U.S. NIH Grant/Contract)

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: NO
• IPD Plan Description: Results from this study will be disseminated through conference presentations and peer-reviewed publications. However, IPD will not be shared outside the study team.

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: Yes