Immersive Virtual Reality and Ocular Tracking for Brain Mapping During Awake Surgery: Prospective Evaluation Study

Abstract

Background: Language mapping during awake brain surgery is currently a standard procedure. However, mapping is rarely performed for other cognitive functions that are important for social interaction, such as visuospatial cognition and nonverbal language, including facial expressions and eye gaze. The main reason for this omission is the lack of tasks that are fully compatible with the restrictive environment of an operating room and awake brain surgery procedures.

Objective: This study aims to evaluate the feasibility and safety of a virtual reality headset equipped with an eye-tracking device that is able to promote an immersive visuospatial and social virtual reality (VR) experience for patients undergoing awake craniotomy.

Methods: We recruited 15 patients with brain tumors near language and/or motor areas. Language mapping was performed with a naming task, DO 80, presented on a computer tablet and then in 2D and 3D via the VRH. Patients were also immersed in a visuospatial and social VR experience.

Results: None of the patients experienced VR sickness, whereas 2 patients had an intraoperative focal seizure without consequence; there was no reason to attribute these seizures to virtual reality headset use. The patients were able to perform the VR tasks. Eye tracking was functional, enabling the medical team to analyze the patients' attention and exploration of the visual field of the virtual reality headset directly.

Conclusions: We found that it is possible and safe to immerse the patient in an interactive virtual environment during awake brain surgery, paving the way for new VR-based brain mapping procedures.

Trial registration: ClinicalTrials.gov NCT03010943; https://ichgcp.net/clinical-trials-registry/NCT03010943.

Keywords: awake surgery; brain mapping; eye tracking; mobile phone; nonverbal language; virtual reality; visuospatial cognition.

Conflict of interest statement

Conflicts of Interest: RS is a cofounder of Dynamixyz, which markets the facial expression transfer tool used to animate avatars. He reports personal fees from Dynamixyz. None of the other authors have any conflicts of interest to declare.

©Morgane Casanova, Anne Clavreul, Gwénaëlle Soulard, Matthieu Delion, Ghislaine Aubin, Aram Ter Minassian, Renaud Seguier, Philippe Menei. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 24.03.2021.

Figures

Figure 1

(A) Patient wearing the virtual reality headset. (B) and (C) Example of the item “phone” in the DO 80 naming task presented in 2D (B) and 3D (C) with the virtual reality headset. The green spot indicates the patient’s gaze.

Figure 2

Left: view of the operating room during the procedure. (A) Head of the patient wearing the virtual reality headset; (B) application of direct electrical stimulation to the exposed brain; (C) screen showing what the patient sees in the virtual reality headset, his gaze materialized by a green spot; (D) neuronavigational system showing brain white matter fascicles and the position of the electrode. Right: example of a layout after the virtual reality task simulating a visuospatial and social experience. (E) The image that is visualized and analyzed on the screen (C). The movement of the patient’s gaze is visualized as a blue line (with the starting point in green and the endpoint in pink). The green box indicates the avatar making eye contact. The white arrow indicates the avatar on which the patient focuses for more than 0.6 seconds (triggering the expression of a dynamic facial emotion). In this example, the patient identified the avatar making eye contact in 2.53 seconds and indicated the emotion expressed 3.77 seconds later.