Extended Reality in Medical Practice

Abstract

Purpose of review: Advances in display technology and computing have led to new devices capable of overlaying digital information onto the physical world or incorporating aspects of the physical world into virtual scenes. These combinations of digital and physical environments are referred to as extended realities. Extended reality (XR) devices offer many advantages for medical applications including realistic 3D visualization and touch-free interfaces that can be used in sterile environments. This review introduces extended reality and describes how it can be applied to medical practice.

Recent findings: The 3D displays of extended reality devices are valuable in situations where spatial information such as patient anatomy and medical instrument position is important. Applications that take advantage of these 3D capabilities include teaching and pre-operative planning. The utility of extended reality during interventional procedures has been demonstrated with through 3D visualizations of patient anatomy, scar visualization, and real-time catheter tracking with touch-free software control. Extended reality devices have been applied to education, pre-procedural planning, and cardiac interventions. These devices excel in settings where traditional devices are difficult to use, such as in the cardiac catheterization lab. New applications of extended reality in cardiology will continue to emerge as the technology improves.

Keywords: Display technology; Extended reality devices; Informatics.

Figures

Figure 1.

Extended reality devices and interfaces. A. Conventional computing (left) displays images on a 2D screen with a clear separation between physical and digital realities. User interaction with conventional computing devices generally requires keyboard and mouse or touch screen. Extended reality devices (right) mix digital and physical realities. 2D augmented reality (AR) devices may be head-mounted or displayed on a phone or tablet screen by using a camera to display physical reality. 3D AR devices incorporate spatial mapping to display 3D objects in the user’s physical space. Virtual reality (VR) devices occlude the user’s vision but may incorporate views of the user’s surroundings. B. Extended reality interfaces. VR controllers (left) use optical tracking and button proximity sensors to display the user’s hand position and posture accurately in the VR headset. Gaze and gesture interfaces (right) enable users to select 3D objects by targeting them with a visual cursor (the white ring) and performing a pinch gesture to interact with the object.