Mixed reality applications in urology: Requirements and future potential

Gerd Reis, Mehmet Yilmaz, Jason Rambach, Alain Pagani, Rodrigo Suarez-Ibarrola, Arkadiusz Miernik, Paul Lesur, Nareg Minaskan, Gerd Reis, Mehmet Yilmaz, Jason Rambach, Alain Pagani, Rodrigo Suarez-Ibarrola, Arkadiusz Miernik, Paul Lesur, Nareg Minaskan

Abstract

Background: Mixed reality (MR), the computer-supported augmentation of a real environment with virtual elements, becomes ever more relevant in the medical domain, especially in urology, ranging from education and training over surgeries. We aimed to review existing MR technologies and their applications in urology.

Methods: A non-systematic review of current literature was performed using the PubMed-Medline database using the medical subject headings (MeSH) term "mixed reality", combined with one of the following terms: "virtual reality", "augmented reality", ''urology'' and "augmented virtuality". The relevant studies were utilized.

Results: MR applications such as MR guided systems, immersive VR headsets, AR models, MR-simulated ureteroscopy and smart glasses have enormous potential in education, training and surgical interventions of urology. Medical students, urology residents and inexperienced urologists can gain experience thanks to MR technologies. MR applications are also used in patient education before interventions.

Conclusions: For surgical support, the achievable accuracy is often not sufficient. The main challenges are the non-rigid nature of the genitourinary organs, intraoperative data acquisition, online and multimodal registration and calibration of devices. However, the progress made in recent years is tremendous in all respects and the gap is constantly shrinking.

Keywords: Augmented reality; Augmented virtuality; Mixed reality; Urology; Virtual reality.

Conflict of interest statement

The authors declare no conflict of interest.

© 2021 The Authors.

References

    1. Milgram P., Takemura H., Utsumi A., Kishino F. Augmented reality: a class of displays on the reality-virtuality continuum. Telemanipulator and Telepresence Technologies. 1994:2351.
    1. Li P, Qin T, Hu B, Zhu F, Shen S. Monocular Visual-Inertial State Estimation for Mobile Augmented Reality2017. 11-21 p.
    1. Rambach J, Pagani A, Stricker D. [POSTER] Augmented Things: Enhancing AR Applications Leveraging the Internet of Things and Universal 3D Object Tracking2017.
    1. Wuest H, Vial F, Stricker D. Adaptive Line Tracking with Multiple Hypotheses for Augmented Reality2005. 62-69 p.
    1. Rambach J, Deng C, Pagani A, Stricker D. Learning 6DoF Object Poses from Synthetic Single Channel Images2018.
    1. Forster C, Pizzoli M, Scaramuzza D. SVO: Fast Semi-direct Monocular Visual Odometry2014.
    1. Mur-Artal R., Montiel J., Tardos J., Orb S.L.A.M. A versatile and accurate monocular SLAM system. IEEE Trans. Robot. 2015;31:1147–1163.
    1. Newcombe R, Davison A, Izadi S, et al. KinectFusion: Real-Time Dense Surface Mapping and Tracking2011. 127-136 p.
    1. Rambach J, Lesur P, Pagani A, Stricker D. SlamCraft: Dense Planar RGB Monocular SLAM2019.
    1. Zhi S., Bloesch M., Leutenegger S., Davison A. 2019. SceneCode: monocular dense semantic reconstruction using Learned Encoded Scene Representations; pp. 11768–11777.
    1. Codd A., Choudhury B. Virtual reality anatomy: is it comparable with traditional methods in the teaching of human forearm musculoskeletal anatomy? Anat. Sci. Educ. 2011;4:119–125.
    1. Lorenzo-Alvarez R., Rudolphi-Solero T., Ruiz-Gómez M.J., Sendra Portero F. Medical student education for abdominal radiographs in a 3D virtual classroom versus traditional classroom: a randomized controlled trial. Am. J. Roentgenol. 2019;213:1–7.
    1. Kugelmann D., Stratmann L., Nühlen N. An augmented reality magic mirror as additive teaching device for gross anatomy. Annals of anatomy = Anatomischer Anzeiger: official organ of the Anatomische Gesellschaft. 2017;215
    1. Moro C., Štromberga Z., Raikos A., Stirling A. The effectiveness of virtual and augmented reality in health sciences and medical anatomy. Anat. Sci. Educ. 2017;10
    1. Schoeb D.S., Schwarz J., Hein S. Mixed reality for teaching catheter placement to medical students: a randomized single-blinded, prospective trial. BMC Med. Educ. 2020;20:510.
    1. Parkhomenko E., O'Leary M., Safiullah S. Pilot assessment of immersive virtual reality renal models as an educational and preoperative planning tool for percutaneous nephrolithotomy. J. Endourol. 2019;33:283–288.
    1. Wake N., Rosenkrantz A., Huang R. Patient-specific 3D printed and augmented reality kidney and prostate cancer models: impact on patient education. 3D Printing in Medicine. 2019;5
    1. Al Janabi H., Aydın A., Palaneer S. Effectiveness of the hololens mixed reality headset in minimally invasive surgery: a simulation-based feasibility study. Surg. Endosc. 2020;34
    1. Hung A.J., Shah S.H., Dalag L., Shin D., Gill I.S. Development and validation of a novel robotic procedure specific simulation platform: partial nephrectomy. J. Urol. 2015;194:520–526.
    1. Feifer A., Delisle J., Anidjar M. Hybrid augmented reality simulator: preliminary construct validation of laparoscopic smoothness in a urology residency program. J. Urol. 2008;180:1455–1459.
    1. Kuronen-Stewart C., Ahmed K., Aydin A. Holmium laser enucleation of the prostate: simulation-based training curriculum and validation. Urology. 2015;86:639–646.
    1. Meccariello G., Faedi F., AlGhamdi S. An experimental study about haptic feedback in robotic surgery: may visual feedback substitute tactile feedback? J Robot Surg. 2016;10:57–61.
    1. Våpenstad C., Hofstad E., Bø L.E. Lack of transfer of skills after virtual reality simulator training with haptic feedback. Minim Invasive Ther. Allied Technol. 2017;26:1–9.
    1. Overtoom E., Horeman T., Jansen F.-W., Dankelman J., Schreuder H.W.R. Haptic feedback, force feedback, and force-sensing in simulation training for laparoscopy: a systematic overview. J. Surg. Educ. 2018;76
    1. Yamada Y., Inoue Y., Kaneko M., Fujihara A., Hongo F., Ukimura O. Virtual reality of three‐dimensional surgical field for surgical planning and intraoperative management. Int. J. Urol. 2019;26
    1. Porpiglia F., Amparore D., Checcucci E. Current use of three-dimensional model technology in urology: a road map for personalised surgical planning. European Urology Focus. 2018;4
    1. Antonelli A., Veccia A., Palumbo C. Holographic reconstructions for preoperative planning before partial nephrectomy: a head-to-head comparison with standard CT scan. Urol. Int. 2018;102:1–6.
    1. Checcucci E., Amparore D., Pecoraro A. 3D mixed reality holograms for preoperative surgical planning of nephron-sparing surgery: evaluation of surgeons’ perception. Minerva Urol. Nefrol. 2019 doi: 10.23736/S0393-2249.19.03610-5.
    1. Xie L., O’Leary M., Jefferson F.A. Interactive virtual reality renal models as an educational and preoperative planning tool for laparoscopic donor nephrectomy. Urology. 2021 doi: 10.1016/j.urology.2020.12.046. In press.
    1. Ma Y. A review of virtual cutting methods and technology in deformable objects. Int. J. Med. Robot. Comput. Assist. Surg. 2018;14
    1. Borgmann H., Rodríguez Socarrás M., Salem J. Feasibility and safety of augmented reality-assisted urological surgery using smartglass. World J. Urol. 2017;35:967–972.
    1. Schiavina R., Bianchi L., Lodi S. Real-time augmented reality three-dimensional guided robotic radical prostatectomy: preliminary experience and evaluation of the impact on surgical planning. Eur Urol Focus. 2020 doi: 10.1016/j.euf.2020.08.004.
    1. Amirabdollahian F., Livatino S., Vahedi B. Prevalence of haptic feedback in robot-mediated surgery: a systematic review of literature. J Robotic Surgery. 2018;12
    1. Bouget D., Allan M., Stoyanov D., Jannin P. Vision-based and marker-less surgical tool detection and tracking: a review of the literature. Med. Image Anal. 2016;35
    1. Zhang J., Zhong Y., Gu C. Soft tissue deformation modelling through neural dynamics-based reaction-diffusion mechanics. Med. Biol. Eng. Comput. 2018;56:2163–2176.
    1. Kong S.-H., Haouchine N., Soares R. Robust augmented reality registration method for localization of solid organs' tumors using CT-derived virtual biomechanical model and fluorescent fiducials. Surg. Endosc. 2017;31
    1. Yu F., Song E., Liu H., Li Y., Zhu J., Hung C.-C. An augmented reality endoscope system for ureter position detection. J. Med. Syst. 2018;42

Source: PubMed

스폰서 및 공동 작업자

건강 상태

약물 개입

3
구독하다