Integration of Square Fiducial Markers in Patient-Specific Instrumentation and Their Applicability in Knee Surgery

Vicente J León-Muñoz, Joaquín Moya-Angeler, Mirian López-López, Alonso J Lisón-Almagro, Francisco Martínez-Martínez, Fernando Santonja-Medina, Vicente J León-Muñoz, Joaquín Moya-Angeler, Mirian López-López, Alonso J Lisón-Almagro, Francisco Martínez-Martínez, Fernando Santonja-Medina

Abstract

Computer technologies play a crucial role in orthopaedic surgery and are essential in personalising different treatments. Recent advances allow the usage of augmented reality (AR) for many orthopaedic procedures, which include different types of knee surgery. AR assigns the interaction between virtual environments and the physical world, allowing both to intermingle (AR superimposes information on real objects in real-time) through an optical device and allows personalising different processes for each patient. This article aims to describe the integration of fiducial markers in planning knee surgeries and to perform a narrative description of the latest publications on AR applications in knee surgery. Augmented reality-assisted knee surgery is an emerging set of techniques that can increase accuracy, efficiency, and safety and decrease the radiation exposure (in some surgical procedures, such as osteotomies) of other conventional methods. Initial clinical experience with AR projection based on ArUco-type artificial marker sensors has shown promising results and received positive operator feedback. Once initial clinical safety and efficacy have been demonstrated, the continued experience should be studied to validate this technology and generate further innovation in this rapidly evolving field.

Keywords: augmented reality (AR); extended reality (XR); immersive technology; knee; patient-specific instrumentation (PSI); surgical navigation system; virtual reality (VR).

Conflict of interest statement

The authors declare no conflict of interest.

Figures

Figure 1
Figure 1
Images of the virtual model with the planning of the tibial osteotomy and the tunnels for the anterior and posterior anchorage of the medial meniscal transplant.
Figure 2
Figure 2
Detail of the jig design to perform the osteotomy with the ArUco markers.
Figure 3
Figure 3
Virtual modelling of the plate and screw placement avoiding the tunnel trajectory for anchoring the medial meniscal transplant.
Figure 4
Figure 4
Details of the analysis of the rotational distortion on the virtual model and of the proposed corrections.
Figure 5
Figure 5
Cutting jig with ArUco markers for guiding femoral osteotomy under AR control.

References

    1. Gao J., Dong S., Li J.J., Ge L., Xing D., Lin J. New technology-based assistive techniques in total knee arthroplasty: A Bayesian network meta-analysis and systematic review. Int. J. Med. Robot. Comput. Assist. Surg. 2020;17:e2189. doi: 10.1002/rcs.2189.
    1. Batailler C., Parratte S. Assistive technologies in knee arthroplasty: Fashion or evolution? Rate of publications and national registries prove the Scott Parabola wrong. Arch. Orthop. Trauma Surg. 2021;141:2027–2034. doi: 10.1007/s00402-021-04051-3.
    1. Amiot L.-P., Poulin F. Computed Tomography-Based Navigation for Hip, Knee, and Spine Surgery. Clin. Orthop. Relat. Res. 2004;421:77–86. doi: 10.1097/01.blo.0000126866.29933.42.
    1. Lustig S., Fleury C., Goy D., Neyret P., Donell S.T. The accuracy of acquisition of an imageless computer-assisted system and its implication for knee arthroplasty. Knee. 2011;18:15–20. doi: 10.1016/j.knee.2009.12.010.
    1. Jones C.W., Jerabek S.A. Current Role of Computer Navigation in Total Knee Arthroplasty. J. Arthroplast. 2018;33:1989–1993. doi: 10.1016/j.arth.2018.01.027.
    1. Nam D., Cody E.A., Nguyen J.T., Figgie M.P., Mayman D.J. Extramedullary Guides Versus Portable, Accelerometer-Based Navigation for Tibial Alignment in Total Knee Arthroplasty: A Randomized, Controlled Trial: Winner of the 2013 HAP PAUL Award. J. Arthroplast. 2014;29:288–294. doi: 10.1016/j.arth.2013.06.006.
    1. Kim G.B., Lee S., Kim H., Yang D.H., Kim Y.-H., Kyung Y.S., Kim C.-S., Choi S.H., Kim B.J., Ha H., et al. Three-Dimensional Printing: Basic Principles and Applications in Medicine and Radiology. Korean J. Radiol. 2016;17:182. doi: 10.3348/kjr.2016.17.2.182.
    1. Batailler C., Shatrov J., Sappey-Marinier E., Servien E., Parratte S., Lustig S. Artificial intelligence in knee arthroplasty: Current concept of the available clinical applications. Arthroplasty. 2022;4:17. doi: 10.1186/s42836-022-00119-6.
    1. Alpaugh K., Ast M.P., Haas S.B. Immersive technologies for total knee arthroplasty surgical education. Arch. Orthop. Trauma Surg. 2021;141:2331–2335. doi: 10.1007/s00402-021-04174-7.
    1. Tsukada S., Ogawa H., Nishino M., Kurosaka K., Hirasawa N. Augmented Reality-Assisted Femoral Bone Resection in Total Knee Arthroplasty. JB JS Open Access. 2021;6:e21.00001. doi: 10.2106/JBJS.OA.21.00001.
    1. Iacono V., Farinelli L., Natali S., Piovan G., Screpis D., Gigante A., Zorzi C. The use of augmented reality for limb and component alignment in total knee arthroplasty: Systematic review of the literature and clinical pilot study. J. Exp. Orthop. 2021;8:52. doi: 10.1186/s40634-021-00374-7.
    1. Goh G.S., Lohre R., Parvizi J., Goel D.P. Virtual and augmented reality for surgical training and simulation in knee arthroplasty. Arch. Orthop. Trauma Surg. 2021;141:2303–2312. doi: 10.1007/s00402-021-04037-1.
    1. Maharjan N., Alsadoon A., Prasad P.W.C., Abdullah S., Rashid T.A. A novel visualization system of using augmented reality in knee replacement surgery: Enhanced bidirectional maximum correntropy algorithm. Int. J. Med. Robot. 2021;17:e2223. doi: 10.1002/rcs.2223.
    1. Fucentese S.F., Koch P.P. A novel augmented reality-based surgical guidance system for total knee arthroplasty. Arch. Orthop. Trauma Surg. 2021;141:2227–2233. doi: 10.1007/s00402-021-04204-4.
    1. Tsukada S., Ogawa H., Nishino M., Kurosaka K., Hirasawa N. Augmented reality-based navigation system applied to tibial bone resection in total knee arthroplasty. J. Exp. Orthop. 2019;6:44. doi: 10.1186/s40634-019-0212-6.
    1. Bagaria V., Sadigale O.S., Pawar P.P., Bashyal R.K., Achalare A., Poduval M. Robotic-Assisted Knee Arthroplasty (RAKA): The Technique, the Technology and the Transition. Indian J. Orthop. 2020;54:745–756. doi: 10.1007/s43465-020-00088-5.
    1. Ha J., Parekh P., Gamble D., Masters J., Jun P., Hester T., Daniels T., Halai M. Opportunities and challenges of using augmented reality and heads-up display in orthopaedic surgery: A narrative review. J. Clin. Orthop. Trauma. 2021;18:209–215. doi: 10.1016/j.jcot.2021.04.031.
    1. Casari F.A., Navab N., Hruby L.A., Kriechling P., Nakamura R., Tori R., de Lourdes Dos Santos Nunes F., Queiroz M.C., Fürnstahl P., Farshad M. Augmented Reality in Orthopedic Surgery Is Emerging from Proof of Concept Towards Clinical Studies: A Literature Review Explaining the Technology and Current State of the Art. Curr. Rev. Musculoskelet. Med. 2021;14:192–203. doi: 10.1007/s12178-021-09699-3.
    1. Furman A.A., Hsu W.K. Augmented Reality (AR) in Orthopedics: Current Applications and Future Directions. Curr. Rev. Musculoskelet. Med. 2021;14:397–405. doi: 10.1007/s12178-021-09728-1.
    1. Jud L., Fotouhi J., Andronic O., Aichmair A., Osgood G., Navab N., Farshad M. Applicability of augmented reality in orthopedic surgery—A systematic review. BMC Musculoskelet. Disord. 2020;21:103. doi: 10.1186/s12891-020-3110-2.
    1. Matthews J.H., Shields J.S. The Clinical Application of Augmented Reality in Orthopaedics: Where Do We Stand? Curr. Rev. Musculoskelet. Med. 2021;14:316–319. doi: 10.1007/s12178-021-09713-8.
    1. Laverdière C., Corban J., Khoury J., Ge S.M., Schupbach J., Harvey E.J., Reindl R., Martineau P.A. Augmented reality in orthopaedics: A systematic review and a window on future possibilities. Bone Jt. J. 2019;101-B:1479–1488. doi: 10.1302/0301-620X.101B12.BJJ-2019-0315.R1.
    1. Lee K. Augmented Reality in Education and Training. TechTrends. 2012;56:13–21. doi: 10.1007/s11528-012-0559-3.
    1. Caudell T.P., Mizell D.W. Augmented reality: An application of heads-up display technology to manual manufacturing processes; Proceedings of the Twenty-Fifth Hawaii International Conference on System Sciences; Kauai, HI, USA. 7–10 January 1992; Manhattan, NY, USA: IEEE; 1992. pp. 659–669.
    1. Milgram P., Kishino F. A taxonomy of mixed reality visual displays. IEICE Trans. Inf. Syst. 1994;77:1321–1329.
    1. Garrido-Jurado S., Muñoz-Salinas R., Madrid-Cuevas F.J., Marín-Jiménez M.J. Automatic generation and detection of highly reliable fiducial markers under occlusion. Pattern Recognit. 2014;47:2280–2292. doi: 10.1016/j.patcog.2014.01.005.
    1. About OpenCV. [(accessed on 21 November 2022)]. Available online:
    1. Ortega G., Wolff A., Baumgaertner M., Kendoff D. Usefulness of a head mounted monitor device for viewing intraoperative fluoroscopy during orthopaedic procedures. Arch. Orthop. Trauma Surg. 2008;128:1123–1126. doi: 10.1007/s00402-007-0500-y.
    1. Chimenti P.C., Mitten D.J. Google Glass as an Alternative to Standard Fluoroscopic Visualization for Percutaneous Fixation of Hand Fractures: A Pilot Study. Plast. Reconstr. Surg. 2015;136:328–330. doi: 10.1097/PRS.0000000000001453.
    1. Keating T.C., Jacobs J.J. Augmented Reality in Orthopedic Practice and Education. Orthop. Clin. N. Am. 2021;52:15–26. doi: 10.1016/j.ocl.2020.08.002.
    1. Ma L., Fan Z., Ning G., Zhang X., Liao H. 3D Visualization and Augmented Reality for Orthopedics. Adv. Exp. Med. Biol. 2018;1093:193–205. doi: 10.1007/978-981-13-1396-7_16.
    1. Song P., Fan Z., Zhi X., Cao Z., Min S., Liu X., Zhang Y., Kong X., Chai W. Study on the accuracy of automatic segmentation of knee CT images based on deep learning. Zhongguo Xiu Fu Chong Jian Wai Ke Za Zhi = Zhongguo Xiufu Chongjian Waike Zazhi=Chin. J. Reparative Reconstr. Surg. 2022;36:534–539. doi: 10.7507/1002-1892.202201072.
    1. Deng Y., Wang L., Zhao C., Tang S., Cheng X., Deng H.-W., Zhou W. A deep learning-based approach to automatic proximal femur segmentation in quantitative CT images. Med. Biol. Eng. Comput. 2022;60:1417–1429. doi: 10.1007/s11517-022-02529-9.
    1. Tang X., Li X., Gu X., Zhao Y., Liu A., Liu Y., Tao Y. Automatic modeling of the knee joint based on artificial intelligence. Zhongguo Xiu Fu Chong Jian Wai Ke Za Zhi = Zhongguo Xiufu Chongjian Waike Zazhi=Chin. J. Reparative Reconstr. Surg. 2023;37:348–352. doi: 10.7507/1002-1892.202212008.
    1. León-Muñoz V.J., Manca S., López-López M., Martínez-Martínez F., Santonja-Medina F. Coronal and axial alignment relationship in Caucasian patients with osteoarthritis of the knee. Sci. Rep. 2021;11:7836. doi: 10.1038/s41598-021-87483-6.
    1. León-Muñoz V.J., López-López M., Martínez-Martínez F., Santonja-Medina F. Comparison of weight-bearing full-length radiographs and computed-tomography-scan-based three-dimensional models in the assessment of knee joint coronal alignment. Knee. 2020;27:543–551. doi: 10.1016/j.knee.2019.11.017.
    1. Liu D., Li Y., Li T., Yu Y., Cai G., Yang G., Wang G. The use of a 3D-printed individualized navigation template to assist in the anatomical reconstruction surgery of the anterior cruciate ligament. Ann. Transl. Med. 2020;8:1656. doi: 10.21037/atm-20-7515.
    1. Zhou F., Duh H.B.-L., Billinghurst M. Trends in augmented reality tracking, interaction and display: A review of ten years of ISMAR; Proceedings of the 2008 7th IEEE/ACM International Symposium on Mixed and Augmented Reality; Washington, DC, USA. 15–18 September 2008; pp. 193–202.
    1. Daniel C., Ramos O. Augmented Reality for Assistance of Total Knee Replacement. J. Electr. Comput. Eng. 2016;2016:9358369. doi: 10.1155/2016/9358369.
    1. Pokhrel S., Alsadoon A., Prasad P.W.C., Paul M. A novel augmented reality (AR) scheme for knee replacement surgery by considering cutting error accuracy. Int. J. Med. Robot. 2019;15:e1958. doi: 10.1002/rcs.1958.
    1. Wang L., Sun Z., Zhang X., Sun Z., Wang J. In: A HoloLens Based Augmented Reality Navigation System for Minimally Invasive Total Knee Arthroplasty BT—Intelligent Robotics and Applications. Yu H., Liu J., Liu L., Ju Z., Liu Y., Zhou D., editors. Springer International Publishing; Cham, Switzerland: 2019. pp. 519–530.
    1. Auvinet E., Maillot C., Uzoho C. Personalized Hip and Knee Joint Replacement. Springer International Publishing; Cham, Switzerland: 2020. Augmented Reality Technology for Joint Replacement; pp. 321–328.
    1. Fallavollita P., Brand A., Wang L., Euler E., Thaller P., Navab N., Weidert S. An augmented reality C-arm for intraoperative assessment of the mechanical axis: A preclinical study. Int. J. Comput. Assist. Radiol. Surg. 2016;11:2111–2117. doi: 10.1007/s11548-016-1426-z.
    1. Su S., Lei P., Wang C., Gao F., Zhong D., Hu Y. Mixed Reality Technology in Total Knee Arthroplasty: An Updated Review With a Preliminary Case Report. Front. Surg. 2022;9:804029. doi: 10.3389/fsurg.2022.804029.
    1. McKnight R.R., Pean C.A., Buck J.S., Hwang J.S., Hsu J.R., Pierrie S.N. Virtual Reality and Augmented Reality-Translating Surgical Training into Surgical Technique. Curr. Rev. Musculoskelet. Med. 2020;13:663–674. doi: 10.1007/s12178-020-09667-3.
    1. Eberlein T.J. A new paradigm in surgical training. J. Am. Coll. Surg. 2014;218:511–518. doi: 10.1016/j.jamcollsurg.2013.12.045.
    1. Edwards T.C., Patel A., Szyszka B., Coombs A.W., Liddle A.D., Kucheria R., Cobb J.P., Logishetty K. Immersive virtual reality enables technical skill acquisition for scrub nurses in complex revision total knee arthroplasty. Arch. Orthop. Trauma Surg. 2021;141:2313–2321. doi: 10.1007/s00402-021-04050-4.
    1. Zaid M.B., Dilallo M., Shau D., Ward D.T., Barry J.J. Virtual Reality as a Learning Tool for Trainees in Unicompartmental Knee Arthroplasty: A Randomized Controlled Trial. J. Am. Acad. Orthop. Surg. 2022;30:84–90. doi: 10.5435/JAAOS-D-20-01357.
    1. Morita K., Nii M., Koh M.-S., Kashiwa K., Nakayama H., Kambara S., Yoshiya S., Kobashi S. Bone Tunnel Placement Determination Method for 3D Images and Its Evaluation for Anterior Cruciate Ligament Reconstruction. Curr. Med. Imaging. 2020;16:491–498. doi: 10.2174/1573405614666181030125846.
    1. Guo N., Yang B., Ji X., Wang Y., Hu L., Wang T. Intensity-based 2D-3D registration for an ACL reconstruction navigation system. Int. J. Med. Robot. 2019;15:e2008. doi: 10.1002/rcs.2008.
    1. Chen F., Cui X., Han B., Liu J., Zhang X., Liao H. Augmented reality navigation for minimally invasive knee surgery using enhanced arthroscopy. Comput. Methods Programs Biomed. 2021;201:105952. doi: 10.1016/j.cmpb.2021.105952.
    1. van der Putten K., Anderson M.B., van Geenen R.C. Looking through the Lens: The Reality of Telesurgical Support with Interactive Technology Using Microsoft’s HoloLens 2. Case Rep. Orthop. 2022;2022:5766340. doi: 10.1155/2022/5766340.
    1. Blasco J., Igual-Camacho C., Blasco M., Antón-Antón V., Ortiz-Llueca L., Roig-Casasús S. The efficacy of virtual reality tools for total knee replacement rehabilitation: A systematic review. Physiother. Theory Pract. 2021;37:682–692. doi: 10.1080/09593985.2019.1641865.
    1. Fung V., Ho A., Shaffer J., Chung E., Gomez M. Use of Nintendo Wii FitTM in the rehabilitation of outpatients following total knee replacement: A preliminary randomised controlled trial. Physiotherapy. 2012;98:183–188. doi: 10.1016/j.physio.2012.04.001.
    1. Piqueras M., Marco E., Coll M., Escalada F., Ballester A., Cinca C., Belmonte R., Muniesa J.M. Effectiveness of an interactive virtual telerehabilitation system in patients after total knee arthoplasty: A randomized controlled trial. J. Rehabil. Med. 2013;45:392–396. doi: 10.2340/16501977-1119.
    1. Christiansen C.L., Bade M.J., Davidson B.S., Dayton M.R., Stevens-Lapsley J.E. Effects of Weight-Bearing Biofeedback Training on Functional Movement Patterns Following Total Knee Arthroplasty: A Randomized Controlled Trial. J. Orthop. Sport. Phys. Ther. 2015;45:647–655. doi: 10.2519/jospt.2015.5593.
    1. Ficklscherer A., Stapf J., Meissner K.M., Niethammer T., Lahner M., Wagenhäuser M., Müller P.E., Pietschmann M.F. Testing the feasibility and safety of the Nintendo Wii gaming console in orthopedic rehabilitation: A pilot randomized controlled study. Arch. Med. Sci. 2016;12:1273–1278. doi: 10.5114/aoms.2016.59722.
    1. Su C.-H. Developing and evaluating effectiveness of 3D game-based rehabilitation system for Total Knee Replacement Rehabilitation patients. Multimed. Tools Appl. 2016;75:10037–10057. doi: 10.1007/s11042-015-2820-1.
    1. Roig-Casasús S., Blasco J.M., López-Bueno L., Blasco-Igual M.C. Balance Training With a Dynamometric Platform Following Total Knee Replacement: A Randomized Controlled Trial. J. Geriatr. Phys. Ther. 2018;41:204–209. doi: 10.1519/JPT.0000000000000121.
    1. Berton A., Longo U.G., Candela V., Fioravanti S., Giannone L., Arcangeli V., Alciati V., Berton C., Facchinetti G., Marchetti A., et al. Virtual Reality, Augmented Reality, Gamification, and Telerehabilitation: Psychological Impact on Orthopedic Patients’ Rehabilitation. J. Clin. Med. 2020;9:2567. doi: 10.3390/jcm9082567.
    1. Li L. Effect of Remote Control Augmented Reality Multimedia Technology for Postoperative Rehabilitation of Knee Joint Injury. Comput. Math. Methods Med. 2022;2022:9320063. doi: 10.1155/2022/9320063.
    1. Chan Z.Y.S., MacPhail A.J.C., Au I.P.H., Zhang J.H., Lam B.M.F., Ferber R., Cheung R.T.H. Walking with head-mounted virtual and augmented reality devices: Effects on position control and gait biomechanics. PLoS ONE. 2019;14:e0225972. doi: 10.1371/journal.pone.0225972.
    1. Nagymáté G., Kiss R.M. Affordable gait analysis using augmented reality markers. PLoS ONE. 2019;14:e0212319. doi: 10.1371/journal.pone.0212319.
    1. Braga Rodrigues T., Ó Catháin C., O’Connor N.E., Murray N. A Quality of Experience assessment of haptic and augmented reality feedback modalities in a gait analysis system. PLoS ONE. 2020;15:e0230570. doi: 10.1371/journal.pone.0230570.
    1. Guerrero G., da Silva F.J.M., Fernández-Caballero A., Pereira A. Augmented Humanity: A Systematic Mapping Review. Sensors. 2022;22:514. doi: 10.3390/s22020514.

Source: PubMed

Sponsors et collaborateurs

Les conditions médicales

Interventions en matière de drogue

3
S'abonner